diff --git a/finetune_demo/README.md b/finetune_demo/README.md index 10aaea1..561e9be 100644 --- a/finetune_demo/README.md +++ b/finetune_demo/README.md @@ -253,7 +253,7 @@ pip install -r requirements.txt 通过以下代码执行 **单机多卡/多机多卡** 运行,这是使用 `deepspeed` 作为加速方案的,您需要安装 `deepspeed`。接着,按照此命令运行: ```shell -OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune_hf.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune +OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune_vision.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune ``` @@ -261,7 +261,7 @@ OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune_ ```shell python finetune.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune -python finetune.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune +python finetune_vision.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune ``` ## 从保存点进行微调 diff --git a/finetune_demo/README_en.md b/finetune_demo/README_en.md index bbad31a..1b2db5d 100644 --- a/finetune_demo/README_en.md +++ b/finetune_demo/README_en.md @@ -258,7 +258,7 @@ Execute **single machine multi-card/multi-machine multi-card** run through the f the acceleration solution, and you need to install `deepspeed`. ```shell -OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune_hf.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune +OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune OMP_NUM_THREADS=1 torchrun --standalone --nnodes=1 --nproc_per_node=8 finetune_vision.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune ``` @@ -266,7 +266,7 @@ Execute **single machine single card** run through the following code. ```shell python finetune.py data/AdvertiseGen/ THUDM/glm-4-9b-chat configs/lora.yaml # For Chat Fine-tune -python finetune.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune +python finetune_vision.py data/CogVLM-311K/ THUDM/glm-4v-9b configs/lora.yaml # For VQA Fine-tune ``` ## Fine-tune from a saved point diff --git a/finetune_demo/finetune_vision.py b/finetune_demo/finetune_vision.py index 6b47e37..c259d8a 100644 --- a/finetune_demo/finetune_vision.py +++ b/finetune_demo/finetune_vision.py @@ -226,7 +226,6 @@ class DataManager(object): orig_dataset = self._get_dataset(split) if orig_dataset is None: return - if remove_orig_columns: remove_columns = orig_dataset.column_names else: @@ -236,6 +235,10 @@ class DataManager(object): batched=batched, remove_columns=remove_columns, num_proc=self._num_proc, + # This is default params of orig_dataset.map, and you can change it smaller + # https://github.com/THUDM/GLM-4/issues/277 + writer_batch_size=1000, + batch_size=1000, )