317 lines
11 KiB
Markdown
317 lines
11 KiB
Markdown
---
|
||
frameworks:
|
||
- Pytorch
|
||
license: other
|
||
tasks:
|
||
- text-generation
|
||
---
|
||
# Model Card for CodeFuse-CodeGeeX2-6B
|
||
<p align="center">
|
||
<img src="https://modelscope.cn/api/v1/models/codefuse-ai/CodeFuse-CodeGeeX2-6B/repo?Revision=master&FilePath=LOGO.jpg&View=true" width="800"/>
|
||
<p>
|
||
|
||
[[中文]](#chinese) [[English]](#english)
|
||
|
||
#### Clone with HTTP
|
||
```bash
|
||
git clone https://www.modelscope.cn/codefuse-ai/CodeFuse-CodeGeeX2-6B.git
|
||
```
|
||
|
||
<a id="english"></a>
|
||
|
||
## Model Description
|
||
|
||
CodeFuse-CodeGeeX2-6B is a 6B Code-LLM finetuned by LoRA of multiple code tasks on the base model CodeGeeX2.
|
||
|
||
<br>
|
||
|
||
## News and Updates
|
||
|
||
🔥🔥 2023-11-10 CodeFuse-CodeGeeX2-6B has been released, achieving a pass@1 (greedy decoding) score of 45.12% on HumanEval, which is a 9.22% increase compared to CodeGeeX2 35.9%.
|
||
|
||
🔥🔥 2023-10-20 CodeFuse-QWen-14B technical documentation has been released. For those interested, please refer to the CodeFuse article on our WeChat official account via the provided link.(https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
|
||
|
||
🔥🔥 2023-10-16 CodeFuse-QWen-14B has been released, achieving a pass@1 (greedy decoding) score of 48.78% on HumanEval, which is a 16% increase compared to Qwen-14b's 32.3%.
|
||
|
||
🔥🔥 2023-09-27 CodeFuse-StarCoder-15B has been released, achieving a pass@1 (greedy decoding) score of 54.9% on HumanEval, which is a 21% increase compared to StarCoder's 33.6%.
|
||
|
||
🔥🔥🔥 2023-09-26 We are pleased to announce the release of the [4-bit quantized version](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary) of [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary). Despite the quantization process, the model still achieves a remarkable 73.8% accuracy (greedy decoding) on the HumanEval pass@1 metric.
|
||
|
||
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary) has achived 74.4% of pass@1 (greedy decoding) on HumanEval, which is SOTA results for openspurced LLMs at present.
|
||
|
||
<br>
|
||
|
||
## Code Community
|
||
|
||
**Homepage**: 🏡 https://github.com/codefuse-ai (**Please give us your support with a Star🌟 + Fork🚀 + Watch👀**)
|
||
|
||
+ If you wish to fine-tune the model yourself, you can visit ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
|
||
|
||
+ If you wish to deploy the model yourself, you can visit ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
|
||
|
||
+ If you wish to see a demo of the model, you can visit ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
|
||
|
||
<br>
|
||
|
||
## Performance
|
||
|
||
|
||
| Model | HumanEval(pass@1) | Date |
|
||
|:----------------------------|:-----------------:|:-------:|
|
||
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 |
|
||
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 |
|
||
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 |
|
||
| GPT-4(zero-shot) | 67.0% | 2023.3 |
|
||
| PanGu-Coder2 15B | 61.6% | 2023.8 |
|
||
| CodeLlama-34b-Python | 53.7% | 2023.8 |
|
||
| CodeLlama-34b | 48.8% | 2023.8 |
|
||
| GPT-3.5(zero-shot) | 48.1% | 2022.11 |
|
||
| OctoCoder | 46.2% | 2023.8 |
|
||
| StarCoder-15B | 33.6% | 2023.5 |
|
||
| Qwen-14b | 32.3% | 2023.10 |
|
||
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 |
|
||
| **CodeFuse-QWen-14B** | **48.78%** | 2023.10 |
|
||
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 |
|
||
|
||
|
||
<br>
|
||
|
||
## Requirements
|
||
|
||
* python>=3.8
|
||
* pytorch>=2.0.0
|
||
* transformers==4.33.2
|
||
* Sentencepiece
|
||
* CUDA 11.4
|
||
<br>
|
||
|
||
## Inference String Format
|
||
|
||
The inference string is a concatenated string formed by combining conversation data(system, human and bot contents) in the training data format. It is used as input during the inference process.
|
||
Here is an example format of the concatenated string:
|
||
|
||
```python
|
||
"""
|
||
<s>system
|
||
System instruction
|
||
<s>human
|
||
Human 1st round input
|
||
<s>bot
|
||
Bot 1st round output<|endoftext|>
|
||
<s>human
|
||
Human 2nd round input
|
||
<s>bot
|
||
Bot 2nd round output<|endoftext|>
|
||
...
|
||
...
|
||
...
|
||
<s>human
|
||
Human nth round input
|
||
<s>bot
|
||
{Bot output to be genreated}<|endoftext|>
|
||
"""
|
||
```
|
||
|
||
When applying inference, you always make your input string end with "\<s\>bot" to ask the model generating answers.
|
||
|
||
|
||
## Quickstart
|
||
|
||
|
||
```bash
|
||
pip install transformers modelscope cpm_kernels -U
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
```python
|
||
import torch
|
||
from modelscope import (
|
||
AutoTokenizer,
|
||
AutoModelForCausalLM,
|
||
snapshot_download
|
||
)
|
||
model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
|
||
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
|
||
tokenizer.padding_side = "left"
|
||
tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids("<unk>")
|
||
tokenizer.eos_token_id = tokenizer.convert_tokens_to_ids("</s>")
|
||
tokenizer.pad_token = "<unk>"
|
||
tokenizer.eos_token = "</s>"
|
||
# try 4bit loading if cuda memory not enough
|
||
model = AutoModelForCausalLM.from_pretrained(model_dir,
|
||
trust_remote_code=True,
|
||
load_in_4bit=False,
|
||
device_map="auto",
|
||
torch_dtype=torch.bfloat16)
|
||
model.eval()
|
||
|
||
HUMAN_ROLE_START_TAG = "<s>human\n"
|
||
BOT_ROLE_START_TAG = "<s>bot\n"
|
||
|
||
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}"
|
||
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
|
||
outputs = model.generate(
|
||
inputs=inputs["input_ids"],
|
||
attention_mask=inputs["attention_mask"],
|
||
max_new_tokens=512,
|
||
top_p=0.95,
|
||
temperature=0.1,
|
||
do_sample=True,
|
||
eos_token_id=tokenizer.eos_token_id,
|
||
pad_token_id=tokenizer.pad_token_id
|
||
)
|
||
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
|
||
print(gen_text)
|
||
```
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<a id="chinese"></a>
|
||
|
||
## 模型简介
|
||
|
||
CodeFuse-CodeGeeX2-6B 是一个通过LoRA对基座模型CodeGeeeX2进行多代码任务微调的代码大模型。
|
||
<br>
|
||
|
||
## 新闻
|
||
|
||
🔥🔥 2023-11-10 开源了CodeFuse-CodeGeeX2-6B模型，在HumanEval pass@1(greedy decoding)上可以达到48.12%, 比CodeGeeX2提高了9.22%的代码能力（HumanEval）
|
||
|
||
🔥🔥 2023-10-20 公布了CodeFuse-QWen-14B技术文档，感兴趣详见微信公众号CodeFuse文章：https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw
|
||
|
||
🔥🔥 2023-10-16开源了CodeFuse-QWen-14B模型，在HumanEval pass@1(greedy decoding)上可以达到48.78%, 比Qwen-14b提高了16%的代码能力（HumanEval）
|
||
|
||
🔥🔥 2023-09-27开源了CodeFuse-StarCoder-15B模型，在HumanEval pass@1(greedy decoding)上可以达到54.9%, 比StarCoder提高了21%的代码能力（HumanEval）
|
||
|
||
🔥🔥🔥 2023-09-26 [CodeFuse-CodeLlama-34B 4bits](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary)量化版本发布，量化后模型在HumanEval pass@1指标为73.8% (贪婪解码)。
|
||
|
||
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary)发布，HumanEval pass@1指标达到74.4% (贪婪解码), 为当前开源SOTA。
|
||
|
||
<br>
|
||
|
||
## 代码社区
|
||
**大本营**： 🏡 https://github.com/codefuse-ai （**请支持我们的项目Star🌟 + Fork🚀 + Watch👀**）
|
||
|
||
+ 如果您想自己微调该模型，可以访问 ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
|
||
|
||
+ 如果您想自己部署该模型，可以访问 ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
|
||
|
||
+ 如果您想观看该模型示例，可以访问 ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
|
||
|
||
<br>
|
||
|
||
|
||
## 评测表现
|
||
|
||
### 代码
|
||
|
||
|
||
| 模型 | HumanEval(pass@1) | 日期 |
|
||
|:----------------------------|:-----------------:|:-------:|
|
||
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 |
|
||
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 |
|
||
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 |
|
||
| GPT-4(zero-shot) | 67.0% | 2023.3 |
|
||
| PanGu-Coder2 15B | 61.6% | 2023.8 |
|
||
| CodeLlama-34b-Python | 53.7% | 2023.8 |
|
||
| CodeLlama-34b | 48.8% | 2023.8 |
|
||
| GPT-3.5(zero-shot) | 48.1% | 2022.11 |
|
||
| OctoCoder | 46.2% | 2023.8 |
|
||
| StarCoder-15B | 33.6% | 2023.5 |
|
||
| Qwen-14b | 32.3% | 2023.10 |
|
||
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 |
|
||
| **CodeFuse-QWen-14B** | **48.78%** | 2023.8 |
|
||
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 |
|
||
|
||
|
||
## Requirements
|
||
|
||
* python>=3.8
|
||
* pytorch>=2.0.0
|
||
* transformers==4.33.2
|
||
* Sentencepiece
|
||
* CUDA 11.4
|
||
<br>
|
||
|
||
## 推理数据格式
|
||
|
||
推理数据为模型在训练数据格式下拼接的字符串形式，它也是推理时输入prompt拼接的方式：
|
||
|
||
```python
|
||
"""
|
||
<s>system
|
||
这是System指令
|
||
<s>human
|
||
这是第1轮用户输入的问题
|
||
<s>bot
|
||
这是第1轮模型生成的内容<|endoftext|>
|
||
<s>human
|
||
这是第2轮用户输入的问题
|
||
<s>bot
|
||
这是第2轮模型生成的内容<|endoftext|>
|
||
...
|
||
...
|
||
...
|
||
<s>human
|
||
这是第n轮用户输入的问题
|
||
<s>bot
|
||
{模型现在要生成的内容}<|endoftext|>
|
||
"""
|
||
```
|
||
|
||
推理时，请确保拼接的prompt字符串以"\<s\>bot\n"结尾，引导模型生成回答。
|
||
|
||
## 快速使用
|
||
|
||
|
||
```bash
|
||
pip install transformers modelscope cpm_kernels -U
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
```python
|
||
import torch
|
||
from modelscope import (
|
||
AutoTokenizer,
|
||
AutoModelForCausalLM,
|
||
snapshot_download
|
||
)
|
||
model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
|
||
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
|
||
tokenizer.padding_side = "left"
|
||
tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids("<unk>")
|
||
tokenizer.eos_token_id = tokenizer.convert_tokens_to_ids("</s>")
|
||
tokenizer.pad_token = "<unk>"
|
||
tokenizer.eos_token = "</s>"
|
||
# try 4bit loading if cuda memory not enough
|
||
model = AutoModelForCausalLM.from_pretrained(model_dir,
|
||
trust_remote_code=True,
|
||
load_in_4bit=False,
|
||
device_map="auto",
|
||
torch_dtype=torch.bfloat16)
|
||
model.eval()
|
||
|
||
HUMAN_ROLE_START_TAG = "<s>human\n"
|
||
BOT_ROLE_START_TAG = "<s>bot\n"
|
||
|
||
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}"
|
||
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
|
||
outputs = model.generate(
|
||
inputs=inputs["input_ids"],
|
||
attention_mask=inputs["attention_mask"],
|
||
max_new_tokens=512,
|
||
top_p=0.95,
|
||
temperature=0.1,
|
||
do_sample=True,
|
||
eos_token_id=tokenizer.eos_token_id,
|
||
pad_token_id=tokenizer.pad_token_id
|
||
)
|
||
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
|
||
print(gen_text)
|
||
```
|