first commit
This commit is contained in:
xxl
parent
1e514f0724
commit
ced0816f93
385
README.md
385
README.md
|
|
@ -1,3 +1,384 @@
|
|||
# lb-reranker-0.5B-v1.0
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
- es
|
||||
- de
|
||||
- ar
|
||||
- ru
|
||||
- ja
|
||||
- ko
|
||||
- hi
|
||||
- sk
|
||||
- vi
|
||||
- tr
|
||||
- fi
|
||||
- id
|
||||
- fa
|
||||
- 'no'
|
||||
- th
|
||||
- sv
|
||||
- pt
|
||||
- da
|
||||
- bn
|
||||
- te
|
||||
- ro
|
||||
- it
|
||||
- fr
|
||||
- nl
|
||||
- sw
|
||||
- pl
|
||||
- hu
|
||||
- cs
|
||||
- el
|
||||
- uk
|
||||
- mr
|
||||
- ta
|
||||
- tl
|
||||
- bg
|
||||
- lt
|
||||
- ur
|
||||
- he
|
||||
- gu
|
||||
- kn
|
||||
- am
|
||||
- kk
|
||||
- hr
|
||||
- uz
|
||||
- jv
|
||||
- ca
|
||||
- az
|
||||
- ms
|
||||
- sr
|
||||
- sl
|
||||
- yo
|
||||
- lv
|
||||
- is
|
||||
- ha
|
||||
- ka
|
||||
- et
|
||||
- bs
|
||||
- hy
|
||||
- ml
|
||||
- pa
|
||||
- mt
|
||||
- km
|
||||
- sq
|
||||
- or
|
||||
- as
|
||||
- my
|
||||
- mn
|
||||
- af
|
||||
- be
|
||||
- ga
|
||||
- mk
|
||||
- cy
|
||||
- gl
|
||||
- ceb
|
||||
- la
|
||||
- yi
|
||||
- lb
|
||||
- tg
|
||||
- gd
|
||||
- ne
|
||||
- ps
|
||||
- eu
|
||||
- ky
|
||||
- ku
|
||||
- si
|
||||
- ht
|
||||
- eo
|
||||
- lo
|
||||
- fy
|
||||
- sd
|
||||
- mg
|
||||
- so
|
||||
- ckb
|
||||
- su
|
||||
- nn
|
||||
datasets:
|
||||
- lightblue/reranker_continuous_filt_max7_train
|
||||
base_model:
|
||||
- Qwen/Qwen2.5-0.5B-Instruct
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- reranker
|
||||
widget:
|
||||
- text: "<<<Query>>>\nHow many languages has LB-Reranker been trained on?\n\n\n<<<Context>>>\nLB-Reranker has been trained on more than 95 languages."
|
||||
example_title: Positive example (7/7)
|
||||
- text: "<<<Query>>>\nHow many languages has LB-Reranker been trained on?\n\n\n<<<Context>>>\nAA-Reranker is applicable to a broad range of use cases."
|
||||
example_title: Negative example (2/7)
|
||||
|
||||
lb-reranker-0.5B-v1.0
|
||||
---
|
||||
|
||||
# LB Reranker v1.0
|
||||
|
||||
<div style="width: 100%; height: 160px;
|
||||
display: flex; align-items: center;
|
||||
justify-content: center;
|
||||
border: 8px solid black;
|
||||
font-size: 120px; font-weight: bold;
|
||||
text-align: center;
|
||||
color: #438db8;
|
||||
font-family: 'Helvetica Neue', sans-serif;">
|
||||
LBR
|
||||
</div>
|
||||
|
||||
The LB Reranker has been trained to determine the relatedness of a given query to a piece of text, therefore allowing it to be used as a ranker or reranker in various retrieval-based tasks.
|
||||
|
||||
This model is fine-tuned from a [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) model checkpoint and was trained for roughly 5.5 hours using the 8 x L20 instance ([ecs.gn8is-8x.32xlarge](https://www.alibabacloud.com/help/en/ecs/user-guide/gpu-accelerated-compute-optimized-and-vgpu-accelerated-instance-families-1)) on [Alibaba Cloud](https://www.alibabacloud.com/).
|
||||
|
||||
The training data for this model can be found at [lightblue/reranker_continuous_filt_max7_train](https://huggingface.co/datasets/lightblue/reranker_continuous_filt_max7_train) and the code for generating this data as well as running the training of the model can be found on [our Github repo](https://github.com/lightblue-tech/lb-reranker).
|
||||
|
||||
Trained on data in over 95 languages, this model is applicable to a broad range of use cases.
|
||||
|
||||
This model has three main benefits over comparable rerankers.
|
||||
1. It has shown slightly higher performance on evaluation benchmarks.
|
||||
2. It has been trained on more languages than any previous model.
|
||||
3. It is a simple Causal LM model trained to output a string between "1" and "7".
|
||||
|
||||
This last point means that this model can be used natively with many widely available inference packages, including vLLM and LMDeploy.
|
||||
This in turns allows our reranker to benefit from improvements to inference as and when these packages release them.
|
||||
|
||||
Update: We have also found that this model works pretty well as a code snippet reranker too (P@1 of 96%)! See our [Colab](https://colab.research.google.com/drive/1ABL1xaarekLIlVJKbniYhXgYu6ZNwfBm?usp=sharing) for more details.
|
||||
|
||||
# How to use
|
||||
|
||||
The model was trained to expect an input such as:
|
||||
|
||||
```
|
||||
<<<Query>>>
|
||||
{your_query_here}
|
||||
|
||||
<<<Context>>>
|
||||
{your_context_here}
|
||||
```
|
||||
|
||||
And to output a string of a number between 1-7.
|
||||
|
||||
In order to make a continuous score that can be used for reranking query-context pairs (i.e. a method with few ties), we calculate the expectation value of the scores.
|
||||
|
||||
We include scripts to do this in vLLM, LMDeploy, and OpenAI (hosted for free on Huggingface):
|
||||
|
||||
|
||||
<ul>
|
||||
<li><b>vLLM</b>
|
||||
|
||||
Install [vLLM](https://github.com/vllm-project/vllm/) using `pip install vllm`.
|
||||
|
||||
<details open>
|
||||
<summary>Show vLLM code</summary>
|
||||
|
||||
```python
|
||||
from vllm import LLM, SamplingParams
|
||||
import numpy as np
|
||||
|
||||
def make_reranker_input(t, q):
|
||||
return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"
|
||||
|
||||
def make_reranker_inference_conversation(context, question):
|
||||
system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."
|
||||
|
||||
return [
|
||||
{"role": "system", "content": system_message},
|
||||
{"role": "user", "content": make_reranker_input(context, question)},
|
||||
]
|
||||
|
||||
def get_prob(logprob_dict, tok_id):
|
||||
return np.exp(logprob_dict[tok_id].logprob) if tok_id in logprob_dict.keys() else 0
|
||||
|
||||
llm = LLM("lightblue/lb-reranker-v1.0")
|
||||
sampling_params = SamplingParams(temperature=0.0, logprobs=14, max_tokens=1)
|
||||
tok = llm.llm_engine.tokenizer.tokenizer
|
||||
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]
|
||||
|
||||
query_texts = [
|
||||
("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
]
|
||||
|
||||
chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
|
||||
responses = llm.chat(chats, sampling_params)
|
||||
probs = np.array([[get_prob(r.outputs[0].logprobs[0], y) for y in idx_tokens] for r in responses])
|
||||
|
||||
N = probs.shape[1]
|
||||
M = probs.shape[0]
|
||||
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)
|
||||
|
||||
expected_vals = (probs * idxs).sum(axis=1)
|
||||
print(expected_vals)
|
||||
# [6.66570732 1.86686378 1.01102923]
|
||||
```
|
||||
|
||||
</details></li>
|
||||
<li><b>LMDeploy</b>
|
||||
|
||||
Install [LMDeploy](https://github.com/InternLM/lmdeploy) using `pip install lmdeploy`.
|
||||
|
||||
<details>
|
||||
<summary>Show LMDeploy code</summary>
|
||||
|
||||
```python
|
||||
# Un-comment this if running in a Jupyter notebook, Colab etc.
|
||||
# import nest_asyncio
|
||||
# nest_asyncio.apply()
|
||||
|
||||
from lmdeploy import GenerationConfig, ChatTemplateConfig, pipeline
|
||||
import numpy as np
|
||||
|
||||
def make_reranker_input(t, q):
|
||||
return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"
|
||||
|
||||
def make_reranker_inference_conversation(context, question):
|
||||
system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."
|
||||
|
||||
return [
|
||||
{"role": "system", "content": system_message},
|
||||
{"role": "user", "content": make_reranker_input(context, question)},
|
||||
]
|
||||
|
||||
def get_prob(logprob_dict, tok_id):
|
||||
return np.exp(logprob_dict[tok_id]) if tok_id in logprob_dict.keys() else 0
|
||||
|
||||
pipe = pipeline(
|
||||
"lightblue/lb-reranker-v1.0",
|
||||
chat_template_config=ChatTemplateConfig(
|
||||
model_name='qwen2d5',
|
||||
capability='chat'
|
||||
)
|
||||
)
|
||||
tok = pipe.tokenizer.model
|
||||
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]
|
||||
|
||||
query_texts = [
|
||||
("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
]
|
||||
|
||||
chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
|
||||
responses = pipe(
|
||||
chats,
|
||||
gen_config=GenerationConfig(temperature=1.0, logprobs=14, max_new_tokens=1, do_sample=True)
|
||||
)
|
||||
probs = np.array([[get_prob(r.logprobs[0], y) for y in idx_tokens] for r in responses])
|
||||
|
||||
N = probs.shape[1]
|
||||
M = probs.shape[0]
|
||||
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)
|
||||
|
||||
expected_vals = (probs * idxs).sum(axis=1)
|
||||
print(expected_vals)
|
||||
# [6.66415229 1.84342025 1.01133205]
|
||||
```
|
||||
|
||||
</details></li>
|
||||
<li><b>OpenAI (Hosted on Huggingface)</b>
|
||||
|
||||
Install [openai](https://github.com/openai/openai-python) using `pip install openai`.
|
||||
|
||||
<details>
|
||||
<summary>Show OpenAI + Huggingface Inference code</summary>
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
import numpy as np
|
||||
from multiprocessing import Pool
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
client = OpenAI(
|
||||
base_url="https://api-inference.huggingface.co/v1/",
|
||||
api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Change this to an access token from https://huggingface.co/settings/tokens
|
||||
)
|
||||
|
||||
def make_reranker_input(t, q):
|
||||
return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"
|
||||
|
||||
def make_reranker_inference_conversation(context, question):
|
||||
system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."
|
||||
|
||||
return [
|
||||
{"role": "system", "content": system_message},
|
||||
{"role": "user", "content": make_reranker_input(context, question)},
|
||||
]
|
||||
|
||||
def get_reranker_score(context_question_tuple):
|
||||
question, context = context_question_tuple
|
||||
|
||||
messages = make_reranker_inference_conversation(context, question)
|
||||
|
||||
completion = client.chat.completions.create(
|
||||
model="lightblue/lb-reranker-0.5B-v1.0",
|
||||
messages=messages,
|
||||
max_tokens=1,
|
||||
temperature=0.0,
|
||||
logprobs=True,
|
||||
top_logprobs=5, # Max allowed by the openai API as top_n_tokens must be >= 0 and <= 5. If this gets changed, fix to > 7.
|
||||
)
|
||||
|
||||
logprobs = completion.choices[0].logprobs.content[0].top_logprobs
|
||||
|
||||
calculated_score = sum([int(x.token) * np.exp(x.logprob) for x in logprobs])
|
||||
|
||||
return calculated_score
|
||||
|
||||
query_texts = [
|
||||
("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
|
||||
]
|
||||
|
||||
with Pool(processes=16) as p: # Allows for parallel processing
|
||||
expected_vals = list(tqdm(p.imap(get_reranker_score, query_texts), total=len(query_texts)))
|
||||
|
||||
print(expected_vals)
|
||||
# [6.64866580, 1.85144404, 1.010719508]
|
||||
```
|
||||
|
||||
</details></li>
|
||||
</ul>
|
||||
|
||||
# Evaluation
|
||||
|
||||
We perform an evaluation on 9 datasets from the [BEIR benchmark](https://github.com/beir-cellar/beir) that none of the evaluated models have been trained upon (to our knowledge).
|
||||
|
||||
* Arguana
|
||||
* Dbpedia-entity
|
||||
* Fiqa
|
||||
* NFcorpus
|
||||
* Scidocs
|
||||
* Scifact
|
||||
* Trec-covid-v2
|
||||
* Vihealthqa
|
||||
* Webis-touche2020
|
||||
|
||||
We evaluate on a subset of all queries (the first 250) to save evaluation time.
|
||||
|
||||
We find that our model performs similarly or better than many of the state-of-the-art reranker models in our evaluation, without compromising on inference speed.
|
||||
|
||||
We make our evaluation code and results available [on our Github](https://github.com/lightblue-tech/lb-reranker/blob/main/run_bier.ipynb).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As we can see, this reranker attains greater IR evaluation metrics compared to the two benchmarks we include for all positions apart from @1.
|
||||
|
||||
|
||||
|
||||
We also show that our model is, on average, faster than the BGE reranker v2.
|
||||
|
||||
# License
|
||||
|
||||
We share this model under an Apache 2.0 license.
|
||||
|
||||
# Developed by
|
||||
|
||||
<a href="https://www.lightblue-tech.com">
|
||||
<img src="https://www.lightblue-tech.com/wp-content/uploads/2023/08/color_%E6%A8%AA%E5%9E%8B-1536x469.png" alt="Lightblue technology logo" width="400"/>
|
||||
</a>
|
||||
|
||||
This model was trained by Peter Devine ([ptrdvn](https://huggingface.co/ptrdvn)) for Lightblue
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
{
|
||||
"</tool_call>": 151658,
|
||||
"<tool_call>": 151657,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
{
|
||||
"_name_or_path": "lightblue/reranker_0.5_cont_filt_7max",
|
||||
"architectures": [
|
||||
"Qwen2ForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 896,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 4864,
|
||||
"max_position_embeddings": 32768,
|
||||
"max_window_layers": 21,
|
||||
"model_type": "qwen2",
|
||||
"num_attention_heads": 14,
|
||||
"num_hidden_layers": 24,
|
||||
"num_key_value_heads": 2,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.47.1",
|
||||
"use_cache": false,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"bos_token_id": 151643,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
151645,
|
||||
151643
|
||||
],
|
||||
"pad_token_id": 151643,
|
||||
"repetition_penalty": 1.1,
|
||||
"temperature": 0.7,
|
||||
"top_k": 20,
|
||||
"top_p": 0.8,
|
||||
"transformers_version": "4.47.1"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
|
@ -0,0 +1,31 @@
|
|||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
|
@ -0,0 +1,209 @@
|
|||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"padding_side": "right",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue