wsgtest
/
Test-wsg-0001
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add special_tokens_map.json 

Signed-off-by: wsgtest <4918b15d@leinao.ai>
This commit is contained in:
wsgtest 2024-07-05 15:45:28 +08:00
parent d3160fd800
commit 7a7f0cf3dc
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
special_tokens_map.json Normal file
View File

@ -0,0 +1 @@
{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>", "pad_token": "<|endoftext|>"}