forked from ailab/bert-base-chinese
76 lines
1.8 KiB
Markdown
76 lines
1.8 KiB
Markdown
|
---
|
||
|
|
language: zh
|
||
|
|
---
|
||
|
|
|
||
|
|
# Bert-base-chinese
|
||
|
|
|
||
|
|
## Table of Contents
|
||
|
|
- [Model Details](#model-details)
|
||
|
|
- [Uses](#uses)
|
||
|
|
- [Risks, Limitations and Biases](#risks-limitations-and-biases)
|
||
|
|
- [Training](#training)
|
||
|
|
- [Evaluation](#evaluation)
|
||
|
|
- [How to Get Started With the Model](#how-to-get-started-with-the-model)
|
||
|
|
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
### Model Description
|
||
|
|
|
||
|
|
This model has been pre-trained for Chinese, training and random input masking has been applied independently to word pieces (as in the original BERT paper).
|
||
|
|
|
||
|
|
- **Developed by:** HuggingFace team
|
||
|
|
- **Model Type:** Fill-Mask
|
||
|
|
- **Language(s):** Chinese
|
||
|
|
- **License:** [More Information needed]
|
||
|
|
- **Parent Model:** See the [BERT base uncased model](https://huggingface.co/bert-base-uncased) for more information about the BERT base model.
|
||
|
|
|
||
|
|
### Model Sources
|
||
|
|
- **Paper:** [BERT](https://arxiv.org/abs/1810.04805)
|
||
|
|
|
||
|
|
## Uses
|
||
|
|
|
||
|
|
#### Direct Use
|
||
|
|
|
||
|
|
This model can be used for masked language modeling
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
## Risks, Limitations and Biases
|
||
|
|
**CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.**
|
||
|
|
|
||
|
|
Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).
|
||
|
|
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
#### Training Procedure
|
||
|
|
* **type_vocab_size:** 2
|
||
|
|
* **vocab_size:** 21128
|
||
|
|
* **num_hidden_layers:** 12
|
||
|
|
|
||
|
|
#### Training Data
|
||
|
|
[More Information Needed]
|
||
|
|
|
||
|
|
## Evaluation
|
||
|
|
|
||
|
|
#### Results
|
||
|
|
|
||
|
|
[More Information Needed]
|
||
|
|
|
||
|
|
|
||
|
|
## How to Get Started With the Model
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForMaskedLM
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("bert-base-chinese")
|
||
|
|
|
||
|
|
model = AutoModelForMaskedLM.from_pretrained("bert-base-chinese")
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|