- **Demo [optional]:** [More Information Needed] -->
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
The intended use of the Helium model is research and development of natural language processing systems, including but not limited to language generation and understanding.
The model can be used in English, French, German, Italian, Portuguese and Spanish.
For most downstream use cases, the model should be aligned with supervised fine-tuning, RLHF or related methods.
### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
The model should not be used in other languages than the ones on which it was trained.
The model is not intended to be used for any malicious or illegal activities of any kind.
The model was not fine-tuned to follow instructions, and thus should not be used as such.
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
Helium-1 preview is a base language model, which was not aligned to human preferences.
As such, the model can generate incorrect, biased, harmful or generally unhelpful content.
Thus, the model should not be used for downstream applications without further alignment, evaluations and mitigations of risks.
<!-- Thus, it should not be used without further evaluations of risks and mitigations. -->
## How to Get Started with the Model
Use the code below to get started with the model.
```python
import torch
from transformers import pipeline
model_id = "kyutai/helium-1-preview-2b"
pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
text = pipe("Hello, today is a great day to")
```
## Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
Helium-1 preview was trained on a mix of data including: Wikipedia, Stack Exchange, open-access scientific articles (from peS2o) and Common Crawl.
<!--#### Training Hyperparameters
- **Training regime:** [More Information Needed] -->
