---
license: apache-2.0
library_name: transformers
base_model:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
---

**Model Name**: Tiny-R1-32B-Preview   
**Title**: SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.


## Introduction
We introduce our first-generation reasoning model, Tiny-R1-32B-Preview, which outperforms the 70B model Deepseek-R1-Distill-Llama-70B and nearly matches the full R1 model in math.

We applied supervised fine-tuning (SFT) to Deepseek-R1-Distill-Qwen-32B across three target domains—Mathematics, Code, and Science — using the [360-LLaMA-Factory](https://github.com/Qihoo360/360-LLaMA-Factory/) training framework to produce three domain-specific models. We used questions from open-source data as seeds. Meanwhile, responses for mathematics, coding, and science tasks were generated by R1, creating specialized models for each domain. Building on this, we leveraged the Mergekit tool from the Arcee team to combine multiple models, creating Tiny-R1-32B-Preview, which demonstrates strong overall performance.

*Note:* We fix a tokenizer config bug that exists from 2025.2.26 21:00 PM UTC+8 to 2025.2.27 22:31 PM UTC+8, which will degrade model performance; refer to [Hotfix](##Hotfix).

## Evaluation 
| Model                           | Math (AIME 2024)        | Coding (LiveCodeBench)  | Science (GPQA-Diamond) |
| ------------------------------- | ------------------- | ----------------------- | ---------------------- |
| Deepseek-R1-Distill-Qwen-32B    | 72.6        	| 57.2           	  | 62.1            |
| Deepseek-R1-Distill-Llama-70B   | 70.0                  |  57.5                   | 65.2                   |
| Deepseek-R1                     | 79.8                | 65.9                    | 71.5               |
| Tiny-R1-32B-Preview (Ours)       | 78.1                | 61.6                    | 65.0      

All scores are reported as pass@1.
For AIME 2024, we sample 16 responses, and for GPQA-Diamond, we sample 4 responses, both using average overall accuracy for stable evaluation.


We merged the models trained separately in three directions into a single model. Below are the comparison results.  
| Model                           | Math (AIME 2024)        | Coding (LiveCodeBench)  | Science (GPQA-Diamond) |
| ------------------------------- | ------------------- | ----------------------- | ---------------------- |
| Math-Model                | 73.1                | -                       | -                      |
| Code-Model               | -                   | 63.4                    | -                      |
| Science-Model             | -                   | -                       | 64.5                   |
| Merged-Model (Tiny-R1-32B-Preview)              | 78.1                | 61.6                    | 65.0 


## Quickstart

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "qihoo360/TinyR1-32B-Preview"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Please reason step by step, and put your final answer within \\boxed{}. Solve the integral:  \[I = \int \frac{x^2}{(x+1)^3} \,dx\]"
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=4000
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)
```


## Data

#### 1. Math
58.3k CoT trajectories from [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), default subset

#### 2. Coding
19k CoT trajectories [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k), coding subset

#### 3. Science
8.6k CoT trajectories:
- 2.7k CoT trajectories from [simplescaling/data_ablation_full59K](https://huggingface.co/datasets/simplescaling/data_ablation_full59K), science and health science subset
- 4.9k CoT trajectories from [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k), science subset
- 1.0k CoT trajectories from [simplescaling/s1K-1.1](https://huggingface.co/datasets/simplescaling/s1K-1.1), all


## Open Source Plan
We will publish a technical report as soon as possible and open-source our training and evaluation code, selected training data, and evaluation logs. Having benefited immensely from the open-source community, we are committed to giving back in every way we can.

## Caveats
TinyR1-32B-Preview is an experimental research model designed to advance AI reasoning capabilities. As a preview release, it has demonstrated higher evaluation scores on some benchmarks but is not intended for general user applications. Key limitations include:

1. Incorrect parameter configurations may result in repetitive output loops, similar to R1. We recommend setting the temperature to 0.6 and top-p to 0.95, in line with R1's configuration.
2. The model currently omits the `<think>` token, which indicates the reasoning start, and only outputs the `</think>` token to signal the end of the thinking process. This will be addressed soon in the next version.
3. The model may generate overly lengthy reasoning chains for simple queries. We are working on improvements.
4. Benchmark coverage is limited. We encourage users to evaluate the model on additional benchmarks, and we will continue to expand our benchmark results.
5. The model requires enhanced safety measures to ensure reliable and secure performance.
## Hotfix
Serious Tokenizer Config BUG:  
Around 21:00 Beijing Time (UTC+8) on February 26, 2025, we made adjustments in commit `910e373a8247c59633a9deba39b4aa9ad8e05c09` in an attempt to reduce excessive repetition encountered by users. However, this tokenizer configuration significantly lowered our scores (dropping by nearly 10 points on AIME and GPQA). We have already updated `tokenizer_config.json` and are working urgently to fix the remaining issues. Currently, we have received feedback from some companies confirming our model evaluation scores.

## Contributors

*360 Team:*
Lin Sun,
Guangxiang Zhao,
Xiaoqi Jian,
Weihong Lin,
Yongfu Zhu,
Change Jia,
Linglin Zhang,
Jinzhu Wu,
Sai-er Hu,
Xiangzheng Zhang

*PKU Team:*
Yuhan Wu,
Zihan Jiang,
Wenrui Liu,
Junting Zhou,
Bin Cui,
Tong Yang


## Citation
```
@misc{tinyr1proj,
      title={SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.}, 
      author={TinyR1 Team},
      year={2025},
      eprint={},
      archivePrefix={},
      primaryClass={},
      url={https://huggingface.co/qihoo360/TinyR1-32B-Preview}, 
}
```