あらゆるLLMにDNA知識を学習させるための最大規模のデータセット（FineDNA）
huggingface.co
モデルの学習レシピ、推論、評価コードが含まれるリポジトリ
huggingface/carbon
FineDNAに関するRedditの議論スレッド
reddit.com
187 views
@JiahanCed The data is already public here huggingface.co
The training recipe is in the blog: huggingface.co
Training code here huggingface/carbon
3 likes · 31 views
FineDNA: the largest dataset to teach any LLM DNA (midtraining, SFT, RL)
Hi everyone, we're releasing FineDNA, all the data we used to turn Qwen3-14B into Carbon-X, the strongest open language model on DNA. It contains:
- Midtraining: 300B+ tokens of genomes and DNA-rich text
- SFT: ~200K conversations
- RL: ~98K verifiable prompts
Code and recipe are open too, so you can apply it to other LLMs.
Dataset: huggingface.co
Code: huggingface/carbon
65 points · 5 comments
Open data and code to teach LLMs to read DNA
3 points
Everything is open: the data, the models and the full training recipe.
Code: huggingface/carbon
Models & dataset: huggingface.co
Enjoy this late halloween treat!
4 likes · 143 views
Hugging Face releases Carbon-X, an open 14.8B LLM that reads raw DNA and answers in English (66.7% avg on 5 DNA benchmarks)
Hugging Face Bio, with Zhongguancun Academy and TIGEM / University of Naples Federico II, released Carbon-X. It is a 14.8B decoder-only model built from Qwen3-14B. You paste DNA into a normal chat and it predicts variant effects, flags regulatory regions, classifies organisms and predicts next bases. Weights are Apache 2.0.
What it does
- Hybrid tokenizer: Qwen3 BPE for text plus 5,515 DNA tokens (4,096 non-overlapping 6-mers, 1,364 tail tokens, 3 markers)
- DNA and text share 1 decoder, no separate DNA encoder
- Training: ~10B-token embedding warmup, 302B tokens of mid-training (55% DNA), 3 epochs of SFT, then 2 rounds of async GRPO with code-checked rewards
- 40,960-token context; post-trained and evaluated with thinking disabled
Key numbers (self-reported by Hugging Face)
- Average across 5 DNA benchmarks: 66.7% vs GPT-6.1 Sol (max effort) 67.5%
- BioReason Coding 74.9% and NextBase 39.4%, best of all models tested
- 27 points above Qwen3-14B
- IFEval: 79.3% prompt-level strict
- Runs on 1 x 80 GB GPU in bf16; official Q8_0 GGUF is 15.8 GB
- Estimated ~$300 to score 2.4M ClinVar VUS vs $32,000 to $4M on frontier models (H200 at $5/hour)
Full analysis: t.co
Model: t.co
Blog: t.co
Repo: t.co
GGUF: t.co
50 points
@anshulkundaje As for the evals, I think we covered a wide range of tasks with some established VEP benchmarks like ClinVar but we're always happy to improve! If you have another list of evals you'd recommend, could you open an issue here with the list here huggingface/carbon Thanks!
On BioReason, agreed, we found some serious issues like answer leakage in some prompts. We fixed them and will share the details when we release the eval code. We also opened an issue in their repo BioReason#26
Re AlphaGenome evals, I'll take a closer look at the paper but they have ClinVar eval which we also have. It's the FrontierVEP task where we wrap ClinVar sequence in Natural Language text so it fits the DNA + text format we're targeting.
We had TraitGym Mendelian in Carbon-3B the results were in line with other VEP tasks in terms of model ranking, we could include it again, after the Natural language transformation.
2 likes · 107 views
Weights, model card and the one-GPU serving recipe: huggingface.co
Code: huggingface/carbon
Wired's write-up is what put it on my radar today.
4 views
Mentions
あらゆるLLMにDNA知識を学習させるための最大規模のデータセット（FineDNA） huggingface.co モデルの学習レシピ、推論、評価コードが含まれるリポジトリ huggingface/carbon FineDNAに関するRedditの議論スレッド reddit.com
@JiahanCed The data is already public here huggingface.co The training recipe is in the blog: huggingface.co Training code here huggingface/carbon
FineDNA: the largest dataset to teach any LLM DNA (midtraining, SFT, RL) Hi everyone, we're releasing FineDNA, all the data we used to turn Qwen3-14B into Carbon-X, the strongest open language model on DNA. It contains: - Midtraining: 300B+ tokens of genomes and DNA-rich text - SFT: ~200K conversations - RL: ~98K verifiable prompts Code and recipe are open too, so you can apply it to other LLMs. Dataset: huggingface.co Code: huggingface/carbon
Open data and code to teach LLMs to read DNA
Everything is open: the data, the models and the full training recipe. Code: huggingface/carbon Models & dataset: huggingface.co Enjoy this late halloween treat!
Hugging Face releases Carbon-X, an open 14.8B LLM that reads raw DNA and answers in English (66.7% avg on 5 DNA benchmarks) Hugging Face Bio, with Zhongguancun Academy and TIGEM / University of Naples Federico II, released Carbon-X. It is a 14.8B decoder-only model built from Qwen3-14B. You paste DNA into a normal chat and it predicts variant effects, flags regulatory regions, classifies organisms and predicts next bases. Weights are Apache 2.0. What it does - Hybrid tokenizer: Qwen3 BPE for text plus 5,515 DNA tokens (4,096 non-overlapping 6-mers, 1,364 tail tokens, 3 markers) - DNA and text share 1 decoder, no separate DNA encoder - Training: ~10B-token embedding warmup, 302B tokens of mid-training (55% DNA), 3 epochs of SFT, then 2 rounds of async GRPO with code-checked rewards - 40,960-token context; post-trained and evaluated with thinking disabled Key numbers (self-reported by Hugging Face) - Average across 5 DNA benchmarks: 66.7% vs GPT-6.1 Sol (max effort) 67.5% - BioReason Coding 74.9% and NextBase 39.4%, best of all models tested - 27 points above Qwen3-14B - IFEval: 79.3% prompt-level strict - Runs on 1 x 80 GB GPU in bf16; official Q8_0 GGUF is 15.8 GB - Estimated ~$300 to score 2.4M ClinVar VUS vs $32,000 to $4M on frontier models (H200 at $5/hour) Full analysis: t.co Model: t.co Blog: t.co Repo: t.co GGUF: t.co
@anshulkundaje As for the evals, I think we covered a wide range of tasks with some established VEP benchmarks like ClinVar but we're always happy to improve! If you have another list of evals you'd recommend, could you open an issue here with the list here huggingface/carbon Thanks! On BioReason, agreed, we found some serious issues like answer leakage in some prompts. We fixed them and will share the details when we release the eval code. We also opened an issue in their repo BioReason#26 Re AlphaGenome evals, I'll take a closer look at the paper but they have ClinVar eval which we also have. It's the FrontierVEP task where we wrap ClinVar sequence in Natural Language text so it fits the DNA + text format we're targeting. We had TraitGym Mendelian in Carbon-3B the results were in line with other VEP tasks in terms of model ranking, we could include it again, after the Natural language transformation.
Weights, model card and the one-GPU serving recipe: huggingface.co Code: huggingface/carbon Wired's write-up is what put it on my radar today.