Dolma

github.com ↗

(AI2, 2023-2024) - 3T-token open pretraining corpus with documented filtering pipeline.