Interactive overview of the LUNA model program: the LUNA_PreTrain pretraining corpus (4.5B deduplicated tokens) and the LUNA-RAG-MCP-SFT-10M instruction-tuning dataset (RAG + MCP knowledge injection).
Sources: Wikipedia · FineWeb-Edu (score ≥ 3.5) · OpenWebText · legacy cleaned web. English-only, zero source overlap, tokenized to the LUNA tokenizer (vocab 50,277).
| Property | Value |
|---|---|
| tokens_post_dedup | 4,515,286,950 |
| tokens_pre_dedup | 5,092,876,500 |
| docs | 5,832,141 |
| chunks | 270 |
| block_size | 1025 |
| vocab_size | 50277 |
| dedup_docs_removed | 567,611 (8.87%) |
| dedup_tokens_removed | 577,589,550 (11.34%) |
| boilerplate_stripped | 25,535,904 chars |
| Source | Est. tokens |
|---|---|
| General web (cleaned legacy corpus) | ~2.94B |
| litdata_english_500m (Wiki+FineWeb) | ~515M |
| litdata_combined (Wiki+FineWeb) | ~257M |
| Wikipedia (articles id ≥ 200,001) | ~365M |
| FineWeb-Edu (edu score ≥ 3.5) | ~445M |
| OpenWebText (Reddit-curated, no overlap) | ~445M |
Pre-dedup total: 5,092,876,500 tokens → post-dedup 4,515,286,950. Full audit in BUILD_REPORT.txt / DEDUP_REPORT.txt in the dataset repo.
english_1b_clean/)Human-readable cleaned corpus: Simple Wikipedia + Wikipedia long-tail + FineWeb-Edu tail, stored as 105 parquet files. Tokenized mirror in litdata_english_1b/.
| Property | Value |
|---|---|
| docs_in | 525,000 |
| docs_out | 524,344 |
| tokens_in | 394,305,085 |
| tokens_out | 392,392,118 |
| parquet_files | 105 |
Cleaning: emails removed (2,158) · number tables stripped (71,386) · punctuation fixes (45,574) · orphan parens removed (24,034) · code/import lines removed · boilerplate stripped.
Alpaca-style records (instruction / input / output), English-only, exactly deduplicated, ≤1,024 tokens/sample. Grounded in curated topic cards from MCP & RAG documentation.
| Property | Value |
|---|---|
| Realized tokens | 10,000,168 |
| Train samples | 60,647 |
| Validation samples | 1,237 |
| Total samples | 61,884 |
| Avg tokens / sample | 161.6 |
| Max window (tokens) | 1024 |
| Task style | Samples |
|---|---|
| qna | 15639 |
| description | 12424 |
| comparison | 9368 |
| scenario | 9241 |
| checklist | 9050 |
| clarification | 6162 |
| Topic | Samples |
|---|---|
| MCP | 25242 |
| RAG | 21022 |
| Bridge | 6252 |
| MCP+MCP | 2087 |
| RAG+MCP | 1887 |
| MCP+RAG | 1824 |
| RAG+RAG | 1407 |
| Bridge+MCP | 592 |
| MCP+Bridge | 561 |
| RAG+Bridge | 467 |
| Bridge+RAG | 447 |
| Bridge+Bridge | 96 |
Browse 20 preview records from the RAG+MCP SFT dataset with full metadata. Filter by task style or topic.