3,000 legacy examples
Issue and grammar fixes improve existing V1 supervision.
V2 research · 24 Oct 2026
SamLingo — A Language Model for Sambalpuri
English-to-Sambalpuri translation, written in Odia script. V2 continues training from the saved V1 checkpoint with corrected examples, new sentences, and dictionary entries.
SamLingo
Explore further: Data audit · Training setup · Model & report
01From V1 to a stronger translation checkpoint
The research question
Can a targeted mix of corrected legacy examples, new sentence coverage, and lexical supervision improve the saved V1 system?
SamLingo V2 studies continued supervised fine-tuning in an NLLB-200-based translation setup. The comparison is between two checkpoints from this project, with a documented data update, training configuration, and evaluation protocol.
My contribution
I developed the model, trained and evaluated the checkpoints, analyzed the outputs, and prepared the research report.
The V2 work connects corpus refinement, source-grouped validation, single-GPU training, and automatic translation metrics. Shivangi Dwivedi contributed to dataset collection.
Source: Sambalpuri Neural Translation: Data Refinement and Continued Fine-Tuning from V1 to V2, 24 October 2026.
02A focused 6,400-record update
The package combines 3,000 revised V1 examples, 2,800 new sentence pairs, and 600 dictionary entries. Of the 6,400 raw records, 3,400 are newly added; the rest correct legacy material.
2,800 new sentence pairs
New English–Sambalpuri examples expand sentence-level coverage.
600 dictionary entries
Word-level supervision complements the sentence pairs.
6,335 unique records
Normalization and exact deduplication remove 65 duplicate records. Multiple target translations remain available.
View the full data audit
| Origin | Raw | Unique | Train | Validation | Excluded |
|---|---|---|---|---|---|
| Revised V1 | 3,000 | 2,993 | 2,983 | 0 | 10 |
| New sentences | 2,800 | 2,742 | 2,462 | 280 | 0 |
| Dictionary | 600 | 600 | 549 | 51 | 0 |
| Total | 6,400 | 6,335 | 5,994 | 331 | 10 |
Cleaning & source grouping
Unicode NFC, whitespace normalization, and case folding precede exact deduplication by source, target, and record type. A source-grouped holdout on fresh records uses seed 42; 10 overlapping legacy records are excluded from training.
What the audit establishes
The current training and validation frames have no normalized-English-source overlap. The notebook also flags 277 sources with multiple translations for review; these are not 277 confirmed errors. The full historical V1 corpus was not audited.
How V1 history relates to these counts
The earlier manuscripts report 15,000 parallel pairs. Their record-level corpus was unavailable for audit, so that historical count is not added to the V2 package as a verified cumulative total. Earlier qualitative accuracy claims are not carried into the V2 results.
03Continued fine-tuning on one A100
V2 starts from the saved V1 checkpoint and completes five epochs of supervised sequence-to-sequence training. The selected model is checkpoint-1875, chosen by minimum validation loss.
English in, Sambalpuri in Odia script out
The tokenizer routes English through eng_Latn and generation through ory_Orya, with target BOS token ID 256136. Sambalpuri adaptation is learned from project examples; no dedicated Sambalpuri language token is introduced.
Documented model family
The report identifies an NLLB-200-based setup; the executed checkpoint reports model_type = m2m_100. The exact upstream identifier, immutable revision, and parameter count remain unspecified.
17.29 minutes of training
The run takes 1,037.43 seconds on one NVIDIA A100-SXM4-40GB with BF16. Peak allocated GPU memory is 12.856 GiB, with 1,875 optimizer updates and an effective batch of 16.
Training configuration & five-epoch loss trace
Optimizer & learning rate
AdamW (adamw_torch), initial learning rate 0.00001; linear decay with 94 warmup updates (~5%).
Batch & regularization
4 examples per device × 4 accumulation steps; weight decay 0.01, maximum gradient norm 1.0.
Sequence limits
128 source tokens and 128 target tokens.
Checkpoint selection
Seed 42; evaluation and saving each epoch; lowest validation loss. Early-stopping patience 2, with all five epochs completed; gradient checkpointing off.
Recorded software
Transformers 5.20.2 · SacreBLEU 2.6.0 · Python 3.13.16. These are report-logged versions.
| Epoch | Logged training loss | Validation loss |
|---|---|---|
| 1 | 4.399788 | 1.159955 |
| 2 | 4.215046 | 1.037327 |
| 3 | 3.903251 | 0.984181 |
| 4 | 3.296158 | 0.962923 |
| 5 | 3.417338 | 0.956640 |
Validation loss falls 17.53% from epoch 1 to epoch 5. Logged training loss rises slightly in the final epoch. The two loss series use different collection conditions and do not establish a calibrated generalization gap.
04The V2 research workflow
- 01
Assemble
3,000 revised records
3,400 new records - 02
Normalize
NFC & whitespace
6,335 unique records - 03
Split
5,994 train · 331 validation
10 legacy records excluded - 04
Continue
Saved V1 checkpoint
5 epochs on one A100 - 05
Select
Lowest validation loss
checkpoint-1875 - 06
Compare
149 final reference pairs
331 validation records
Evaluation boundary
The final benchmark and development validation are separate populations. Neither is independently certified as unseen to both checkpoints.
05V1 → V2: measured results
On 149 unique English-reference pairs, V2 improves the reported translation-overlap metrics and exact match. These are within-project comparisons on the measured sample.
Evaluation at a glance
- Baseline
- Saved V1 checkpoint vs. V2 checkpoint-1875, selected by lowest validation loss.
- Final sample
- 149 reference pairs · 142 distinct English sources · inputs of 1–11 whitespace tokens. Separate development set: 331 records.
- Shared decoding
- Batch 8 · beam 4 · 128 source tokens · up to 128 new tokens.
Reproduction is incomplete: full prediction exports, checkpoint hashes and metric signatures are not published with this page. The V1 BLEU value is author-corrected. Read the limitations ↓
| Metric | V1 | V2 | Change |
|---|---|---|---|
| BLEU ↑ | 6.78 | 33.54 | +26.76 points |
| chrF++ ↑ | 39.69 | 58.91 | +19.22 points |
| TER ↓ | 78.15 | 49.67 | −28.48 points |
| Exact match ↑ | 5.37% | 17.45% | +12.08 percentage points |
| Inference time ↓ | 7.13 s | 5.53 s | −1.60 s |
| Examples / second ↑ | 20.89 | 26.96 | +6.07 examples/s |
↑ Higher is better · ↓ Lower is better. BLEU is an overlap score, not a percentage of translation accuracy. The V1 BLEU value follows the author’s correction and was not recomputed from a complete prediction export.
How to read this comparison
The 149 pairs contain 142 distinct English sources, with inputs of 1–11 whitespace tokens. Complete historical train–test overlap is unknown, so these scores do not establish performance on an independently audited unseen test set or on long passages.
Decoding uses batch size 8, beam size 4, a 128-token source limit, and up to 128 new tokens. Timing excludes model loading and reflects one recorded run per checkpoint.
Separate development evaluation: 331 records
This set contains 280 sentences and 51 dictionary entries. It was also used to select the checkpoint by validation loss, so it measures development performance. Across all 331 records, BLEU rises from 7.27 to 22.18 and chrF++ from 35.58 to 49.93.
View validation scores by subset
| Subset / model | Records | BLEU | chrF++ | Normalized exact match |
|---|---|---|---|---|
| All / V1 | 331 | 7.27 | 35.58 | 2.11% (7/331) |
| All / V2 | 331 | 22.18 | 49.93 | 7.25% (24/331) |
| Sentences / V1 | 280 | 7.41 | 36.10 | 0.00% (0/280) |
| Sentences / V2 | 280 | 22.36 | 50.69 | 4.29% (12/280) |
| Words / V1 | 51 | 2.88* | 19.17 | 13.73% (7/51) |
| Words / V2 | 51 | 0.00* | 20.98 | 23.53% (12/51) |
*Word-only corpus BLEU can be zero when higher-order word n-grams are unavailable; it does not mean zero lexical accuracy. Validation exact match uses NFC, whitespace normalization, and case folding; the final benchmark strips edge whitespace only.
V2 gains 18 newly exact outputs and has one regression, for a net gain of 17 normalized exact matches. These are string-match transitions, not counts of human-verified semantic corrections.
Qualitative examples & remaining lexical errors
“Papaya”
The report describes a V2 output naming onion instead of the reference fruit. A concrete lexical substitution remains.
“Your (Informal)”
V1 retains the English annotation; V2 matches the reference without it in the displayed example.
“Mita’s phone is charging.”
The displayed V2 output matches its reference exactly; V1 does not. This is one local success, not a success-rate estimate.
Selected observations from the saved notebook, summarized in report page 7. Target strings are described in English; these examples were not a blinded random human-evaluation sample.
06Limitations & next steps
The combined update improves both reported comparisons. Establishing the same gain on a locked, independently audited test set is the next step.
Lock an independent test set
Audit source overlap against all V1 and V2 material, then freeze the test set before further tuning.
Separate the update components
Compare corrected-only, new-sentence-only, and combined training under fixed settings. The current run does not isolate each component’s contribution.
Expand coverage & human review
Evaluate lexical adequacy and longer passages, with blinded bilingual ratings for meaning and fluency.
Complete the reproducibility record
Publish full prediction exports, checkpoint hashes, upstream revision, manifests, and metric signatures. Repeat seeds and timing trials and report uncertainty.
Model & project resources
Technical research report · 10 pages · 24 October 2026. The Hugging Face link is the project’s public model repository; this page does not verify its checkpoint revision against the V2 run.
What this work brings together
Better data. Measured translation progress.
Corpus refinement, continued fine-tuning, and transparent checkpoint comparisons for a low-resource language.
Evidence scope: this page summarizes the owner-supplied V2 technical report dated 24 October 2026. Training and generation were not rerun for the website update. Reported results compare project checkpoints and do not establish a universal quality rating or state-of-the-art performance.