ptpt-scratch-lm
Brazil has twenty times Portugal’s speakers and roughly six times its share of the crawled web, so a small Portuguese model learns to write Brazilian. This one did not. Here is the model, the Brazilian-trained control that proves the corpus did it, and the three times the measuring instrument was wrong before any of it could be believed.
Stated plainly, because the rest of the page assumes it.
Why a single model proves nothing.
If the European model prefers autocarro, that could be the corpus. It could also be the tokenizer splitting one variant more cheaply, or the carrier sentence, or Portuguese in general. Those explanations are properties of the pair, identical for any model, so they cancel when the control is subtracted:
score = Δ(pt‑PT model) − Δ(pt‑BR control)
The reported number is that difference of differences. A single model’s raw gap is descriptive only.
Three results and one reason not to over-read them.
Where the +12.39 above comes from, pair by pair. The sign test runs over all 211 = 2,048 assignments.
What the numbers in the first two columns are.
Each model scores both sentences of a pair and the entry is log P(European) − log P(Brazilian), averaged over that pair’s carrier sentences. Positive means the model finds the European version more likely. The units are nats — natural log of a probability ratio — so the size is read by exponentiating. Taking the autocarro row:
| pt-PT model | +3.324 | finds the autocarro sentence about 28× more likely |
| pt-BR control | −7.963 | finds the ônibus sentence about 2,900× more likely |
| score | 11.29 | the gap between the two models — what the corpus did |
The first column alone is not the result, because it is inflated or deflated by things that have nothing to do with variety: how long each phrase is, how the tokenizer splits it, how common the words are in Portuguese generally. Those apply equally to both models, so subtracting the control removes them.
| pair | pt-PT model | pt-BR control | score |
|---|---|---|---|
| pequeno-almoço / café da manhã | −0.897 | −20.673 | 19.78 |
| telemóvel / celular | +5.877 | −11.749 | 17.63 |
| rapariga / garota | +3.259 | −13.652 | 16.91 |
| comboio / trem | +6.943 | −8.709 | 15.65 |
| sumo / suco | +4.469 | −9.667 | 14.14 |
| autocarro / ônibus | +3.324 | −7.963 | 11.29 |
| está a fazer / está fazendo | +3.737 | −7.395 | 11.13 |
| frigorífico / geladeira | +7.611 | −2.572 | 10.18 |
| casa de banho / banheiro | +1.139 | −8.270 | 9.41 |
| estão a preparar / estão preparando | +2.624 | −3.967 | 6.59 |
| talho / açougue | +3.798 | +0.247 | 3.55 |
Positive means the European model prefers the European form more than the control does. The two progressive rows are grammar rather than vocabulary — estar a + infinitive against the gerund — and they carry the stronger claim: the model learned European syntax, not a word list.
One row that does not say what it looks like.
On pequeno-almoço the European model still leans Brazilian in absolute terms (−0.897). It scores highest on the table only because the control leans Brazilian far harder (−20.673). Counted absolutely the European model prefers the European form on ten of eleven pairs, not eleven. Relative to the control, eleven. That gap is the whole reason the estimator is a difference of differences, and it is the first thing a careful reader should check.
The table is the measurement. This is the part that persuades a room. Same prompt, same seed, same sampling settings — only the weights differ.
None of that was tested for, and none of it was filtered on. Metro against metrô is a variety marker carried entirely by a circumflex. Partilhar against compartilhar is a real pair that appears in neither the evaluation nor the corpus filter. Carris Metropolitana is the Lisbon-area bus operator, named correctly, unprompted.
The corpus filter selects documents on clitic placement — dá-me against me dá — and the evaluation deliberately scores none of it. Filtering on grammar and testing on vocabulary asks the harder question: did the model learn the feature it was selected on, or the variety? The answer is the variety, and it reaches words and places nobody supplied.
Everything above is a measurement someone else made. This runs the two models on your machine instead, on a prompt you choose — the European one and the Brazilian-trained control, same seed, same sampling, only the weights different.
This downloads about 167 MB to your device.
These are the published models at four bits, not the published models. The result in the table above survives that — all eleven pairs, the same p — but token‑for‑token the quantised model agrees with the original only about 69% of the time. It prefers the same words. It does not write the same text.
starting…
Both models are now on your device and run here, in this tab. There is no server doing this — what you type is not sent anywhere, and it keeps working with the network off. An 84M‑parameter model on one CPU thread writes at about three words a second, so give it a moment.
Writing on your machine…
Temperature 0.8, top‑k 40, the same seed for both — so the only difference between those two lines is which corpus the model read. Both are running at four bits: they prefer the same words as the models behind the table above, and they do not write the same text. Each line stops when it reaches the end of a sentence — the model has no way to stop itself, so that is the page's decision, not its own.
Four files came from this domain and nothing else was contacted: the two models (75.5 MB each), the tokenizer, and ONNX Runtime Web 1.29 (14 MB of WebAssembly), which is the only third‑party code involved. It executes in a worker, so the page stays responsive while it writes.
The models are the ones trained for this write‑up: 84.4M parameters, 13 layers, RoPE and RMSNorm, exported to ONNX and stored with 4‑bit weights and a half‑precision embedding table — the arithmetic still runs in 32‑bit, the weights are only stored small. The tokenizer is about 170 lines checked token‑for‑token against the Python one the models were trained with.
There is no inference server, no API key and no telemetry: your prompt is turned into numbers in this tab and never leaves it. Every figure on this page — including what the four‑bit version costs — was measured rather than estimated, and the limits are stated above.
This needs a browser with WebAssembly and enough memory; if it did not start, the samples above are the same thing, generated earlier.
Each of these produced a confident, wrong, entirely plausible number. None of them failed loudly. They are the reason this page exists at all rather than a retraction.
1. Tela is not a Brazilian word.
The first contamination check scored a clean European corpus at 49.2% pt-PT — chance — which read as a dead project. Twenty-eight of thirty-two supposed Brazilian hits were the single word tela, ordinary European Portuguese for canvas or screen. Descarga (a flush) and celular (the adjective, as in membrana celular) are European too. Corrected, the same corpus reads 95.3%. The same list feeds the evaluation, so left alone it would have reported that the model failed to learn pt-PT when the model was fine.
2. The corpus was never 97% junk.
A prose-quality filter built on words-per-line and sentence-ending punctuation rejected 97% of documents — including 100% of real news articles. WET extraction breaks paragraphs at every HTML element boundary, so line-based metrics measure markup, not writing. The underlying error was cruder: documents had been judged by their first 300 characters, which on any web page is the navigation menu. A school blog condemned that way turned out to be exactly the target register, enclisis and all.
3. Brazilian pages write siga-nos.
The mirrored variety gate kept 31% of the control corpus against 64% of the European one, which would have left the control both smaller and selected for strong proclisis — more emphatically Brazilian than Brazilian Portuguese is, inflating the very contrast being measured. The detector was not at fault: enclisis matching is clean, 670 hits of contacte-nos and siga-nos in the European corpus against five in the Brazilian. The threshold was. Brazilian commercial pages do write siga-nos, and a single such token gave a share of 0/1 and condemned the document. Requiring three markers before judging moved the control from 59.4% kept to 87.8% while the European corpus moved 98.5% to 99.5%.
A fourth belongs with them, on the training side. PyTorch initialises nn.Embedding at N(0, 1) where MLX uses roughly N(0, 1/√d), and because the weights are tied that embedding is also the output projection. The same model on the same data began at loss 235.9 in one framework and 10.8 in the other. Left alone, the next section would have compared a well-scaled model against a broken one and called the difference a property of silicon.
The models had to be trained anyway, so a second question came almost free: train the same recipe from the same seed, the same bytes and the same token budget, once on a Mac mini under MLX and once on a rented GPU under CUDA. Do the runs agree?
| seed | MLX / M4 | torch / T4 | difference |
|---|---|---|---|
| 1 | 3.1845 | 3.1765 | −0.0080 |
| 2 | 3.2008 | 3.2016 | +0.0008 |
| 3 | 3.2103 | 3.1940 | −0.0163 |
| 4 | 3.2661 | 3.2728 | +0.0067 |
| 5 | 3.2195 | 3.2143 | −0.0052 |
| 6 | 3.1688 | 3.1598 | −0.0090 |
| mean | 3.2083 | 3.2032 | −0.0052 |
An exact permutation test on the framework label — six per side gives C(12,6) = 924 assignments, enumerated rather than sampled — returns p = 0.7965. The gap between frameworks is 0.14 standard deviations of the seed-to-seed noise. Seeds 4 and 6 differ by 0.0973 on the same hardware, nineteen times the cross-framework gap.
Changing the random seed matters vastly more than changing the chip. That is a null, and a null is only worth reporting when the instrument has been checked: batch-order hashes matched byte-for-byte across both frameworks for all six seeds, and the initialisation scheme was forced identical after the 235.9-against-10.8 discovery above. The noise floor — sd 0.0364 — is the number that makes the null credible, because it says how small an effect this design could have seen. Anything above roughly 0.07 would have shown up.
| Two 84.4M models (Q1) | $72 |
| Twelve models, both frameworks (Q2) | $6.55 |
| Six local seeds, electricity | €0.10 |
| Total | ~$79 |
Ninety-two per cent of the spend is the two artifacts. The entire second experiment — twelve models and the answer to whether hardware matters — cost $6.55.
The expensive input was never AWS. It was streaming 2.4 TB of CommonCrawl to keep 2.2 GB of text: European domains are 0.325% of records, and a third of what survives is not Portuguese prose. The same token count needed 10,373 crawl files on the European side against 1,715 on the Brazilian — the six-to-one density ratio that is the whole reason a small European Portuguese model did not already exist.
One unexpected number, measured rather than assumed: an L40S delivered 3.2× more compute per dollar than a T4 despite costing nearly three times as much per hour, and 17.5× the Mac mini. For training, hourly price is close to uncorrelated with cost efficiency. The crossover below which a laptop wins outright is about one overnight run.