The ArmSTEM verification loop is the interesting engineering here, worth reading before the score table. COPA machine-translated English math and science problems into Armenian, masking numbers and mathematical notation first so the translation could not corrupt them, then asked an independent model to solve the Armenian version and reproduce the original answer [13]. When the solver failed, it got the English original as a control, and items that defeated it in both languages were retained with a `solver_limited` label rather than counted as verified translations [14]. That label is the good part. It separates "the translation broke the problem" from "the problem is hard", a distinction most synthetic-data pipelines collapse into a single discard bucket.
The loop's blind spot is a translation that stays solvable, still lands the original answer, and is wrong in some way the answer never exposes. Human review covered 300 items: two native Armenian speakers assessed logical coherence and correctness separately, agreed on every verdict, accepted 299, and recorded a Cohen's kappa of 1.0 [15]. Against 372,907 problems, that sample is 0.08% [22]. 324,323 of the problems carry step-by-step solutions [12], leaving 48,584 without them [21]. COPA itself notes the process remains partly dependent on model-based judgment [19].
ArmWeb is 4.37 million documents and about 3.3 billion tokens measured under the Gemma-4 tokenizer [6], roughly 755 tokens per document [20], which is news-article length and what a 15-year crawl of Armenian news sites should produce [7]. Set against a 9-billion-parameter model, 3.3 billion tokens is 0.37 tokens per parameter [24]. This is an adaptation corpus, and anyone planning to train Armenian from scratch on it should price the shortfall before starting.
The contamination figures are what should change how you read the ranking. COPA reports benchmark matches in 7.9% of CulturaX-hy documents, 10.9% of HPLT-v2-hy and 17.4% of FineWeb-2-hy [9]; its own post-split pool sat at 3.3%, or 147,101 documents, which 13-gram decontamination then removed [10]. FineWeb-2-hy's rate is 5.3 times COPA's pre-cleaning rate [23]. COPA says arm-gemma-e4b recorded the highest mean score among the open Armenian models it evaluated [4]. For that to transfer, the evaluated set has to contain the model you would otherwise have picked, and you have to accept a decontaminated model being scored beside models whose Armenian slices overlapped the test items. Both conditions weigh against the ranking's transferability.
On provenance the release goes further than most: each released record carries source and URL metadata [8], and COPA describes this as the first open Armenian LLM published with its complete training data and recipe [5]. Provenance metadata establishes where a document came from; whether it can be redistributed is a separate question, and the material as published does not set out licensing terms for fifteen years of crawled news [25]. The validation approach has history behind it, too: Arakelyan's SynDARin paper proposed generating and validating question-answering datasets in low-resource languages and tested it on a 1,200-sample Armenian set [17]. COPA sells decision-intelligence products to institutional and commercial clients [18], so the release doubles as a credential. A team defending a model to a regulator will want provenance and licence both, and this release has shipped just the one.
