Why Bahasa Indonesia needs its own fine-tuning at all
Most general-purpose embedding models sold by US and Chinese vendors are trained on corpora that are 70–95% English, with Bahasa Indonesia often representing well under 1% of the total token count. In practice this means a base sentence-transformer or BGE model will score Bahasa Indonesia pairs 10–20 points lower on MIRACL and MTEB-Indonesian tasks than the published English averages, and will routinely confuse Malay-Indonesian code-mixed text because the shared lexical root is overweighted. Fine-tuning a model on Bahasa Indonesia data fixes the most common failure modes: informal Jakarta slang (bahasa gaul), regional insertions (Javanese, Sundanese, Batak nouns inside Indonesian sentences), and the agglutinative morphology that English-tokenizers routinely split into three or four fragments.
Also worth reading: Why do Obama and IShowSpeed champion Bahasa Indonesia globally? · How much does B2B AI pricing model Indonesia cost? · How Can B2B AI SaaS Indonesia Market Leaders Adapt To The SaaS Apocalypse?
For B2B market-intelligence and knowledge-ops teams operating in Indonesia and SEA, the difference between a generic and a fine-tuned embedder is the difference between a search box that returns 50% irrelevant Indonesian documents and one that returns 85%+ precision on the same query. That is not a marginal UX improvement, it is the difference between the tool being adopted and being quietly abandoned after the second weekly retro.
What the realistic data sources look like in 2026
The Indonesian open dataset ecosystem has matured enough that a competent team can assemble 200,000–1,000,000 high-quality sentence pairs without paying any licensing fee. The standard pipeline in 2026 combines four buckets: (1) Indonesian Wikipedia (around 740,000 articles, ~115 million words) used as a positive-pair source for adjacent-sentence training; (2) the id_nusantara_mteb corpus released via Hugging Face in late 2025 with roughly 320,000 curated query-document pairs drawn from Indonesian e-commerce, news, and government portals; (3) CC-licensed news (Kompas, Tempo, CNN Indonesia) filtered for paragraphs above 80 words; and (4) internal company data such as Slack exports, Confluence pages, and Jira ticket descriptions, deduplicated and PII-stripped.
One frequently overlooked source is the parallel-corpus data produced by the now-defunct Jawi-to-Latin script transliteration projects that surfaced in the late 2010s. While the underlying theological texts are not useful for embeddings, the alignment files contain millions of Indonesian-Indonesian paraphrase pairs produced by the original translators. Teams that have mined these for clean Indonesian sentence pairs report a 3–5 point lift on domain-internal retrieval benchmarks.
Choosing a base model: what actually works
As of September 2026, three base models dominate Indonesian fine-tuning work. The first is BGE-m3-en-id, a 568M-parameter model that ships with an Indonesian-adapted tokenizer and reaches 64.2 on the MIRACL-id benchmark before any fine-tuning. The second is the E5-mistral-7b-instruct-id-v2, a 7B-parameter instruct-tuned model that scores 71.8 out of the box but costs roughly four times as much to fine-tune and requires an A100 80GB or H100 for sensible batch sizes. The third is a region-specific model called NusantaraEmbed-110M, which is small enough to run on CPU for inference but plateaus at 58.1 on the same benchmark regardless of how much Indonesian data is thrown at it.
| Feature | BGE-m3-en-id | E5-mistral-7b-instruct-id-v2 | NusantaraEmbed-110M |
|---|---|---|---|
| Parameters | 568M | 7B | 110M |
| MIRACL-id baseline | 64.2 | 71.8 | 58.1 |
| VRAM needed for FT | 24 GB | 80 GB | 8 GB |
| Inference cost / 1k tokens | $0.0008 | $0.0031 | $0.0002 |
| Best for | Mid-market SaaS | Enterprise accuracy | Edge / on-prem |
| Open license | MIT | Apache 2.0 | CC-BY-SA 4.0 |
The fine-tuning loop, step by step
The actual technical loop has not changed much since 2024, but the tooling around it has. Step one is to define a hard evaluation set of 500–2,000 Indonesian query-document pairs sampled from real user traffic; without this, every later claim of improvement is anecdotal. Step two is to generate or mine hard negatives using a BM25 index plus a top-100 retrieval from the base model, then label the false positives. Step three is to fine-tune using the standard InfoNCE contrastive loss with a temperature of 0.05, batch size of 64–128, and learning rate of 2e-5 for the encoder layers and 1e-4 for the pooling head. Step four is to evaluate on the held-out set and on MIRACL-id, then repeat.
In practice a single fine-tuning run on BGE-m3-en-id with 500,000 pairs takes 14–18 hours on a single A100 80GB and produces a model that scores 70–73 on MIRACL-id, a 6–9 point lift over the baseline. Going from 73 to 75 is harder: it requires either a much larger hard-negative pool or a curated set of 50,000+ expert triplets, which is where most teams quietly stop because the marginal cost per point begins to outpace the marginal business value.
Common mistakes that quietly wreck results
The single most common error is fine-tuning on English-translated pairs. Bilingual teams often run their Indonesian corpus through Google Translate to produce English pairs and then train on those, but this destroys the morphology that fine-tuning is supposed to preserve. The second most common error is using a tokenizer that has not been re-fitted to Indonesian; pre-2024 BGE variants tokenize common Indonesian words like "perkembangan" as five sub-pieces, which means every embedding is computed on a noisier signal. The third mistake is ignoring code-switching: roughly 38% of Indonesian business chat in Jakarta and Surabaya contains English technical terms, and a model trained on monolingual Indonesian will treat these as out-of-vocabulary noise. The fourth mistake is evaluating on the training distribution: if the eval set is drawn from the same Wikipedia dump used for training, the gains will not transfer to customer data.
A subtler mistake is over-training on hard negatives that are too hard. Pushing the model to distinguish between two paragraphs that differ by only a single word teaches it to overweight surface form at the expense of semantic content, and downstream retrieval precision on long queries drops by 4–7 points even as MIRACL-id scores climb. A reasonable rule of thumb is to cap the hardest negative at BM25 rank 50 and never use negatives from rank 1–10.
When fine-tuning is the wrong move
Fine-tuning an embedder is the right answer when retrieval precision on Indonesian content is the bottleneck and the team has at least 100,000 domain-specific sentence pairs available. It is the wrong answer in three situations. First, when the corpus is small (under 20,000 documents); a better baseline is hybrid BM25 + dense retrieval with the off-the-shelf BGE-m3-en-id. Second, when the business problem is actually classification or extraction rather than retrieval, in which case a fine-tuned classifier or a generative model will outperform a fine-tuned embedder by a wide margin. Third, when the team is on a tight maintenance budget: a fine-tuned embedding model needs to be re-evaluated every quarter as Indonesian usage shifts, and the operational cost of that cycle is rarely budgeted for in advance.
For most early-stage SEA knowledge-ops products, the honest answer is that off-the-shelf BGE-m3-en-id plus a well-tuned BM25 hybrid retriever will deliver 80–85% of the value of a fine-tuned model at 10% of the operational cost, and the remaining 15–20% is rarely worth the engineering hours until the product has reached product-market fit.
Cost and timing reality in September 2026
A single fine-tuning run on BGE-m3-en-id costs roughly $180–$260 in cloud GPU time on AWS p4d.24xlarge or Lambda Labs, plus 40–80 engineering hours for data preparation, evaluation harness, and deployment plumbing. A full E5-mistral-7b fine-tune runs $1,400–$2,100 for compute alone and requires a minimum of three weeks of iteration because each run takes 5–6 days. Most B2B teams in 2026 should budget $25,000–$60,000 for the first serious Indonesian fine-tuning project when including internal time, and plan to repeat the evaluation cycle every 90 days.
The timeline is realistic if scoped tightly: two weeks to build the eval harness and mine hard negatives, two weeks to run three rounds of fine-tuning, one week to A/B test in production behind a feature flag, and one week to roll back if the numbers regress on live traffic. A team that promises a working production Indonesian embedding model in under six weeks is almost certainly cutting corners on the eval set.
What to do this quarter
If the team has not yet run an Indonesian retrieval evaluation, the first concrete step is to download the MIRACL-id dev set, run BGE-m3-en-id against a representative 50,000-document slice of production data, and report the nDCG@10 honestly. Anything below 55 means fine-tuning is justified; anything above 65 means the engineering hours are better spent on a hybrid retriever or on query rewriting. If fine-tuning is justified, the next step is to lock down the eval set before touching the model, because every iteration after that will be evaluated against it and a moving target makes every later decision suspect.