Translation Workflows & CI/CD Pipeline Sync ¶

Continuous localization treats translation as a pipeline stage — keys are extracted from source, pushed to a translation platform, translated, pulled back, and built into the release — so locale files stay in sync with code on every merge instead of drifting into a separate manual track. This page covers the full loop: deterministic key extraction, two-way sync with a translation management system (TMS) such as Crowdin or Weblate, CI gates that block untranslated or malformed keys, bridging between PO and XLIFF formats, machine-translation pre-fill, and the governance rules that keep all of it reproducible.

If the work in core i18n architecture and locale negotiation defines what a translatable string is and how a locale is resolved at runtime, and the framework i18n and component routing area decides where strings live in your React, Vue, Angular, or SvelteKit code, then this area answers how strings travel — from a developer’s commit, out to translators, and back into a deployable artifact without a human copying files by hand.

Architecture overview ¶

A localization pipeline has one job: keep the set of translatable strings in your repository and the set of translations in your TMS consistent, automatically, on every change. Everything else — format conversion, machine-translation pre-fill, review gates — hangs off that core sync loop.

The loop has five repeating phases. Extract scans source code and emits a canonical source catalog (usually the English/base locale). Push uploads any changed strings to the TMS so translators see new work. Translate is where humans and machine translation fill in target locales. Pull brings completed translations back into the repository as committed files. Gate runs in CI before the artifact is built, rejecting anything malformed or untranslated above your threshold.

This area sits downstream of the other two. The shape of an extractable key — its namespace, its ICU MessageFormat plural arms, its placeholder names — is decided in the core architecture area, and the extractor only mirrors those decisions. Likewise, the file layout you extract into (one JSON namespace per route, a flat PO file per locale) is dictated by how your framework’s i18n routing loads bundles. A good pipeline is deliberately “dumb”: it moves strings between systems deterministically and never invents structure of its own.

Three properties make the loop trustworthy. It must be idempotent — running sync twice with no source changes uploads nothing. It must be deterministic — the same source always produces the same catalog (sorted keys, stable separators, normalized line endings), so diffs stay small and reviewable. And it must be atomic at deploy time — all locale assets for a release ship together, so users never see a half-German, half-English screen during a rollout.

The boundary between “source” and “target” is the one rule that, once broken, unravels everything else. Source strings flow one way: code → extractor → catalog → TMS. Target strings flow the other way: TMS → pull → repository → build. The moment a human edits a target file directly in the repository, or the extractor starts emitting target locales, the two systems begin overwriting each other and you lose the ability to say which copy is authoritative. Every design choice below — hash-gated uploads, pulling onto a dedicated branch, extracting only the base locale — exists to keep those two flows from crossing.

Concept 1 — Deterministic key extraction ¶

Extraction is the seam between code and translation. If it is non-deterministic, every build produces a noisy diff, translators re-translate strings that did not change, and CI cannot tell a real drift from formatting churn. The fix is to pin the extractor’s output to a canonical form.

Follow these steps to make extraction reproducible:

1. Choose one extractor per framework and pin its version. For i18next codebases use i18next-parser; for React Intl or Lingui projects use their respective extract commands. Mixing extractors guarantees divergent output. The deeper i18next-parser key extraction guide walks through lexer selection for .ts/.tsx files.
2. Force a canonical sort and separator set. Sort keys, fix keySeparator/namespaceSeparator, and pick a single indentation. Two machines must emit byte-identical catalogs.
3. Normalize line endings to lf. A Windows checkout emitting CRLF is the single most common source of phantom diffs.
4. Mark missing strings with a sentinel, not an empty string. A value like __STRING_NOT_TRANSLATED__ is greppable in CI; an empty string is indistinguishable from a deliberately blank label.
5. Commit the source catalog, never the target locales, from the extract job. Extraction owns the base locale only; targets come back from the TMS on pull.

// i18next-parser.config.js — deterministic output for CI
module.exports = {
  contextSeparator: '_',
  createOldCatalogs: false,        // don't litter _old.json files
  defaultNamespace: 'common',
  defaultValue: '__STRING_NOT_TRANSLATED__', // greppable sentinel
  indentation: 2,
  keepRemoved: false,              // drop keys no longer in source
  keySeparator: '.',
  namespaceSeparator: ':',
  lexers: {
    ts: ['JavascriptLexer'],
    tsx: ['JsxLexer'],
    default: ['JavascriptLexer'],
  },
  lineEnding: 'lf',                // stop CRLF phantom diffs
  locales: ['en'],                // extract base locale ONLY
  output: 'locales/$LOCALE/$NAMESPACE.json',
  sort: true,                      // stable key order → small diffs
};

With this config, npx i18next-parser 'src/**/*.{ts,tsx}' is a pure function of your source tree. The choice of namespace and key shape should match the conventions your fallback chain configuration expects at lookup time, so a missing target key resolves to the base locale rather than rendering the raw key.

Concept 2 — Two-way TMS sync (Crowdin / Weblate) ¶

Once the source catalog is canonical, the pipeline pushes it to a TMS and pulls completed translations back. The hard part is not the upload — it is making the round trip idempotent so that pushing an unchanged catalog is a no-op and pulling does not clobber in-flight local edits.

The mechanism that makes this safe is a content hash. Compute a hash of the source catalog, compare it to the hash the TMS last ingested, and only upload on a mismatch. This single check turns a chatty, every-commit upload into a quiet, change-driven one.

1. Hash the source catalog and compare before uploading. Skip the push when hashes match.
2. Push on a branch that mirrors your Git branch. Both Crowdin and Weblate support per-branch projects, so release/* work does not contaminate main translations.
3. Pull on a schedule or webhook, into a dedicated l10n/sync branch. Never pull straight onto main; open a pull request so the translation diff is reviewed.
4. Let the TMS own targets; let Git own source. This division prevents the two systems from fighting over the same keys.

# .github/workflows/i18n-sync.yml
name: i18n extract and sync
on:
  push:
    branches: [main, 'release/*']
    paths: ['src/**']
jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx i18next-parser 'src/**/*.{ts,tsx}'
      - name: Compute source hash
        run: sha256sum locales/en/common.json | cut -d' ' -f1 > .src_hash
      - name: Upload only on drift
        run: |
          REMOTE=$(curl -s -H "Authorization: Bearer $TMS_TOKEN" "$TMS_API/hash")
          if [ "$(cat .src_hash)" != "$REMOTE" ]; then
            curl -X POST "$TMS_API/upload" \
              -H "Authorization: Bearer $TMS_TOKEN" \
              -F "file=@locales/en/common.json" \
              -F "branch=${{ github.ref_name }}"
          else
            echo "No source drift; skipping upload."
          fi
        env:
          TMS_TOKEN: ${{ secrets.TMS_TOKEN }}
          TMS_API: ${{ vars.TMS_API }}

Teams on the SaaS side typically follow the Crowdin integration for dev teams path and wire the round trip directly into pull requests, as covered in connecting the Crowdin API to GitHub pull requests. Teams that need data residency or air-gapped operation run a Weblate self-hosted setup and drive the same loop through Weblate webhooks for auto-translation. If you are still choosing, the Crowdin vs Weblate comparison for self-hosted teams lays out the trade-offs around hosting, cost, and connector maturity.

Concept 3 — CI gates for untranslated and malformed keys ¶

The pull step brings translations back; the gate decides whether they are allowed to ship. A localization CI gate runs after the pull and before the build, and it answers two questions: is every required key present and translated? and is every string well-formed? A failure on either should block the merge, not just warn.

There are three categories of check, in increasing strictness:

1. Completeness — are there keys still holding the __STRING_NOT_TRANSLATED__ sentinel for a locale you consider release-blocking? A simple grep over the pulled catalog catches these. The failing-build-on-untranslated-keys page tunes which locales block versus warn.
2. Structural integrity — do the placeholders and ICU arms in each target match the source? A German plural that drops the other arm, or a translation that loses a {count} placeholder, is a runtime crash waiting to happen.
3. Format validity — does every PO/XLIFF/JSON file still parse?

#!/usr/bin/env bash
# scripts/i18n-gate.sh — run after pull, before build
set -euo pipefail
BLOCKING_LOCALES=("de" "fr" "ja")
fail=0

for loc in "${BLOCKING_LOCALES[@]}"; do
  # 1. completeness: no untranslated sentinels allowed
  if grep -rq "__STRING_NOT_TRANSLATED__" "locales/$loc/"; then
    echo "::error::Untranslated keys remain in $loc"
    fail=1
  fi
  # 2. placeholder parity vs base locale
  node scripts/check-placeholders.mjs "locales/en" "locales/$loc" || fail=1
done

# 3. ICU + JSON validity across every locale
node scripts/validate-icu.mjs 'locales/**/*.json' || fail=1

exit $fail

Wire this as a required status check on the protected branch so a red gate physically prevents merge. The GitHub Actions i18n CI gates topic covers matrix parallelization across locales and how to split blocking errors from advisory warnings without doubling pipeline latency. Placeholder-parity checks lean directly on the pluralization rules across languages — a locale’s required CLDR plural categories define which ICU arms the gate must demand, which is why a German or Polish target that ships only an other arm should fail even when every key is technically “translated”.

Concept 4 — PO and XLIFF format bridging ¶

Pipelines rarely live in one format. Frontend frameworks emit JSON or ICU; gettext-based backends and most professional translation tools speak PO and XLIFF. The pipeline becomes the bridge, and the danger is silent data loss — comments, context notes, plural forms, and translation states that one format carries and another drops.

The two formats encode different metadata. gettext PO carries msgctxt for disambiguation, #. developer comments, and nplurals plural forms governed by a per-language Plural-Forms header. XLIFF 2.1 carries translation state (initial, translated, reviewed, final), <note> elements, and segment-level IDs. Converting blindly in either direction strips whatever the target format cannot represent.

The rule is to convert through a tool that preserves the superset and to assert no loss afterward:

1. Map plural categories explicitly. A PO file’s numbered msgstr[0]/msgstr[1] slots must map to the correct CLDR categories (one, few, many, other) for the target language, not to positional indices.
2. Carry context across. PO msgctxt becomes an XLIFF <note> or unit attribute; never drop it.
3. Preserve translation state. A reviewed XLIFF segment must not silently downgrade to “needs translation” in PO.
4. Round-trip test. Convert A→B→A and diff against the original.

# PO ↔ XLIFF 2.1 round trip with loss assertion
po2xliff -i messages.de.po -o messages.de.xlf --version 2.1
xliff2po  -i messages.de.xlf -o messages.de.roundtrip.po

# fail if the round trip lost anything semantically meaningful
diff <(msgcat messages.de.po) <(msgcat messages.de.roundtrip.po) \
  && echo "round trip clean" \
  || { echo "::error::PO/XLIFF conversion lost data"; exit 1; }

The full procedure, including how to preserve plural arms and context through gettext, lives in converting gettext PO to XLIFF 2.1 without data loss, part of the broader work on PO / XLIFF format bridging. Keep the bridge inside the pipeline rather than running it by hand: a one-off manual conversion drifts the moment a translator adds context in the TMS, whereas a converter invoked on every pull keeps both representations consistent and lets the round-trip assertion guard the seam continuously.

Concept 5 — Machine-translation pre-fill ¶

Machine translation in a pipeline is a pre-fill, not a publish. Its job is to give translators a non-empty starting draft so they edit rather than type from scratch, and to fill low-risk strings (status messages, generic UI labels) that rarely need human nuance. The two non-negotiables are that MT output is always marked as unreviewed, and that it passes the same structural gate as human work.

A safe pre-fill stage works like this:

1. Select only untranslated, unlocked keys. Never overwrite a human or reviewed string with MT.
2. Attach context to the request. Send the key, the source string, the component path, and any max_length or do-not-translate terms so the engine respects glossary and length constraints.
3. Write results with state="initial" (XLIFF) or a needs-review flag. MT must enter the review queue, never the “done” column.
4. Run the structural gate on MT output. An engine that drops a {count} placeholder or mangles an ICU plural arm must fail the same check a human typo would.
5. Feed post-edits back into translation memory so the next pre-fill reuses corrected phrasing and costs less.

// mt-prefill.ts — fill only empty keys, mark unreviewed, keep placeholders
import Bottleneck from 'bottleneck';
const limiter = new Bottleneck({ minTime: 120, maxConcurrent: 4 });

type Unit = { key: string; source: string; target: string; locked: boolean };

export async function prefill(units: Unit[], target: string): Promise<Unit[]> {
  const todo = units.filter(u => !u.locked && u.target.includes('__STRING_NOT_TRANSLATED__'));
  return Promise.all(todo.map(u => limiter.schedule(async () => {
    const mt = await translate(u.source, target);   // any MT/DeepL provider
    if (!placeholdersMatch(u.source, mt)) {         // guard: never drop {vars}
      return { ...u, target: u.source };            // fall back to source, flag later
    }
    return { ...u, target: mt, state: 'initial' };  // enters review queue, not "done"
  })));
}

The end-to-end design, including a quality gate on the MT draft before it reaches translators, is detailed in the machine-translation pre-fill workflows area and the DeepL pre-translation quality gate walkthrough. Corrections captured here are exactly what the translation memory and glossary management area consumes to keep future drafts on-brand. Treat the pre-fill as a cost-and-latency lever, not a quality one: it shrinks the empty-string backlog translators face, but the human review step is still where correctness is decided.

Cross-cutting concerns ¶

Atomic, versioned releases. Tag locale bundles with a version aligned to the application build, and deploy them together. A mixed-version deploy — new code, old strings — surfaces as missing keys or, worse, plausible-but-wrong text. Pin the locale tag in your release manifest so a rollback restores the matching strings.

APP=$(jq -r '.version' package.json)
git tag -a "l10n-$APP" -m "locale bundle for v$APP"
git push origin "l10n-$APP"

Edge caching with correct keys. Locale bundles are excellent CDN candidates, but only with content-addressed filenames (common.<hash>.json) and a Vary strategy that never serves one locale’s bundle for another. Cache the bundle, not the negotiated response; locale negotiation itself belongs at the edge or origin per the locale negotiation strategies area, and the bundle URL it resolves to should be immutable and far-future cacheable.

Accessibility and layout safety. A pipeline that ships untested target strings ships broken UIs. Run locale-aware visual checks in CI for the highest-expansion languages (German, Finnish) and right-to-left scripts (Arabic, Hebrew), so a 40%-longer German label or an un-mirrored RTL layout fails before release rather than in production.

Compliance and audit. For SOC 2 / ISO 27001 scope, every push, pull, and approval should emit a structured, append-only audit event. Because the pipeline is the only path strings take between systems, instrumenting it gives you a complete, tamper-evident trail of who changed which translation when.

Secret hygiene. TMS tokens and MT API keys live in CI secrets, are scoped to the narrowest project, and are rotated on a schedule. Never let a token reach the source catalog or a build artifact.

Five non-negotiable engineering principles ¶

1. Extraction is deterministic or the pipeline is noise. Sorted keys, fixed separators, lf line endings, and a pinned extractor version are prerequisites, not polish — without them every diff is unreviewable.
2. Sync is idempotent. Pushing unchanged source uploads nothing; pulling unchanged targets commits nothing. Enforce this with a content hash, not by hoping nobody runs the job twice.
3. The CI gate blocks, it does not warn. Release-blocking locales with untranslated or malformed keys must turn the build red as a required status check. A warning that ships anyway is not a gate.
4. Machine translation is always unreviewed until a human signs off. MT enters the review queue with an explicit initial/needs-review state and passes the same structural checks as human translation.
5. Locale releases are atomic and versioned. All target bundles for a release deploy together, tagged to the application version, so users never see a mixed-language screen and rollbacks restore matching strings.

Troubleshooting & gotchas ¶

FAQ ¶

Related ¶

• Crowdin integration for dev teams — wiring the SaaS sync loop directly into pull requests.
• Weblate self-hosted setup — running the same loop behind a VPC for data-residency requirements.
• GitHub Actions i18n CI gates — blocking untranslated and malformed keys before they ship.
• Core i18n architecture and locale negotiation — the upstream area that defines key shape, ICU format, and fallback behavior the pipeline mirrors.
• Framework i18n and component routing — where strings live in your app and how bundles load, which dictates your extraction layout.

Part of the i18n & l10n Pipelines documentation.