GitHub Actions matrix jobs per locale

The localization gate runs as one job that loops over every locale. Polish fails, the loop exits, and the report says “Polish coverage 91%, below 95%”. Someone fixes Polish, pushes, and now Japanese fails for a different reason. Three round trips later, the pull request merges.

Every one of those failures existed in the first run. A single job reported one of them.

Single job compared with a per-locale matrix A single job looping over locales produces one ordered log and one cache restore, but stops at the first failure and takes as long as the sum of all locales. A matrix runs each locale as its own job, so every locale reports independently and all failures are visible, at the cost of repeating setup per job. One job or one job per locale Single job, loop over locales · One log to read, in order · First failure hides the rest · Serial: time = sum of locales · One cache restore Matrix, one job per locale · Every locale reports independently · All failures visible in one run · Parallel: time = slowest locale · One cache restore per job Matrix when locales fail independently; a single job when the check is cross-locale by nature.
The deciding question is whether a failure in one locale tells you anything about another.

Root cause: a loop reports the first failure, not all of them

A shell loop over locales exits on the first non-zero status, which is correct behaviour for a script and unhelpful behaviour for a report. The information the author needs is the full set of what is wrong, and a loop structurally cannot produce it — the second locale never runs.

Working around it inside the loop, by collecting failures and exiting at the end, gets closer but keeps the second problem: everything is serial. Ten locales at forty seconds each is nearly seven minutes of wall-clock time, on work that has no dependencies between locales at all.

A matrix solves both. Each locale becomes its own job, all of them run in parallel, and each reports its own status — so the pull request shows exactly which locales fail and the run takes as long as the slowest one rather than the sum.

Per-locale checks against cross-locale checks Coverage thresholds, ICU parsing and format round-trip integrity are per-locale properties and fan out well. Key parity and duplicate detection compare locales or namespaces against one another and must run once over the whole set, since a per-locale job cannot see the comparison. Which checks belong in a matrix Matrix? Why Coverage threshold yes each locale has its own floor ICU parse yes one locale broken says nothing about others Key parity no compares locales against each other Duplicate keys no a cross-namespace property Round-trip integrity yes per-file, independent
Fan out what is independent; keep together what is a comparison.

Not every check belongs in the matrix

The split is between checks that are per locale and checks that are about the relationship between locales.

Coverage is per locale: each has its own threshold and its own result. ICU parsing is per locale: a syntax error in the Polish catalogue says nothing about Japanese. Format round-trip integrity is per file.

Key parity is not. It asks whether every locale has the same key set as the source, which is a comparison across the whole set — a job that only sees Polish cannot answer it. Duplicate key detection is the same: it is a property of the namespaces taken together.

Putting a cross-locale check inside a per-locale matrix produces one of two failures. Either each job re-runs the whole comparison, wasting time and reporting the same failure ten times, or each job compares only its own locale and the check silently stops testing what it was written to test.

The workflow 

name: i18n gates
on: pull_request

jobs:
  locales:
    runs-on: ubuntu-latest
    outputs:
      list: ${{ steps.read.outputs.list }}
    steps:
      - uses: actions/checkout@v4
      - id: read
        # Generated from the config, so a new locale is tested the day it is added.
        run: echo "list=$(jq -c '[.locales[].code]' i18n.config.json)" >> "$GITHUB_OUTPUT"

  per-locale:
    needs: locales
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false                       # one failing locale must not cancel the others
      matrix:
        locale: ${{ fromJSON(needs.locales.outputs.list) }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: pnpm }
      - run: pnpm install --frozen-lockfile
      - run: pnpm i18n:gate --locale=${{ matrix.locale }}

  cross-locale:
    needs: locales
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: pnpm }
      - run: pnpm install --frozen-lockfile
      - run: pnpm i18n:parity && pnpm i18n:duplicates

  gates:
    needs: [per-locale, cross-locale]
    if: always()
    runs-on: ubuntu-latest
    steps:
      # One status for the branch protection rule to require.
      - run: |
          [ "${{ needs.per-locale.result }}" = success ] \
            && [ "${{ needs.cross-locale.result }}" = success ] || exit 1

Two details carry most of the value. fail-fast: false is what makes the matrix report everything rather than cancelling siblings on the first failure — with the default, a matrix behaves almost exactly like the loop it replaced. And the locale list is read from configuration rather than written in the workflow, so adding a locale cannot silently leave it untested.

Five steps for a per-locale job matrix A setup job reads the locale list from configuration and emits it as output, the matrix fans out with fail-fast disabled so one failure does not cancel the others, each job restores the same cached dependencies, a summary job collapses the results into one required status, and checks that compare locales run once in their own job. Building the matrix from the locale list 1 Generate the locale list as job output read it from the config, never hardcode 2 Fan out with fail-fast disabled so one locale does not cancel the rest 3 Give each job the same cached setup install once, restore per job 4 Summarise into one required check a single status the branch rule can require 5 Keep cross-locale checks in a separate job parity and duplicates run once
Generating the list is what stops a new locale being silently untested.

The summary job, and why branch protection needs it

A matrix produces one check per locale, and the names include the locale — per-locale (pl), per-locale (ja). Branch protection rules require checks by name, so a rule listing today’s locales stops covering tomorrow’s, and a locale added next month is unprotected without anyone noticing.

A summary job that depends on the matrix and collapses its result into a single status solves it: the branch rule requires one stable name, and the set behind it grows automatically. The if: always() is necessary because a summary job with default conditions is skipped when a dependency fails — and a skipped required check does not block a merge, which is precisely backwards.

This pattern is worth applying to every fan-out in a pipeline, not only localization, and the localization matrix is usually where a team meets it first because locales are the thing that grows.

Making the report readable

Fanning out gives you every failure at once, which is only an improvement if the result is legible. A matrix of twenty red checks with generic names is not obviously better than one red check.

Three things make the difference. Name the job by what a reader needs to know — the locale code, not the step. A check named per-locale (pl) is scannable in a list of twenty; one named gate repeated twenty times is not.

Write the actionable detail into the job summary rather than the log. A failing coverage job should state the locale, the measured percentage, the threshold and the two or three keys that most contribute to the gap. That turns a red check into an instruction, and it removes the step where someone opens the log and scrolls.

Annotate the diff where the problem is. A malformed ICU message has a file and a line, and emitting a workflow annotation puts the error next to the string in the pull request view. For catalogue problems this is unusually effective, because the reviewer looking at the diff is the person who can fix it.

The general principle is that a fan-out multiplies both the information and the noise, and the work of keeping it useful scales with the number of jobs. A matrix over five locales needs almost no attention; one over thirty needs the summary and the annotations, or people will stop reading it and start re-running it hoping for green.

Verification 

# Every locale in the configuration produces a job
gh run view --json jobs -q '[.jobs[].name] | map(select(startswith("per-locale")))'
#   ["per-locale (en)","per-locale (de)","per-locale (fr)","per-locale (ja)","per-locale (pl)"]

# A deliberate break in one locale must fail exactly one matrix job
#   and must not cancel the others

The second check is worth running once, by hand, when the workflow is introduced. A matrix that cancels siblings looks fine until the day two locales are broken, and that is a poor day to discover the configuration is wrong.

When to escalate

If the matrix is slower than the loop it replaced, the per-job setup dominates. Installing dependencies ten times costs more than the checks themselves, and the answer is a shared setup — a prepared artifact or a container image — rather than fewer jobs.

If jobs fail with rate limits or quota errors, the parallelism is exceeding an external limit, typically a translation service API. Bounding the matrix with max-parallel keeps the reporting benefit while respecting the limit.

If a locale passes in the matrix and the merged result is still broken, the failure is cross-locale and belongs in the other job — the parity check that per-locale jobs are structurally unable to perform, described in GitHub Actions i18n CI gates.

FAQ

Should every locale block a merge?

Not necessarily. A common arrangement holds primary markets at one hundred percent coverage and lets newer locales run as non-blocking reports, which is expressed by allowing specific matrix entries to fail without failing the summary. What matters is that the policy is explicit rather than emerging from which checks happen to be required.

How many parallel jobs is too many?

The practical limits are your runner concurrency and any external API in the checks. Beyond about twenty locales the matrix is usually still fine, and the constraint that appears first is the shared setup cost rather than the parallelism itself.

Can the matrix include locale-specific thresholds?

Yes, and it is the neatest way to express them: emit objects rather than strings from the setup job, so each matrix entry carries its own threshold alongside its code. The gate then reads the threshold from the matrix context instead of looking it up.

Does this work for the pseudo-locale too?

It does, and including it is worthwhile — the pseudo bundle should parse and preserve arguments like any other locale, so a matrix entry for it catches transform regressions in the same run as everything else, as described in localization testing and pseudolocalization.

Part of GitHub Actions i18n CI Gates.