PHPUnit CI jobs cut 52%; rerun flakiness halved

In recent years, the CI load across core WordPress GitHub repos increased as the project evolved. This is normal for an active project.

But, the load can be especially high when running the full PHPUnit matrix at once on many PRs, such as during a release with many backport branches. Cases like this mean the number of jobs can strain the capacity of the system.

Ahead of 7.1, we’ve landed a round of trims and reliability fixes aimed at increasing capacity and running leaner while ensuring quality checks still run.

This CI work builds on the test-suite optimization work from many contributors in the past.

Two key changes

1. Trimmed the PHPUnit matrix to boundary PHP versions (#12719 on trunk, #12720 for 7.0). Full PHP coverage kept, redundant database combinations dropped. Per run: ~52% fewer jobs and ~54% fewer job-minutes.
2. Fetch the Gutenberg build once per run (#12701) instead of once per job, plus bounded retries on Docker image pulls (#12703). Runs needing a rerun to go green roughly halved, from ~68% to ~36%.

Source: measured per run from the GitHub Actions API (job counts and durations, and rerun counts) across comparable runs before / after the changes.

Next up

• Trim the 6.8 matrix (#12726).
• Pilot a dedicated larger-runner pool for use during release windows to help with concurrency issues (more on that in a future update).

Note: The PHPUnit tests themselves could be further tuned. The trims above cut job count, not job duration—so we could give more attention to the tests themselves to find more efficiency.

Thanks to contributions from adrianmoldovanwp, barry, garyj, @johnbillion, @jonsurrell, @jorbin, lucasbustamante, and @mukesh27.