Shared file version history restored after build change…

Since the block editor was first merged into wordpress/wordpress-develop (leading up to the WordPress 5.0 release), there has always been a considerable amount of manual work required to sync the necessary changes in the gutenberg GitHub repository into SVN. During the early phases of the 7.0 release cycle, #64393 worked on making changes to the wordpress-develop build scripts with the goal of simplifying this process.

While the initial iteration of these changes committed in [61438] removed some of that manual friction, specific parts of the change disrupted a few important contributor workflows due to the way that PHP files were removed from version control and added to the .gitignore file. These particular PHP files were shared in both repositories, and the idea of removing them was to ensure that the source of truth remains clearly in the Gutenberg repository. Instead of existing in both repositories, a script would create the files in the working copy of the wordpress/wordpress-develop repo.

What went wrong?

Unfortunately, there were a number of unanticipated side-effects of this change:

A number of existing integrations started failing where the build script was not run due to the fact that WordPress calls require directly on these files to load them. This included the distributed hosting tests.
Several files that were previously minified by the build script no longer were (see #65007, #64909)
While the source of truth of these files lives in Gutenberg, it’s valuable to know when the updates were brought into Core and what those updates were. Development in the Gutenberg plugin runs at a different pace than in Core, which means that diagnosing changes to these files from the Gutenberg side is considerably more difficult and time-consuming than when there are changes in version control, which provide pins for when defects are introduced or resolved.
Technically, there was a dual source of truth for these files before the change. The ultimate copies which made it into a WordPress release were those which had been committed into SVN, while the Gutenberg plugin might ship changed or updated files. After the change, that “ultimate” copy no longer existed in SVN, except in the final release bundles. The “source of truth” for the files in both contexts (WordPress and the Gutenberg plugin) lived solely in Gutenberg, but might actually be different from each other due to the new commit hash pinning in wordpress-develop.
Because these files were removed and the change added this new requirement to run the build step, major delays were introduced to any workflow which steps through commits. Even after a series of optimizations, this added over nine hours of runtime just to walk the 800+ commits from 6.8.5 to 6.9.0.
Version history for these files was severed in the change, making it suddenly look like the files were never part of the repository. Any attempts to review the history required custom git commands not usually available in IDEs or GitHub’s UI.
Files that were removed from version control were abandoned on the build server, which did not clean up properly. This resulted in removed files persisting and being included unintentionally in the beta and RC versions (see #64716 and #65418).
While it happens from time to time that someone accidentally updates the code in one of these files in the Core repository instead of in Gutenberg where it should be done, the .gitignore directives hide any of those accidental changes, frustrating an already confusing situation. All git-based tooling (IDEs, git, git GUIs, linters, etc…) is unable to see or revert changes to these files, giving a false sense of safety when someone checks in code that will unexpectedly fail once running on a different computer.

What was resolved?

Two fundamental problems needed to be resolved:

Files that are required by WordPress’ boot sequence needed to be restored.
The version history continuity for these files would ideally need to restored.

On one hand, reverting the original change would have been an easy way to restore the missing files. But doing so would make it appear as though the revert commit was introducing brand new files with the same name as the ones which had been deleted, rather than restoring the deleted files. The history before January 5 would have remained lost. A revert or a new commit which copies the files back remained an easy option, but something else was worth pursuing: restore the files with their history.

The chosen resolution was to create a branch of the code from the revision just before [61438], restore all of the files which were removed, and then perform a merge of the branch back into trunk to reconnect the version history. Until this point, creating branches was normal in the WordPress codebase for each release, but none had been reintegrated into trunk through a merge.

The merge commit provided an opportunity to restore the file contents and connect them to their history. While the original plan was to create a branch with a single commit (the merge commit itself) @desrosj had the suggestion that it would be valuable if the intervening history of changes could also be represented. In other words, to create the restore branch as an effective revert of the decision to remove all these files, not just the file objects themselves.

The result is that the restore branch contains a sequence of commits, where each is associated with a Gutenberg-sync commit in the upstream trunk branch. These commits modify the deleted files and introduce new files that would have been added had it not been for the .gitignore and svn:ignore changes. This is tantamount to checking out trunk at each Gutenberg sync commit, reverting the .gitignore changes, running npm run build:dev, and then adding the no-longer-ignored files (but the details are more complicated than this).

After the final merge, it’s now possible to examine one of the affected files and immediately know at which Gutenberg sync commit it changed and how. Supposing someone discovered that an issue appeared within the first quarter of 2026, this additional granularity might save the need to scan through hundreds of commits when debugging.

What was the process for the resolution?

The original change itself serves as a cautionary guide on how far-reaching the effects can be for fundamental changes to WordPress’ development infrastructure. Because no restoration-merge had ever occurred, it was worth performing full diligence to prevent similar blowback while trying to fix the repository.

The approach was discussed early on and remained open for a long time to allow for rarer implications to surface — many did. The topic was brought up in Trac, in the #core channel in Slack, and during weekly Developer Chats in #core. The solution was brought to the attention of the systems team for their input, and as much as was possible, the restore was simulated at every stage to catch any failures which could be reasonably anticipated – many were.

git is a more powerful companion in situations like this than svn is, but ultimately all of WordPress’ code must begin its life in svn. Therefore, a script was created to prototype a resolution in git to produce the desired outcome in a way that could be tested in local checkouts as well as run the test suite. The goalpost for this test branch was that, if done correctly, in the merge with trunk inside of its PR, there would be no follow-up commits from the bot that runs the build command (because no files would be changed through the build). The simulation script made it easy to adjust strategies and add missing steps once certain problems surfaced. One such problem was that some files didn’t make it at the right time into one of the grunt lists when it should have; this could have involved a lot of manual work, but it didn’t because of the script. The script still involved a lot of effort, but as many times as it ran, rebuilding the branch within a few minutes, it was clear that it was paying off.

Once the git branch was ready it was time to ensure that the steps in svn would actually produce that desired outcome once synchronized. For this, @abbe provided a test copy of the develop.wordpress.org Subversion repo that could be used to verify the flow. A second script stepped through each commit from the git branch and created mirrored commits on the svn side, then performed a merge into the test repo’s trunk branch. The goalpost for this side was that the end result matched identically to the final exploratory git branch. In the end, it took fifteen full iterations of the simulated branch creation to get the expected result. Imagine finding out all of the little nuances only after deploying to production!

With all of the details then ready, it was go-time and time to deploy for real. Sadly, not all aspects of the flow were testable beforehand, and problems did arise which stalled the restoration. The “build” server2 got stuck while processing commits due to a fatal Grunt error and @aidvu had to step in and manually clear up the issues on that server. However, while delayed, all it took to reach the successful end was running the script once more. Thankfully, after preparing each commit and before pushing them, a pause step was built-in for manual review before committing to the commit, and this allowed coordination with systems (it’s ideal to build natural throttles in to automation to avoid billions of errors per second).

What is important to understand about this merge?

When working with merged branches, git automatically represents commits from all of the branches in history, but svn doesn’t. This means that git-based workflows should be working without any trouble. But for those working in svn, a basic svn log will not show the corresponding sync-commits from the branch. One has to run svn -g log to see them. Any svn-based tooling will therefore hide the commits by default.

There is one solvable nuance that was cut from scope for this resolution: commit authors and timestamps. While it was demonstrated to be technically possible to match the sync commits with their associated commits in the upstream trunk branch, this would have required rewriting metadata in svn after making the commits, and there was no certainty that this wouldn’t interrupt the git-svn sync. Therefore, while each sync-commit matches another commit from January 5 through March 26, each of the restore-branch sync-commits appears in a bunch on March 26. This is a minor issue; regrettable but acceptable given the sensitivity of the work done to bring the repository back into a proper state.

What still remains to be done?

A few issues arose during the restoration, some of which highlighted pre-existing problems.

The git and subversion “ignores” lists are incompatible and out of sync. These need to be harmonized and it would be ideal to have an automated process flag discrepancies before they are committed (#64971).
A number of files have been deleted from the git repository over time which were never removed from the subversion side. This leaves stale files in svn and on the build server. These need to be identified, removed, and added to the $_old_files array in wp-admin/includes/update-core.php (see #65418). Automated detection would be helpful here as well (see #64878).
The continued discrepancy between WordPress Coding Standards preferences between the Gutenberg and Core repositories remains an obstacle to harmony between the projects. Rulesets should be normalized or removed from the repositories so that accepted changes don’t suddenly reject PRs when synchronizing. This issue also recently came up when restoring the WordPress documentation, as PHP code which was accepted in Gutenberg broke the docs’ generation process once built in Core. This was due to the use of syntax forms which were already known to break tooling and integrations.
Functions and hooks maintained upstream in the gutenberg repository that are built from the new template files in the wp-build package lack proper PHP Docblock comments. Mainly, action and filter hooks had incorrectly formatted Docblocks (fixed in Gutenberg-78826), and missing @since tags (see Gutenberg-76727).

A few other unrelated issues persist as a result of the build change.

The workflow for developing simultaneously with the Core and Gutenberg repositories needs to be restored, as the traditional approach of mounting the Gutenberg plugin into the wp-content/plugins/ directory was severed in the change.

Thanks you!

A heartfelt “thank you” goes out to everyone who helped with overhauling the build script to reduce friction between the two code bases, and/or this effort to restore file history:

@4thhubbard, @762e5e74, @adamsilverstein, @aidvu, @amykamala, @bernhard-reiter, @dd32, @desrosj, @dmsnell, @ellatrix, @gaisma22, @isabel_brison, @johnbillion, @jonsurrell, @jorbin, @jorgefilipecosta, @jsnajdr, @jtquip88, @mamaduka, @manhar, @manzoorwanijk, @mcsf, @mywp459, @ntsekouras, @peterwilsoncc, @sabernhardt, @SirLouen, @swissspidy, @tobiasbg, @tyxla, @westonruter, @wildworks, and @youknowriad.

Timeline

January 5 — The change was committed.
January 6 — The first set of problems was reported.
February 23 — An initial git branch was proposed which restored the files in their final state.
March 6 — The restore branch was created in Subversion, and successfully synchronized to git.
March 18 — An expanded git branch contained a commit for every associated Gutenberg sync commit in Core.
March 24 — Systems provided a test svn repository from a backup of develop.svn.wordpress.org where a final merge could be tested before being applied to the production repository.
March 26 — The branch was merged into Core, restoring the files and their change history.

Props to @amykamala and @desrosj who reviewed this post before publishing.

The wordpress.org build server checks out each commit to the wordpress-develop SVN repository, runs npm install, npm run build, and then commits any changes to the build directory to the core.svn.wordpress.org repository. All release packages are created from this repository. ︎

#build-test-tools, #git, #post-mortem, #svn

Shared file version history restored after build change disruption