New sync process: Gutenberg code now imported via GitHub Container Registry zip

During the 7.0 release cycle, the way code maintained in the Gutenberg repository is imported into the wordpress-develop repository changed from using published npm packages to downloading a zip file of built assets published to the GitHub Container Registry by the build-plugin-zip.yml workflow file in Gutenberg (see #64393, Gutenberg-75844).

There were two bugs preventing wordpress-develop from being updated with the latest changes (Gutenberg-76715 and #65418). These have been fixed and after [62577-62578,62580-62584], trunk is now in sync with the most recent gutenberg release (currently 23.4.0).

To set expectations and establish some consistency going forward, this post outlines the process for syncing the two repositories going forward, and how to perform the syncing process.

Syncing Practices

The following sections aim to define when and how to sync changes from the gutenberg repository into the wordpress-develop repository.

Cadence During Alpha Periods

For the 7.1 release cycle, syncing will happen one week after each general release of Gutenberg. This ensures that trunk is reasonably up to date with the latest changes, but still allows some time for any follow-up bug fixes that are required. The goal is to eventually sync weekly, or even daily.

Syncing During The Release Cycle Beta/RC Phase

Once the Beta 1 point is reached for a release, the SHA value pinned to gutenberg.sha in package.json for trunk will be updated to one belonging to the release’s corresponding wp/X.Y branch in the Gutenberg repository when the next syncing occurs. trunk will remain pinned to a wp/X.Y branch hash value until branching occurs. This prevents new feature work not intended for the upcoming WordPress release from leaking into the SVN code base.

Because individual changes targeted for an upcoming WordPress release are cherry-picked into each wp/X.Y branch, syncing can happen as often as necessary. However, the most recent changes must be synced prior to each beta and RC release.

Branching in WordPress SVN

After branching is performed in WordPress core SVN, the new numbered branch in SVN will remain pinned to the corresponding wp/X.Y branch in the Gutenberg repository.

After branching, the trunk branch should be bumped to X.Y+1-alpha (example [62161]) and a second commit should be made changing the pinned SHA value back to the most recent Gutenberg plugin release in trunk, thus syncing all of the changes since Beta 1. Making two commits creates two distinct reference points: one for bumping the version, one for documenting all of the changes being synced into the code base from Gutenberg.

Note: Branching typically happens immediately after the RC1 release is published for an upcoming major version. But in some cases, branching can be delayed or moved up based on factors unique to the current release.

Post-Branching in WordPress SVN

After branching has occurred for an upcoming major release and the numbered branch is pinned to the corresponding wp/X.Y branch, syncing should happen as often as necessary to ensure the latest changes targeted for that release are included in each Beta and RC release.

Committers should balance the frequency of updating with the net benefit after considering other factors, such as the severity of the fix being merged, the non-zero amount of noise each commit makes, contributors needing to pull updates/merge the latest into their pull requests, the volume of related reports being made, etc. Syncing solely to pull in a typo fix is probably unnecessary. But syncing to only pull in a bug fix for an API that was broken is worth considering.

This process will continue during maintenance releases.

trunk will return to being synced during the week opposite Gutenberg plugin releases.

Example Timeline

Using the upcoming 7.1 release as an example, here is a timeline of events:

• Gutenberg: Version 23.6.0 of the plugin is released.
• Gutenberg: The wp/7.1 branch is created in the gutenberg repository.
• WP SVN: Prior to 7.1-beta1, trunk is updated to the most recent hash in the wp/7.1 branch of gutenberg.
• WP SVN: trunk is synced with wp/7.1 before every beta or rc release (and as often as necessary in between).
• WP SVN: After RC1 the 7.1 branch is created.
• WP SVN: trunk is updated to 7.2-alpha.
• WP SVN: trunk is updated to sync version 23.7.0 of Gutenberg.
• WP SVN: The 7.1 branch continues to be synced prior to each RC and before final release.
• WP SVN: trunk returns to being updated one week after each general release of Gutenberg.
• WP SVN: WordPress 7.1 is released. The 7.1 branch is updated prior to beta/RC versions for minor releases, and whenever necessary going forward (remaining pinned to wp/7.1).

Minor Releases & Backporting In WordPress SVN

The process for merging commits into a numbered branch for a minor release will remain the same.

1. Commit the change to trunk.
2. Mark the Trac ticket for backport consideration by moving to the minor release milestone with fixed-major, and commit dev-feedback for sign off from a second committer to backport.
3. Merge to the numbered branch after a second sign off is added with the commit dev-reviewed keywords.

However, there is one small change that will be required to this workflow. Because numbered branches now use SHA values from the corresponding wp/X.Y branches and trunk has the latest changes from trunk in Gutenberg, it is likely not possible to merge a single fix into trunk first.

Therefore, commits changing pinned SHA values to numbered branches will be allowed provided the double signoff process is followed.

Note: If any PHP changes are required that will not be included in the sync commit after bumping the pinned SHA value, they should be committed to trunk first and follow the backporting process.

Merging Changes

Creating A Sync Pull Request

To create a pull request for syncing the two repositories, find the full-length hash value for the version of Gutenberg being targeted for syncing and update the gutenberg.sha value in the package.json file.

Running build:dev locally will update every built file with the corresponding changes. However, there is a GitHub Actions workflow that pushes these changes back to a PR’s HEAD branch automatically.

Reviewing A Sync Pull Request

When the value of gutenberg.sha is updated, one or more Gutenberg plugin releases are merged into wordpress-develop. As a result the number of modified/added/deleted files in the PR itself will be quite high and validating every single one is not possible. However, the files updated should only consist of those modified by the build script (mainly build:dev). Any changes to files managed manually must be made separately.

When reviewing a sync PR, the main things to verify are:

• No new changes exist locally after running build:dev.
• The changed files line up with the changes listed for
• Does WordPress run as expected locally using the PR?

Who Is Responsible For Syncing?

Anyone can create the pull request to update the hash value pinned in wordpress-develop! The contributor with the best working knowledge of the changes included in a given Gutenberg plugin version is the contributor leading that release.

To start, creating a ticket on Trac for syncing and the initial pull request for the release will be added as items in the Gutenberg Plugin Releases page in the handbook.

There are opportunities to automate parts of this process, but more time is needed to get that working properly.

Allowed Hash Values In wordpress-develop Commits

There are several ways to pull in code from the gutenberg repository by specifying different values for gutenberg.sha:

• Full-length commit SHA
• Plugin release-specific tags such as release-23.4 (after the release/23.4 branch is created)
• WordPress version-specific tags such as wp-7.1 (after the wp/7.1 branch is created)
• Pull request-specific tags such as pr-123456
• Bleeding edge changes using trunk

Each reference type above is updated after each commit. The build script in wordpress-develop will always attempt to fetch the most recent version before building.

While these tags are helpful for local development, their mutable nature does not guarantee idempotency. Full-length commit hash values are the only immutable references. Given this, only full-length SHA values are allowed to be used as values for gutenberg.sha in the package.json file.

Trac Tickets And Merge Commits

Because these merges include many different features and bug fixes, it can quickly become difficult to track when certain specific changes are merged into wordpress-develop from gutenberg.

To improve clarity, Trac tickets should be created and utilized as follows:

• All changes and updates to files not managed by the build script require individual tickets (current practice).
• A new ticket should be created for every hash bump during the alpha period (new practice).
• A single ticket can be used for all hash bumps between each beta and RC release (new practice).

Examples: A single “Gutenberg Syncs for Beta 2” ticket can be used for all hash bumps between beta1 and beta2. A single “Gutenberg Syncs for RC2” ticket can be used for all hash bumps between RC1 and RC2. But hash bump A and hash bump B during the alpha period must have separate tickets.

This helps to avoid Trac tickets with 100s of comments, and 10s of associated PRs, and 10s of commits and creates a single point of tracking for each merge point.

Commit Messages

The following commit message format should be used when committing a pinned SHA update:

Component: Bump the pinned hash from the Gutenberg repository.

(WITH versions aligning with tags)
This updates the pinned commit hash of the Gutenberg repository from `%%OLD_FULL_HASH%%` (version `v25.0.0`) to `%%NEW_FULL_HASH%%` (version `26.0.0`). (versions are required when the hashes correspond to one, but optional when not directly associated with a specific release tag)

(Without versions aligning with tags)
This updates the pinned commit hash of the Gutenberg repository from `%%OLD_FULL_HASH%%` to `%%NEW_FULL_HASH%%` and merges all of the changes that were cherry-picked to the `wp/7.0` branch between WordPress `7.0-beta1` and today (preparing for `7.0-beta2`).

A full list of changes included in this commit can be found on GitHub: %%LINK%%.

The following commits are included:
- Pattern Editing: The best pattern feature yet! (https://github.com/WordPress/gutenberg/pull/#####)
- Global Styles: Adding support for feature X within the block styles. (https://github.com/WordPress/gutenberg/pull/#####
- etc..

Follow-up to [27195], [41062]. (optional)

Reviewed by a-fellow-committer, maybe-multiple.
Merges [26851] to the x.x branch. (both of these are only required when backporting from `trunk`)

Props person, another.
Fixes #30000. See #20202, #105.

The following command can be used to generate the list of changes being included (the two dot comparison is intentional):
git log --reverse --format="- %s" OLDHASH..NEWHASH | sed 's|#\([0-9][0-9]*\)|https://github.com/WordPress/gutenberg/pull/\1|g; /github\.com\/WordPress\/gutenberg\/pull/!d' | pbcopy

Next Steps

• Document the various ways to pull in changes from the gutenberg repository upstream (see Gutenberg-78211).
• Update the Core Handbook’s Best Practices for Commit Messages page to include the merge commit formatting.
• Update the Branching Before Release section of the Releasing Major Versions page in the Core Handbook to include the new steps and adjustments detailed above.
• Update other release checklists (both major and minor)
• Submit a PR to add new steps to the Gutenberg Plugin Release page of the Block Editor Handbook.

Summary

After considering different options and examining how all the moving pieces work, this process was chosen as a way to balance moving faster while also encouraging stability, and continues to follow long-established historical practices dictating how code is managed from release to release.

Any necessary adjustments can be made as needed and everyone’s feedback is welcome!

Props: @adamsilverstein, @aduth, @annezazu, @ellatrix, @jeffpaul, @jonsurrell, @mamaduka, @tyxla, @wildworks, @youknowriad for peer review and discussing aspects of this post before publishing.