Branch-Assigned Queues
Warning
This is not authoritative documentation. These features are not currently available in Zuul. They may change significantly before final implementation, or may never be fully completed.
The following document describes a change to Zuul’s shared change queue behavior to support a wider variety of project organizational structures and development methodologies.
Introduction
We describe Zuul as a project gating system because the main purpose is to control which changes are merged. A key to how Zuul performs this activity is controlling the order in which changes are merged. By doing so, Zuul is able to stack multiple changes on top of each other when it builds its proposed future states of multiple repositories.
In order to ensure the correct co-gating behavior between multiple repositories, all of the repositories which could affect the outcome of integration test jobs must share a single queue so that if changes to project A and project B are to be merged, they are tested together.
We found early on that it is worthwhile to minimize the number of projects involved in a single queue. The queue must include all the projects in the integration tests that cover the projects, but ideally no more than that. If project C is not involved in the testing of projects A and B, then it is counter-productive for a change to C to wait in the queue behind changes to A and B.
In Zuul, a pipeline defines a workflow action, and it contains one or more queues that it uses when performing that action. A “gate” pipeline in Zuul which is responsible for merging changes may have a single shared queue for all projects, or it may have multiple queues, one for each individual project, or, more typically, a handful of queues each for a collection of projects.
There are currently two styles of shared change queues. The traditional behavior is that all changes for all projects that are added to the queue enter that single queue. That means that changes that involve different branches share a single queue. This is to support a development process where branches interact with each other. For example, OpenStack ensures that it can upgrade from the most recent stable branch to master. There is an upgrade job that acts as an integration test between these two branches (just as other jobs act as an integration test between different projects). In order to ensure changes to either branch don’t break the upgrade test job, changes to both branches must be sequenced in the shared change queue.
The other style is a “per-branch” queue. This facilitates a development process where different branches do not interact with each other. For example, an embedded system manufacturer may have a branch for one hardware revision, and a different branch for another. There is no need to coordinate changes between these branches, so the “per-branch” setting can be enabled which means that changes to branch X of any involved project go into one queue, and changes to branch Y of any involved project go into another. This allows developers of different hardware to work independently while still performing integration testing of their whole system.
A third style of development is currently unsupported: systems where projects have multiple independent branches, but the coordination of those branches across multiple projects is not done by sharing a common branch name. If we consider the previous “per-branch” case, that works under the assumption that all of the branches involved in hardware revision 1 use a name like “hw1” and all the branches involved in revision 2 use “hw2”. But if a system is composed from arbitrary branches of multiple repositories, there is no way to build corresponding change queues in Zuul. This document proposes a way to do that.
Proposal
The proposed change is simple: to allow users a third way of composing change queues where individual project-branch combinations can be assigned to specific queues. In contrast to the traditional and “per-branch” queues, we will call this a “branch-assigned” queue.
The default state of any project that is not assigned to a queue is for Zuul to dynamically create a queue that contains only that project. If a user adds a project to a traditional queue, then all branches of that project are added to that queue. If the user adds a project to a per-branch queue, then each branch of the project is added to the queue for that branch.
When a user assigns a project-branch combination to a branch-assigned queue, only that specific project-branch will be added to that queue. Each other project-branch combination may be added to a different queue. Any project-branch combinations not explicitly specified will be subject to a fallback behavior described below.
If any branch of the project specifies either a traditional or per-branch queue, then that will be considered the default queue for the project. If the user does neither, then the default queue for the project will be (as it is now) an automatically created traditional queue that contains only that project. Any project-branch combination that does not specify a branch-assigned queue will use the default queue for the project.
The behavior when determining what queue a change for a given project-branch will be assigned to is therefore:
If this project-branch was assigned to a branch-assigned queue, use that
If the project was assigned to a traditional or per-branch queue, use that
Use the automatically created traditional queue
User Interface
Currently, a project may only be assigned to a single queue, therefore
when we first see a queue attribute in a project stanza for a
given project, we take that as the project’s queue and ignore any
subsequent appearances. That means that if a project has a project
stanza in every branch of its repo, we consider only the one on the
master branch and ignore the rest.
The simplest and most intuitive way to support a branch-assigned queue
is to lift this restriction, and therefore, if the project stanza
on the master branch declares queue: foo and on the stable branch
declares queue: bar, then the master branch will be in foo and
the stable branch in bar.
However, that may be an unexpected behavior change for existing users
since they may be relying on the current behavior to ignore obsolete
values in older branches. Or they may remove queue attributes
from project stanzas in older branches. Therefore, we should
expect users to opt-in to this behavior. We will add a new attribute
to the queue object itself to indicate what type of queue it is.
We will add a type attribute to the queue object which can be one
of the following values: all-branches, per-branch, and
branch-assigned.
The all-branches value will be the default and is the traditional
queue behavior. The per-branch value will be a synonym for
per-branch: true. The per-branch attribute will be
deprecated.
We will use the new type attribute instead of adding a new
branch-assigned: true attribute because per-branch and
branch-assigned should be mutually exclusive (and neither one is
also a traditional all-branches queue).
It will be an error to specify both type and per-branch
attributes.
If a project stanza declares a queue assignment, when
considering what project-branch is involved, we will consider the name
of the project in the project stanza. If the name is a regular
expression, then it will apply to all projects that match the regular
expression. If the project stanza omits the branches
attribute, then we will use the current branch if it appears in an
untrusted project. If it appears in a config project, then it will
apply to all branches of the project. If the branches attribute
is a regular expression, then it will apply to all branches of the
project that match the regular expression. In other words, we will
follow the same procedure as outlined in project.branches.
Only the first queue assignment for a given project-branch combination will be honored; subsequent appearances will be ignored.
The following example would assign project1@legacy to the legacy queue, while all other branches of the project are assigned to the general queue:
- queue:
name: general
- queue:
name: legacy-queue
type: branch-assigned
- project:
name: project1
queue: general
- project:
name: project1
branches: legacy
queue: legacy-queue