Release Notes

3.5.0

New Features

  • The executor.min_avail_mem setting now takes cgroup limits into account. There is also a new metric zuul.executor.<executor>.pct_used_ram_cgroup available.

  • The job.secrets.pass-to-parent attribute has been added to allow secrets to be made available to playbooks in parent jobs (for example, to allow for jobs which are designed to use secrets, but leave it to child jobs to actually supply them). See also the Secret documentation.

  • The restriction on using known_hosts in playbooks has been lifted.

Upgrade Notes

  • The zuul_return module has been converted to an Ansible action plugin. Job playbooks no longer need to use delegate_to or a localhost only play with this module as action plugin by default run on localhost.

Security Issues

  • Jobs with secrets in untrusted projects now automatically set allowed-projects.

    It is possible to circumvent the use of allowed-projects in untrusted projects by creating a change which Depends-On a change which alters a project definition. This behavior may be unexpected, so documentation has been updated with warnings to avoid relying on it in sensitive cases.

    It may have been possible to expose a secret, or use resources protected by a secret, if a job using a secret was defined in an untrusted project on a system with an independent pre-merge post-review pipeline – that is, a pipeline with post-review set to true, manager set to independent, and which operated on changes before they merged.

    To prevent disclosure or use in this situation, allowed-projects is now automatically set to the current project when a secret is used in a job defined in an untrusted project, and it can not be overridden.

Bug Fixes

  • Untrusted playbooks no longer see ‘Adding hosts ssh with ansible_user to the inventory is prohibited’ when using the add_host Ansible task on localhost.

  • Fixed an issue where a trailing slash on the baseurl of a git driver connection config could cause an indefinite job hang.

3.4.0

New Features

  • Jobs may now return artifact URLs and they will be stored in the SQL database (if configured). See Returning artifact URLs for usage.

  • One or more zuul executors can now be added to a executor.zone. This is helpful if a cloud does not have any public IP addresses for nodepool nodes. Now you’ll be able to have a zuul executor on the same private network as your nodepool nodes.

  • The Ansible inventory file now includes nodepool.host_id variable if the node was launched using the OpenStack Nodepool driver.

  • A new scheduler option, scheduler.relative_priority, can be used to instruct Nodepool to fulfull requests from less-busy projects more quickly.

Upgrade Notes

  • The zuul.nodepool statistics have been moved under zuul.nodepool.requests to allow sub-stats to work correctly. For example zuul.nodepool.requested has become zuul.nodepool.requests.requested.total. The previously missing label and size counters are now available at zuul.nodepool.requests.<state>.<size|label>. For more info see the monitoring documentation.

  • The zuul-web service now requires access to ZooKeeper, as a result you may need to update your firewall allow access from the service. Additionally, zuul.conf should now contain a zookeeper configuration section.

3.3.1

New Features

  • A job using a semaphore now can configure if it should acquire the it before requesting resources or just before running.

Upgrade Notes

  • The acquiring behavior of jobs with semaphores has been changed. Up to now a job requested resources and aquired the semaphore just before it started to run. However this could lead to a high amount of resource waste. Instead jobs now acquire the semaphore before requesting the resources by default. This behavior can be overridden by jobs using job.semaphore.resources-first if some waste of resources is acceptable.

Security Issues

  • The add_host module options are restricted to a hostname, port, user and password. Previously, malicious options could be used to bypass protection and execute tasks on the executor. Only ssh and kubectl connection are authorized.

Bug Fixes

  • Jobs that encountered unreachable nodes are now correctly detected and retried.

3.3.0

New Features

  • A local project key file URI (eg. file:///path/to/key.pub) is now supported by the encrypt_secret.py tool. This allows encrypting secrets without directly accessing the Zuul web API to retrieve the project key.

  • Zuul now supports triggering a full reconfiguration by using the command zuul-scheduler full-reconfigure.

  • The Gerrit driver can now (optionally) report via HTTP instead of SSH. In the future, this will be used to report file and line comments (the SSH API only supports review messages).

  • Zuul now supports leaving file comments, though currently only when using the Gerrit driver. See the documentation for zuul_return for details.

  • A job now can pause itself using Return Values and let the child jobs run until they are finished. This can be used to serve some service which can be used by the child jobs.

  • An SSH keypair is now generated for every project and may be used in post-review jobs to access systems which trust that project.

  • The Zuul web dashboard has been rewritten in React.

  • The restriction on using add_host in playbooks has been lifted.

  • A new Build page in the web interface enable displaying a single Build information.

  • A new Notification Drawer and a ConfigErrors page in the web interface enable displaying the config-errors endpoint data.

  • A new Job page in the web interface enable browsing through job configuration.

  • Client certificate locations to be used by winrm connections can be configured now.

Upgrade Notes

  • The Zuul web dashboard is now a single index.html and static offload server needs new rewrite rules. Check the install instruction for backward compatible rewrite rules. Note that serving the web dashboard from a sub-directory requires the application to be rebuilt using the desired homepage location.

Deprecation Notes

  • Signal based triggering of a full reconfiguration via sending SIGHUP to the zuul-scheduler PID is deprecated. Use the command zuul-scheduler full-reconfigure now.

3.2.0

New Features

  • Zuul client got a new command ‘tenant-conf-check’. This command validates the schema of the tenant configuration and exits -1 if errors have been detected.

  • A new Ansible inventory variable zuul.child_jobs which is a list of the first level child jobs to be run after a job has finished successfully.

  • Jobs are now able to use the job.extra-vars which will use the –extra-vars flag when a job runs.

  • Project and project-templates may now create variables via a vars configuration entry. Jobs can access these at runtime in the same manner as job variables.

  • The supercedent pipeline manager has been added. It is designed to make post-merge artifact build pipelines more efficient.

  • The dequeue command has been added to the Zuul CLI. It allows operators to stop a given buildset at will.

  • It is now possible to use zuul_return to skip child jobs. You can use the zuul.child_jobs inventory variable to get a list of child jobs configured to run, then use zuul_return to modify the list. Any child job not in zuul_return zuul.child_jobs will be skipped. See Return Values for examples.

Bug Fixes

  • Project Templates are now branch-aware and behave more like project stanzas. If a template is defined on a branch, it will only apply to changes to that branch.

  • The timer trigger does not enqueue an event for every branch of every project anymore and it now only processes projects actually using the pipeline triggered by a timer.

3.1.0

New Features

  • The GitHub driver can determine the required status checks of pull requests which are needed for entering a gate pipeline. This eliminates the need to hard code required status checks in the gate pipeline and makes interoperation with other GitHub apps much more flexible.

  • Zuul is now ables to start with an invalid configuration. When reading configuration files from project repositories, if an issue is detected, Zuul will store the issue and skip the broken block of configuration. Issues are then reported in the scheduler log at the end of the configuration phase.

  • The json log now also contains the role name and the uuid similar to the task entry.

Upgrade Notes

  • Files (and irrelevant-files) matchers are now overridable. Zuul now uses only branch matchers to collect job variants. Once those variants are collected, they are combined, and the files and irrelevant-files attributes are inherited and overridden as any other job attribute. The final values are used to determine whether the job should ultimately run.

  • Zuul now uses Ansible 2.5.

Security Issues

  • Tobias Henkel (BMW Car IT GmbH) discovered a vulnerability which is fixed in this release. If nodes become offline during the build, the no_log attribute of a task is ignored. If the unreachable error occurred in a task used with a loop variable (e.g., with_items), the contents of the loop items would be printed in the console. This could lead to accidentally leaking credentials or secrets. MITRE has assigned CVE-2018-12557 to this vulnerability.

Bug Fixes

  • Untrusted playbooks no longer see ‘Executing local code is prohibited’ when using the zuul_return Ansible task.

3.0.3

New Features

  • The project.default-branch option is now documented. It has been supported since version 3.0.0, but was omitted from the documentation.

  • Project stanzas now support regex matching of project.name. This can be used to apply project pipelines to many projects at once.

Deprecation Notes

  • The merge-mode and default-branch attributes may no longer appear in a Project Template stanza.

Bug Fixes

  • Configuration loading for dynamic configuration changes (i.e., changes to zuul.yaml files) is now significantly more CPU and memory efficient, incurring only a slight penalty compared to normal changes.

3.0.2

New Features

Upgrade Notes

  • The fb-re2 python library is added as a dependency; this may required the installation of the re2 library and header files in order to build.

Bug Fixes

  • Story 2001441 is fixed. Failure by one Zuul reporter will not short circuit the reporting of other reporters. This ensures as much information as possible is reported for each change even if some failures occur. Note that the build set status is changed to ‘ERROR’ after the first failed reporter.

  • The zuul-changes.py script has been adapted to the new zuul-web api routes.

3.0.1

New Features

  • Git repositories will have a origin remote with refs pointing to the previous change in the speculative state.

    This allows jobs to determine the commits that are part of a change, which was not possible before. The remote URL is set to a bogus value which won’t work with git commands that need to talk to the remote repository.

  • PostgreSQL is now officially supported as database backend. See <sql connection> on how to configure database connections.

Upgrade Notes

  • The alembic version table is fixed to being prefixed too. This is necessary when using <sql connection>.table_prefix. However if you are already using table_prefix you will need to rename the table alembic_version to <prefix>alembic_version before starting Zuul. Otherwise zuul will try to create the tables again and fail. If you’re not using table_prefix you can safely ignore this.

Bug Fixes

  • Zuul role repository checkouts now honor job.override-checkout.

    Previously, when a Zuul role was specified for a job, Zuul would usually checkout the master branch, unless that repository appeared in the dependency chain for a patch. It will now follow the usual procedure for determining the branch to check out, including honoring job.override-checkout options.

    This may alter the behavior of currently existing jobs. Depending on circumstances, you may need to set job.override-checkout or copy roles to other branches of projects.