Zuul Client

Zuul includes a simple command line client that may be used to affect Zuul’s behavior while running. It must be run on a host with access to Zuul’s web server.

Configuration

The client uses the same zuul.conf file as the server, and will look for it in the same locations if not specified on the command line.

The webclient section is required.

It is also possible to run the client without a configuration file, by using the --zuul-url option to specify the base URL of the Zuul web server.

Note

Not all commands are available through the REST API.

Usage

The general options that apply to all subcommands are:

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul [-h] [-c CONFIG] [--version] [-v] [--auth-token AUTH_TOKEN]
            [--zuul-url ZUUL_URL] [--insecure]
            {autohold,autohold-delete,autohold-info,autohold-list,enqueue,enqueue-ref,dequeue,promote,show,tenant-conf-check,create-auth-token,import-keys,export-keys,copy-keys,delete-keys,delete-state,delete-pipeline-state}
            ...

Zuul CLI client.

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG             specify the config file
  --version             show zuul version
  -v                    verbose output
  --auth-token AUTH_TOKEN
                        Authentication Token, needed if using theREST API
  --zuul-url ZUUL_URL   Zuul API URL, needed if using the REST API without a
                        configuration file
  --insecure            Do not verify SSL connection to Zuul, when using the
                        REST API (Defaults to False)

commands:
  valid commands

  {autohold,autohold-delete,autohold-info,autohold-list,enqueue,enqueue-ref,dequeue,promote,show,tenant-conf-check,create-auth-token,import-keys,export-keys,copy-keys,delete-keys,delete-state,delete-pipeline-state}
                        additional help
    autohold            hold nodes for failed job
    autohold-delete     delete autohold request
    autohold-info       retrieve autohold request detailed info
    autohold-list       list autohold requests
    enqueue             enqueue a change
    enqueue-ref         enqueue a ref
    dequeue             dequeue a buildset by its change or ref
    promote             promote one or more changes
    show                show current statuses
    tenant-conf-check   validate the tenant configuration
    create-auth-token   create an Authentication Token for the web API
    import-keys         import project keys to ZooKeeper
    export-keys         export project keys from ZooKeeper
    copy-keys           copy keys from one project to another
    delete-keys         delete project keys
    delete-state        delete ephemeral ZooKeeper state
    delete-pipeline-state
                        delete single pipeline ZooKeeper state

The following subcommands are supported:

Autohold

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul autohold [-h] --tenant TENANT --project PROJECT --job JOB
                     [--change CHANGE] [--ref REF] --reason REASON
                     [--count COUNT]
                     [--node-hold-expiration NODE_HOLD_EXPIRATION]

optional arguments:
  -h, --help            show this help message and exit
  --tenant TENANT       tenant name
  --project PROJECT     project name
  --job JOB             job name
  --change CHANGE       specific change to hold nodes for
  --ref REF             git ref to hold nodes for
  --reason REASON       reason for the hold
  --count COUNT         number of job runs (default: 1)
  --node-hold-expiration NODE_HOLD_EXPIRATION
                        how long in seconds should the node set be in HOLD
                        status (default: scheduler's default_hold_expiration
                        value)

Example:

zuul autohold --tenant openstack --project example_project --job example_job --reason "reason text" --count 1

Autohold Delete

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul autohold-delete [-h] REQUEST_ID

positional arguments:
  REQUEST_ID  the hold request ID

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul autohold-delete --id 0000000123

Autohold Info

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul autohold-info [-h] REQUEST_ID

positional arguments:
  REQUEST_ID  the hold request ID

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul autohold-info --id 0000000123

Autohold List

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul autohold-list [-h] --tenant TENANT

optional arguments:
  -h, --help       show this help message and exit
  --tenant TENANT  tenant name

Example:

zuul autohold-list --tenant openstack

Dequeue

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul dequeue [-h] --tenant TENANT --pipeline PIPELINE --project PROJECT
                    [--change CHANGE] [--ref REF]

optional arguments:
  -h, --help           show this help message and exit
  --tenant TENANT      tenant name
  --pipeline PIPELINE  pipeline name
  --project PROJECT    project name
  --change CHANGE      change id
  --ref REF            ref name

Examples:

zuul dequeue --tenant openstack --pipeline check --project example_project --change 5,1
zuul dequeue --tenant openstack --pipeline periodic --project example_project --ref refs/heads/master

Enqueue

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul enqueue [-h] --tenant TENANT [--trigger TRIGGER] --pipeline
                    PIPELINE --project PROJECT --change CHANGE

optional arguments:
  -h, --help           show this help message and exit
  --tenant TENANT      tenant name
  --trigger TRIGGER    trigger name (deprecated and ignored. Kept only for
                       backward compatibility)
  --pipeline PIPELINE  pipeline name
  --project PROJECT    project name
  --change CHANGE      change id

Example:

zuul enqueue --tenant openstack --trigger gerrit --pipeline check --project example_project --change 12345,1

Note that the format of change id is <number>,<patchset>.

Enqueue-ref

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul enqueue-ref [-h] --tenant TENANT [--trigger TRIGGER] --pipeline
                        PIPELINE --project PROJECT --ref REF [--oldrev OLDREV]
                        [--newrev NEWREV]

Submit a trigger event

Directly enqueue a trigger event.  This is usually used
to manually "replay" a trigger received from an external
source such as gerrit.

optional arguments:
  -h, --help           show this help message and exit
  --tenant TENANT      tenant name
  --trigger TRIGGER    trigger name
  --pipeline PIPELINE  pipeline name
  --project PROJECT    project name
  --ref REF            ref name
  --oldrev OLDREV      old revision
  --newrev NEWREV      new revision

This command is provided to manually simulate a trigger from an external source. It can be useful for testing or replaying a trigger that is difficult or impossible to recreate at the source. The arguments to enqueue-ref will vary depending on the source and type of trigger. Some familiarity with the arguments emitted by gerrit update hooks such as patchset-created and ref-updated is recommended. Some examples of common operations are provided below.

Manual enqueue examples

It is common to have a release pipeline that listens for new tags coming from gerrit and performs a range of code packaging jobs. If there is an unexpected issue in the release jobs, the same tag can not be recreated in gerrit and the user must either tag a new release or request a manual re-triggering of the jobs. To re-trigger the jobs, pass the failed tag as the ref argument and set newrev to the change associated with the tag in the project repository (i.e. what you see from git show X.Y.Z):

zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline release --project openstack/example_project --ref refs/tags/X.Y.Z --newrev abc123..

The command can also be used asynchronosly trigger a job in a periodic pipeline that would usually be run at a specific time by the timer driver. For example, the following command would trigger the periodic jobs against the current master branch top-of-tree for a project:

zuul enqueue-ref --tenant openstack --trigger timer --pipeline periodic --project openstack/example_project --ref refs/heads/master

Another common pipeline is a post queue listening for gerrit merge results. Triggering here is slightly more complicated as you wish to recreate the full ref-updated event from gerrit. For a new commit on master, the gerrit ref-updated trigger expresses “reset refs/heads/master for the project from oldrev to newrev” (newrev being the committed change). Thus to replay the event, you could git log in the project and take the current HEAD and the prior change, then enqueue the event:

NEW_REF=$(git rev-parse HEAD)
OLD_REF=$(git rev-parse HEAD~1)

zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/example_project --ref refs/heads/master --newrev $NEW_REF --oldrev $OLD_REF

Note that zero values for oldrev and newrev can indicate branch creation and deletion; the source code is the best reference for these more advanced operations.

Promote

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul promote [-h] --tenant TENANT --pipeline PIPELINE --changes CHANGES
                    [CHANGES ...]

optional arguments:
  -h, --help            show this help message and exit
  --tenant TENANT       tenant name
  --pipeline PIPELINE   pipeline name
  --changes CHANGES [CHANGES ...]
                        change ids

Example:

zuul promote --tenant openstack --pipeline gate --changes 12345,1 13336,3

Note that the format of changes id is <number>,<patchset>.

The promote action is used to reorder the changes in a pipeline, by putting the provided changes at the top of the queue.

The most common use case for the promote action is the need to merge an urgent fix when the gate pipeline has several patches queued ahead. This is especially needed if there is concern that one or more changes ahead in the queue may fail, thus increasing the time to land for the fix; or concern that the fix may not pass validation if applied on top of the current patch queue in the gate.

Any items in a dependent pipeline which have had items ahead of them changed will have their jobs canceled and restarted based on the new ordering.

If items in independent pipelines are promoted, no jobs will be restarted, but their change queues within the pipeline will be re-ordered so that they will be processed first and their node request priorities will increase.

tenant-conf-check

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul tenant-conf-check [-h]

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul tenant-conf-check

This command validates the tenant configuration schema. It exits ‘-1’ in case of errors detected.

create-auth-token

Note

This command is only available if an authenticator is configured in zuul.conf. Furthermore the authenticator’s configuration must include a signing secret.

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul create-auth-token [-h] --auth-config AUTH_CONFIG --tenant TENANT
                              --user USER [--expires-in EXPIRES_IN]

Create an Authentication Token for the administration web API

Create a bearer token that can be used to access Zuul's
administration web API. This is typically used to delegate
privileged actions such as enqueueing and autoholding to
third parties, scoped to a single tenant.
At least one authenticator must be configured with a secret
that can be used to sign the token.

optional arguments:
  -h, --help            show this help message and exit
  --auth-config AUTH_CONFIG
                        The authenticator to use. Must match an authenticator
                        defined in zuul's configuration file.
  --tenant TENANT       tenant name
  --user USER           The user's name. Used for traceability in logs.
  --expires-in EXPIRES_IN
                        Token validity duration in seconds (default: 600)

Example:

zuul create-auth-token --auth-config zuul-operator --user alice --tenant tenantA --expires-in 1800

The return value is the value of the Authorization header the user must set when querying a protected endpoint on Zuul’s REST API.

Example:

bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vbWFuYWdlc2Yuc2ZyZG90ZXN0aW5zdGFuY2Uub3JnIiwienV1bC50ZW5hbnRzIjp7ImxvY2FsIjoiKiJ9LCJleHAiOjE1Mzc0MTcxOTguMzc3NTQ0fQ.DLbKx1J84wV4Vm7sv3zw9Bw9-WuIka7WkPQxGDAHz7s

export-keys

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul export-keys [-h] path

Export project secret keys from ZooKeeper

This command exports project secret keys from ZooKeeper
and writes them to a file which is suitable for backing
up and later use with the import-keys command.

The key contents are still protected by the keystore
password and can not be used or decrypted without it.

positional arguments:
  path        key export file path

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul export-keys /var/backup/zuul-keys.json

import-keys

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul import-keys [-h] [--force] path

Import previously exported project secret keys to ZooKeeper

Given a file with previously exported project keys, this
command will import them into ZooKeeper.  Existing keys
will not be overwritten; to overwrite keys, add the
--force flag.

positional arguments:
  path        key export file path

optional arguments:
  -h, --help  show this help message and exit
  --force     overwrite existing keys

Example:

zuul import-keys /var/backup/zuul-keys.json

copy-keys

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul copy-keys [-h]
                      src_connection src_project dest_connection dest_project

Copy secret keys from one project to another

When projects are renamed, this command may be used to
copy the secret keys from the current name to the new name
in order to avoid service interruption.

positional arguments:
  src_connection   original connection name
  src_project      original project name
  dest_connection  new connection name
  dest_project     new project name

optional arguments:
  -h, --help       show this help message and exit

Example:

zuul copy-keys gerrit old_project gerrit new_project

delete-keys

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul delete-keys [-h] connection project

Delete the ssh and secrets keys for a project

positional arguments:
  connection  connection name
  project     project name

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul delete-keys gerrit old_project

delete-state

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul delete-state [-h]

Delete all ephemeral state stored in ZooKeeper

Zuul stores a considerable amount of ephemeral state
information in ZooKeeper.  Generally it should be able to
detect and correct any errors, but if the state becomes
corrupted and it is unable to recover, this command may be
used to delete all ephemeral data from ZooKeeper and start
anew.

Do not run this command while any Zuul component is
running (perform a complete shutdown first).

This command will only remove ephemeral Zuul data from
ZooKeeper; it will not remove private keys or Nodepool
data.

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul delete-state

delete-pipeline-state

/home/zuul/src/opendev.org/zuul/zuul/.tox/docs/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
usage: zuul delete-pipeline-state [-h] tenant pipeline

Delete a single pipeline state stored in ZooKeeper

In the unlikely event that a bug in Zuul or ZooKeeper data
corruption occurs in such a way that it only affects a
single pipeline, this command might be useful in
attempting to recover.

The circumstances under which this command will be able to
effect a recovery are very rare and even so it may not be
sufficient.  In general, if an error occurs it is better
to shut Zuul down and run "zuul delete-state".

This command will lock the specified tenant and
then completely delete the pipeline state.

positional arguments:
  tenant      tenant name
  pipeline    pipeline name

optional arguments:
  -h, --help  show this help message and exit

Example:

zuul delete-pipeline-state tenant pipeline