General Purpose Roles

add-authorized-keys

Install SSH public key(s) on all hosts

This role is intended to be run at the end of a failed job for which the build node set will be held with zuul’s autohold command.

It copies the public key(s) into the authorized_keys file of every host in the inventory, allowing privileged users to access the node set for debugging or post-mortem analysis.

Add this stanza at the end of your project’s base post playbook to activate this functionality:

- hosts: all
  roles:
    - role: add-authorized-keys
      public_keys:
        - public_key: ssh-rsa AAAAB... venkman@parapsy.columbia.edu
        - public_key: ssh-rsa AAAAB... spengler@parapsy.columbia.edu
      when: not zuul_success | bool

Caution

Including this role earlier in any playbook may allow the keys’ owners to tamper with the execution of the jobs. It is strongly advised against doing so.

Role Variables

ssh_public_keys

A list of keys to inject.

ssh_public_keys.public_key

A public key to inject into authorized_keys, or a URL to a public key.

add-build-sshkey

Generate and install a build-local SSH key on all hosts

This role is intended to be run on the Zuul Executor at the start of every job. It generates an SSH keypair and installs the public key in the authorized_keys file of every host in the inventory. It then removes the Zuul master key from this job’s SSH agent so that the original key used to log into all of the hosts is no longer accessible (any per-project keys, if present, remain available), then adds the newly generated private key.

Role Variables

zuul_temp_ssh_key

Where to put the newly-generated SSH private key.

zuul_build_sshkey_cleanup
Default: false

Remove previous build sshkey. Set it to true for single use static node. Do not set it to true for multi-slot static nodes as it removes the build key configured by other jobs.

add-gpgkey

Install a GPG private key onto a host.

Role Variables

gpg_key

Complex argument which contains the GPG private key. It is expected that this argument comes from a Secret.

gpg_key.private

The ascii-armored contents of the GPG private key.

add-sshkey

Add an ssh key to the host so that non-ansible ssh connections can be made.

Role Variables

ssh_key

Complex argument which contains the ssh key information. It is expected that this argument comes from a Secret.

ssh_key.ssh_known_hosts

String containing known host signature for the remote host.

ssh_key.ssh_private_key

Contents of the ssh private key to use.

ssh_key.fqdn

The FQDN of the remote host.

bindep

Installs distro packages using bindep tool

Looks for a bindep.txt in a project’s source directory, or failing that a other-requirements.txt. If one exists, run bindep on the file to produce a list of required distro packages that do not exist and then install the missing packages.

Role Variables

bindep_dir
Default: {{ zuul.project.src_dir }}

The directory to look for bindep files in.

bindep_profile
Default: test

A specific bindep profile to request.

bindep_file

Path to a specific bindep file to read from.

bindep_command

Path to the bindep command. Defaults to unset which will look for a system installed bindep. If bindep_command is not found, bindep will be installed into a temporary virtualenv.

bindep_fallback

Path to a bindep fallback file to be used if no bindep file can be found in bindep.bindep_dir.

buildset-artifacts-location

Return the location of buildset logs

When a ‘buildset’ directory exists in the job logs, then use zuul_return to set buildset_artifacts_url for children jobs.

rpm-build job:

  • Create a repository

  • Fetch the repository to “{{ zuul.executor.log_root }}/buildset”

  • Use the buildset-artifacts-location role

rpm-test jobs:

  • Install “{{ buildset_artficats_url }}” yum repository

  • Run integration tests

layout:
  jobs:
    - rpm-build
    - rpm-test1:
        dependencies:
          - rpm-build
    - rpm-test2:
        dependencies:
          - rpm-build

Role Variables

zuul_log_url

Base URL where logs are to be found.

zuul_log_path

Path of the logs. This optional when the role is used after ‘upload-logs’.

configure-mirrors

An ansible role to configure services to use mirrors.

Role Variables

mirror_fqdn
Default: {{ zuul_site_mirror_fqdn }}

The base host for mirror servers.

pypi_mirror

URL to override the generated pypi mirror url based on configure-mirrors.mirror_fqdn.

set_apt_mirrors_trusted
Default: False

Set to True in order to tag APT mirrors as trusted, needed when accessing unsigned mirrors with newer releases like Ubuntu Bionic.

copy-build-sshkey

Copy a build-local SSH key to a defined user on all hosts

This role is intended to be run on the Zuul Executor. It copies a generated build specific ssh key to a user and adds it to the authorized_keys file of every host in the inventory.

Role Variables

zuul_temp_ssh_key
Default: "{{ zuul.executor.work_root }}/{{ zuul.build }}_id_rsa"

Where to source the build private key

copy_sshkey_target_user
Default: root

The user to copy the sshkey to.

download-artifact

Download artifacts from a completed build of a Zuul job

Given a change, downloads artifacts from a previous build (by default of the current change) into the work directory. This will download as many artifacts as match the selection criteria.

Role Variables

download_artifact_api

The Zuul API endpoint to use. Example: https://zuul.example.org/api/tenant/{{ zuul.tenant }}

download_artifact_pipeline

The pipeline in which the previous build ran.

download_artifact_job

The job of the previous build.

download_artifact_name

The artifact name. This can be a string or a list of strings.

download_artifact_query
Default: change={{ zuul.change }}&patchset={{ zuul.patchset }}&pipeline={{ download_artifact_pipeline }}&job_name={{ download_artifact_job }}

The query to use to find the build. Normally the default is used.

download_artifact_directory
Default: {{ zuul.executor.work_root }}

The directory in which to place the downloaded artifacts.

emit-job-header

Log a few lines about the job.

Role Variables

zuul_log_url

Base URL where logs are to be found.

git-prepare-nodecache

Prepare an archive containing all repositories that are part of the job. This can be used to prepare the repos archive suitable for caching in the node image to be used by prepare-workspace-git.

The path to the resulting archive file will be stored in the git_cache_file variable. That variable can be used to push the archive to a place where it will be picked up to be baked into the node image.

Role variables

git_cache_root
Default: {{ansible_user_dir }}/git-cache"

Directory where the git cache should be prepared. Usually this should not be changed.

log-inventory

Log the inventory used to run the job to the job’s log dir.

This will result in the log collection roles logging the job inventory.

Role Variables

zuul_info_dir
Default: {{ zuul.executor.log_root }}/zuul-info

The directory path to store the inventory file.

mirror-workspace-git-repos

Mirror the local git repos to remote nodes

This role uses git operations (unlike prepare-workspace which uses rsync) to mirror the local prepared git repos to the remote nodes. This may be useful if the remote node already has a copy of some or all of the git repos.

Role Variables

mirror_workspace_quiet
Default: false

If true git operations will be silenced and won’t print every changed reference.

multi-node-bridge

Configures a VXLAN virtual network overlay through an openvswitch network bridge between a ‘switch’ node and ‘peer’ nodes.

This allows members of the bridge to communicate with each other through the virtual network.

By default, this role will:

  • Install and start openvswitch

  • Set up a br-infra bridge on all nodes

  • Set up the connectivity between the switch and the peer with a virtual port

  • Set up an ip address on the bridge interface:

172.24.4.1/23 # switch node
172.41.4.2/23 # first peer
172.41.4.3/23 # second peer
...

Role requirements

This role requires and expects two groups to be set up in the Ansible host inventory in order to work:

  • switch (the node acting as the switch)

  • peers (nodes connected to the virtual switch ports)

Role variables

bridge_vni_offset
Default: 1000000

VXLAN Network Identifier offset (openvswitch key).

bridge_mtu
Default: Smallest mtu less 50 bytes for vxlan overhead

Bridge interface MTU. By default we determine this value by checking all interfaces on host, taking the smallest MTU and subtracting by 50 for vxlan overhead. Can be overridden explicitly if this does not work.

bridge_name
Default: br-infra

Name of the bridge interface.

bridge_configure_address
Default: true

Whether or not to configure an IP address on the bridge interface.

bridge_authorize_internal_traffic
Default: false

When bridge_configure_address is true, whether or not to set up firewall rules in order to allow traffic to flow freely within the bridge subnet (bridge_address_prefix.0/bridge_address_subnet).

bridge_address_prefix
Default: 172.24.4

The IP address range prefix.

bridge_address_offset
Default: 1

The IP address offset, used with bridge_address_prefix to provide the full IP address. The initial offset defines the IP address of the switch node in the virtual network.

bridge_address_subnet
Default: 23

The IP address range CIDR/subnet.

multi-node-firewall

Configures the inventory private and public addresses in a multi-node job in iptables in order to allow traffic to and from each node without restrictions.

multi-node-hosts-file

Configures the inventory hostnames in a multi-node job resolve to their respective private ipv4 addresses through the /etc/hosts file on each node.

multi-node-known-hosts

Configures the file /etc/ssh/ssh_known_hosts on each node in a multi-node job to contain all known host names.

persistent-firewall

Saves current iptables rules for both ipv4 and ipv6 and makes them persistent so that they are available if iptables or the instance is restarted.

This role can be re-used more than once in order to persist new rules.

prepare-workspace

Prepare remote workspaces

This role is intended to run before any other role in a Zuul job.

It starts the Zuul console streamer on every host in the inventory, and then copies the prepared source repos to the working directory on every host.

prepare-workspace-git

Mirror the local git repos to remote nodes

This role uses git operations (unlike prepare-workspace which uses rsync) to mirror the locally prepared git repos to the remote nodes while taking advantage of cached repos on the node if they exist. This role works generically regardless of the existence of a cached repo on the node.

The cached repos need to be placed using the canonical name under the cached_repos_root directory.

Role Variables

cached_repos_root
Default: /opt/git

The root of the cached repos.

remove-build-sshkey

Remove the per-build SSH key from all hosts

The complement to add-build-sshkey. It removes the build’s SSH key from the authorized_keys files of all remote hosts.

Role Variables

zuul_temp_ssh_key

Where the per-build SSH private key was stored.

remove-gpgkey

Remove an added GPG key from the host.

remove-sshkey

Remove an added ssh key from the host.

Role Variables

ssh_key

Complex argument which contains the ssh key information. It is expected that this argument comes from a Secret.

ssh_key.ssh_known_hosts

String containing known host signature for the remote host.

ssh_key.fqdn

The FQDN of the remote host.

revoke-sudo

Remove sudo access for the Zuul user.

If the file /etc/sudoers.d/zuul exists, then it will be removed. This is to facilitate systems which may use the same image for tests which require sudo and those which do not.

This role also asserts that sudo access has been removed and will fail if it has not.

sign-artifacts

Sign artifacts

Role Variables

gpg_key

Complex argument which contains the GPG private key for signing the artifacts. It is expected that this argument comes from a Secret.

gpg_key.private

The ascii-armored contents of the GPG private key.

gpg_artifact_path
Default: "{{ zuul.executor.work_root }}/artifacts/"

Path to a directory containing artifacts to sign.

stage-output

Stage job output on the remote node

Takes as input a dictionary of files/folders named ‘zuul_copy_output’. Copies contents into {{ zuul_output_dir }} on the remote node and is intended to be used before output fetching in a base job’s post-playbook.

Role Variables

zuul_copy_output
Default: None

Dictionary of files and folders to be staged.

The input is a dictionary so that it can accumulated via zuul variable merging. Keys of the dictionary will be things to copy. Valid values describe the type of output to copy:

  • logs

  • artifacts

  • docs

  • null # ansible null, not the string null

null overrides the will of a parent job to copy something instructing not to copy.

If the type is suffixed with _txt, then the item will have .txt appended to its name. For example:

.. code-block:: yaml
zuul_copy_output:

/var/log/syslog: logs_txt

Will copy /var/log/syslog to logs/syslog.txt.

stage_dir
Default: {{ ansible_user_dir }}

The stage directory on the remote node.

extensions_to_txt
Default: null

A dict of file extensions to be replaced with .txt when staging. This can be useful to ensure that text files with an extension not registered in the web server may be viewed via browser when uploaded to a file server.

Note that this is only used for files listed directly in zuul_copy_output and it won’t be applied to files contained in folders listed in zuul_copy_output.

Example:

extensions_to_txt:
  conf: True
  log: True
  txt: False

zuul.conf --(staged as)--> zuul_conf.txt
stage_compress_logs
Default: True

When True, files staged as logs will be compressed individually.

start-zuul-console

Start Zuul console logger

It starts the Zuul console streamer on every host in the inventory.

test-setup

Perform project test setup tasks.

This role assumes that Zuul has checked out a change for a project at src/{{ zuul.project.canonical_name }} and looks for a file named tools/test-setup.sh. If that file exists and is executable, it will be run.

This allows projects to specify test-setup steps (such as creating or initializing a database) in a form that can be easily run by both an automated testing system and developers.

Role Variables

test_setup_environment

Environment variables to pass in to the test-setup script.

test_setup_args

String of optional command line options passed to the test-setup script.

trigger-readthedocs

Trigger readthedocs build for a project

Role Variables

rtd_project_name
Default: ``{{ zuul.project.short_name }}``

The readthedocs project name

rtd_webhook_id

The readthedocs webhook API ID. This needs to be taken from the project’s “Integrations” dashboard page in RTD. The URL will look like readthedocs.org/api/v2/webhook/<project-name>/<id>/.

This may come from a secret, however it can not be triggered without authentication.

rtd_credentials

Complex argument which contains the RTD authentication credentials. This is expected to come from a secret.

rtd_credentials.integration_token

The webhook integration token. You’ll find this value on the project’s “Integrations” dashboard page in RTD. This can be used instead of username/password combo.

rtd_credentials.username

The readthedocs username. If set, this will be used to authenticate in preference to any token set via rtd_integration_token.

rtd_credentials.password

Password for username. Must be set if username is set.

upload-git-mirror

Mirrors a git repository to a remote git server

Meant to be used after a change was successfully merged, this role mirrors a tested git repository to a remote git server over SSH.

The role assumes that git has been previously installed and does not require superuser privileges to run.

Role Variables

git_mirror_credentials

Dictionary that provides the remote git repository credentials

git_mirror_credentials.user

SSH user for the remote git repository

git_mirror_credentials.host

SSH host for the remote git repository

git_mirror_credentials.ssh_key

Literal private key contents. Should start with something like -----BEGIN RSA PRIVATE KEY-----.

git_mirror_credentials.host_key

SSH host key of the remote git server. Can be obtained with ssh-keyscan -H <host>.

git_mirror_repository

Path of the remote git repository

validate-dco-license

Validate all commits have Signed-off-by header

Role Variables

dco_license_failure

Message to display when Signed-off-by header is missing.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to DCO license check in.

validate-host

Log information about the build node

Role Variables

zuul_site_traceroute_host

If defined, a host to run a traceroute against to verify build node network connectivity.

zuul_site_image_manifest_files
Default: ['/etc/dib-builddate.txt', '/etc/image-hostname.txt']

A list of files to read from the filesystem of the build node and whose contents will be logged. The default files are files written to nodes by diskimage-builder.

validate-zone-db

Validate bind zone.db files

This role uses named-checkzone to validate Bind zone.db files.

Role Variables

zone_files
Default: zuul.project.src_dir

Look for zone.db files recursively in this directory. The layout should be domain.xyz/zone.db where a parent directory is named for the zone the child zone.db file describes. This populates the zone_db_files variable. Will not be used if zone_db_files is explicitly set per below.

zone_db_files
Default: []

A list of zone.db files to check. Each entry is a list with the first element the domain, and the second element the path to the zone.db file. If this variable is set, automatic searching described by zone_files will not be performed.

version-from-git

Sets three facts based on information in a git repo.

scm_sha

The short sha found in the repository.

project_ver

A string describing the project’s version. It will either be the value of {{ zuul.tag }} or {{ scm_tag }}.{{ commits_since_tag }}.{{ scm_sha }} otherwise where scm_tag is either the most recent tag or the value of scm_sha if there are no commits in the repo.

commits_since_tag

Number of commits since the most recent tag.

Role Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to run git in.

write-inventory

Write an abbreviated version of the Zuul inventory to a file

This writes the minimal information about hosts from the current Zuul inventory to a file. It may be used to subsequently invoke Ansible with the inventory for the job.

Role Variables

write_inventory_dest

The path of the inventory file to write.

write_inventory_include_hostvars
Type: list

A list of facts about the host to include. By default this parameter is omitted and all variables about a host will be included. To only include certain variables, list them here. The empty list will cause no variables to be included.

write_inventory_exclude_hostvars
Type: list

A list of facts about the host to exclude. By default, all variables about a host will be included. To exclude certain variables, list them here.