Python Roles

build-python-release

Build sdist and wheel for Python projects.

Role Variables

release_python
Default: python

The python interpreter to use. Set it to “python3” to use python 3, for example.

build_wheel
Default: true

Whether to build a wheel. Set to false to just build an sdist tarball.

bdist_wheel_xargs
Default: ''

Extra arguments to pass to the bdist_wheel command when building packages.

build-releasenotes

Build releasenotes for a project, optionally incorporating translations.

Role Variables

zuul_work_virtualenv
Default: ~/.venv

Virtualenv location in which to install things.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to build releasenotes in.

ensure-babel

Ensure babel is installed

Role Variables

constraints_file

Optional path to a pip constraints file for installing python libraries.

ensure-if-python

Install the contents of a directory if they contain a python project.

Installs into a virtualenv.

Role Variables

install_package
Default: true

Flag indicating whether or not the software in the zuul_work_dir should be installed.

error_on_failure

Flag that indicates installation errors should result in failure. Failures in installing the target directory are ignored by default.

constraints_file

Optional path to a pip constraints file to use when installing.

zuul_work_virtualenv
Default: ~/.venv

Virtualenv location in which to install things.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to operate in.

ensure-pip

Ensure pip is available

This role is intended install the requirements for the pip module on hosts.

Jobs that also wish to call pip via shell commands directly can also use this to ensure pip is available. However, it should be noted that calling pip is ambiguous when supporting many platforms. On some platforms it may install the package under the Python 2 interpreter and in others Python 3. You should use a qualified name (pip2 or pip3) to avoid confusion.

This role will also install wheel components sufficient to run bdist_wheel builds or pip wheel on a source tree.

Role Variables

ensure_pip_from_packages
Default: True

Install the distribution packages for Python 3 pip, setuptools and wheel on the system.

ensure_pip_from_packages_with_python2
Default: False

Install the distribution packages for Python 2 pip, setuptools and wheel packages. This defaults to True when Ansible is running under Python 2.

CentOS 7 requies the epel repository if this flag is enabled because python-pip packages come from there.

This may not be valid on distributions that have removed Python 2 support.

ensure_pip_from_upstream
Default: False

Install pip from latest upstream sources locally. Note this is probably not what you want and should be used with extreme caution. The installed pip does not coordinate with the system packaged versions, and can lead to wide variety of problems if CI jobs re-install pip packages, for example.

ensure_pip_from_upstream_interpreters
Default: [ ansible_python_interpreter ]

A list of interpreters to install pip from upstream with. Note that by default the last entry in the list will likely own the /usr/local/bin/pip command; this can create confusion for legacy jobs if they assume pip installs Python 2 libraries but it is actually installing into the Python 3 environment. This role does not install the Python 2 interpreter, which may not be available on the system, so caution should be used when specifying python2 in this list.

Output Variables

ensure_pip_virtualenv_cmd

This variable will be set to a command appropriate for general usage with the pip module virtualenv_command argument on the host. On Python 3 hosts this will be the inbuilt venv module, on Python 2 hosts the virtualenv package will be installed (this is avoided on Python 3 hosts as an unnecessary dependency).

ensure-python

Ensure specified python interpreter and development files are installed

Note

This role is only available for Debian based platforms currently.

There are three ways to install the python interpreter:

1. Using distribution packages: This is the default (python_use_pyenv and python_use_stow are both false``).

2. Install using pyenv.

3. Install using stow.

Note

You cannot use both pyenv and stow method for the same job. That means that python_use_pyenv and python_use_stow cannot be set both to True at the same time.

Role Variables

python_version

Optional version of python interpreter to install, such as 3.7.

python_use_pyenv
Default: False

Whether to optionally use pyenv to install python instead of distro packages.

python_use_stow
Default: False

In case you have image with already prepared python versions, for example used the python-stow-versions element, you can activate them with stow utility by setting this variable to true.

python_stow_dir
Default: /usr/local/stow

Sets the target directory for stow. This should be the path to the directory where prepared python packages are located.

ensure-sphinx

Ensure sphinx is installed

Installs sphinx. Also installs any dependencies needed in the first of doc/requirements.txt, releasenotes/requirements.txt and test-requirements.txt to be found.

All pip installs are done with a provided constraints file, if given.

Role Variables

constraints_file

Optional path to a pip constraints file for installing python libraries.

doc_building_packages
Default: []

List of python packages to install for building docs. The default package list is based on the python version in use.

doc_building_extra_packages
Default: []

List of python additional packages to install for building docs. By default this list is empty.

sphinx_python
Default: python2

Version of python to use, either python2 or python3.

zuul_work_virtualenv
Default: ~/.venv

Virtualenv location in which to install things.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to operate in.

ensure-tox

Ensure tox is installed

Look for tox, and if not found, install it via pip into a virtual environment for the current user.

Role Variables

tox_prefer_python2
Default: False

If tox is not detected, prefer to install tox inside Python 2 instead of Python 3.

If set, ensure-pip.ensure_pip_from_packages_with_python2 will be automatically set to True to enable a Python 2 installation of pip.

ensure_global_symlinks
Default: False

Install a symlink to the tox executable into /usr/local/bin/tox. This can be useful when scripts need to be run that expect to find tox in a more standard location and plumbing through the value of tox_executable would be onerous.

Setting this requires root access, so should only be done in circumstances where root access is available.

Output Variables

tox_executable
Default: tox

After running this role, tox_executable will be set as the path to a valid tox.

At role runtime, look for an existing tox at this specific path. Note the default (tox) effectively means to find tox in the current $PATH. For example, if your base image pre-installs tox in an out-of-path environment, set this so the role does not attempt to install the user version.

ensure-twine

Ensure twine is installed.

Role Variables

twine_python
Default: python

The python interpreter to use to install twine if it is not already installed. Set it to “python3” to use python 3, for example.

Output Variables

twine_excutable
Default: twine

After running this role, twine_executable will be set as the path to a valid twine.

At role runtime, look for an existing twine at this specific path. Note the default (twine) effectively means to find tox in the current $PATH. For example, if your base image pre-installs twine in an out-of-path environment, set this so the role does not attempt to install the user version.

ensure-virtualenv

Ensure virtualenv is available

This role installs the requirements for the virtualenv command on the current distribution.

Users should be aware of some portability issues when using virtualenv:

• Distributions differ on the interpreter that virtualenv is provided by, so by calling virtualenv with no other arguments means that on some platforms you will get a Python 2 environment and others a Python 3 environment.

• If you wish to call virtualenv as a module (e.g. python -m virtualenv) you will need to know which interpreter owns the virtualenv package for your distribution; e.g. on some, such as Bionic, virtualenv is provided by python3-virtualenv but python refers to Python 2, so python -m virtualenv is not a portable way to call virtualenv.

• virtualenv -p python3 is likely the most portable way to consistently get a Python 3 environment. virtualenv -p python2 may not work on some platforms without Python 2.

• If you use Python 3 and do not require the specific features of virtualenv, it is likely easier to use Python’s inbuilt python3 -m venv module to create an isolated environment. If you are using pip: in your Ansible roles and require an environment, see the documentation for ensure-pip.

fetch-coverage-output

Collect output from a coverage run

By default, this copies the output from a coverage run on the worker to the log root of the executor.

Role Variables

zuul_executor_dest
Default: {{ zuul.executor.log_root }}

The destination directory on the executor. By default, the log root.

coverage_output_src
Default: {{ zuul.project.src_dir }}/cover/

The location on the worker from which to fetch the coverage output detail. By default, the cover dir of the current project.

fetch-python-sdist-output

Collect output from a python sdist build

fetch-sphinx-output

Collect output from a sphinx build

By default, this copies the output from the sphinx build on the worker to the log root of the executor.

Role Variables

zuul_executor_dest
Default: {{ zuul.executor.log_root }}

The destination directory on the executor. By default, the log root.

sphinx_build_dir
Default: doc/build

Directory relative to zuul_work_dir where build output will be put.

sphinx_output_suffix
Default: ''

Suffix to use for constructing the path. This is normally an empty string. If set to ‘/’ then, rsync will remove the last part from the original path.

sphinx_output_src
Default: {{ zuul_work_dir }}/{{ sphinx_build_dir }}/html{{ sphinx_output_suffix }}

The location on the worker from which to fetch the generated sphinx content. By default, the HTML doc build dir of the current project.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The location of the main working directory of the job.

zuul_use_fetch_output
Default: false

Whether to synchronize files to the executor work dir, or to only copy them on the test instance.

When set to False, the default, the role synchronizes the tarball archives and extracted documentation files to the executor log_root.

When set to True, the content is copied locally to {{  ansible_user_dir }}/zuul-output/logs/. The fetch-output role needs to be run to copy this output to the executor log_root.

fetch-sphinx-tarball

Collect output from a sphinx build as a tarball

By default, this copies the output from the sphinx build on the worker to the log root of the executor as a tarball, and then extracts the archive into the log root for viewing.

Role Variables

sphinx_build_dir
Default: doc/build

Directory relative to zuul_work_dir where build output should be found.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The location of the main working directory of the job.

sphinx_pdf_files
Default: list

A list of file names of PDF files to collect. By default, the list contains as entry only doc-{{ zuul.project.short_name }}.pdf.

zuul_use_fetch_output
Default: false

Whether to synchronize files to the executor work dir, or to only copy them on the test instance.

When set to False, the default, the role synchronizes the tarball archives and extracted documentation files to the executor log_root.

When set to True, the content is copied locally to {{  ansible_user_dir }}/zuul-output/logs/. The fetch-output role needs to be run to copy this output to the executor log_root.

fetch-subunit-output

Collect subunit outputs

Role Variables

zuul_work_dir
Default: {{ ansible_user_dir }}/{{ zuul.project.src_dir }}

Directory to work in. It has to be a fully qualified path.

fetch_subunit_output_additional_dirs
Default: []

List of additional directories which contains subunit files to collect. The content of zuul_work_dir is always checked, so it should not be added here.

zuul_use_fetch_output
Default: false

Whether to synchronize files to the executor work dir, or to copy them on the test instance. When set to false, the role synchronizes the file to the executor. When set to true, the job needs to use the fetch-output role later.

fetch-tox-output

Collect log output from a tox build

Role Variables

tox_envlist

Comma separated string with test environmens to fetch log output from. ALL fetches all environments while an empty string fetches all test environments configured with envlist in tox.

tox_executable
Default: tox

Location of the tox executable.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory tox was run in.

zuul_use_fetch_output
Default: false

Whether to synchronize files to the executor work dir, or to copy them on the test instance. When set to false, the role synchronizes the file to the executor. When set to true, the job needs to use the fetch-output role later.

find-constraints

Find a pip constraints file

Sets a variable upper_constraints which can be passed to a pip invocation.

Role Variables

constraints_file

Optional path to a pip constraints file for installing python libraries.

sphinx

Run sphinx to generate documentation

Role Variables

sphinx_source_dir
Default: doc/source

Directory relative to zuul_work_dir that contains the Sphinx sources.

sphinx_build_dir
Default: doc/build

Directory relative to zuul_work_dir where build output will be put.

sphinx_builders
Default: ['html']

Which sphinx builders to run.

sphinx_warning_is_error

Whether to treat sphinx build warnings as errors. Defaults to the value of [build_sphinx] warning-is-error in setup.cfg if defined, False if the [build_sphinx] section is present but the warning-is-error option is undefined, or True if the entire section is undefined.

zuul_work_virtualenv
Default: ~/.venv

Virtualenv that sphinx is installed in.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to operate in.

tox

Runs tox for a project

This role overrides Python packages installed into tox environments with corresponding Zuul sibling projects and runs tox tests as follows:

1. Create tox environments.

2. Get Python sibling package names for sibling projects created by Zuul (using required-projects job variable). Package names are searched in following sources:

   • setup.cfg of pbr projects,

   • setup.py,

   • tox_package_name role variable.

3. Remove sibling packages from tox environments.

4. Create temporary constraints file, lines for sibling packages are removed.

5. Install sibling packages from Zuul projects into tox environments with temporary constraints file.

6. Run tox tests.

Role Variables

tox_environment
Type: dict

Environment variables to pass in to the tox run.

tox_envlist

Comma separated string with test environments tox should run. ALL runs all test environments while an empty string runs all test environments configured with envlist in tox.

tox_executable
Default: tox

Location of the tox executable.

tox_config_file

Path to a tox configuration file, or directory containing a tox.ini file. Will be provided to tox via its -c command-line option if set.

tox_extra_args
Default: -vv

String of extra command line options to pass to tox.

tox_constraints_file

Path to a pip constraints file. Will be provided to tox via TOX_CONSTRAINTS_FILE (deprecated but currently still supported name is UPPER_CONSTRAINTS_FILE) environment variable if it exists.

tox_install_siblings
Default: true

Flag controlling whether to attempt to install python packages from any other source code repos zuul has checked out. Defaults to True.

tox_package_name

Allows a user to setup the package name to be used by tox, over reading a setup.cfg file in the project.

zuul_work_dir
Default: {{ zuul.project.src_dir }}

Directory to run tox in.

upload-pypi

Upload python packages to PyPI

Role Variables

pypi_info

Complex argument which contains the information about the PyPI server as well as the authentication information needed. It is expected that this argument comes from a Secret.

pypi_info.username

Username to use to log in to PyPI.

pypi_info.password

Password to use to log in to PyPI.

pypi_info.repository
Default: pypi

Name of the repository to upload to.

pypi_info.repository_url
Default: The built-in twine default for the production pypi.org service.

URL of the PyPI repostory.

pypi_path
Default: src/{{ zuul.project.canonical_name }}/dist

Path containing artifacts to upload.

pypi_twine_executable
Default: twine

Path to twine executable.

pypi_register_first
Default: false

Whether the role should register the package before uploading it. This may be required when uploading for the first time to a devPI instance.