Python Roles¶
-
build-python-release
¶ Build sdist and wheel for Python projects.
Role Variables
-
release_python
¶
Default:python
The python interpreter to use. Set it to “python3” to use python 3, for example.
-
bdist_wheel_xargs
¶
Default:''
Extra arguments to pass to the bdist_wheel command when building packages.
-
-
build-releasenotes
¶ Build releasenotes for a project, optionally incorporating translations.
Role Variables
-
zuul_work_virtualenv
¶
Default:~/.venv
Virtualenv location in which to install things.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory to build releasenotes in.
-
-
ensure-babel
¶ Ensure babel is installed
Role Variables
-
constraints_file
¶ Optional path to a pip constraints file for installing python libraries.
-
-
ensure-if-python
¶ Install the contents of a directory if they contain a python project.
Installs into a virtualenv.
Role Variables
-
install_package
¶
Default:true
Flag indicating whether or not the software in the
zuul_work_dir
should be installed.
-
error_on_failure
¶ Flag that indicates installation errors should result in failure. Failures in installing the target directory are ignored by default.
-
constraints_file
¶ Optional path to a pip constraints file to use when installing.
-
zuul_work_virtualenv
¶
Default:~/.venv
Virtualenv location in which to install things.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory to operate in.
-
-
ensure-pip
¶ Ensure pip is available
This role is intended install the requirements for the pip module on hosts.
Jobs that also wish to call
pip
via shell commands directly can also use this to ensurepip
is available. However, it should be noted that callingpip
is ambiguous when supporting many platforms. On some platforms it may install the package under the Python 2 interpreter and in others Python 3. You should use a qualified name (pip2
orpip3
) to avoid confusion.This role will also install
wheel
components sufficient to runbdist_wheel
builds orpip wheel
on a source tree.Role Variables
-
ensure_pip_from_packages
¶
Default:True
Install the distribution packages for Python 3 pip, setuptools and wheel on the system.
-
ensure_pip_from_packages_with_python2
¶
Default:False
Install the distribution packages for Python 2 pip, setuptools and wheel packages. This defaults to
True
when Ansible is running under Python 2.CentOS 7 requies the
epel
repository if this flag is enabled becausepython-pip
packages come from there.This may not be valid on distributions that have removed Python 2 support.
-
ensure_pip_from_upstream
¶
Default:False
Install pip from latest upstream sources locally. Note this is probably not what you want and should be used with extreme caution. The installed pip does not coordinate with the system packaged versions, and can lead to wide variety of problems if CI jobs re-install
pip
packages, for example.
-
ensure_pip_from_upstream_interpreters
¶
Default:[ ansible_python_interpreter ]
A list of interpreters to install pip from upstream with. Note that by default the last entry in the list will likely own the
/usr/local/bin/pip
command; this can create confusion for legacy jobs if they assumepip
installs Python 2 libraries but it is actually installing into the Python 3 environment. This role does not install the Python 2 interpreter, which may not be available on the system, so caution should be used when specifyingpython2
in this list.
Output Variables
-
ensure_pip_virtualenv_cmd
¶ This variable will be set to a command appropriate for general usage with the
pip
modulevirtualenv_command
argument on the host. On Python 3 hosts this will be the inbuiltvenv
module, on Python 2 hosts thevirtualenv
package will be installed (this is avoided on Python 3 hosts as an unnecessary dependency).
-
-
ensure-python
¶ Ensure specified python interpreter and development files are installed
Note
This role is only available for Debian based platforms currently.
There are three ways to install the python interpreter:
Using distribution packages: This is the default (
python_use_pyenv
andpython_use_stow
are both false``).Install using
pyenv
.Install using
stow
.
Note
You cannot use both
pyenv
andstow
method for the same job. That means thatpython_use_pyenv
andpython_use_stow
cannot be set both toTrue
at the same time.Role Variables
-
python_version
¶ Optional version of python interpreter to install, such as
3.7
.
-
python_use_pyenv
¶
Default:False
Whether to optionally use
pyenv
to install python instead of distro packages.
-
python_use_stow
¶
Default:False
In case you have image with already prepared python versions, for example used the python-stow-versions element, you can activate them with stow utility by setting this variable to
true
.
-
python_stow_dir
¶
Default:/usr/local/stow
Sets the target directory for stow. This should be the path to the directory where prepared python packages are located.
-
ensure-sphinx
¶ Ensure sphinx is installed
Installs sphinx. Also installs any dependencies needed in the first of doc/requirements.txt, releasenotes/requirements.txt and test-requirements.txt to be found.
All pip installs are done with a provided constraints file, if given.
Role Variables
-
constraints_file
¶ Optional path to a pip constraints file for installing python libraries.
-
doc_building_packages
¶
Default:[]
List of python packages to install for building docs. The default package list is based on the python version in use.
-
doc_building_extra_packages
¶
Default:[]
List of python additional packages to install for building docs. By default this list is empty.
-
sphinx_python
¶
Default:python2
Version of python to use, either
python2
orpython3
.
-
zuul_work_virtualenv
¶
Default:~/.venv
Virtualenv location in which to install things.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory to operate in.
-
-
ensure-tox
¶ Ensure tox is installed
Look for
tox
, and if not found, install it viapip
into a virtual environment for the current user.Role Variables
-
tox_prefer_python2
¶
Default:False
If tox is not detected, prefer to install tox inside Python 2 instead of Python 3.
If set, ensure-pip.ensure_pip_from_packages_with_python2 will be automatically set to True to enable a Python 2 installation of pip.
-
ensure_global_symlinks
¶
Default:False
Install a symlink to the tox executable into
/usr/local/bin/tox
. This can be useful when scripts need to be run that expect to find tox in a more standard location and plumbing through the value oftox_executable
would be onerous.Setting this requires root access, so should only be done in circumstances where root access is available.
Output Variables
-
tox_executable
¶
Default:tox
After running this role,
tox_executable
will be set as the path to a validtox
.At role runtime, look for an existing
tox
at this specific path. Note the default (tox
) effectively means to find tox in the current$PATH
. For example, if your base image pre-installs tox in an out-of-path environment, set this so the role does not attempt to install the user version.
-
-
ensure-twine
¶ Ensure twine is installed.
Role Variables
-
twine_python
¶
Default:python
The python interpreter to use to install twine if it is not already installed. Set it to “python3” to use python 3, for example.
Output Variables
-
twine_excutable
¶
Default:twine
After running this role,
twine_executable
will be set as the path to a validtwine
.At role runtime, look for an existing
twine
at this specific path. Note the default (twine
) effectively means to find tox in the current$PATH
. For example, if your base image pre-installs twine in an out-of-path environment, set this so the role does not attempt to install the user version.
-
-
ensure-virtualenv
¶ Ensure virtualenv is available
This role installs the requirements for the
virtualenv
command on the current distribution.Users should be aware of some portability issues when using
virtualenv
:Distributions differ on the interpreter that
virtualenv
is provided by, so by callingvirtualenv
with no other arguments means that on some platforms you will get a Python 2 environment and others a Python 3 environment.If you wish to call
virtualenv
as a module (e.g.python -m virtualenv
) you will need to know which interpreter owns thevirtualenv
package for your distribution; e.g. on some, such as Bionic,virtualenv
is provided bypython3-virtualenv
butpython
refers to Python 2, sopython -m virtualenv
is not a portable way to callvirtualenv
.virtualenv -p python3
is likely the most portable way to consistently get a Python 3 environment.virtualenv -p python2
may not work on some platforms without Python 2.If you use Python 3 and do not require the specific features of
virtualenv
, it is likely easier to use Python’s inbuiltpython3 -m venv
module to create an isolated environment. If you are usingpip:
in your Ansible roles and require an environment, see the documentation for ensure-pip.
-
fetch-coverage-output
¶ Collect output from a coverage run
By default, this copies the output from a coverage run on the worker to the log root of the executor.
Role Variables
-
zuul_executor_dest
¶
Default:{{ zuul.executor.log_root }}
The destination directory on the executor. By default, the log root.
-
coverage_output_src
¶
Default:{{ zuul.project.src_dir }}/cover/
The location on the worker from which to fetch the coverage output detail. By default, the
cover
dir of the current project.
-
-
fetch-python-sdist-output
¶ Collect output from a python sdist build
-
fetch-sphinx-output
¶ Collect output from a sphinx build
By default, this copies the output from the sphinx build on the worker to the log root of the executor.
Role Variables
-
zuul_executor_dest
¶
Default:{{ zuul.executor.log_root }}
The destination directory on the executor. By default, the log root.
-
sphinx_build_dir
¶
Default:doc/build
Directory relative to zuul_work_dir where build output will be put.
-
sphinx_output_suffix
¶
Default:''
Suffix to use for constructing the path. This is normally an empty string. If set to ‘/’ then, rsync will remove the last part from the original path.
-
sphinx_output_src
¶
Default:{{ zuul_work_dir }}/{{ sphinx_build_dir }}/html{{ sphinx_output_suffix }}
The location on the worker from which to fetch the generated sphinx content. By default, the HTML doc build dir of the current project.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
The location of the main working directory of the job.
-
zuul_use_fetch_output
¶
Default:false
Whether to synchronize files to the executor work dir, or to only copy them on the test instance.
When set to
False
, the default, the role synchronizes the tarball archives and extracted documentation files to the executorlog_root
.When set to
True
, the content is copied locally to{{ ansible_user_dir }}/zuul-output/logs/
. Thefetch-output
role needs to be run to copy this output to the executorlog_root
.
-
-
fetch-sphinx-tarball
¶ Collect output from a sphinx build as a tarball
By default, this copies the output from the sphinx build on the worker to the log root of the executor as a tarball, and then extracts the archive into the log root for viewing.
Role Variables
-
sphinx_build_dir
¶
Default:doc/build
Directory relative to zuul_work_dir where build output should be found.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
The location of the main working directory of the job.
-
sphinx_pdf_files
¶
Default:list
A list of file names of PDF files to collect. By default, the list contains as entry only
doc-{{ zuul.project.short_name }}.pdf
.
-
zuul_use_fetch_output
¶
Default:false
Whether to synchronize files to the executor work dir, or to only copy them on the test instance.
When set to
False
, the default, the role synchronizes the tarball archives and extracted documentation files to the executorlog_root
.When set to
True
, the content is copied locally to{{ ansible_user_dir }}/zuul-output/logs/
. Thefetch-output
role needs to be run to copy this output to the executorlog_root
.
-
-
fetch-subunit-output
¶ Collect subunit outputs
Role Variables
-
zuul_work_dir
¶
Default:{{ ansible_user_dir }}/{{ zuul.project.src_dir }}
Directory to work in. It has to be a fully qualified path.
-
fetch_subunit_output_additional_dirs
¶
Default:[]
List of additional directories which contains subunit files to collect. The content of zuul_work_dir is always checked, so it should not be added here.
-
zuul_use_fetch_output
¶
Default:false
Whether to synchronize files to the executor work dir, or to copy them on the test instance. When set to false, the role synchronizes the file to the executor. When set to true, the job needs to use the fetch-output role later.
-
-
fetch-tox-output
¶ Collect log output from a tox build
Role Variables
-
tox_envlist
¶ Comma separated string with test environmens to fetch log output from.
ALL
fetches all environments while an empty string fetches all test environments configured withenvlist
in tox.
-
tox_executable
¶
Default:tox
Location of the tox executable.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory tox was run in.
-
zuul_use_fetch_output
¶
Default:false
Whether to synchronize files to the executor work dir, or to copy them on the test instance. When set to false, the role synchronizes the file to the executor. When set to true, the job needs to use the fetch-output role later.
-
-
find-constraints
¶ Find a pip constraints file
Sets a variable
upper_constraints
which can be passed to a pip invocation.Role Variables
-
constraints_file
¶ Optional path to a pip constraints file for installing python libraries.
-
-
sphinx
¶ Run sphinx to generate documentation
Role Variables
-
sphinx_source_dir
¶
Default:doc/source
Directory relative to zuul_work_dir that contains the Sphinx sources.
-
sphinx_build_dir
¶
Default:doc/build
Directory relative to zuul_work_dir where build output will be put.
-
sphinx_builders
¶
Default:['html']
Which sphinx builders to run.
-
sphinx_warning_is_error
¶ Whether to treat sphinx build warnings as errors. Defaults to the value of
[build_sphinx] warning-is-error
insetup.cfg
if defined,False
if the[build_sphinx]
section is present but thewarning-is-error
option is undefined, orTrue
if the entire section is undefined.
-
zuul_work_virtualenv
¶
Default:~/.venv
Virtualenv that sphinx is installed in.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory to operate in.
-
-
tox
¶ Runs tox for a project
This role overrides Python packages installed into tox environments with corresponding Zuul sibling projects and runs tox tests as follows:
Create tox environments.
Get Python sibling package names for sibling projects created by Zuul (using
required-projects
job variable). Package names are searched in following sources:setup.cfg
of pbr projects,setup.py
,tox_package_name
role variable.
Remove sibling packages from tox environments.
Create temporary constraints file, lines for sibling packages are removed.
Install sibling packages from Zuul projects into tox environments with temporary constraints file.
Run tox tests.
Role Variables
-
tox_environment
¶
Type: dict Environment variables to pass in to the tox run.
-
tox_envlist
¶ Comma separated string with test environments tox should run.
ALL
runs all test environments while an empty string runs all test environments configured withenvlist
in tox.
-
tox_executable
¶
Default:tox
Location of the tox executable.
-
tox_extra_args
¶
Default:-vv
String of extra command line options to pass to tox.
-
tox_constraints_file
¶ Path to a pip constraints file. Will be provided to tox via
TOX_CONSTRAINTS_FILE
(deprecated but currently still supported name isUPPER_CONSTRAINTS_FILE
) environment variable if it exists.
-
tox_install_siblings
¶
Default:true
Flag controlling whether to attempt to install python packages from any other source code repos zuul has checked out. Defaults to True.
-
tox_package_name
¶ Allows a user to setup the package name to be used by tox, over reading a setup.cfg file in the project.
-
zuul_work_dir
¶
Default:{{ zuul.project.src_dir }}
Directory to run tox in.
-
upload-pypi
¶ Upload python packages to PyPI
Role Variables
-
pypi_info
¶ Complex argument which contains the information about the PyPI server as well as the authentication information needed. It is expected that this argument comes from a Secret.
-
pypi_info.
username
¶ Username to use to log in to PyPI.
-
pypi_info.
password
¶ Password to use to log in to PyPI.
-
pypi_info.
repository
¶
Default:pypi
Name of the repository to upload to.
-
pypi_info.
repository_url
¶
Default:The built-in twine default for the production pypi.org service.
URL of the PyPI repostory.
-
-
pypi_path
¶
Default:src/{{ zuul.project.canonical_name }}/dist
Path containing artifacts to upload.
-
pypi_twine_executable
¶
Default:twine
Path to twine executable.
-
pypi_register_first
¶
Default:false
Whether the role should register the package before uploading it. This may be required when uploading for the first time to a devPI instance.
-