Ansible Integration

Zuul contains Ansible modules and plugins to control the execution of Ansible Job content. These break down into two basic categories.

• Restricted Execution on Executors

• Build Log Support

Restricted Execution

Zuul runs ansible-playbook on executors to run job content on nodes. While the intent is that content is run on the remote nodes, Ansible is a flexible system that allows delegating actions to localhost, and also reading and writing files. These actions can be desirable and necessary for actions such as fetching log files or build artifacts, but could also be used as a vector to attack the executor.

For that reason Zuul implements a set of Ansible action plugins and lookup plugins that override and intercept task execution during untrusted playbook execution to ensure local actions are not executed or that for operations that are desirable to allow locally that they only interact with files in the zuul work directory.

class zuul.ansible.action.normal.ActionModule(task, connection, play_context, loader, templar, shared_loader_obj)

Override the normal action plugin

ansible.plugins.normal.ActionModule is run for every module that does not have a more specific matching action plugin.

Our overridden version of it wraps the execution with checks to block undesired actions on localhost.

dispatch_handler()

Run per-action handler if one exists.

handle_file()

Allow file module on localhost if it doesn’t touch unsafe files.

The Ansible file module can be useful in jobs for manipulating logs and artifacts.

Block any access of files outside the zuul work dir.

handle_known_hosts()

Allow known_hosts on localhost

The Ansible known_hosts module can be used to add SSH host keys of a remote system. When run from a executor it can be used with the add_host task to access remote servers. This is needed because ansible on the executor is configured to check host keys by default.

Block any access of files outside the zuul work dir.

handle_stat()

Allow stat module on localhost if it doesn’t touch unsafe files.

The Ansible stat module can be useful in jobs for manipulating logs and artifacts.

Block any access of files outside the zuul work dir.

handle_uri()

Allow uri module on localhost if it doesn’t touch unsafe files.

The Ansible uri module can be used from the executor to do things like pinging readthedocs.org that otherwise don’t need a node. However, it can also download content to a local file, or be used to read from file:/// urls.

Block any use of url schemes other than https, http and ftp. Further, block any local file interaction that falls outside of the zuul work dir.

handle_zuul_return()

Allow zuul_return module on localhost.

run(tmp=None, task_vars=None)

Overridden primary method from the base class.

Build Log Support

Zuul provides realtime build log streaming to end users so that users can watch long-running jobs in progress. As jobs may be written that execute a shell script that could run for a long time, additional effort is expended to stream stdout and stderr of shell tasks as they happen rather than waiting for the command to finish.

Zuul contains a modified version of the Ansible command module that starts a log streaming daemon on the build node.

All jobs run with the zuul.ansible.callback.zuul_stream callback plugin enabled, which writes the build log to a file so that the zuul.lib.log_streamer.LogStreamer can provide the data on demand over the finger protocol. Finally, zuul.web.LogStreamHandler exposes that log stream over a websocket connection as part of zuul.web.ZuulWeb.

class zuul.ansible.callback.zuul_stream.CallbackModule

This is the Zuul streaming callback. It’s based on the default callback plugin, but streams results from shell commands.

class zuul.lib.log_streamer.LogStreamer(host, port, jobdir_root)

Class implementing log streaming over the finger daemon port.

class zuul.web.LogStreamHandler(*args, **kw)

class zuul.web.ZuulWeb(listen_address, listen_port, gear_server, gear_port, ssl_key=None, ssl_cert=None, ssl_ca=None, static_cache_expiry=3600, connections=None, info=None, static_path=None, zk_hosts=None)

In addition to real-time streaming, Zuul also installs another callback module, zuul.ansible.callback.zuul_json.CallbackModule that collects all of the information about a given run into a json file which is written to the work dir so that it can be published along with build logs. Since the streaming log is by necessity a single text stream, choices have to be made for readability about what data is shown and what is not shown. The json log file is intended to allow for a richer more interactive set of data to be displayed to the user.

class zuul.ansible.callback.zuul_json.CallbackModule(display=None)