Ansible Integration

Zuul contains Ansible modules and plugins to control the execution of Ansible Job content. These break down into two basic categories.

Restricted Execution on Executors
Build Log Support

Restricted Execution

Zuul runs ansible-playbook on executors to run job content on nodes. While the intent is that content is run on the remote nodes, Ansible is a flexible system that allows delegating actions to localhost, and also reading and writing files. These actions can be desirable and necessary for actions such as fetching log files or build artifacts, but could also be used as a vector to attack the executor.

For that reason Zuul implements a set of Ansible action plugins and lookup plugins that override and intercept task execution during untrusted playbook execution to ensure local actions are not executed or that for operations that are desirable to allow locally that they only interact with files in the zuul work directory.

class zuul.ansible.base.action.normal.ActionModule(task, connection, play_context, loader, templar, shared_loader_obj)

Override the normal action plugin

ansible.plugins.normal.ActionModule is run for every module that does not have a more specific matching action plugin.

Our overridden version of it wraps the execution with checks to block undesired actions on localhost.

dispatch_handler(): Run per-action handler if one exists.

handle_file()

Allow file module on localhost if it doesn’t touch unsafe files.

The Ansible file module can be useful in jobs for manipulating logs and artifacts.

Block any access of files outside the zuul work dir.

handle_find()

Allow find module on localhost if it doesn’t traverse unsafe files.

The Ansible find module can be used from the executor to gather a list of files.

handle_k8s()

Allow k8s module on localhost if it doesn’t touch unsafe files.

The Ansible k8s module can be used from the executor to modify k8s resources. Several options refer to local paths; check that they are constrained to the work dir.

handle_known_hosts()

Allow known_hosts on localhost

The Ansible known_hosts module can be used to add SSH host keys of a remote system. When run from a executor it can be used with the add_host task to access remote servers. This is needed because ansible on the executor is configured to check host keys by default.

Block any access of files outside the zuul work dir.

handle_stat()

Allow stat module on localhost if it doesn’t touch unsafe files.

The Ansible stat module can be useful in jobs for manipulating logs and artifacts.

Block any access of files outside the zuul work dir.

handle_zuul_return(): Allow zuul_return module on localhost.

run(tmp=None, task_vars=None): Overridden primary method from the base class.

Build Log Support

Zuul provides realtime build log streaming to end users so that users can watch long-running jobs in progress. As jobs may be written that execute a shell script that could run for a long time, additional effort is expended to stream stdout and stderr of shell tasks as they happen rather than waiting for the command to finish.

Zuul contains a modified version of the Ansible command module that starts a log streaming daemon on the build node.

All jobs run with the zuul.ansible.base.callback.zuul_stream callback plugin enabled, which writes the build log to a file so that the zuul.lib.log_streamer.LogStreamer can provide the data on demand over the finger protocol. Finally, zuul.web.LogStreamHandler exposes that log stream over a websocket connection as part of zuul.web.ZuulWeb.

class zuul.ansible.base.callback.zuul_stream.CallbackModule: This is the Zuul streaming callback. It’s based on the default callback plugin, but streams results from shell commands.

class zuul.lib.log_streamer.LogStreamer(host, port, jobdir_root): Class implementing log streaming over the finger daemon port.

class zuul.web.LogStreamHandler(*args, **kw)

class zuul.web.ZuulWeb(listen_address, listen_port, gear_server, gear_port, connections, authenticators, zk_hosts, zk_timeout=10.0, zk_tls_cert=None, zk_tls_key=None, zk_tls_ca=None, ssl_key=None, ssl_cert=None, ssl_ca=None, static_cache_expiry=3600, info=None, static_path=None, command_socket=None)

In addition to real-time streaming, Zuul also installs another callback module, zuul.ansible.base.callback.zuul_json.CallbackModule that collects all of the information about a given run into a json file which is written to the work dir so that it can be published along with build logs. Since the streaming log is by necessity a single text stream, choices have to be made for readability about what data is shown and what is not shown. The json log file is intended to allow for a richer more interactive set of data to be displayed to the user.

class zuul.ansible.base.callback.zuul_json.CallbackModule(display=None)