Ansible Integration
Zuul contains Ansible modules and plugins to control the execution of Ansible Job content. These break down into two basic categories.
Restricted Execution on Executors
Build Log Support
Restricted Execution
Zuul runs ansible-playbook
on executors to run job content on nodes. While
the intent is that content is run on the remote nodes, Ansible is a flexible
system that allows delegating actions to localhost
, and also reading and
writing files. These actions can be desirable and necessary for actions such
as fetching log files or build artifacts, but could also be used as a vector
to attack the executor.
For that reason Zuul implements a set of Ansible action plugins and lookup plugins that override and intercept task execution during untrusted playbook execution to ensure local actions are not executed or that for operations that are desirable to allow locally that they only interact with files in the zuul work directory.
- class zuul.ansible.base.action.normal.ActionModule(task, connection, play_context, loader, templar, shared_loader_obj)
Override the normal action plugin
ansible.plugins.normal.ActionModule
is run for every module that does not have a more specific matching action plugin.Our overridden version of it wraps the execution with checks to block undesired actions on localhost.
- dispatch_handler()
Run per-action handler if one exists.
- handle_file()
Allow file module on localhost if it doesn’t touch unsafe files.
The Ansible file module can be useful in jobs for manipulating logs and artifacts.
Block any access of files outside the zuul work dir.
- handle_find()
Allow find module on localhost if it doesn’t traverse unsafe files.
The Ansible find module can be used from the executor to gather a list of files.
- handle_k8s()
Allow k8s module on localhost if it doesn’t touch unsafe files.
The Ansible k8s module can be used from the executor to modify k8s resources. Several options refer to local paths; check that they are constrained to the work dir.
- handle_known_hosts()
Allow known_hosts on localhost
The Ansible known_hosts module can be used to add SSH host keys of a remote system. When run from a executor it can be used with the add_host task to access remote servers. This is needed because ansible on the executor is configured to check host keys by default.
Block any access of files outside the zuul work dir.
- handle_stat()
Allow stat module on localhost if it doesn’t touch unsafe files.
The Ansible stat module can be useful in jobs for manipulating logs and artifacts.
Block any access of files outside the zuul work dir.
- handle_zuul_return()
Allow zuul_return module on localhost.
- run(tmp=None, task_vars=None)
Overridden primary method from the base class.
Build Log Support
Zuul provides realtime build log streaming to end users so that users can watch long-running jobs in progress. As jobs may be written that execute a shell script that could run for a long time, additional effort is expended to stream stdout and stderr of shell tasks as they happen rather than waiting for the command to finish.
Zuul contains a modified version of the Ansible command module that starts a log streaming daemon on the build node.
All jobs run with the zuul.ansible.base.callback.zuul_stream
callback
plugin enabled, which writes the build log to a file so that the
zuul.lib.log_streamer.LogStreamer
can provide the data on demand
over the finger protocol. Finally, zuul.web.LogStreamHandler
exposes that log stream over a websocket connection as part of
zuul.web.ZuulWeb
.
- class zuul.ansible.base.callback.zuul_stream.CallbackModule
This is the Zuul streaming callback. It’s based on the default callback plugin, but streams results from shell commands.
- class zuul.lib.log_streamer.LogStreamer(host, port, jobdir_root)
Class implementing log streaming over the finger daemon port.
- class zuul.web.LogStreamHandler(*args, **kw)
- class zuul.web.ZuulWeb(config, connections, authenticators, info=None)
In addition to real-time streaming, Zuul also installs another callback module,
zuul.ansible.base.callback.zuul_json.CallbackModule
that collects all
of the information about a given run into a json file which is written to the
work dir so that it can be published along with build logs. Since the streaming
log is by necessity a single text stream, choices have to be made for
readability about what data is shown and what is not shown. The json log file
is intended to allow for a richer more interactive set of data to be displayed
to the user.
- class zuul.ansible.base.callback.zuul_json.CallbackModule(display=None)