Openshift Pods Driver

Selecting the openshift pods driver adds the following options to the providers section of the configuration.

providers.[openshiftpods]
Type: list

The Openshift Pods driver is similar to the Openshift driver, but it only supports pod label. This enables using an unprivileged service account that doesn’t require the self-provisioner role.

Example:

providers:
  - name: cluster
    driver: openshiftpods
    context: unprivileged-context-name
    pools:
      - name: main
        labels:
          - name: openshift-pod
            image: docker.io/fedora:28
providers.[openshiftpods].context (required)

Name of the context configured in kube/config.

Before using the driver, Nodepool services need a kube/config file manually installed. Make sure the context is present in oc config get-contexts command output.

providers.[openshiftpods].launch-retries
Default: 3

The number of times to retry launching a pod before considering the job failed.

providers.[openshiftpods].max-pods
Default: infinite
Type: int

An alias for max-servers.

providers.[openshiftpods].max-cores
Default: unlimited
Type: int

Maximum number of cores usable from this provider’s pools by default. This can be used to limit usage of the openshift backend. If not defined nodepool can use all cores up to the limit of the backend.

providers.[openshiftpods].max-servers
Default: unlimited
Type: int

Maximum number of pods spawnable from this provider’s pools by default. This can be used to limit the number of pods. If not defined nodepool can create as many servers the openshift backend allows.

providers.[openshiftpods].max-ram
Default: unlimited
Type: int

Maximum ram usable from this provider’s pools by default. This can be used to limit the amount of ram allocated by nodepool. If not defined nodepool can use as much ram as the openshift backend allows.

providers.[openshiftpods].max-resources
Default: unlimited
Type: dict

A dictionary of other quota resource limits applicable to this provider’s pools by default. Arbitrary limits may be supplied with the providers.[openshiftpods].pools.labels.extra-resources attribute.

providers.[openshiftpods].pools
Type: list

A pool defines a group of resources from an Openshift provider.

providers.[openshiftpods].pools.name (required)

The project’s (namespace) name that will be used to create the pods.

providers.[openshiftpods].pools.priority
Default: 100
Type: int

The priority of this provider pool (a lesser number is a higher priority). Nodepool launchers will yield requests to other provider pools with a higher priority as long as they are not paused. This means that in general, higher priority pools will reach quota first before lower priority pools begin to be used.

This setting may be specified at the provider level in order to apply to all pools within that provider, or it can be overridden here for a specific pool.

providers.[openshiftpods].pools.node-attributes
Type: dict

A dictionary of key-value pairs that will be stored with the node data in ZooKeeper. The keys and values can be any arbitrary string.

providers.[openshiftpods].pools.max-cores
Type: int

Maximum number of cores usable from this pool. This can be used to limit usage of the kubernetes backend. If not defined nodepool can use all cores up to the limit of the backend.

providers.[openshiftpods].pools.max-servers
Type: int

Maximum number of pods spawnable from this pool. This can be used to limit the number of pods. If not defined nodepool can create as many servers the kubernetes backend allows.

providers.[openshiftpods].pools.max-ram
Type: int

Maximum ram usable from this pool. This can be used to limit the amount of ram allocated by nodepool. If not defined nodepool can use as much ram as the kubernetes backend allows.

providers.[openshiftpods].pools.max-resources
Default: unlimited
Type: dict

A dictionary of other quota resource limits applicable to this pool. Arbitrary limits may be supplied with the providers.[openshiftpods].pools.labels.extra-resources attribute.

providers.[openshiftpods].pools.default-label-cpu
Type: int

Specifies specifies a default value for providers.[openshiftpods].pools.labels.cpu for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.default-label-memory
Type: int

Specifies a default value in MiB for providers.[openshiftpods].pools.labels.memory for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.default-label-storage
Type: int

Specifies a default value in MB for providers.[openshiftpods].pools.labels.storage for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.default-label-cpu-limit
Type: int

Specifies specifies a default value for providers.[openshiftpods].pools.labels.cpu-limit for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.default-label-memory-limit
Type: int

Specifies a default value in MiB for providers.[openshiftpods].pools.labels.memory-limit for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.default-label-storage-limit
Type: int

Specifies a default value in MB for providers.[openshiftpods].pools.labels.storage-limit for all labels of this pool that do not set their own value.

providers.[openshiftpods].pools.labels
Type: list

Each entry in a pool`s labels section indicates that the corresponding label is available for use in this pool.

Each entry is a dictionary with the following keys

providers.[openshiftpods].pools.labels.name (required)

Identifier for this label; references an entry in the labels section.

providers.[openshiftpods].pools.labels.image

The image name.

providers.[openshiftpods].pools.labels.image-pull
Default: IfNotPresent
Type: str

The ImagePullPolicy, can be IfNotPresent, Always or Never.

providers.[openshiftpods].pools.labels.image-pull-secrets
Default: []
Type: list

The imagePullSecrets needed to pull container images from a private registry.

Example:

labels:
  - name: openshift-pod
    type: pod
    image: docker.io/fedora:28
    image-pull-secrets:
      - name: registry-secret
providers.[openshiftpods].pools.labels.labels
Type: dict

A dictionary of additional values to be added to the namespace or pod metadata. The value of this field is added to the metadata.labels field in OpenShift. Note that this field contains arbitrary key/value pairs and is unrelated to the concept of labels in Nodepool.

providers.[openshiftpods].pools.labels.dynamic-labels
Default: None
Type: dict

Similar to providers.[openshiftpods].pools.labels.labels, but is interpreted as a format string with the following values available:

  • request: Information about the request which prompted the creation of this node (note that the node may ultimately be used for a different request and in that case this information will not be updated).

    • id: The request ID.

    • labels: The list of labels in the request.

    • requestor: The name of the requestor.

    • requestor_data: Key/value information from the requestor.

    • relative_priority: The relative priority of the request.

    • event_id: The external event ID of the request.

    • created_time: The creation time of the request.

    • tenant_name: The name of the tenant associated with the request.

For example:

labels:
  - name: pod-fedora
    dynamic-labels:
      request_info: "{request.id}"
providers.[openshiftpods].pools.labels.annotations
Type: dict

A dictionary of additional values to be added to the pod metadata. The value of this field is added to the metadata.annotations field in OpenShift. This field contains arbitrary key/value pairs that can be accessed by tools and libraries. E.g custom schedulers can make use of this metadata.

providers.[openshiftpods].pools.labels.cpu
Type: int

Specifies the number of cpu to request for the pod. If no limit is specified, this will also be used as the limit.

providers.[openshiftpods].pools.labels.memory
Type: int

Specifies the amount of memory in MiB to request for the pod. If no limit is specified, this will also be used as the limit.

providers.[openshiftpods].pools.labels.storage
Type: int

Specifies the amount of ephemeral-storage in MB to request for the pod. If no limit is specified, this will also be used as the limit.

providers.[openshiftpods].pools.labels.extra-resources
Type: dict

Specifies any extra resources that Nodepool should consider in its quota calculation other than the resources described above (cpu, memory, storage).

providers.[openshiftpods].pools.labels.cpu-limit
Type: int

Specifies the cpu limit for the pod.

providers.[openshiftpods].pools.labels.memory-limit
Type: int

Specifies the memory limit in MiB for the pod.

providers.[openshiftpods].pools.labels.storage-limit
Type: int

Specifies the ephemeral-storage limit in MB for the pod.

providers.[openshiftpods].pools.labels.gpu
Type: float

Specifies the amount of gpu allocated to the pod. This will be used to set both requests and limits to the same value, based on how kubernetes assigns gpu resources: https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/.

providers.[openshiftpods].pools.labels.gpu-resource
Type: str

Specifies the custom schedulable resource associated with the installed gpu that is available in the cluster.

providers.[openshiftpods].pools.labels.python-path
providers.[openshiftpods].pools.labels.:type: str
providers.[openshiftpods].pools.labels.:default: auto

The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[openshiftpods].pools.labels.shell-type
Default: sh
Type: str

The shell type of the node’s default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

  • For a windows pod with the experimental connection-type ssh, in which case cmd or powershell should be set and reflect the node’s DefaultShell configuration.

  • If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become

providers.[openshiftpods].pools.labels.env
Default: []
Type: list

A list of environment variables to pass to the Pod.

providers.[openshiftpods].pools.labels.env.name (required)
Type: str

The name of the environment variable passed to the Pod.

providers.[openshiftpods].pools.labels.env.value (required)
Type: str

The value of the environment variable passed to the Pod.

providers.[openshiftpods].pools.labels.node-selector
Type: dict

A map of key-value pairs to ensure the OpenShift scheduler places the Pod on a node with specific node labels.

providers.[openshiftpods].pools.labels.scheduler-name
Type: str

Sets the schedulerName field on the container. Normally left unset for the OpenShift default.

providers.[openshiftpods].pools.labels.privileged
Type: bool

Sets the securityContext.privileged flag on the container. Normally left unset for the OpenShift default.

providers.[openshiftpods].pools.labels.volumes
Type: list

Sets the volumes field on the pod. If supplied, this should be a list of OpenShift Pod Volume definitions.

providers.[openshiftpods].pools.labels.volume-mounts
Type: list

Sets the volumeMounts flag on the container. If supplied, this should be a list of OpenShift Container VolumeMount definitions.

providers.[openshiftpods].pools.labels.spec
Type: dict

This attribute is exclusive with all other label attributes except providers.[openshiftpods].pools.labels.name providers.[openshiftpods].pools.labels.annotations, providers.[openshiftpods].pools.labels.labels and providers.[openshiftpods].pools.labels.dynamic-labels. If a spec is provided, then Nodepool will supply the contents of this value verbatim to OpenShift as the spec attribute of the OpenShift Pod definition. No other Nodepool attributes are used, including any default values set at the provider level (such as default-label-cpu and similar).

This attribute allows for the creation of arbitrary complex pod definitions but the user is responsible for ensuring that they are suitable. The first container in the pod is expected to be a long-running container that hosts a shell environment for running commands. The following minimal definition matches what Nodepool itself normally creates and is recommended as a starting point:

labels:
  - name: custom-pod
    spec:
      containers:
        - name: custom-pod
          image: ubuntu:jammy
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh", "-c"]
          args: ["while true; do sleep 30; done;"]