Azure Compute Driver

Before using the Azure driver, make sure you have created a service principal and saved the credential information in a JSON file. Follow the instructions at Azure CLI and use the --sdk-auth flag:

az ad sp create-for-rbac --name nodepool --sdk-auth

You must also have created a network for Nodepool to use. Be sure to enable IPv6 on the network if you plan to use it.

The Azure driver now uses “Standard” SKU for all public IP addresses. Standard IP addresses block all incoming traffic by default, therefore the use of a Network Security Group is required in order to allow incoming traffic. You will need to create one, add any required rules, and attach it to the subnet created above.

You may also need to register the Microsoft.Compute resource provider with your subscription.

Selecting the azure driver adds the following options to the providers section of the configuration.

providers.[azure] Type: list

An Azure provider’s resources are partitioned into groups called pool, and within a pool, the node types which are to be made available are listed

Note

For documentation purposes the option names are prefixed providers.[azure] to disambiguate from other drivers, but [azure] is not required in the configuration (e.g. below providers.[azure].pools refers to the pools key in the providers section when the azure driver is selected).

Example:

providers:
  - name: azure-central-us
    driver: azure
    location: centralus
    resource-group: nodepool
    resource-group-location: centralus
    auth-path: /path/to/nodepoolCreds.json
    network: nodepool
    cloud-images:
      - name: bionic
        username: zuul
        key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAA...
        image-reference:
          sku: 18.04-LTS
          publisher: Canonical
          version: latest
          offer: UbuntuServer
    pools:
      - name: main
        max-servers: 10
        labels:
          - name: bionic
            cloud-image: bionic
            hardware-profile:
              vm-size: Standard_D1_v2

providers.[azure].name (required): A unique name for this provider configuration.

providers.[azure].location (required): Name of the Azure region to interact with.

providers.[azure].resource-group (required): Name of the Resource Group in which to place the Nodepool nodes.

providers.[azure].resource-group-location (required): Name of the Azure region where the home Resource Group is or should be created.

providers.[azure].auth-path (required): Path to the JSON file containing the service principal credentials. Create with the Azure CLI and the --sdk-auth flag

providers.[azure].network (required)

Network upon which to create VMs. This can either be a string, in which case it must be the name of a network in the provider’s resource group and Nodepool will use the subnet named default, or it can be a dictionary with these keys:

providers.[azure].network.resource-group Default: The provider's resource group: The resource group containing the network.

providers.[azure].network.network (required): The name of the network.

providers.[azure].network.subnet Default: default: The name of the subnet within the network.

providers.[azure].ipv4 Type: bool: Whether to enable IPv4 networking. Defaults to true unless ipv6 is enabled. Enabling this will attach a private IP address.

providers.[azure].ipv6 Default: false Type: bool: Whether to enable IPv6 networking. Enabling this will attach a private IP address.

providers.[azure].public-ipv4 Type: bool: Whether to attach a public IPv4 address to instances. Defaults to true, but will change to false in a future release. Implies ipv4.

providers.[azure].public-ipv6 Default: false Type: bool: Whether to attach a public IPv4 address to instances. Defaults to true, but will change to false in a future release. Implies ipv6.

providers.[azure].use-internal-ip Default: false Type: bool: If a public IP is attached but Nodepool should prefer the private IP, set this to true.

providers.[azure].host-key-checking Default: true Type: bool: Whether to validate SSH host keys. When true, this helps ensure that nodes are ready to receive SSH connections before they are supplied to the requestor. When set to false, nodepool-launcher will not attempt to ssh-keyscan nodes after they are booted. Disable this if nodepool-launcher and the nodes it launches are on different networks, where the launcher is unable to reach the nodes directly, or when using Nodepool with non-SSH node platforms. The default value is true.

providers.[azure].rate Default: 1.0 Type: float: The number of operations per second to perform against the provider.

providers.[azure].boot-timeout Default: 120 Type: int seconds: Once an instance is active, how long to try connecting to the image via SSH. If the timeout is exceeded, the node launch is aborted and the instance deleted.

providers.[azure].launch-timeout Default: 3600 Type: int seconds: The time to wait from issuing the command to create a new instance until that instance is reported as “active”. If the timeout is exceeded, the node launch is aborted and the instance deleted.

providers.[azure].launch-retries Default: 3 Type: int: The number of times to retry launching a server before considering the request failed.

providers.[azure].post-upload-hook Default: None Type: string

Filename of an optional script that can be called after an image has been uploaded to a provider but before it is taken into use. This is useful to perform last minute validation tests before an image is really used for build nodes. The script will be called as follows:

<SCRIPT> <PROVIDER> <EXTERNAL_IMAGE_ID> <LOCAL_IMAGE_FILENAME>

If the script returns with result code 0 it is treated as successful otherwise it is treated as failed and the image gets deleted.

providers.[azure].cloud-images Type: list

Each entry in this section must refer to an entry in the labels section.

cloud-images:
  - name: bionic
    username: zuul
    image-reference:
      sku: 18.04-LTS
      publisher: Canonical
      version: latest
      offer: UbuntuServer
  - name: windows-server-2016
    username: zuul
    image-reference:
       sku: 2016-Datacenter
       publisher: MicrosoftWindowsServer
       version: latest
       offer: WindowsServer

Each entry is a dictionary with the following keys

providers.[azure].cloud-images.name (required) Type: string: Identifier to refer this cloud-image from labels section. Since this name appears elsewhere in the nodepool configuration file, you may want to use your own descriptive name here.

providers.[azure].cloud-images.username (required) Type: str: The username that should be used when connecting to the node.

providers.[azure].cloud-images.password Type: str: If booting a Windows image, an administrative password is required. Either supply it here, or set providers.[azure].cloud-images.generate-password. Nodepool does not provide the password to requesting clients; to be used it must be provided in some other manner.

providers.[azure].cloud-images.generate-password Type: bool: If booting a Windows image, an administrative password is required. If the password is not actually used (e.g., the image has key-based authentication enabled), a random password can be provided by enabling this option. The password is not stored anywhere and is not retrievable.

providers.[azure].cloud-images.key Type: str: The SSH public key that should be installed on the node.

providers.[azure].cloud-images.connection-type Type: string: The connection type that a consumer should use when connecting to the node. For most diskimages this is not necessary. However when creating Windows images this could be winrm to enable access via ansible.

providers.[azure].cloud-images.connection-port Default: 22 / 5986 Type: int: The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

providers.[azure].cloud-images.python-path Default: auto Type: str: The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[azure].cloud-images.shell-type Type: str

The shell type of the node’s default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

For a windows image with the experimental connection-type ssh in which case cmd or powershell should be set and reflect the node’s DefaultShell configuration.
If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become

providers.[azure].cloud-images.image-filter Type: dict

Specifies a private image to use via filters. Either this field, providers.[azure].cloud-images.shared-gallery-image, providers.[azure].cloud-images.community-gallery-image, providers.[azure].cloud-images.image-reference, or providers.[azure].cloud-images.image-id must be provided.

If a filter is provided, Nodepool will list all of the images in the provider’s resource group and reduce the list using the supplied filter. All items specified in the filter must match in order for an image to match. If more than one image matches, the images are sorted by name and the last one matches.

Example:

cloud-images:
  - name: image-by-name
    image-filter:
      name: test-image
  - name: image-by-tag
    image-filter:
      tags:
        foo: bar

The following filters are available:

providers.[azure].cloud-images.image-filter.name Type: str: The name of the image.

providers.[azure].cloud-images.image-filter.location Type: str: The location of the image.

providers.[azure].cloud-images.image-filter.tags Type: dict: The image tags.

providers.[azure].cloud-images.image-id Type: str: Specifies a private image to use by ID. Either this field, providers.[azure].cloud-images.shared-gallery-image, providers.[azure].cloud-images.community-gallery-image, providers.[azure].cloud-images.image-reference, or providers.[azure].cloud-images.image-filter must be provided.

providers.[azure].cloud-images.shared-gallery-image Type: dict

Specifies a shared gallery image to use by ID. Either this field, providers.[azure].cloud-images.community-gallery-image, providers.[azure].cloud-images.image-reference, providers.[azure].cloud-images.image-id, or providers.[azure].cloud-images.image-filter must be provided.

providers.[azure].cloud-images.shared-gallery-image.gallery-name (required) Type: str: The name of the image gallery.

providers.[azure].cloud-images.shared-gallery-image.name (required) Type: str: The name of the image.

providers.[azure].cloud-images.shared-gallery-image.version Type: str: The image version. Omit to use the latest version.

providers.[azure].cloud-images.community-gallery-image Type: dict

Specifies a community gallery image to use by ID. Either this field, providers.[azure].cloud-images.shared-gallery-image, providers.[azure].cloud-images.image-reference, providers.[azure].cloud-images.image-id, or providers.[azure].cloud-images.image-filter must be provided.

providers.[azure].cloud-images.community-gallery-image.gallery-name (required) Type: str: The name of the image gallery.

providers.[azure].cloud-images.community-gallery-image.name (required) Type: str: The name of the image.

providers.[azure].cloud-images.community-gallery-image.version Type: str: The image version. Omit to use the latest version.

providers.[azure].cloud-images.image-reference Type: dict

Specifies a public image to use. Either this field, providers.[azure].cloud-images.shared-gallery-image, providers.[azure].cloud-images.community-gallery-image, providers.[azure].cloud-images.image-id, or providers.[azure].cloud-images.image-filter must be provided.

providers.[azure].cloud-images.image-reference.sku (required) Type: str: Image SKU

providers.[azure].cloud-images.image-reference.publisher (required) Type: str: Image Publisher

providers.[azure].cloud-images.image-reference.offer (required) Type: str: Image offers

providers.[azure].cloud-images.image-reference.version (required) Type: str: Image version

providers.[azure].diskimages Type: list

Each entry in a provider’s diskimages section must correspond to an entry in diskimages. Such an entry indicates that the corresponding diskimage should be uploaded for use in this provider. Additionally, any nodes that are created using the uploaded image will have the associated attributes (such as flavor or metadata).

If an image is removed from this section, any previously uploaded images will be deleted from the provider.

diskimages:
  - name: bionic
    pause: False
  - name: windows
    connection-type: winrm
    connection-port: 5986

Each entry is a dictionary with the following keys

providers.[azure].diskimages.name (required) Type: string: Identifier to refer this image from providers.[azure].pools.labels and diskimages sections.

providers.[azure].diskimages.pause Default: False Type: bool: When set to True, nodepool-builder will not upload the image to the provider.

providers.[azure].diskimages.username Type: str: The username that should be used when connecting to the node.

Warning

This option is deprecated. Specify the username on the diskimage definition itself instead.

providers.[azure].diskimages.key Type: str: The SSH public key that should be installed on the node.

providers.[azure].diskimages.connection-type Type: string: The connection type that a consumer should use when connecting to the node. For most diskimages this is not necessary. However when creating Windows images this could be winrm to enable access via ansible.

providers.[azure].diskimages.connection-port Default: 22 / 5986 Type: int: The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

providers.[azure].diskimages.python-path Default: auto Type: str: The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[azure].diskimages.tags Default: None Type: dict: A dictionary of tags to add to uploaded images. This will be merged with any existing metadata from the global diskimage configuration for this image. Avoid the use of nodepool_ as a key prefix since Nodepool uses this for internal values.

providers.[azure].pools Type: list

A pool defines a group of resources from an Azure provider. Each pool has a maximum number of nodes which can be launched from it, along with a number of cloud-related attributes used when launching nodes.

providers.[azure].pools.name (required): A unique name within the provider for this pool of resources.

providers.[azure].pools.priority Default: 100 Type: int

The priority of this provider pool (a lesser number is a higher priority). Nodepool launchers will yield requests to other provider pools with a higher priority as long as they are not paused. This means that in general, higher priority pools will reach quota first before lower priority pools begin to be used.

This setting may be specified at the provider level in order to apply to all pools within that provider, or it can be overridden here for a specific pool.

providers.[azure].pools.node-attributes Type: dict: A dictionary of key-value pairs that will be stored with the node data in ZooKeeper. The keys and values can be any arbitrary string.

providers.[azure].pools.ipv4 Type: bool: Whether to enable IPv4 networking. Defaults to true unless ipv6 is enabled. Enabling this will attach a private IP address.

providers.[azure].pools.ipv6 Default: false Type: bool: Whether to enable IPv6 networking. Enabling this will attach a private IP address.

providers.[azure].pools.public-ipv4 Type: bool: Whether to attach a public IPv4 address to instances. Defaults to true, but will change to false in a future release. Implies ipv4.

providers.[azure].pools.public-ipv6 Default: false Type: bool: Whether to attach a public IPv4 address to instances. Defaults to true, but will change to false in a future release. Implies ipv6.

providers.[azure].pools.use-internal-ip Default: false Type: bool: If a public IP is attached but Nodepool should prefer the private IP, set this to true.

providers.[azure].pools.host-key-checking Default: true Type: bool: Whether to validate SSH host keys. When true, this helps ensure that nodes are ready to receive SSH connections before they are supplied to the requestor. When set to false, nodepool-launcher will not attempt to ssh-keyscan nodes after they are booted. Disable this if nodepool-launcher and the nodes it launches are on different networks, where the launcher is unable to reach the nodes directly, or when using Nodepool with non-SSH node platforms. The default value is true.

providers.[azure].pools.labels Type: list

Each entry in a pool’s labels section indicates that the corresponding label is available for use in this pool. When creating nodes for a label, the flavor-related attributes in that label’s section will be used.

labels:
  - name: bionic
    cloud-image: bionic
    hardware-profile:
      vm-size: Standard_D1_v2

Each entry is a dictionary with the following keys:

providers.[azure].pools.labels.name (required) Type: str: Identifier for this label.

providers.[azure].pools.labels.cloud-image (required) Type: str: Refers to the name of an externally managed image in the cloud that already exists on the provider. The value of cloud-image should match the name of a previously configured entry from the cloud-images section of the provider.

providers.[azure].pools.labels.diskimage (required) Type: str: Refers to provider’s diskimages, see providers.[azure].diskimages. Mutually exclusive with providers.[azure].pools.labels.cloud-image

providers.[azure].pools.labels.hardware-profile (required)

providers.[azure].pools.labels.hardware-profile.vm-size (required) Type: str: VM Size of the VMs to use in Azure. See the VM size list on azure.microsoft.com for the list of sizes availabile in each region.

providers.[azure].pools.labels.tags Default: None Type: dict: A dictionary of tags to add to newly created VMs.

providers.[azure].pools.labels.dynamic-tags Default: None Type: dict

Similar to providers.[azure].pools.labels.tags, but is interpreted as a format string with the following values available:

request: Information about the request which prompted the creation of this node (note that the node may ultimately be used for a different request and in that case this information will not be updated).
- id: The request ID.
- labels: The list of labels in the request.
- requestor: The name of the requestor.
- requestor_data: Key/value information from the requestor.
- relative_priority: The relative priority of the request.
- event_id: The external event ID of the request.
- created_time: The creation time of the request.
- tenant_name: The name of the tenant associated with the request.

For example:

labels:
  - name: precise
    dynamic-tags:
      request_info: "Created for request {request.id}"

providers.[azure].pools.labels.user-data Default: None Type: str: The Azure User Data value for newly created VMs.

providers.[azure].pools.labels.custom-data Default: None Type: str: The Azure Custom Data value for newly created VMs.

providers.[azure].pools.labels.volume-size Type: int: If given, the size of the operating system disk, in GiB.