AWS EC2 Driver

Selecting the aws driver adds the following options to the providers section of the configuration.

providers.[aws]
Type: list

An AWS provider’s resources are partitioned into groups called pool (see providers.[aws].pools for details), and within a pool, the node types which are to be made available are listed (see providers.[aws].pools.labels for details).

See Boto Configuration for information on how to configure credentials and other settings for AWS access in Nodepool’s runtime environment.

Note

For documentation purposes the option names are prefixed providers.[aws] to disambiguate from other drivers, but [aws] is not required in the configuration (e.g. below providers.[aws].pools refers to the pools key in the providers section when the aws driver is selected).

Example:

providers:
  - name: ec2-us-west-2
    driver: aws
    region-name: us-west-2
    cloud-images:
      - name: debian9
        image-id: ami-09c308526d9534717
        username: admin
    pools:
      - name: main
        max-servers: 5
        subnet-id: subnet-0123456789abcdef0
        security-group-id: sg-01234567890abcdef
        labels:
          - name: debian9
            cloud-image: debian9
            instance-type: t3.medium
            iam-instance-profile:
              arn: arn:aws:iam::123456789012:instance-profile/s3-read-only
            key-name: zuul
            tags:
              key1: value1
          - name: debian9-large
            cloud-image: debian9
            instance-type: t3.large
            key-name: zuul
            tags:
              key1: value1
              key2: value2
providers.[aws].name (required)

A unique name for this provider configuration.

providers.[aws].region-name (required)

Name of the AWS region to interact with.

providers.[aws].profile-name

The AWS credentials profile to load for this provider. If unspecified the boto3 library will select a profile.

See Boto Configuration for more information.

providers.[aws].boot-timeout
Default: 60
Type: int seconds

Once an instance is active, how long to try connecting to the image via SSH. If the timeout is exceeded, the node launch is aborted and the instance deleted.

providers.[aws].launch-retries
Default: 3

The number of times to retry launching a node before considering the job failed.

providers.[aws].cloud-images
Type: list

Each entry in this section must refer to an entry in the labels section.

cloud-images:
  - name: ubuntu1804
    image-id: ami-082fd9a18128c9e8c
    username: ubuntu
  - name: ubuntu1804-by-filters
    image-filters:
      - name: name
        values:
         - named-ami
    username: ubuntu
  - name: my-custom-win2k3
    connection-type: winrm
    username: admin

Each entry is a dictionary with the following keys

providers.[aws].cloud-images.name (required)
Type: string

Identifier to refer this cloud-image from providers.[aws].pools.labels section. Since this name appears elsewhere in the nodepool configuration file, you may want to use your own descriptive name here and use image-id to specify the cloud image so that if the image id changes on the cloud, the impact to your Nodepool configuration will be minimal. However, if image-id is not provided, this is assumed to be the image id in the cloud.

providers.[aws].cloud-images.image-id
Type: str

If this is provided, it is used to select the image from the cloud provider by ID.

providers.[aws].cloud-images.image-filters
Type: list

If provided, this is used to select an AMI by filters. If the filters provided match more than one image, the most recent will be returned. image-filters are not valid if image-id is also specified.

Each entry is a dictionary with the following keys

providers.[aws].cloud-images.image-filters.name (required)
Type: str

The filter name. See Boto describe images for a list of valid filters.

providers.[aws].cloud-images.image-filters.values (required)
Type: list

A list of str values to filter on

providers.[aws].cloud-images.username
Type: str

The username that a consumer should use when connecting to the node.

providers.[aws].cloud-images.python-path
Default: auto
Type: str

The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[aws].cloud-images.connection-type
Type: str

The connection type that a consumer should use when connecting to the node. For most images this is not necessary. However when creating Windows images this could be ‘winrm’ to enable access via ansible.

providers.[aws].cloud-images.connection-port
Default: 22/ 5986
Type: int

The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

providers.[aws].cloud-images.shell-type
Default: sh
Type: str

The shell type of the node’s default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

  • For a windows image with the experimental connection-type ssh in which case cmd or powershell should be set and reflect the node’s DefaultShell configuration.

  • If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become

providers.[aws].pools
Type: list

A pool defines a group of resources from an AWS provider. Each pool has a maximum number of nodes which can be launched from it, along with a number of cloud-related attributes used when launching nodes.

providers.[aws].pools.name (required)

A unique name within the provider for this pool of resources.

providers.[aws].pools.node-attributes
Type: dict

A dictionary of key-value pairs that will be stored with the node data in ZooKeeper. The keys and values can be any arbitrary string.

providers.[aws].pools.subnet-id

If provided, specifies the subnet to assign to the primary network interface of nodes.

providers.[aws].pools.security-group-id

If provided, specifies the security group ID to assign to the primary network interface of nodes.

providers.[aws].pools.public-ip-address
Default: True
Type: bool

Specify if a public ip address shall be attached to nodes.

providers.[aws].pools.host-key-checking
Default: True
Type: bool

Specify custom behavior of validation of SSH host keys. When set to False, nodepool-launcher will not ssh-keyscan nodes after they are booted. This might be needed if nodepool-launcher and the nodes it launches are on different networks. The default value is True.

providers.[aws].pools.labels
Type: list

Each entry in a pool’s labels section indicates that the corresponding label is available for use in this pool. When creating nodes for a label, the flavor-related attributes in that label’s section will be used.

labels:
  - name: bionic
    instance-type: m5a.large

Each entry is a dictionary with the following keys

providers.[aws].pools.labels.name (required)
Type: str

Identifier to refer this label. Nodepool will use this to set the name of the instance unless the name is specified as a tag.

providers.[aws].pools.labels.cloud-image (required)
Type: str

Refers to the name of an externally managed image in the cloud that already exists on the provider. The value of cloud-image should match the name of a previously configured entry from the cloud-images section of the provider. See providers.[aws].cloud-images.

providers.[aws].pools.labels.ebs-optimized
Default: False
Type: bool

Indicates whether EBS optimization (additional, dedicated throughput between Amazon EC2 and Amazon EBS,) has been enabled for the instance.

providers.[aws].pools.labels.instance-type (required)
Type: str

Name of the flavor to use.

providers.[aws].pools.labels.iam-instance-profile
Type: dict

Used to attach an iam instance profile. Useful for giving access to services without needing any secrets.

providers.[aws].pools.labels.iam-instance-profile.name

Name of the instance profile. Mutually exclusive with providers.[aws].pools.labels.iam-instance-profile.arn

providers.[aws].pools.labels.iam-instance-profile.arn

ARN identifier of the profile. Mutually exclusive with providers.[aws].pools.labels.iam-instance-profile.name

providers.[aws].pools.labels.key-name (required)
Type: string

The name of a keypair that will be used when booting each server.

providers.[aws].pools.labels.volume-type
Type: string

If given, the root EBS volume type

providers.[aws].pools.labels.volume-size
Type: int

If given, the size of the root EBS volume, in GiB.

providers.[aws].pools.labels.userdata
Default: None
Type: str

A string of userdata for a node. Example usage is to install cloud-init package on image which will apply the userdata. Additional info about options in cloud-config: https://cloudinit.readthedocs.io/en/latest/topics/examples.html

providers.[aws].pools.labels.tags
Default: None
Type: dict

A dictionary of tags to add to the EC2 instances