Ansible Dynamic Inventory

Static inventory will not serve the needs when tracking and deploying to multiple sources with hosts spinning up and shutting down in response to business demands. The solution to such a changing infrastructure would be ansible dynamic inventory. To accomplish this, you will need a cloud provider ini file (e.g aws.ini) and ec2.py (which ansible use to communicate to the AWS API to fetch the metadata for all the instances). This post focus on AWS(Amazon Web Service).

ec2.ini file provided by Ansible. All the cloud provider .ini can be found here https://github.com/ansible/ansible/tree/stable-2.9/contrib/inventory

ec2.py: EC2 external inventory script which dynamically fetches data from AWS API

To set up the ec2 external inventory script, copy the script to /etc/ansible/ec2.py and run chmod +x /etc/ansible/ec2.py. You will also need to copy the ec2.ini file to /etc/Ansible/ec2.ini. Then run ansible-playbook normally, the --limit args can be parsed.

To make a successful call to AWS API, configure boto. There are a variety of methods available, you can add setup your aws credential by setting up your ~/.aws/credentials or export the following environment variables:

export AWS_ACCESS_KEY_ID=’mYaWeSoMeAcCeSsKeY′
export AWS_SECRET_ACCESS_KEY=’IcAnToTaLlYpUtMySeCrEtHeRe′

Set up a few more environment variables to the inventory management script such as

$ export ANSIBLE_HOSTS=/PATH/ec2.py
$ export EC2_INI_PATH=/PATH/ec2.ini

This variable sets ansible-playbook to use the dynamic EC2 script instead of a static /etc/ansible/hosts file. Open up ec2.py in a text editor and make sure that the path to ec2.ini config file is defined correctly at the top of the script:

EC2_INI_PATH instructs ec2.py where the ec2.ini config file is located.

Make the script executable and test it by running
./ec2.py –list

$ ./ec2.py --list
{
  "_meta": {
    "hostvars": {
      "1.9O.1OO.12O": {
        "ansible_host": "1.9O.1OO.12O",
        "ec2__in_monitoring_element": false,
        "ec2_account_id": "19OP23I42OF3",
        "ec2_ami_launch_index": "0",
        ....
        "ec2_tag_App": "olonje",
        "ec2_tag_Env": "Staging",
        "ec2_tag_Name": "Bami Gbe",
        "ec2_virtualization_type": "hvm",
        "ec2_vpc_id": "vpc-IaMhIdDeN"
      },
      "4.OOO.122.1OO9": {
        "ansible_host": "4.OOO.122.1OO9",
        "ec2__in_monitoring_element": false,
        ...
        "ec2_key_name": "emi-moni-profile",
        "ec2_root_device_name": "/dev/xvda",
        "ec2_tag_App": "ounje2",
        "ec2_tag_Env": "prod",
        "ec2_tag_Name": "Production Stack",
        "ec2_virtualization_type": "hvm",
        "ec2_vpc_id": "vpc-IaMhIdDeN"
      }
    }
  },
  "ami_1k02nd83js84k6apO": [
    "1.9O.1OO.12O",
    "4.OOO.122.1OO9"
  ],
  "tag_App_olonje": [
    "1.9O.1OO.12O"
  ],
  "tag_App_order2": [
    "4.OOO.122.1OO9"
  ],
  "tag_Env_Staging": [
    "1.9O.1OO.12O"
  ],
  "tag_Env_prod": [
    "4.OOO.122.1OO9"
  ],
  "tag_Name_app_Worker": [
    "1.9O.1OO.12O"
  ],
  "tag_Name_Production_Stack": [
    "4.OOO.122.1OO9"
  ],
  "vpc_id_vpc_IaMhIdDeN": [
    "1.9O.1OO.12O",
    "4.OOO.122.1OO9"
  ]
}

To fine-tune the output and other features that aren’t applicable refer to the ec2.ini file. For multiple AWS accounts, you can pass --profile PROFILE name to the ec2.py script.

$ cat ~/.aws/credentials
[default]
aws_access_key_id = <default access key>
aws_secret_access_key = <default secret key>

[profile tito]
aws_access_key_id = <tito access key>
aws_secret_access_key = <tito secret key>

[profile ounje]
aws_access_key_id = <ounje access key>
aws_secret_access_key = <ounje secret key>

To get the inventory for the ounje account run AWS_PROFILE=ounje ansible-playbook -i /etc/ansible/ec2.py myplaybook.yml. Since the EC2 external inventory provides mappings to instances from several groups, we can filter the target instance based on resource tag. Speaking of Tags.

TESTING OUR DYNAMIC INVENTORY ABOVE

Running ansible-playbook deployment. Using sample.yml

- name: Deploy Ounje Artifact
  hosts: all
  vars:
    - app: "e-status "
    - env: "staging"
  tasks:
    - name: Get Instance IDs
      ec2_remote_facts:
        aws_access_key: "moYaCoverSecretMi"
        aws_secret_key: "amalaShitaLoSureJuMafo"
        region: "us-east-1"
        filters:
          "tag:Env": "{{ env }}"
      register: gbebodi

    - name: Store ounje hosts
      debug: var=gbebodi

    - name: Store Ounje EC2 instance ID
      set_fact:
         instance_id: "{{ item.id }}"
      with_items: "{{ gbebodi.instances }}"

    - name: Store EC2 instance ID in var
      debug: var=instance_id

Run command:

$ ansible-playbook -i /etc/ansible/ec2.py --limit "tag_App_order2" -u ec2-user sample.yml -e env=prod

In the above example:

ansible-playbook is the standard command to run an ansible playbook
/etc/ansible/ec2.py points ansible inventory to use the ec2.py to callout to aws API to get the ec2 instance metadata
--limit "tag_App_ounje2” This limit the host you want to target by instructing ansible-playbook to pull the metadata of all instances with a tag of key App and value ounje2. This enables the ansible-playbook to know where to run the tasks, know which instance you want to access and limit the activities to the only ounje2
sample.yml is the name of our playbook file
–e a way of parsing env variable to the playbook, or to override the default env variable
-u indicates the user ansible needs to use to login to the instance.

output

$ ansible-playbook -i /etc/ansible/ec2.py --limit "tag_App_ounje2" -u ec2-user sample.yml -e env=prod

[DEPRECATION WARNING]: ec2_remote_facts is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
PLAY [Deploy Ounje Artifact] ***********************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************
Monday 23 September 2O19 O1:17:25 -O4OO (O:OO:OO.141) O:OO:OO.141 ******
ok: [1.OO.1O9.111]
TASK [Get Instance IDs] *****************************************************************************************************************************************
Monday 23 September 2O19 O1:17:28 -O4OO (O:OO:O2.234) O:OO:O2.376 ******
ok: [1.OO.1O9.111]
TASK [Store ounje hosts] ***************************************************************************************************************************************
Monday 23 September 2O19 O1:17:29 -O4OO (O:OO:O1.481) O:OO:O3.857 ******
ok: [1.OO.1O9.111] => { "ounje": { "changed": false, "failed": false, "instances": [ { "ami_launch_index": "O", "architecture": "x86_94", "block_device_mapping": [ { "attach_time": "2O19-O6-23TO4:54:57.OOOZ", "delete_on_termination": true, "device_name": "/dev/xvda", "status": "attached", "volume_id": "vol-298hfn3f348ff4f348f" } ], "client_token": "", "ebs_optimized": false, "groups": [ { "id": "sg-Of11425adOe7f7d1a", "name": "launch-wizard-13" } ], "hypervisor": "xen", "id": "i-1O293847561OO293", "image_id": "ami-1k02nd83js84k6apO", "instance_profile": null, "interfaces": [ { "id": "eni-nf32f2fn48489r348", "mac_address": "41:9O:i7:dc:69:lO" } ], "kernel": null, "key_name": "emi-moni-profile", "launch_time": "2O19-O6-23TO4:54:56.OOOZ", "monitoring_state": "disabled", "persistent": false, "placement": { "tenancy": "default", "zone": "us-east-1c" }, "private_dns_name": "ip-1-OO-1O9-111.ec2.internal", "private_ip_address": "OO7-OO-69-42O", "public_dns_name": "ec2-1-OO-1O9-111.compute-1.amazonaws.com", "public_ip_address": "1.OO.1O9.111", "ramdisk": null, "region": "us-east-1", "requester_id": null, "root_device_type": "ebs", "source_destination_check": "true", "spot_instance_request_id": null, "state": "running", "tags": { "App": "ounje2", "Env": "prod", "Name": "Production Stack" }, "virtualization_type": "hvm", "vpc_id": "vpc-IaMhIdDeN" } ] }
}
TASK [Store EC2 instance ID] ************************************************************************************************************************************
Monday 23 September 2O19 O1:17:29 -O4OO (O:OO:OO.145) O:OO:O4.OO3 ******
ok: [1.OO.1O9.111] => (item={u'ramdisk': None, u'kernel': None, u'instance_profile': None, u'root_device_type': u'ebs', u'private_dns_name': u'ip-1-OO-1O9-111.ec2.internal', u'block_device_mapping': [{u'status': u'attached', u'device_name': u'/dev/xvda', u'delete_on_termination': True, u'attach_time': u'2O19-O6-23TO4:54:57.OOOZ', u'volume_id': u'vol-298hfn3f348ff4f348f'}], u'key_name': u'emi-moni-profile', u'interfaces': [{u'id': u'eni-nf32f2fn48489r348', u'mac_address': u'41:9O:i7:dc:69:lO'}], u'persistent': False, u'image_id': u'ami-1k02nd83js84k6apO', u'groups': [{u'id': u'sg-Of11425adOe7f7d1a', u'name': u'launch-wizard-13'}], u'spot_instance_request_id': None, u'requester_id': None, u'source_destination_check': u'true', u'id': u'i-1O293847561OO293', u'tags': {u'App': u'ounje2', u'Name': u'Production Stack', u'Env': u'prod'}, u'public_ip_address': u'1.OO.1O9.111', u'monitoring_state': u'disabled', u'placement': {u'tenancy': u'default', u'zone': u'us-east-1c'}, u'ami_launch_index': u'O', u'hypervisor': u'xen', u'region': u'us-east-1', u'ebs_optimized': False, u'launch_time': u'2O19-O6-23TO4:54:56.OOOZ', u'public_dns_name': u'ec2-1-OO-1O9-111.compute-1.amazonaws.com', u'state': u'running', u'architecture': u'x86_94', u'private_ip_address': u'OO7-OO-69-42O', u'vpc_id': u'vpc-IaMhIdDeN', u'client_token': u'', u'virtualization_type': u'hvm'})
TASK [Store EC2 instance ID in var] *****************************************************************************************************************************
Monday 23 June 2O19 O1:17:29 -O4OO (O:OO:OO.173) O:OO:O4.176 ******
ok: [1.OO.1O9.111] => { "instance_id": "i-1O293847561OO293"
}
PLAY RECAP ******************************************************************************************************************************************************
1.OO.1O9.111 : ok=5 changed=O unreachable=O failed=O
Monday 23 June 2O19 O1:17:29 -O4OO (O:OO:OO.11O) O:OO:O4.287 ******
===============================================================================
Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------------ 2.23s
Get Instance IDs ----------------------------------------------------------------------------------------------------------------------------------------- 1.48s
Store EC2 instance ID ------------------------------------------------------------------------------------------------------------------------------------ O.17s
Store ounje hosts --------------------------------------------------------------------------------------------------------------------------------------- O.15s
Store EC2 instance ID in var ----------------------------------------------------------------------------------------------------------------------------- O.11s
Playbook run took O days, O hours, O minutes, 4 seconds

With no hostfile to maintain, you can deploy to freshly created instances spun up by terraform without having to worry about maintaining an hostfile.

Blog

Ansible Dynamic Inventory

iamtito

Tags

TESTING OUR DYNAMIC INVENTORY ABOVE

Join Our Newsletter. No Spam, Only the good stuff.

Related