read variable names from csv file with ansible and substitute with values

Question

I am trying to create an environment specific configuration file consisting of key = value pairs that are read from a csv file. The csv file contains configuration settings for many environments and I need to generate a configuration file for each environment depending on inventory used by playbook.

The CSV file settings.csv

Key|siteA-test|siteA-prod|siteB-test|siteB-prod|Comment
param1|true|true|false|false|Comment for param1
param2|http://test|http://prod|||Comment for param2
param3|ansible_variable_A_test|ansible_variable_A_prod|ansible_variable_B_test|ansible_variable_B_prod|Variable must be taken from secret vault
param4|ansible_variable_C_test|ansible_variable_C_prod|ansible_variable_D_test| ansible_variable_D_prod|Another variable from variable file

Intended file for siteA-test would look like:

param1=true # Comment for param1
param2=http://test # Comment for param2
param3=secret_value_from_vault # Variable must be taken from secret vault 
param4=1984 # Another variable from variable file

I will write this out to a file using lineinfile module.

So far I have (somewhat ugly) figured out how to get values for specific environments, by creating a dictionary of keys first and then iterating over that dict. I don't like that I have to read CSV file twice, but this works.

- hosts: hostgroupA
  tasks:
  - name: Read CSV file and return a dictionary
    community.general.read_csv:
      path: settings.csv
      delimiter: '|'
      key: Key
      strict: yes
    register: config
    delegate_to: localhost

  - ansible.builtin.debug:
      msg: "Key {{ lookup('csvfile', '{{item.key}} file=settings.csv col=1 delimiter=|') }}, value: {{ lookup('csvfile', '{{item.key}} file=settings.csv col=2 delimiter=|')}} "
    loop: "{{ config.dict| dict2items }}"
    delegate_to: localhost

The problem is that I don't know if it is possible to replace variable names for param3 and param4 with real values from vaults or other variable files.

Frankly, it seems you're trying to get Ansible to do things it's not really good at. You could have a play calling the shell module to parse the file with sed, awk, cut, tr, or other tools and write the vars file in yaml format, then have another play to load that newly created file. — Jack
– Jack, Commented May 3, 2021 at 15:24

Vladimir Botka · Accepted Answer · 2021-05-03 17:08:56Z

2

Use lookup plugin vars. See ansible-doc -t lookup vars. For example, given simplified inventory. (Encrypt test*_var if you want to. The best option might be to put the variables into encrypted files stored in host_vars).

shall> cat hosts
testA testA_var="A"
testB testB_var="B"

and the simplified CSV file settings.csv, e.g.

shell> cat settings.csv
Key|testA|testB|Comment
param1|true|false|Comment for param1
param2|testA_var|testB_var|Var from vault

The playbook

- hosts: all
  tasks:
  - read_csv:
      path: settings.csv
      delimiter: '|'
      key: Key
      strict: yes
    register: config
    delegate_to: localhost
    run_once: true

  - debug:
      msg: "param1: {{ _p1 }}, param2: {{ _p2 }}"
    vars:
      _p1: "{{ config.dict.param1[inventory_hostname] }}"
      _p2: "{{ lookup('vars', config.dict.param2[inventory_hostname]) }}"

gives

ok: [testB] => 
  msg: 'param1: false, param2: B'
ok: [testA] => 
  msg: 'param1: true, param2: A'

If you want to iterate the hosts, e.g.

  - debug:
      msg: "{{ item }} param1: {{ _p1 }}, param2: {{ _p2 }}"
    loop: "{{ ansible_play_hosts }}"
    vars:
      _p1: "{{ config.dict.param1[item] }}"
      _varname: "{{ config.dict.param2[item] }}"
      _p2: "{{ hostvars[item][_varname] }}"
    run_once: true

gives

  msg: 'testA param1: true, param2: A'
  msg: 'testB param1: false, param2: B'

edited May 3, 2021 at 17:08

answered May 3, 2021 at 16:18

Vladimir Botka

70k7 gold badges45 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

shpokas Over a year ago

Yes, this is getting me much closer to what I wanted to achieve! Thanks, much appreciated!

Collectives™ on Stack Overflow

read variable names from csv file with ansible and substitute with values

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related