Using jq to parse json output of AWS CLI tools

Question

I would like to use jq (http://stedolan.github.io/jq/) to parse the json output from aws ec2 describe-instances and return the Instance ID and Tags only where Tags contains Name. owner, costcentre.

Here is partial redacted json representing the source output:

{
  "OwnerId":"121456789127",
  "ReservationId":"r-48465168",
  "Groups":[

  ],
  "Instances":[
    {
      "Monitoring":{
        "State":"disabled"
      },
      "PublicDnsName":null,
      "RootDeviceType":"ebs",
      "State":{
        "Code":16,
        "Name":"running"
      },
      "EbsOptimized":false,
      "LaunchTime":"2014-03-19T09:16:56.000Z",
      "PrivateIpAddress":"10.250.171.248",
      "ProductCodes":[
        {
          "ProductCodeId":"aacglxeowvn5hy8sznltowyqe",
          "ProductCodeType":"marketplace"
        }
      ],
      "VpcId":"vpc-86bab0e4",
      "StateTransitionReason":null,
      "InstanceId":"i-1234576",
      "ImageId":"ami-b7f6c5de",
      "PrivateDnsName":"ip-10-120-134-248.ec2.internal",
      "KeyName":"Test_Virginia",
      "SecurityGroups":[
        {
          "GroupName":"Test",
          "GroupId":"sg-12345b"
        }
      ],
      "ClientToken":"VYeFw1395220615808",
      "SubnetId":"subnet-12345314",
      "InstanceType":"t1.micro",
      "NetworkInterfaces":[
        {
          "Status":"in-use",
          "SourceDestCheck":true,
          "VpcId":"vpc-123456e4",
          "Description":"Primary network interface",
          "NetworkInterfaceId":"eni-3619f31d",
          "PrivateIpAddresses":[
            {
              "Primary":true,
              "PrivateIpAddress":"10.120.134.248"
            }
          ],
          "Attachment":{
            "Status":"attached",
            "DeviceIndex":0,
            "DeleteOnTermination":true,
            "AttachmentId":"eni-attach-9210dee8",
            "AttachTime":"2014-03-19T09:16:56.000Z"
          },
          "Groups":[
            {
              "GroupName":"Test",
              "GroupId":"sg-123456cb"
            }
          ],
          "SubnetId":"subnet-31236514",
          "OwnerId":"109030037527",
          "PrivateIpAddress":"10.120.134.248"
        }
      ],
      "SourceDestCheck":true,
      "Placement":{
        "Tenancy":"default",
        "GroupName":null,
        "AvailabilityZone":"us-east-1c"
      },
      "Hypervisor":"xen",
      "BlockDeviceMappings":[
        {
          "DeviceName":"/dev/sda",
          "Ebs":{
            "Status":"attached",
            "DeleteOnTermination":false,
            "VolumeId":"vol-37ff097b",
            "AttachTime":"2014-03-19T09:17:00.000Z"
          }
        }
      ],
      "Architecture":"x86_64",
      "KernelId":"aki-88aa75e1",
      "RootDeviceName":"/dev/sda1",
      "VirtualizationType":"paravirtual",
      "Tags":[
        {
          "Value":"Server for testing RDS feature in us-east-1c AZ",
          "Key":"Description"
        },
        {
          "Value":"RDS_Machine (us-east-1c)",
          "Key":"Name"
        },
          {
          "Value":"1234",
          "Key":"Cost.centre"
        },
        {
          "Value":"Jyoti Bhanot",
          "Key":"Owner"
        }
      ],
      "AmiLaunchIndex":0
    }
  ]
}

I am using this command :

 aws ec2 describe-instances | jq '.Instances[] | select(.Tags==["Name","Owner","cost.centre") | .InstanceId, .Tags'

But its giving me error :

                                                           ^
   error: Invalid character
.Reservations[].Instances | map(select(has("Tags"))) |.Tags[].Key=="cost.centre")) | map(select(has("InstanceId"))) | .[].InstanceId,.[].Tags
                                                                                ^
error: Invalid character
.Reservations[].Instances | map(select(has("Tags"))) |.Tags[].Key=="cost.centre")) | map(select(has("InstanceId"))) | .[].InstanceId,.[].Tags
                                                                                 ^
2 compile errors

[Errno 32] Broken pipe

Kindly help me resolve this error

phatypus · Accepted Answer · 2015-09-20 23:06:11Z

3

Try this:

$ aws ec2 describe-instances --output json | jq '.Reservations[].Instances[] | (.Tags | from_entries) as $tags | select($tags.Owner != null) | {InstanceID: .InstanceId, Name: $tags.Name, Owner: $tags.Owner, CostCenter: $tags."cost.center"}'

... which returns:

{
  "InstanceID": "i-734cbc51",
  "Name": "someserver001",
  "Owner": "john.smith@null",
  "CostCenter": "ABC001"
}

answered Sep 20, 2015 at 23:06

phatypus

7723 gold badges9 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

phatypus Over a year ago

That works on the current the output of aws ec2 describe-instances, but I just noticed that it needs tweaking for your sample input; this works for your sample: cat sample | jq '.Instances[] | (.Tags | from_entries) as $tags | select($tags.Owner != null) | {InstanceID: .InstanceId, Name: $tags.Name, Owner: $tags.Owner, CostCenter: $tags."Cost.centre"}'

sorohan · Accepted Answer · 2014-03-31 03:09:22Z

0

The main problem is that your query is missing a closing ']', however I think it's a bit trickier to filter on EC2 instances that have those tags.

This might be a bit convoluted, but something like this works for me:

aws ec2 describe-instances | jq '.Instances | map(select(has("Tags"))) | map(select(.Tags[].Key=="Name" and .Tags[].Key=="Owner" and .Tags[].Key=="Cost.centre")) | map(select(has("InstanceId"))) | .[].InstanceId,.[].Tags'

edited Mar 31, 2014 at 3:09

answered Mar 30, 2014 at 9:46

sorohan

8668 silver badges13 bronze badges

4 Comments

sorohan Over a year ago

Sorry, my aws command has a "Reservations" top level element, and the "cost.centre" should have been Cost.centre". I've updated the command to match the JSON you provided.

user3086014 Over a year ago

can you do it for case sensitive also ?

sorohan Over a year ago

@user3086014 I can't see support in jq for case insensitive.

sorohan Over a year ago

@MeghaSharma can you tell me the error? It is working for me with the JSON you provided.

Collectives™ on Stack Overflow

Using jq to parse json output of AWS CLI tools

2 Answers 2

1 Comment

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related