boto3 dict to json to text format in Python

Question

I'm trying to create a text file from my boto3 response in a AWS Lambda function which I want to upload to my S3. Here's what I have so far.

          import boto3
          import json
          s3 = boto3.client('s3')
          clientVPN = boto3.client('ec2')
          clientVPN = clientVPN.export_client_vpn_client_configuration(ClientVpnEndpointId='arn-of-clientVPN-id', DryRun=False)
              clientVPN_string = json.dumps(clientVPN, indent=2)
              response = s3.put_object(Bucket=myBucket, Body=clientVPN_string, Key='configuration.ovpn')

With what I have above, I'm able to write the file into S3. However, the file looks like below:

{
  "ClientConfiguration": "client\ndev tun\nproto udp\nremote xxxx.us-east-1.amazonaws.com 443\nremote-random-hostname\nresolv-retry infinite\nnobind\nremote-cert-tls server\ncipher AES-256-GCM\nverb 3\n<ca>\n-----BEGIN CERTIFICATE-----\nxx/2x\ncxxBr\nkw==\n-----END CERTIFICATE-----\n\n</ca>\n\n\nreneg-sec 0",
  "ResponseMetadata": {
    "RequestId": "40b87d13-edb9-4f3e-85e0-9b9f233518d9",
    "HTTPStatusCode": 200,
    "HTTPHeaders": {
      "x-amzn-requestid": "40b87d13-edb9-4f3e-85e0-9b9f233518d9",
      "cache-control": "no-cache, no-store",
      "strict-transport-security": "max-age=31536000; includeSubDomains",
      "vary": "accept-encoding",
      "content-type": "text/xml;charset=UTF-8",
      "transfer-encoding": "chunked",
      "date": "Sat, 26 Feb 2022 19:25:22 GMT",
      "server": "AmazonEC2"
    },
    "RetryAttempts": 0
  }
}

However, the output from AWS CLI gives me a neat txt format which would look as below:

client
dev tun
proto udp
remote xxxx.amazonaws.com 443
remote-random-hostname
resolv-retry infinite
nobind
remote-cert-tls server
cipher AES-256-GCM
verb 3
<ca>
-----BEGIN CERTIFICATE-----
xx
-----END CERTIFICATE-----

</ca>


reneg-sec 0

As we can see in the output, I would like to extract only the "ClientConfiguration" and neatly formatted as lines as provided by the AWS CLI output. I've tried several ways to do it like below but unfortunately I'm not having luck.

   clientVPN_string = json.dumps(clientVPN, indent=2)
   for key, value in clientVPN_string.items():
     with open('client.ovpn', 'a') as the_file:
       print(value)

Is there any way to convert the output am getting with boto3 to the txt format like how AWS CLI provides? Thank you!

valdeci · Accepted Answer · 2022-03-03 23:40:50Z

1

Using the methods decode("utf-8") and json.loads() will work for you.

Example:

def get_data_from_s3(name):
    s3 = boto3.client('s3')
    bucket = 'bucket_name'
    result = s3.list_objects(Bucket=bucket)
    for o in result.get('Contents'):
        data = s3.get_object(Bucket=bucket, Key=o.get('Key'))
        contents = data['Body'].read()
        client_vpn_body = contents.decode("utf-8")
        client_vpn_json = json.loads(client_vpn_body)
        print(client_vpn_json['ClientConfiguration'])

answered Mar 3, 2022 at 23:40

valdeci

15.6k7 gold badges64 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ramesh Over a year ago

Thank you. I'm not reading anything from S3 though. I want to write to s3 the output of export_client_vpn_client_configuration in the text format.

Collectives™ on Stack Overflow

boto3 dict to json to text format in Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related