How to save json data as it is without data type conversion in dynamo db using python

Question

I want to store key-value JSON data in aws DynamoDB where key is a date string in YYYY-mm-dd format and value is entries which is a python dictionary. When I used boto3 client to save data there, it saved it as a data type object, which I don't want. My purpose is simple: Store JSON data against a key which is a date, so that later I will query the data by giving that date. I am struggling with this issue because I did not find any relevant link which says how to store JSON data and retrieve it without any conversion. I need help to solve it in Python.

What I am doing now:

item = {
  "entries": [
    {
      "path": [
        {
          "name": "test1",
          "count": 1
        },
        {
          "name": "test2",
          "count": 2
        }
      ],
      "repo": "test3"
    }
  ],
  "date": "2022-10-11"
}
dynamodb_client = boto3.resource('dynamodb')
table = self.dynamodb_client.Table(table_name)
response = table.put_item(Item = item)

What actually saved:

[{"M":{"path":{"L":[{"M":{"name":{"S":"test1"},"count":{"N":"1"}}},{"M":{"name":{"S":"test2"},"count":{"N":"2"}}}]},"repo":{"S":"test3"}}}]

But I want to save exactly the same JSON data as it is, without any conversion at all.

When I retrieve it programmatically, you see the difference of single quote, count value change.

response = table.get_item(
    Key={
        "date": "2022-10-12"
    }
)

Output
{'Item': {'entries': [{'path': [{'name': 'test1', 'count': Decimal('1')}, {'name': 'test2', 'count': Decimal('2')}], 'repo': 'test3'}], 'date': '2022-10-12} }

Sample picture:

What does your create_table command look like for this table? Do you tell it to use date in your KeySchema? What do you define in AttributeDefinitions; all attributes or just the date (it should be the latter: stackoverflow.com/a/54155980/361842) — JohnLBevan
– JohnLBevan, Commented Oct 11, 2022 at 8:03
I am creating table through code and date schema is string is good for me. The major issue is with count field data type, single quote. If you see the output above you find that it is not what I saved in it. — Ashish Sharma
– Ashish Sharma, Commented Oct 11, 2022 at 8:10

hunterhacker · Accepted Answer · 2022-10-11 08:12:32Z

3

Why not store it as a single attribute of type string? Then you’ll get out exactly what you put in, byte for byte.

answered Oct 11, 2022 at 8:12

hunterhacker

7,2801 gold badge19 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ashish Sharma Over a year ago

I think I am doing that. in the example given above key is "2022-10-12" and values is List

hunterhacker Over a year ago

Right, a List. Not a string. Make entries a string and it will come back as a string, unchanged.

Leeroy Hannigan · Accepted Answer · 2022-10-11 07:56:23Z

1

When you store this in DynamoDB you get exactly what you want/have provided. Key is your date and you have a list of entries.

If you need it to store in a different format you need to provide the JSON which correlates with what you need. It's important to note that DynamoDB is a key-value store not a document store. You should also look up the differences in these.

answered Oct 11, 2022 at 7:56

Leeroy Hannigan

20.5k4 gold badges23 silver badges46 bronze badges

3 Comments

Ashish Sharma Over a year ago

I have updated my question with output I am getting. You can see the difference between what I saved in it and what I retrieved from it.

Leeroy Hannigan Over a year ago

What you see stored is DynamoDB JSON and all data is stored in DynamoDB that way. When you read the item back using the Resource client in boto3 you will get back the same data as you saved which is native JSON.

Ashish Sharma Over a year ago

but it is not happening as you can see in the output above. single quote and count value get changed.

Ashish Sharma · Accepted Answer · 2022-10-11 08:45:00Z

I figured out how to solve this issue. I have two column name date and entries in my dynamo db (also visible in screenshot in ques). I convert entries values from list to string then saved it in db. At the time of retrival, I do the same, create proper json response and return it. I am also sharing sample code below so that anybody else dealing with the same situation can have atleast one option.

# While storing:

entries_string = json.dumps([
    {
      "path": [
        {
          "name": "test1",
          "count": 1
        },
        {
          "name": "test2",
          "count": 2
        }
      ],
      "repo": "test3"
    }
  ])
item = {
  "entries": entries_string,
  "date": "2022-10-12"
}
dynamodb_client = boto3.resource('dynamodb')
table = dynamodb_client.Table(<TABLE-NAME>)

-------------------------
# While fetching:

response = table.get_item(
    Key={
        "date": "2022-10-12"
    }
)['Item']
entries_string=response['entries']
entries_dic = json.loads(entries_string)
response['entries'] = entries_dic
print(json.dumps(response))

Collectives™ on Stack Overflow

How to save json data as it is without data type conversion in dynamo db using python

3 Answers 3

2 Comments

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related