DynamoDB BatchGetItem Keys Array generation with multiple sort keys in python

Question

I might be missing something, but I struggle to generate multiple sort keys dict for use with Boto3 Batch get Item in python.

For all other DB, which don't use sort key, I have something like this:

I have a method that queries dynamo, but I use helper that populates the keys node:

            DYNAMODB_TABLE: {
                'Keys': generate_keydicts(myids, table_name)
            }
        })

and:

 if table_to_query == 'BossTable':
        return [{'id': myid} for myid in myids]

And that works beautiful - as charm.

Now, this other table that is using sort key would need to have additional "json" object in the loop, something like:

 if table_to_query == 'SortedKeyBossTable':
    keys = []
    for myid in myids:
          single_key = {{'id': myid},{"sortkey":"sortkey1"}, 
                        {'id': myid},{"sortkey":"sortkey2"}}
    keys.append(single_key)
  return keys

But this is not a valid, hashable JSON - so it fails.

If I remove outside curly brackets - then it's a tuple - again fails.

I've tried to pass that manually, in code - and it worked, but I don't know how to generate this "json" that is not valid json, or object that is a list of objects.

I'm not even sure how is that called, and how can that be achieved.

Before anyone asks - I have 2k IDs that I need to check, out of 2 million entries, and - I guess I should have done regular query per ID, but since I have the code for batch item in place, and all post processing that I'm doing - I hoped it would be matter of adding one line.

I've also looked here: Is it possible to use multiple sort values in aws sdk dynamodb batchGetItem?

And some other places, but can't find a working example how to generate that object in python.

Balkyto · Accepted Answer · 2023-10-06 12:39:19Z

0

If anyone comes to this problem...

Suggestion from linked example with pseudo code is missleading, same as (for me) confusing suggestion on AWS pages.

However - it's not a nested "double json", it requires just a list of all possible jsons with their id's and sort keys.

I've ended up creating a list of sort keys, and double loop:

sort_items = ['sort1','sort2','sort3']

if table_to_query == 'SortedKeyBossTable':
        blah = [{'id': myid, 'sortkey': sorter } for sorter in sort_items for myid in myids]
        return blah

answered Oct 6, 2023 at 12:39

Balkyto

1,5204 gold badges22 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Leeroy Hannigan Over a year ago

While this works, if you are reading all items related to a partition key, then its simpler and more cost efficient to use Query. However, if you are only reading a subset of items related to an id, then BatchGetItem is fine.

Collectives™ on Stack Overflow

DynamoDB BatchGetItem Keys Array generation with multiple sort keys in python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related