Empty string for DynamoDB index key in Golang

Question

I've been using Protobuf for object definitions to be used when communicating with DynamoDB. Until now, such objects would look like this:

type Record struct {
    state         protoimpl.MessageState
    sizeCache     protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields

    Id               string        `protobuf:"bytes,2,opt,name=id,json=id,proto3" dynamodbav:"id,omitempty" json:"id,omitempty"`
    Name             string        `protobuf:"bytes,3,opt,name=name,json=name,proto3" dynamodbav:"name,omitempty" json:"name,omitempty"`
    OrgId            string        `protobuf:"bytes,4,opt,name=org_id,json=orgId,proto3" dynamodbav:"org_id,omitempty" json:"org_id,omitempty"`
    AccountId        string        `protobuf:"bytes,7,opt,name=account_id,json=accountId,proto3" dynamodbav:"account_id,omitempty" json:"account_id,omitempty"`
    Address          string        `protobuf:"bytes,9,opt,name=address,proto3" dynamodbav:"address,omitempty" json:"address,omitempty"`
    BillingTitle     string        `protobuf:"bytes,10,opt,name=billing_title,json=billingTitle,proto3" dynamodbav:"billing_title,omitempty" json:"billing_title,omitempty"`
    Language         string        `protobuf:"bytes,11,opt,name=language,proto3" dynamodbav:"language,omitempty" json:"language,omitempty"`
    Personal         string        `protobuf:"bytes,12,opt,name=personal,proto3" dynamodbav:"personal,omitempty" json:"personal,omitempty"`
    Phonenumber      string        `protobuf:"bytes,13,opt,name=phonenumber,proto3" dynamodbav:"phonenumber,omitempty" json:"phonenumber,omitempty"`
    Postalcode       string        `protobuf:"bytes,14,opt,name=postalcode,proto3" dynamodbav:"postalcode,omitempty" json:"postalcode,omitempty"`
    ProjectId        string        `protobuf:"bytes,15,opt,name=project_id,json=projectId,proto3" dynamodbav:"project_id,omitempty" json:"project_id,omitempty"`
    Remarks          string        `protobuf:"bytes,16,opt,name=remarks,proto3" dynamodbav:"remarks,omitempty" json:"remarks,omitempty"`
}

The table I'm adding these objects to has a global secondary index with project_id as the sort-key but this field isn't always populated. So, when I went to do a PutObject operation on the table, like this:

record := Record {
    Id: "A000",
    Name: "Some Record",
    OrgId: "O000"
}

attrs, err := dynamodbattribute.MarshalMap(&record)
if err != nil {
    panic(err)
}

if _, err := conn.PutItem(&dynamodb.PutItemInput{
    TableName: aws.String(tableName),
    Item: attrs
}); err != nil {
    panic(err)
}

this code wouldn't panic and I'd see the value in DynamoDB. However, after removing the omitempty clause from all the dynamodbav tags, I notice that this now fails with the following error:

ValidationException: Invalid attribute value type
        status code: 400, request id: 6a626232-fcd4-4999-afe4-3df5769ce1b2

After some further investigation, I see that the object is serialized as:

map[account_id:{
  NULL: true
} address:{
  NULL: true,
} billing_title:{
  NULL: true
} id:{
  S: "A000"
} name:{
  S: "Some Record"
} language:{
  NULL: true
} org_id:{
  S: "O000"
} personal:{
  NULL: true
} phonenumber:{
  NULL: true
} postalcode:{
  NULL: true
} project_id:{
  NULL: true
} remarks:{
  NULL: true
}

when omitempty is not included but serializes to:

map[id:{
  S: "A000"
} name:{
  S: "Some Record"
} org_id:{
  S: "O000"
}

when omitempty is included.

I'd like to get the former with default values rather than NULL: true but I don't see any serialization options for this. Is there a way to do this without implementing the DynamoDB marshaler interface?

Woody1193 · Accepted Answer · 2021-11-08 07:00:50Z

2

After some investigation, I found the source code for the Marshal function, here. According to this, there's a field called NullEmptyString which I can set to tell the encoder that I want empty strings to be sent as strings, rather than as null attributes. However, if I set this I cannot use Marshal, MarshalMap and MarshalList directly. So, I copied them and wrote my own:

func MarshalMap(in interface{}) (map[string]*dynamodb.AttributeValue, error) {
    av, err := getEncoder().Encode(in)
    if err != nil || av == nil || av.M == nil {
        return map[string]*dynamodb.AttributeValue{}, err
    }

    return av.M, nil
}

func getEncoder() *dynamodbattribute.Encoder {
    encoder := dynamodbattribute.NewEncoder()
    encoder.NullEmptyString = false
    return encoder
}

answered Nov 8, 2021 at 7:00

Woody1193

8,2029 gold badges65 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Empty string for DynamoDB index key in Golang

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related