How to solve golang fatal error: concurrent map write

Question

I am using google pubsub to publish a lot of records which is happening concurrently, but after few dozens of publishes, I get an error : concurrent map write.

I have read that I am suppose to use sync.Map{} or sync.Mutex, but I haven't found an example that helped me understand how to use it in my specific situation.

Here is a code snippet that causes me problems:

md := metadata.Collect(ctx)
    records, err := s.emailRepository.SetCurrentVersionWithEnsure(ctx, md.GetTransactionId(), valid)
    if err != nil {
        return err
    }

    for _, value := range records {
        go s.pub.PublishAction(md, value)
    }

    return nil

Variable records is of type map[string][]byte

If anyone can help with this, I would be grateful. Cheers !

"How to solve golang fatal error: concurrent map write" don't write concurently to a map. "I have read that I am suppose to use sync.Map{} or sync.Mutex, but I haven't found an example that helped me understand how to use it in my specific situation." Your situation is not specific. You'll have to learn how to synchronise access. — Volker
– Volker, Commented Jun 23, 2023 at 11:43

Poala Astrid · Accepted Answer · 2023-06-27 19:46:20Z

1

Multiple goroutines are trying to read and write on your map concurrently or simultaneously. Yes, sync.Mutex is one of the workarounds that can fix that error because it ensures that only one goroutine can access the map. Aside from sync.Mutex, you can also use sync.RWMutex. For accessing the map concurrently, you can use sync.Map. You can refer to this SO case.

Posting this answer as a Community Wiki since this is the possible known workaround and for the benefit of the community that might encounter this use case in the future.

Please feel free to edit this answer for additional information and if there are other possible workarounds/direct solutions for this use case.

answered Jun 27, 2023 at 19:46

community wiki

Poala Astrid

Sign up to request clarification or add additional context in comments.

Comments

David · Accepted Answer · 2023-06-28 20:35:45Z

0

I have found a problem for my particular case.

In method PublishAction there is a line of code that checks if pubsub topic exists. This line does that by pinging google server, but there is a limit of how many times you can hit that server in a minute.

answered Jun 28, 2023 at 20:35

David

771 silver badge6 bronze badges

Collectives™ on Stack Overflow

How to solve golang fatal error: concurrent map write

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related