Using msgp with interfaces and maps in Go

Question

I have a map that uses an interface as the key. The map is defined like this MyMap map[Signature]Packets. The interface is Signature, and there will be two structs A and B that implement this interface. I am also using msgp to serialize these two structs.

My issue is that msgp automatically generates methods that use a pointer as the type of the function receiver, which I think will make the key Signature receive pointers. If that was the case, then the key would be different every single time since pointers are different, even though the underlying values are the same. So, every time, I would be creating a new entry instead of finding the existing one and modifying it.

I wonder:

Is there a way to force msgp to generate methods purely with function receivers of the concrete type? Currently, I can only modify the function receivers of auto-generated methods like MarshalMsg and UnmarshalMsg to the concrete type (A or B instead of *A or *B). By doing that, the key of the map is either of type A or of type B, and the map MyMap works fine. However, I know I should not modify the auto-generated code. So, I wonder whether there is an acceptable way to do that.
If there is no way to do 1., is there any workaround to solve this problem? I really need some polymorphic feature of the map's key with the use of msgp.

UPDATE 1 (Apr. 12): Thanks for sharing your thoughts and offering solutions. Here are some details about my question.

The background is that the map is used for collecting different network events. The two structs implementing the interface Signature are EventSignatureIPv4 and EventSignatureIPv6

type EventSignatureIPv4 struct {
    SourceIPv4 [4]byte
    Port     uint16
    Traffic  TrafficType
}

type EventSignatureIPv6 struct {
    SourceIPv6 [16]byte
    Port     uint16
    Traffic  TrafficType
}

and Signature is holding common methods shared between IPv4 and IPv6 data. So, essentially, I want to collect and group corresponding IPv4/v6 events at the runtime. The key of the map is to identify the same source, and the value of the map is to collect events with different destinations.

The msgp library I am using is this one https://pkg.go.dev/github.com/tinylib/[email protected]/msgp
Correct me if I am wrong. For compositions in Go, if one of the methods in the method set has a function receiver of the pointer type, then the instance would only be of the pointer type? So here, as I have

func (z *EventSignatureIPv6) MarshalMsg(b []byte) (o []byte, err error) {
    /* Auto-generated code */
}

whenever I use Signature to receive the struct EventSignatureIPv6, the struct would only be of type *EventSignatureIPv6?

@Sebastian In Golang since Struct is a value type using them as Key of the Map is possible. You are correct that if pointers are used you will get a different key every time for the same data. So if possible instead of saving pointers directly may be you can get the value of the pointer using *pointer syntax. Also if you can include some code it would be possible to suggest a better work around. — Sai Ravi Teja K
– Sai Ravi Teja K, Commented Apr 12, 2021 at 13:04
@Christian The msgp library I am using is pkg.go.dev/github.com/tinylib/[email protected]/msgp — Sebastian
– Sebastian, Commented Apr 12, 2021 at 13:33
@SaiRaviTejaK I think this is a different case from what you said about. I have updated my question and please take another look and see if you have any other thoughts. — Sebastian
– Sebastian, Commented Apr 12, 2021 at 13:35
@blackgreen I see your point. Let me make this clearer by modifying your code. It is true that the hash will be entirely different forever, so that is why I want some feature like this where I do not need any of the pointer receivers. In that case, the result is what I expect. — Sebastian
– Sebastian, Commented Apr 12, 2021 at 14:54

blackgreen · Accepted Answer · 2021-04-12 14:46:11Z

2

You are right, "two pointer values are equal if they point to the same variable.", so if you are looking to compare interfaces that may hold pointers to different types, e.g. *A and *B, you are already in trouble.

With that said, I don't think it's an amazing idea to use interface types as map keys in the first place, because you have to deal with some caveats, the first is that:

The comparison operators == and != must be fully defined for operands of the key type

And now you need to be careful about the types that implement the interface. In theory, nobody stops a client from implementing your interface on a defined type with underlying unhashable type, e.g. type UncomparableSignature []int

So you would probably have to add an unexported method on your interface, so that client code outside that package can't implement it. But still, nothing stops code within the same package from implementing it, so this is, at best, maintenance overhead.

Then if the interface holds pointers to zero-values, it's even dependant on the implementation of the specs:

Pointers to distinct zero-size variables may or may not be equal.

Furthermore, you open yourself up to pesky bugs, like variables of type Signature that holds a nil will overwrite each other's values:

var foo Signature
var bar Signature

myMap[foo] = &Packet{/*pretending to have value 1*/}
myMap[bar] = &Packet{/*pretending to have value 2*/}

fmt.Println(myMap[foo]) // 2

A possible solution is, you could replace the map key with a unique id, and you enforce implementors to provide it by declaring the appropriate method on the interface Signature (this still assumes that the implementors can be coordinated to provide unique ids across all of them):

type Signature interface {
    UniqueIdent() uint64 // or string, if you prefer
    // ...other methods
}

and then

packet := myMap[someSignature.UniqueIdent()]

edited Apr 12, 2021 at 14:46

answered Apr 12, 2021 at 12:53

blackgreen♦

46.9k29 gold badges171 silver badges162 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sebastian Over a year ago

Thanks for your answer. I have given more details about my question and want to ask for more suggestions. The unique id idea is great, but I would like to keep the overhead as small as possible, and that is also why I am using interfaces to deal with IPv4/v6 events. The comparison is really a problem here as I am comparing pointers right now, but the nullness is not a big deal I would say based off my error checking

Collectives™ on Stack Overflow

Using msgp with interfaces and maps in Go

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related