Go: JSON marshalling nested struct; incorrectly omitting outer fields

Question

I'm trying to marshal nested structs. See a NON-FUNCTIONING example here (I can't import "compute" and "pretty" in the Go playground, but I've recreated my test logic and pasted the output).

package main

import (
    "encoding/json"
    "fmt"

    "github.com/kylelemons/godebug/pretty"
    compute "google.golang.org/api/compute/v1"
)

type CreateInstance struct {
    compute.Instance

    // Additional metadata to set for the instance.
    Metadata map[string]string `json:"metadata,omitempty"`
    // OAuth2 scopes to give the instance. If none are specified
    // https://www.googleapis.com/auth/devstorage.read_only will be added.
    Scopes []string `json:",omitempty"`

    // StartupScript is the Sources path to a startup script to use in this step.
    // This will be automatically mapped to the appropriate metadata key.
    StartupScript string `json:",omitempty"`
    // Project to create the instance in, overrides workflow Project.
    Project string `json:",omitempty"`
    // Zone to create the instance in, overrides workflow Zone.
    Zone string `json:",omitempty"`
    // Should this resource be cleaned up after the workflow?
    NoCleanup bool
    // Should we use the user-provided reference name as the actual resource name?
    ExactName bool

    // The name of the disk as known internally to Daisy.
    daisyName string
}

func main() {
    ci := <a *CreateInstance part of a larger data structure>
    j, _ := json.MarshalIndent(ci, "", "  ")
    fmt.Println(string(j))
    pretty.Print(ci)  # Pretty prints the struct.
}


##### OUTPUT #####
{
  "disks": [
    {
      "source": "disk"
    }
  ],
  "machineType": "${machine_type}",
  "name": "${instance_name}"
}
{Instance:      {CanIpForward:      false,
                 CpuPlatform:       "",
                 CreationTimestamp: "",
                 Description:       "",
                 Disks:             [{AutoDelete:        false,
                                      Boot:              false,
                                      DeviceName:        "",
                                      DiskEncryptionKey: nil,
                                      Index:             0,
                                      InitializeParams:  nil,
                                      Interface:         "",
                                      Kind:              "",
                                      Licenses:          [],
                                      Mode:              "",
                                      Source:            "disk",
                                      Type:              "",
                                      ForceSendFields:   [],
                                      NullFields:        []}],
                 Id:                0,
                 Kind:              "",
                 MachineType:       "${machine_type}",
                 Metadata:          nil,
                 Name:              "${instance_name}",
                 NetworkInterfaces: [],
                 Scheduling:        nil,
                 SelfLink:          "",
                 ServiceAccounts:   [],
                 Status:            "",
                 StatusMessage:     "",
                 Tags:              nil,
                 Zone:              "",
                 ServerResponse:    {HTTPStatusCode: 0,
                                     Header:         {}},
                 ForceSendFields:   [],
                 NullFields:        []},
 Metadata:      {},
 Scopes:        [],
 StartupScript: "",
 Project:       "",
 Zone:          "",
 NoCleanup:     false,
 ExactName:     false}

Basically, I have a struct, CreateInstance, that embeds an Instance struct from the Google Compute Engine API client lib. CreateInstance also has two bool fields with no JSON tags, ExactName and NoCleanup.

When I try to marshal a CreateInstance, ExactName and NoCleanup are omitted, be they true or false.

Good approximation. No, MashalIndent does not return an error. — crunk1
– crunk1, Commented Jun 7, 2017 at 2:59
So, I'm currently passing in a *CreateInstance. When I deref it, it works. — crunk1
– crunk1, Commented Jun 7, 2017 at 5:00
No, this is more like what's happening: play.golang.org/p/6RCsQ3AM03 — Mr_Pink
– Mr_Pink, Commented Jun 7, 2017 at 13:34

Mr_Pink · Accepted Answer · 2017-06-07 13:32:58Z

2

A compute.Instance is a json.Marshaler, so by embedding that type you are in essence giving your CreateInstance the MarshalJSON method from compute.Instance, which of course isn't going to output any of the fields in your CreateInstance struct.

You could define your own MarshalJSON method, and try to marshal the Instance manually or re-assign it to a new type that will use the default json output, but the API may be relying on the internal MarshalJSON behavior, so that's not guaranteed to be compatible, currently or with future versions.

Embedding types for json marshaling is is best avoided because it's all too easy to create confusing bugs like this. I would try to compose them in a different manner.

answered Jun 7, 2017 at 13:32

Mr_Pink

110k17 gold badges287 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

crunk1 Over a year ago

Hey thank you! Unfortunately, trying to "compose them in a different manner" is a bit difficult since we want to expose a JSON config schema to users. Embedding is nicer than an inner compute.Instance instance or pointer in the JSON config, since an inner compute.Instance would require a nested object.

Collectives™ on Stack Overflow

Go: JSON marshalling nested struct; incorrectly omitting outer fields

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related