find and delete nested json object in Go

Question

I have a json document of a Kubernetes Pod, here's an example: https://github.com/itaysk/kubectl-neat/blob/master/test/fixtures/pod-1-raw.json

I'd like to traverse spec.containers[i].volumeMounts and delete those volumeMount objects where the .name starts with "default-token-". Note that both containers and volumeMounts are arrays.

Using jq it took me 1 min to write this 1 line: try del(.spec.containers[].volumeMounts[] | select(.name | startswith("default-token-"))). I'm trying to rewrite this in Go.

While looking for a good json library I settled on gjson/sjson. Since sjson doesn't support array accessors (the # syntax), and gjson doesn't support getting the path of result, I looked for workarounds.
I've tried using Result.Index do delete the the result from the byte slice directly, and succeeded, but for the query I wrote (spec.containers.#.volumeMounts.#(name%\"default-token-*\")|0) the Index is always 0 (I tried different variations of it, same result).
So currently I have some code 25 line code that uses gjson to get spec.containers.#.volumeMounts and iterate it's way through the structure and eventually use sjson.Delete to delete. It works, but it feels way more complicated then I expected it to be.

Is there a better way to do this in Go? I'm willing to switch json library if needed.

EDIT: I would prefer to avoid using a typed schema because I may need to perform this on different types, for some I don't have the full schema.
(also removed some distracting details about my current implemetation)

icio · Accepted Answer · 2019-08-11 20:05:17Z

3

The easiest thing to do here is parse the JSON into an object, work with that object, then serialise back into JSON.

Kubernetes provides a Go client library that defines the v1.Pod struct you can Unmarshal onto using the stdlib encoding/json:

// import "k8s.io/api/core/v1"
var pod v1.Pod
if err := json.Unmarshal(podBody, &pod); err != nil {
    log.Fatalf("parsing pod json: %s", err)
}

From there you can read pod.Spec.Containers and their VolumeMounts:

// Modify.
for c := range pod.Spec.Containers {
    container := &pod.Spec.Containers[c]
    for i, vol := range container.VolumeMounts {
        if strings.HasPrefix(vol.Name, "default-token-") {
            // Remove the VolumeMount at index i.
            container.VolumeMounts = append(container.VolumeMounts[:i], container.VolumeMounts[i+1:]...)
        }
    }
}

https://play.golang.org/p/3r5-XKIazhK

If you're worried about losing some arbitrary JSON which might appear in your input, you may instead wish to define var pod map[string]interface{} and then type-cast each of the properties within as spec, ok := pod["spec"].(map[string]interface{}), containers, ok := spec["containers"].([]map[string]interface) and so on.

Hope that helps.

ps. The "removing" is following https://github.com/golang/go/wiki/SliceTricks#delete

answered Aug 11, 2019 at 20:05

icio

3,13821 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

itaysk Over a year ago

Thanks. I should have mentioned that I would like to avoid using a typed schema because I may need to perform this on different types, some of them I don't have the full schema (custom resources). The map[string]interface{} solution can work, and it's quite similar to the solution I already have using gjson and iterators, except the deletion technique which I'll adopt but it's not the essence of the question, traversing is. Gonna let this sink for a while to see if a better answer arrives and if not I'll accept this.

icio · Accepted Answer · 2019-08-11 22:17:14Z

To take a totally different approach from before, you could create a

type Root struct {
    fields struct {
        Spec *Spec `json:"spec,omitempty"`
    }
    other map[string]interface{}
}

with custom UnmarshalJSON which unmarshals into both fields and other, and custom MarshalJSON which sets other["spec"] = json.RawMessage(spec.MarshalJSON()) before returning json.Marshal(other):

func (v *Root) UnmarshalJSON(b []byte) error {
    if err := json.Unmarshal(b, &v.fields); err != nil {
        return err
    }
    if v.other == nil {
        v.other = make(map[string]interface{})
    }
    if err := json.Unmarshal(b, &v.other); err != nil {
        return err
    }
    return nil
}

func (v *Root) MarshalJSON() ([]byte, error) {
    var err error
    if v.other["spec"], err = rawMarshal(v.fields.Spec); err != nil {
        return nil, err
    }
    return json.Marshal(v.other)
}

func rawMarshal(v interface{}) (json.RawMessage, error) {
    b, err := json.Marshal(v)
    if err != nil {
        return nil, err
    }
    return json.RawMessage(b), nil
}

You then define these sort of types all of the way down through .spec.containers.volumeMounts and have a Container.MarshalJSON which throws away and VolumeMounts we don't like:

func (v *Container) MarshalJSON() ([]byte, error) {
    mounts := v.fields.VolumeMounts
    for i, mount := range mounts {
        if strings.HasPrefix(mount.fields.Name, "default-token-") {
            mounts = append(mounts[:i], mounts[i+1:]...)
        }
    }

    var err error
    if v.other["volumeMounts"], err = rawMarshal(mounts); err != nil {
        return nil, err
    }
    return json.Marshal(v.other)
}

Full playground example: https://play.golang.org/p/k1603cchwC7

I wouldn't do this.

Collectives™ on Stack Overflow

find and delete nested json object in Go

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related