Nested object of unequal length into array of objects

Question

So I have an interesting problem which I have been able to solve, but my solution is not elegant in any way or form, so I was wondering what others could come up with :)

The issue is converting this response here

const response = {
        "device": {
            "name": "Foo",
            "type": "Bar",
            "telemetry": [
                {
                    "timeStamp": "2022-06-01T00:00:00.000Z",
                    "temperature": 100,
                    "pressure": 50
                },
                {
                    "timeStamp": "2022-06-02T00:00:00.000Z",
                    "temperature": 100,
                    "pressure": 50
                },
                {
                    "timeStamp": "2022-06-03T00:00:00.000Z",
                    "temperature": 100,
                    "pressure": 50
                },
                {
                    "timeStamp": "2022-06-04T00:00:00.000Z",
                    "temperature": 100,
                    "pressure": 50
                },
                {
                    "timeStamp": "2022-06-05T00:00:00.000Z",
                    "temperature": 100,
                    "pressure": 50
                }
            ]
        }
};

Given this selection criteria

const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature']

and the goal is to return something like this

[
  {"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z", "device/telemetry/temperature": 100},
  {"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z", "device/telemetry/temperature": 100},
  {"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z", "device/telemetry/temperature": 100},
 ...,
  {"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z", "device/telemetry/temperature": 100},

]

If you are interested, here is my horrible brute force solution, not that familiar with typescript yet, so please forgive the horribleness :D

EDIT #1 So some clarifications might be needed. The response can be of completely different format, so we can't use our knowledge of how the response looks like now, the depth can also be much deeper.

What we can assume though is that even if there are multiple arrays in the reponse (like another telemetry array called superTelemetry) then the selection criteria will only choose from one of these arrays, never both :)

function createRecord(key: string, value: any){
  return new Map<string, any>([[key, value]])
}

function getNestedData (data: any, fieldPath: string, records: Map<string, any[]>=new Map<string, any[]>()) {
    let dataPoints: any = [];
    const paths = fieldPath.split('/')
    paths.forEach((key, idx, arr) => {
      if(Array.isArray(data)){
        data.forEach(
          (row: any) => {
            dataPoints.push(row[key])
          }
        )
      } else {
        data = data[key]
        if(idx + 1== paths.length){
          dataPoints.push(data);
        }
      }
    })
    records.set(fieldPath, dataPoints)
    return records
  }

  function getNestedFields(data: any, fieldPaths: string[]){
    let records: Map<string, any>[] = []
    let dataset: Map<string, any[]> = new Map<string, any[]>()
    let maxLength = 0;
    // Fetch all the fields
    fieldPaths.forEach((fieldPath) => {
      dataset = getNestedData(data, fieldPath, dataset)
      const thisLength = dataset.get(fieldPath)!.length;
      maxLength = thisLength > maxLength ? thisLength : maxLength;
    })
    for(let i=0; i<maxLength; i++){
      let record: Map<string, any> = new Map<string, any>()
      for(let [key, value] of dataset){
        const maxIdx = value.length - 1;
        record.set(key, value[i > maxIdx ? maxIdx : i])
      }
      records.push(record)
    }
    // Normalize into records
    return records
  }

Are there typos in this? Why "name": "Foo" instead of "device.name": "Foo" and why slashes instead of dots? — jcalz
– jcalz, Commented Aug 5, 2022 at 13:54
I don't see any obvious simple implementation so far; I've got this but I don't know if it's any better than yours (I'm not exactly sure what makes yours "horrible" and "brute force"). You'd need to test it against yours extensively to make sure they behave the same (or that any difference is a difference you want to see). Recursive applications of Cartesian products can get messy. Do you want to see if that works for you and if so I can post it as an answer? And if not could you tell me what I'm missing? — jcalz
– jcalz, Commented Aug 5, 2022 at 15:01
@jcalz - I just always assume there is a better way using map, reduce, recursion etc :) I like your implementation and at least it seems faster than mine, so I'll accept that as an answer. This was a pretty open question, more like a coding challenge I guess than anything else :) — Dammi
– Dammi, Commented Aug 5, 2022 at 15:37

Rohìt Jíndal · Accepted Answer · 2022-08-05 14:07:25Z

2

As per my understanding you are looking for a solution to construct the desired result as per the post. If Yes, you can achieve this by using Array.map() along with the Array.forEach() method.

Try this :

const response = {
  "device": {
    "name": "Foo",
    "type": "Bar",
    "telemetry": [
      {
        "timeStamp": "2022-06-01T00:00:00.000Z",
        "temperature": 100,
        "pressure": 50
      },
      {
        "timeStamp": "2022-06-02T00:00:00.000Z",
        "temperature": 100,
        "pressure": 50
      },
      {
        "timeStamp": "2022-06-03T00:00:00.000Z",
        "temperature": 100,
        "pressure": 50
      },
      {
        "timeStamp": "2022-06-04T00:00:00.000Z",
        "temperature": 100,
        "pressure": 50
      },
      {
        "timeStamp": "2022-06-05T00:00:00.000Z",
        "temperature": 100,
        "pressure": 50
      }
    ]
  }
};

const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature'];

const res = response.device.telemetry.map(obj => {
  const o = {};
  fields.forEach(item => {
    const splittedItem = item.split('/');
    o[item] = (splittedItem.length === 2) ? response[splittedItem[0]][splittedItem[1]] : obj[splittedItem[2]];
  });
  return o;
})

console.log(res);

answered Aug 5, 2022 at 14:07

Rohìt Jíndal

27.3k16 gold badges80 silver badges133 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dammi Over a year ago

Very elegant solution Rohìt! However, I can't assume that "response.device.telemetry" is always going to be there, it might not be in some responses, or the array can have a different name as well. But what we can assume is that the selection criteria will only select from the same array, I'll update the question with that information.

Rohìt Jíndal Over a year ago

In that case you can check the type of key and then apply map on that key instead of using hardcoded telemetry.

Dammi Over a year ago

There is also the added complexity that the depth of the response object can vary, it can be much deeper, so one would have to go dynamically deep into the object as well. Thanks for your input, very much appreciate it :)

Rohìt Jíndal Over a year ago

@Dammi As nested properties might be an array, we can not directly access like this device.telemetry.timeStamp. Hence, to iterate it dynamically we have to first find out the type of the property and then we can iterate and bind dynamically

jcalz · Accepted Answer · 2022-08-05 21:11:43Z

In what follows I will be concerned with just the implementation and runtime behavior, and not so much the types. I've given things very loose typings like any and string instead of the relevant generic object types. Here goes:

function getNestedFields(data: any, paths: string[]): any[] {

If data is an array, we want to perform getNestedFields() on each element of the array, and then concatenate the results together into one big array. So the first thing we do is check for that and make a recursive call:

  if (Array.isArray(data)) return data.flatMap(v => getNestedFields(v, paths));

Now that we know data is not an array, we want to start gathering the pieces of the answer. If paths is, say, ['foo/bar', 'foo/baz/qux', 'x/y', 'x/z'], then we want to make recursive calls to getNestedFields(data.foo, ["bar", "baz/qux"]) and to getNestedFields(data.x, ["y", "z"]). In order to do this we have to split each path element at its first slash "/", and collect the results into a new object whose keys are the part to the left of the slash and whose values are arrays of parts to the right. In this example it would be {foo: ["bar", "baz/qux"], x: ["y", "z"]}.

Some important edge cases: for every element of paths with no slash, then we have a key with an empty value... that is, ["foo"] should result in a call like getNestedFields(data.foo, [""]). And if there is an element of paths that's just the empty string "", then we don't want to do a recursive call; the empty path is the base case and implies that we're asking about data itself. That is, instead of a recursive call, we can just return [{"": data}]. So we need to keep track of the empty path (hence the emptyPathInList variable below).

Here's how it looks:

  const pathMappings: Record<string, string[]> = {};
  let emptyPathInList = false;
  paths.forEach(path => {
    if (!path) {
      emptyPathInList = true;
    } else {
      let slashIdx = path.indexOf("/");
      if (slashIdx < 0) slashIdx = path.length;
      const key = path.substring(0, slashIdx);
      const restOfPath = path.substring(slashIdx + 1);
      if (!(key in pathMappings)) pathMappings[key] = [];
      pathMappings[key].push(restOfPath);
    }
  })

Now, for each key-value pair in pathMappings (with key key and with value restsOfPath) we need to call getNestedFields() recursively... the results will be an array of objects whose keys are relative to data[key], so we need to prepend key and a slash to their keys. Edge cases: if there's an empty path we shouldn't add a slash. And if data` is nullish then we will have a runtime error recursing down into it, so we might want to do something else there (although a runtime error might be fine since it's a weird input):

  const subentries = Object.entries(pathMappings).map(([key, restsOfPath]) =>
    (data == null) ? [{}] : // <-- don't recurse down into nullish data
      getNestedFields(data[key], restsOfPath)
        .map(nestedFields =>
          Object.fromEntries(Object.entries(nestedFields)
            .map(([path, value]) =>
              [key + (path ? "/" : "") + path, value])))
  )

Now subentries is an array of all the separate recursive call results, with the proper keys. We want to add one more entry correpsonding to data if emptyPathInList is true:

  if (emptyPathInList) subentries.push([{ "": data }]);

And now we need to combine these sub-entries by taking their Cartesian product and spreading into a single object for each entry. By Cartesian product I mean that if subentries looks like [[a,b],[c,d,e],[f]] then I need to get [[a,c,f],[a,d,f],[a,e,f],[b,c,f],[b,d,f],[b,e,f]], and then for each of those we spread into single entries. Here's that:

  return subentries.reduce((a, v) => v.flatMap(vi => a.map(ai => ({ ...ai, ...vi }))), [{}])
}

Okay, so let's test it out:

console.log(getNestedFields(response, fields));
/* [{
  "device/name": "Foo",
  "device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z",
  "device/telemetry/temperature": 100
}, {
  "device/name": "Foo",
  "device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z",
  "device/telemetry/temperature": 100
}, {
  "device/name": "Foo",
  "device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z",
  "device/telemetry/temperature": 100
}, {
  "device/name": "Foo",
  "device/telemetry/timeStamp": "2022-06-04T00:00:00.000Z",
  "device/telemetry/temperature": 100
}, {
  "device/name": "Foo",
  "device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z",
  "device/telemetry/temperature": 100
}]   */

That's what you wanted. Even though you said you will never walk into different arrays, this version should support that:

console.log(getNestedFields({
  a: [{ b: 1 }, { b: 2 }],
  c: [{ d: 3 }, { d: 4 }]
}, ["a/b", "c/d"]))
/* [
  { "a/b": 1, "c/d": 3 }, 
  { "a/b": 2, "c/d": 3 }, 
  { "a/b": 1, "c/d": 4 }, 
  { "a/b": 2, "c/d": 4 }
]*/

There are probably all kinds of crazy edge cases, so anyone using this should test thoroughly.

Playground link to code

Collectives™ on Stack Overflow

Nested object of unequal length into array of objects

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related