Search recursively through a deeply nested object/array and collect values that include a specified string without knowing the key(s)

Question

Let's say I have a deeply nested object or array, returned from an API, which may look like this:

const array = [
  {
    key1: 'dog',
    key2: 'cat'
  },
  {
    key1: 'dog',
    key2: 'cat'
    key3: [
      {
        keyA: 'mouse',
        keyB: 'https://myURL.com/path/to/file/1'
        keyC: [
          {
            keyA: 'tiger',
            keyB: 'https://myURL.com/path/to/file/2'
          }
        ]
      }
    ]
  },
  {
    key4: 'dog',
    key5: 'https://myURL.com/path/to/file/3'
  }
]

I want to traverse the object/array recursively and build an array of all the values that are strings and include the substring https://myURL.com, and building an array with the matches as follows:

let matches = [
   'https://myURL.com/path/to/file/1',
   'https://myURL.com/path/to/file/2',
   'https://myURL.com/path/to/file/3'
]

An important note is that I DO NOT know in advance what the relevant key(s) are. A value that is a URL could be on any key, potentially. So, in this example I cannot simply look for keyB or key5 - I have to test every key:value pair for values that include a URL string.

Question

How do I create a recursive function to search the object for values that .includes() string 'https://myURL.com'? Happy to use a lodash method, so feel free to submit solutions that depend on lodash.

Context

The data comes from a CMS API and I will be using the array of URLS (matches above, in this example) to add assets to cache with Cache API, async in background, for offline functionality for a PWA.

Many thanks in advance.

@DeanVanGreunen The problem I have is that is that I don't know what the possible names of the property keys in advance, so I essentially need to test every key:value pair for value that include a URL string. — Rhecil Codes
– Rhecil Codes, Commented Jul 8, 2020 at 13:53

Mulan · Accepted Answer · 2020-07-08 16:07:59Z

2

Here's a generic approach using a values and filter generator -

const values = function* (t)
{ if (Object(t) === t)
    for (const v of Object.values(t))
      yield* values(v)
  else
    yield t
}

const filter = function* (test, t)
{ for (const v of values(t))
    if (test(v))
      yield v
}

We can now write a simple query using filter -

const myArray =
  [ ... ]

const query =
  filter
    ( v =>
        String(v) === v
          && v.startsWith("https://myURL.com")
    , myArray
    )

const result =
  Array.from(query)

console.log(result)
// [ 'https://myURL.com/path/to/file/1'
// , 'https://myURL.com/path/to/file/2'
// , 'https://myURL.com/path/to/file/3'
// ]

Using generics like filter we can write specialised variants like searchByString -

const searchByString = (test, t) =>
  filter                                     // <-- specialisation
    ( v => String(v) === v && test(v)        // <-- auto reject non-strings
    , t
    )

Now querying is cleaner at the call site -

const myArray =
  [ ... ]

const query =
  searchByString                              // <-- specialised
    ( v => v.startsWith("https://myURL.com")  // <-- simplified filter
    , myArray
    )

const result =
  Array.from(query)

console.log(result)
// [ 'https://myURL.com/path/to/file/1'
// , 'https://myURL.com/path/to/file/2'
// , 'https://myURL.com/path/to/file/3'
// ]

Expand the snippet below to verify the results in your browser -

const myArray =
  [{key1:'dog',key2:'cat'},{key1:'dog',key2:'cat',key3:[{keyA:'mouse',keyB:'https://myURL.com/path/to/file/1',keyC:[{keyA:'tiger',keyB:'https://myURL.com/path/to/file/2'}]}]},{key4:'dog',key5:'https://myURL.com/path/to/file/3'}]
  
const values = function* (t)
{ if (Object(t) === t)
    for (const v of Object.values(t))
      yield* values(v)
  else
    yield t
}

const filter = function* (test, t)
{ for (const v of values(t))
    if (test(v))
      yield v
}

const searchByString = (test, t) =>
  filter
    ( v =>
        String(v) === v && test(v)
    , t
    )

const query =
  searchByString
    ( v => v.startsWith("https://myURL.com")
    , myArray
    )

const result =
  Array.from(query)

console.log(result)
// [ 'https://myURL.com/path/to/file/1'
// , 'https://myURL.com/path/to/file/2'
// , 'https://myURL.com/path/to/file/3'
// ]

edited Jul 8, 2020 at 16:07

answered Jul 8, 2020 at 16:02

Mulan

136k35 gold badges240 silver badges276 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Scott Sauyet Over a year ago

You've been saying it for a while, but I finally realized how much your Object (obj) === obj formulation simplifies things. The update in my answer really demonstrates it, by recognizing that Object.values unifies array and object processing, again, what you've been saying for a while, and I've just been missing it.

Rhecil Codes Over a year ago

Thank you. Beautifully concise, but have awarded to Scott as he was first and also good answer.

Mulan Over a year ago

Scott, tbh it's sad we're forced to read between the lines with Object(t) === t. As a dynamic language, I wish JS had a better api for runtime type checks. Eg isObject(t) or better is(Object, t), is(Array, t), is(String, t), ...

Scott Sauyet · Accepted Answer · 2020-07-08 19:47:57Z

This recursive function finds those leaf nodes that match a predicate:

const getMatches = (pred) => (obj) =>
  obj == null ?
    [] 
  : Array .isArray (obj)
    ? obj .flatMap (getMatches (pred))
  : typeof obj == 'object'
    ? Object .values (obj) .flatMap (getMatches (pred))
  : pred (obj)
    ? obj
  : []


const array = [{key1: 'dog', key2: 'cat'},{key1: 'dog', key2: 'cat', key3: [{keyA: 'mouse', keyB: 'https://myURL.com/path/to/file/1', keyC: [{keyA: 'tiger', keyB: 'https://myURL.com/path/to/file/2'}]}]}, {key4: 'dog',key5: 'https://myURL.com/path/to/file/3'}]

const check = val => typeof val == 'string' && val .includes ('https://myURL.com')

console .log (
  getMatches (check) (array)
)

We could modify it to also include the non-leaf nodes as well. But this seems to be what you need for this case.

We simply pass it a predicate to test our node and pass to the resulting function an object or array you want to test. It will return the matching nodes.

You can save the intermediate function, if you like, by simply not passing the object directly:

const getMyUrls = getMatches (check)
// ... later
const urls = getMyUrls (someObject);

Update

The answer from Thank you made me realize that my function could be further simplified. This is a better version of it:

const getMatches = (pred) => (obj) =>
  Object (obj) === obj
    ? Object .values (obj) .flatMap (getMatches (pred))
  : pred (obj)
    ? obj
  : []

Collectives™ on Stack Overflow

Search recursively through a deeply nested object/array and collect values that include a specified string without knowing the key(s)

2 Answers 2

3 Comments

Update

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Update

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related