Why and when do we need to flatten JSON objects? [closed]

I realize this is a 5 year old question at this point, but I figured, I'd add my thoughts to it, in case someone runs into a similar use case and finds this useful.

One of the use cases why you would want to flatten a JSON object, is for dynamic template binding via Regular Expression (RegEx) string interpolation. Well wasn't that a mouthful 👀😅? It simply translates to "template filling a string without hardcoding".

Ok Imagine a scenario, you have a template string like this for an email:

Hello {{firstName}},

It is amazing you chose to join our site. We are happy to have you on board. 
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{confirm_url}}.

Welcome aboard

The Team!

Given the following JSON object in memory:

{
   "user" : {
               "prefix"      : "Dr.",
               "firstName"   : "Awah",
               "lastName"    : "Teh",
               "email"       : "[email protected]",
               "address"     : {
                                  "street": "100 Main St",
                                  "city"  : "PleasantVille",
                                  "state" : "NY",
                                  "phone" : "+1-212-555-1212"
                               }
            },
   "meta" : {
               "confirm_url" : "http://superduperubercoolsite.com/confirm/ABC123"
            }
}

it seems super simple to do a Regular Expression replace like so (assuming our email template string was stored in a variable named template and the json object was stored in a variable called templateData:

template = template.replace(new RegExp('{{firstName}}', 'g'), templateData.user.firstName);
template = template.replace(new RegExp('{{confirm_url}}', 'g'), templateData.meta.confirm_url);

Easy right? --> Actually yes! How about this email had 10 templated fields, or you wanted to decouple the template from the code, by storing it in a separate system like SendGrid, where your cool head of marketing can access the template and make changes to the copy-language, without having to call someone from engineering to make changes to the code, test the code and redeploy to production (what a hassle).

This is exactly where flattening of the JSON comes save the day!

Now there are many ways to flatten JSON, I have attached a link to a codepen I wrote that has logic to flatten JSON (actually, I demonstrate two similar but different approaches in the methods flattenJSONIntoKVP and flattenJSONIntoRAW check 'em out!).

That said, there are other implementations out there, and it is worth remembering that the focus on this post is to discuss the WHY JSON flattening could be useful, not the HOW.

Moving on! Assume you flattened the JSON from above (using my implementation that results in key value pairs) to something like this:

[
   { "key": "user.prefix",         "value": "Dr."},
   { "key": "user.firstName",      "value": "Awah"},
   { "key": "user.lastName",       "value": "Teh"},
   { "key": "user.email",          "value": "[email protected]"},
   { "key": "user.address.street", "value": "100 Main St"},
   { "key": "user.address.city",   "value": "{PleasantVille"},
   { "key": "user.address.state",  "value": "NY"},
   { "key": "user.address.phone",  "value": "+1-212-555-1212"},
   { "key": "meta.confirm_url",    "value": "http://superduperubercoolsite.com/confirm/ABC123"},
]

Now, my friend, you are cooking with GAS!

Why, because now you can dynamically interpolate the template string with values from the JSON object without giving too much worry to the structure of the JSON (if it changes due to the application evolving, you don't have to also remember to come down here and change this interpolation code -- you simply just have to update the email template itself, which mind you, is on SendGrid [per this example]).

So how to do it you say?: Simple, iteratively. Let's assume that flattened from above was stored in a variable called flatJSON:

///Notice how I use Javascript's native string interpolation to create my RegExp

///Also note that I am replacing the dot (.) in my flattened JSON variable names with a double underscore (__), I only do this because my intended target is SendGrid, and I don't believe it likes dots in its template placeholders.
flatJSON.forEach(kvp=>template = template.replace(new RegExp(`{{${kvp.key.replace(/\./g, '__'}}}`, 'g'), kvp.value));

That's it, one line of code to replace possibly 10 or even hundreds or even thousands (ok.. maybe not thousands, but you get the point).

Ohh! almost forgot, we need to update our template string.

Notice how now, in our new templated string we can use a somewhat FQDN style variable to map back to our original JSON (Ideally if SendGrid supported dots in their template placeholders, this would look super sweet but alas, can't always win everything!😭.

Hello {{user__firstName}},

It is amazing you chose to join our site. We are happy to have you on board. 
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{meta__confirm_url}}.

Welcome aboard {{user__prefix}} {{user__lastName}}!

The Team!

Et Voila!

Just like that, we have accomplished some good here today; we have:

1. Answered the WHY of flattening JSON objects
2. We dibble-dabbled into the how, with the codepen example
3. And we even overviewed a use case where taking advantage of JSON flattening can help you write durable dynamic code, that evolves as your underlying object structures change -- and that doesn't require you to leverage the big bad ugly eval method (we can talk about big bad ugly eval on another post).

Here's a simple scenario: In a web app you have an HTTP POST that is updating a complex relational object.

POST
update=1
&user.id=12345
&[email protected]
&user.profile.name=Mr. Test
&user.profile.age=42
&[email protected]
&[email protected]
&[email protected]
&user.profile.skill.0.id=100
&user.profile.skill.0.name=javascript
&user.profile.skill.1.id=200
&user.profile.skill.1.name=piano

Everything is already in a flat structure, so why not have a simple one-to-one binding? If you had a list of constraints or security requirements that you needed to enforce you could validate them by searching directly on the sorted key list.

Flat structures are easier for people to understand and work with there's even some cross-over with database de-normalisation. It also allows for context specific security and constraints to be implemented in a readable, but more verbose way.

When showing a user's view in full you may want to hide the display of the primary key ID for the user's list of skills.

"user.profile.skill.#.id": { hidden: true, readonly: true }

But when looking directly at a skill (to possibly edit it as an administrator) you may want to see the ID.

"skill.id": { readonly: true }

If you were writing a user-centric/self-service type CMS application you'd get more users on board and able to contribute using a straightforward flat model (flat abstraction of the underlying nested relational model) than you would with just the nested model.

TLDR: Flat is easier to read than nested. While programmers can handle nested schemas, recursive parsing and processing; end-users and admins usually prefer that part abstracted away.

Flattening JSON means that you convert a potentially hierarchical JSON into a flat one, i.e., an object of key-value pairs where the values are non-composite.

Since the process of flattening is removing the hierarchical nature of the JSON involved, flattening it only makes sense if there is a hierarchy, that is, your object has at least a composite attribute and does not contain a cycle.

Therefore all examples that one can provide where flattening is useful involves an undesired hierarchy in the input JSON.

Think of a JSON for example which contains some composite attributes, such as geographical coordinates, dates, etc, like:

{
    date: { year: 2025, month: 05, day: 24 },
    coordinate: { x: 23.34, y:47.23 },
    name: { first_name: 'John', last_name: 'Doe' }
}

So far so good. And how would you loop the attributes? In the current format you need to implement an algorithm that traverses a graph (a tree to be precise). Instead, you would like a for loop. Or you want to get all attributes. Or you want to get the value of an attribute.

Of course, while flattening a JSON you may have to deal with the issue of having attributes of the same name at different parts of the hierarchy. The solution to such problems will always depend on your exact needs.

Iterate JSONObject and check for existence of JSONObject or String and invoke this repeatedly to parse entire JSON

Below code works fine public JSONObject parseJSON(JSONObject jsonObject,String prefix) { if (parsedJSONObject == null) parsedJSONObject = new JSONObject(); Iterator iteration = jsonObject.keys();

                while(iteration.hasNext()) {
                    String key = (String)iteration.next();
                    System.out.println(key);
                    String correctedPrefix = (prefix != null && prefix.length()> 0)?(prefix+"."+key):(key);
                    if(jsonObject.get(key) instanceof JSONObject) {
        //              System.out.println("JSON Object" + key);
                        parseJSON((JSONObject)jsonObject.get(key), correctedPrefix);
                    }else if(jsonObject.get(key) instanceof JSONArray) {
                        System.out.println(" JSON Array " + key);
                    }else if(jsonObject.get(key) instanceof String) {
        //              System.out.println(" String "+ key);
                        parsedJSONObject.put(correctedPrefix, jsonObject.get(key));
                    }
                    
                }
                return parsedJSONObject;

Flattening JSON Objects is the easiest solution to editing values anywhere on a JSON Object without having to rewrite the entire structure of the JSON Object yourself.

{
  "document": {
    "dataNodeA": [
      {
        "datalistelement": "element1"
      },
      {
        "datalistelement": "element2"
      }
    ]
  }
}

if you try to edit the value of the second entry in that list element from "element2" to for example "foobar" you need to flatten it out otherwise you need to rewrite every list on the path of the actual keys value you try to edit/update because thats unfortunately how lists work in JSON you need to rewrite the entire list otherwise the structure/order of the list element you just edited changes. Flattening works the same way.

{
  "document.dataNodeA.0.datalistelement": "element1",
  "document.dataNodeA.1.datalistelement": "element2"
}

Lets think about JSON Objects for a second: JSON Objects are nothing else than a tree i.e. like a filesystem.

The problem is you can't edit them without being in the same directory/node. This is where flattening comes in handy. Flattening is nothing else than creating filepaths for files. Flattening out a dictionary/JSON Object/tree gives you the chance to easier edit just what you want and let the system/program do the rest.

unfortunately if you search how to update value in a nested JSON Object. you do not find this post, nor an explaination what flattening is and how it works.

The answer how to do it: You flatten the json object, then you update the value of that one specific key and then you unflatten the flattened JSON