How to transform an array of file paths into a hierarchical JSON structure

Question

I am trying to create a JSON of a machine's directory structure given an array of all its files and paths.

The array looks something like this:

string[] dirArray = {
"./proc/15/task/15/exe",
"./proc/15/task/15/mounts/mounts.xml",
"./proc/15/task/15/mountinfo/mountinfo.xml",
"./proc/15/task/15/clear_refs/clear_ref.xml",
"./proc/14/loginuid/loginuid.xml",
"./proc/14/sessionid/sessionid.xml",
"./proc/14/coredump_filter/coredump_filter.xml",
"./proc/14/io/io.xml"
}

The target JSON I am aiming for is something like this:

{
   ".":
   {
        "file":
        {
            "name":"fileInRoot.xml"
        },
        "proc":
        {
            "file":
            {
                "name":"fileInProc.xml"
            },
            "15":
            {
                "file":
                {
                    "name":"fileIn15.xml"
                },
                "task":
                {
                    "file":
                    {
                        "name":"fileInTask.xml"
                    },
                    "15":
                    {
                        "file":
                        {
                            "name":"fileInTask.xml"
                        },
                        "mounts":
                        {
                            "file":
                            {
                                "name":"fileInMounts.xml"
                            }
                        },
                        "mountsInfo":
                        {
                            "file":
                            {
                                "name":"fileInMountsInfo.xml"
                            }
                        },
                        "clear_refs":
                        {
                            "file":
                            {
                                "name":"fileInClear_Refs.xml"
                            }
                        }                   
                    }
                }
            },
            "14":
            {
                "file":
                {
                    "name":"fileIn14.xml"
                },
                "task":
                {
                    "file":
                    {
                        "name":"fileInTask.xml"
                    },
                    "loginUid":
                    {
                        "file":
                        {
                            "name":"fileInloginUid.xml"
                        }                   
                    },
                    "sessionid":
                    {
                        "file":
                        {
                            "name":"fileInsessionid.xml"
                        }                   
                    },
                    "coreDump_filter":
                    {
                        "file":
                        {
                            "name":"fileIncoreDump_filter.xml"
                        }                   
                    },
                    "io":
                    {
                        "file":
                        {
                            "name":"fileInIo.xml"
                        }                   
                    }
                }
            }
        }
    }
}

I want to create a JSON file that allows consumer component of the JSON to navigate through this directory structure. I have been trying to use the Directory, File and Path classes but perhaps the best way is to use the java.serializor(?) class to construct the JSON as I loop through the array, parsing its directories and files as i go?

Can you edit your question to show an example of the JSON output you would want to create, given the string array shown above? — Brian Rogers
– Brian Rogers, Commented Oct 28, 2014 at 19:20
Where do the file names come from? Are they just assumed from the directory names (one file per directory)? — Brian Rogers
– Brian Rogers, Commented Oct 29, 2014 at 12:09
They are example file names. I put them in as i noted that the example had no files mentioned.. Example updated — Fearghal
– Fearghal, Commented Oct 29, 2014 at 12:57
OK, so the file names will be part of the dirArray. Can I assume there will always be exactly one file per directory? If not, then the JSON will need to change (you would need an array for the files in the JSON at each directory level). Also, can I assume there will be no directory called "file"? If any directory can be called "file", then you will need to change the JSON to account for that as well since there can't be two keys with the same name at the same level in the JSON. — Brian Rogers
– Brian Rogers, Commented Oct 29, 2014 at 13:19

Brian Rogers · Accepted Answer · 2014-10-30 19:17:25Z

11

+50

I think I would approach this problem by breaking it into two parts. First, we need a way to parse apart the array of directory/file paths and put it into a hierarchical structure. Second, we need to take that structure and turn it into JSON. (I wasn't entirely sure from your question which serializer you wanted to use, so for this answer I will assume that Json.Net is OK.)

For the first part, I would create a Dir class which has a name, a dictionary of child directories (for easy lookup) and a set of files. We can make a method in this class which will parse apart a path and either find or add the appropriate child objects.

class Dir
{
    public string Name { get; set; }
    public Dictionary<string, Dir> Dirs { get; set; }
    public HashSet<string> Files { get; set; }

    public Dir(string name)
    {
        Name = name;
        Dirs = new Dictionary<string, Dir>();
        Files = new HashSet<string>();
    }

    public Dir FindOrCreate(string path, bool mightBeFile = true)
    {
        int i = path.IndexOf('/');
        if (i > -1)
        {
            Dir dir = FindOrCreate(path.Substring(0, i), false);
            return dir.FindOrCreate(path.Substring(i + 1), true);
        }

        if (path == "") return this;

        // if the name is at the end of a path and contains a "." 
        // we assume it is a file (unless it is "." by itself)
        if (mightBeFile && path != "." && path.Contains("."))
        {
            Files.Add(path);
            return this;
        }

        Dir child;
        if (Dirs.ContainsKey(path))
        {
            child = Dirs[path];
        }
        else
        {
            child = new Dir(path);
            Dirs.Add(path, child);
        }
        return child;
    }
}

Using this class, we can easily loop through the dirArray given in your question and make the directory hierarchy:

Dir root = new Dir("");
foreach (string dir in dirArray)
{
    root.FindOrCreate(dir);
}

So at this point, root now has the entire directory hierarchy. If you wanted to, you could simply serialize this object directly with Json.Net to get a reasonable JSON structure. However, it will be a little more verbose than what you described in your question. Here is the JSON that would be produced:

{
  "Name": "",
  "Dirs": {
    ".": {
      "Name": ".",
      "Dirs": {
        "proc": {
          "Name": "proc",
          "Dirs": {
            "15": {
              "Name": "15",
              "Dirs": {
                "task": {
                  "Name": "task",
                  "Dirs": {
                    "15": {
                      "Name": "15",
                      "Dirs": {
                        "exe": {
                          "Name": "exe",
                          "Dirs": {},
                          "Files": []
                        },
                        "mounts": {
                          "Name": "mounts",
                          "Dirs": {},
                          "Files": [
                            "mounts.xml"
                          ]
                        },
                        "mountinfo": {
                          "Name": "mountinfo",
                          "Dirs": {},
                          "Files": [
                            "mountinfo.xml",
                            "moremountinfo.xml"
                          ]
                        },
                        "clear_refs": {
                          "Name": "clear_refs",
                          "Dirs": {},
                          "Files": [
                            "clear_ref.xml"
                          ]
                        }
                      },
                      "Files": []
                    }
                  },
                  "Files": []
                }
              },
              "Files": []
            },
            "14": {
              "Name": "14",
              "Dirs": {
                "loginuid": {
                  "Name": "loginuid",
                  "Dirs": {},
                  "Files": [
                    "loginuid.xml"
                  ]
                },
                "sessionid": {
                  "Name": "sessionid",
                  "Dirs": {},
                  "Files": [
                    "sessionid.xml"
                  ]
                },
                "coredump_filter": {
                  "Name": "coredump_filter",
                  "Dirs": {},
                  "Files": [
                    "coredump_filter.xml"
                  ]
                },
                "io": {
                  "Name": "io",
                  "Dirs": {},
                  "Files": [
                    "io.xml"
                  ]
                }
              },
              "Files": []
            }
          },
          "Files": []
        }
      },
      "Files": []
    }
  },
  "Files": []
}

To get the JSON you are aiming for, we will need a JsonConverter class:

class DirConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(Dir));
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        Dir dir = (Dir)value;
        JObject obj = new JObject();
        if (dir.Files.Count > 0)
        {
            JArray files = new JArray();
            foreach (string name in dir.Files)
            {
                files.Add(new JValue(name));
            }
            obj.Add("list_of_files", files);
        }
        foreach (var kvp in dir.Dirs)
        {
            obj.Add(kvp.Key, JToken.FromObject(kvp.Value, serializer));
        }
        obj.WriteTo(writer);
    }

    public override bool CanRead
    {
        get { return false; }
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

We can serialize the directory hierarchy using the converter like this:

JsonSerializerSettings settings = new JsonSerializerSettings();
settings.Converters.Add(new DirConverter());
settings.Formatting = Formatting.Indented;

string json = JsonConvert.SerializeObject(root, settings);

Here is the output. Note that I changed the "file" properties in your original JSON to arrays, and renamed it to "list_of_files" to accommodate the possibility of multiple files per directory, per your comments. I am also assuming that there will never be an actual directory called "list_of_files". If that is a possibility, you would need to change the name of the files array to something else that will not collide with any of your directory names. (If you ever encounter an error that says "Can not add property list_of_files to Newtonsoft.Json.Linq.JObject. Property with the same name already exists on object" that means you have a directory somewhere in your data with the name "list_of_files".)

{
  ".": {
    "proc": {
      "15": {
        "task": {
          "15": {
            "exe": {},
            "mounts": {
              "list_of_files": [
                "mounts.xml"
              ]
            },
            "mountinfo": {
              "list_of_files": [
                "mountinfo.xml"
              ]
            },
            "clear_refs": {
              "list_of_files": [
                "clear_ref.xml"
              ]
            }
          }
        }
      },
      "14": {
        "loginuid": {
          "list_of_files": [
            "loginuid.xml"
          ]
        },
        "sessionid": {
          "list_of_files": [
            "sessionid.xml"
          ]
        },
        "coredump_filter": {
          "list_of_files": [
            "coredump_filter.xml"
          ]
        },
        "io": {
          "list_of_files": [
            "io.xml"
          ]
        }
      }
    }
  }
}

Fiddle: https://dotnetfiddle.net/ConJiu

edited Oct 30, 2014 at 19:17

answered Oct 29, 2014 at 16:12

Brian Rogers

130k31 gold badges315 silver badges315 bronze badges

Sign up to request clarification or add additional context in comments.

18 Comments

Fearghal Over a year ago

Wow, this looks brilliant, i will take some time to look over it and integrate it and test. Il get back to you when iv done that :)

Fearghal Over a year ago

Hey Brian, i get the following error at JSON.Convert.Serialize(root, settings - Can not add property files to Newtonsoft.Json.Linq.JObject. Property with the same name already exists on object.) I'm guessing its the json has 2 objects with the same name?

Fearghal Over a year ago

In fact it has to be the actual data in the array that is issue as yourexample works...

Fearghal Over a year ago

I cant seem to find the issue, if there is a way to send you my arraylist let me know.

Brian Rogers Over a year ago

From the error, I'm guessing that you have a directory called "files" somewhere in your array. Remember when I asked whether I could assume there would be no such directory and you said I could make that assumption? To fix, change the code in the converter so that it writes a property name of _files (or something else of your choosing) instead. I've edited my answer.

|

Collectives™ on Stack Overflow

How to transform an array of file paths into a hierarchical JSON structure

1 Answer 1

18 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

18 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related