Merge Json arrays using Jolt transform in NIFI

Question

The Jolt transform should modify the input as shown, I need to push the data aggregated like that in to the DB based on the Key value as identifier. Input Json :

[
  {
    "Key": "991641500~2167767723",
    "itemNm": "2067875000"
  },
  {
    "Key": "991641500~2167767724",
    "itemNm": "2067875085"
  },
  {
    "Key": "991641500~2167767723",
    "itemNm": "2067875063"
  },
  {
    "Key": "991641500~2167767724",
    "itemNm": "2067875004"
  }
]

Output JSON:

The output JSON should be merged as follows, I need to have the key and the contents of those as elements of a JSon Array.

[
  {
    "Key": "991641500~2167767723",
    "Items": [
      {
        "itemNm": "2067875004"
      },
      {
        "itemNm": "2067875085"
      }
    ]
  },
  {
    "Key": "991641500~2167767724",
    "Items": [
      {
        "itemNm": "2067875000"
      },
      {
        "itemNm": "2067875063"
      }
    ]
  }
]

Barbaros Özhan · Accepted Answer · 2022-09-26 08:37:47Z

2

You can use the following transformation spec :

[
  {
   // group attributes under common objects by their Key(@(1,Key))
    "operation": "shift",
    "spec": {
      "*": {
        "Key": "@(1,Key).&",
        "itemNm": "@(1,Key).items[&1].&"
      }
    }
  },
  {
   // get rid of object labels
    "operation": "shift",
    "spec": {
      "*": ""
    }
  },
  {
    // get rid of redundant null components of the arrays
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": "=recursivelySquashNulls"
    }
  },
  {
   // pick only single one from identical components of each "Key" array 
    "operation": "cardinality",
    "spec": {
      "*": {
        "Key": "ONE"
      }
    }
  }
]

the demo on the site http://jolt-demo.appspot.com/ is :

answered Sep 26, 2022 at 8:37

Barbaros Özhan

65.9k11 gold badges36 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Merge Json arrays using Jolt transform in NIFI

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related