Powershell - Parsing inconsistent Json Data

Question

I got a CSV file with an ID and Json data. I need to parse the data into a CustomerData class.

However, I am having problem accessing the value for a given key

I got this CSV file. The Json might contain some keys in some records and might not be there in the other. Basically just fields that have been updated exists.

ID;CHANGES
713422;[{"key":"zipCode","updatedValue":""},,{"key":"language","updatedValue":"EN"},{"key":"coaddress","updatedValue":""},{"key":"SSN","updatedValue":""},{"key":"msisdn","updatedValue":"1114455789"}]
114365;[{"key":"city","previousValue":"New York","updatedValue":"Palm City"},{"key":"zipcode","previousValue":"100012","updatedValue":"02118"},{"key":"coaddress","updatedValue":""},{"key":"streetaddress","previousValue":"9253 Del Monte Road"updatedValue":"90 Kent Ave"},{"key":"SSN","updatedValue":""},{"key":"companyName","previousValue":"Nutrics","updatedValue":"NutriTiger"}]
114365;[{"key":"zipCode","updatedValue":""},{"key":"coaddress","updatedValue":""},{"key":"SSN","updatedValue":""},{"key":"companyName","previousValue":"NutriTiger","updatedValue":"Nutri-Tiger"}]
713422;[{"key":"zipCode","updatedValue":""},{"key":"coaddress","updatedValue":"Roady Road"},{"key":"SSN","updatedValue":""},{"key":"msisdn","updatedValue":""}]

What I want to do is something like this. There is 8 keys in the Json data that might occur and I need to check if it exists and grab the value if it does.

$city
if (TryParse(Json.city.GetValue, $city )
{
    CustomerData.$city = Json.city.GetValue
}

Here is what I am right now. I am stuck in the If statement, where I try to access the $jsondata in different ways. I've tried $jsonData.city $jSonData.getValue('city') Different kinds of piping the $jsonData and do selects

But I just can't seem to get the value for a given key.

What I eventually need is to build up a new CSV file where I search for the ID and then update the given field, like the city if it has been updated.

class CustomerData
{
    [int]$Id = 0

    [string]$companyName = ""
    [string]$ssn = ""
    [string]$msisdn = ""
    [string]$language = ""

    [string]$city = ""
    [string]$coaddress = ""
    [string]$streetaddress = ""
    [string]$zipCode = ""
}

$inputdata = ".\Testdata.csv"

$iso8859_1 = [System.Text.Encoding]::GetEncoding('ISO-8859-1')

    $reader = New-Object -TypeName System.IO.StreamReader($inputdata, $iso8859_1)
    [int]$counter = 0

    while ($line = $reader.ReadLine() )
    {   

        if ($counter -gt 0)
        {
            $lines_split = $line.Split(';')
            CustomerData.$Id = $lines[0]
            $jsondata = ConvertFrom-Json $lines[1] 
        }
        $counter++;
    }
    $reader.Close()

Thanks for the tip! I can't use it for this, but I will definitely check it out. Looks like something that's nice to have in the toolbox. — SuperKyllingen
– SuperKyllingen, Commented Nov 4, 2018 at 18:37

mklement0 · Accepted Answer · 2018-11-04 14:46:33Z

Andrei Odegov's answer contains helpful improvements to your code.
As currently posted, the primary problem with your code is a variable-name confusion: You save the fields to array $lines_split, and then mistakenly access a different variable, $lines.

Generally, speaking, at least by default there is no need for something akin to TryParse(): you can simply access a property path, and if it doesn't exist, $null will be returned:

$bar = ('{ "foo": 1 }' | ConvertFrom-Json).bar  # $bar will be $null

If Set-StrictMode -Version 2 or higher is in effect, accessing a nonexistent property does cause a (statement-terminating) error; using try / catch is the simplest way to handle this, which also allows you to specify a default value:

$bar = try { ('{ "foo": 1 }' | ConvertFrom-Json).bar } catch { 0 }  # $bar will be 0

That said, your particular JSON input is structured in a way that the target property names are in JSON property values, which prevents direct access such as $obj.City, so you need to filter the objects.

However, instead of looking for specific values, consider iterating over them:

# Instantiate a new customer-data object.
$customer = [CustomerData]::new()

# Parse the JSON data in custom objects ([pscustomobject]).
# In this case, you'll get a single-element array containing an array
# of [pscustomobject] instances.
$jsondata = ConvertFrom-Json '[{"key":"city","previousValue":"New York","updatedValue":"Palm City"},{"key":"zipcode","previousValue":"100012","updatedValue":"02118"},{"key":"coaddress","updatedValue":""},{"key":"streetaddress","previousValue":"9253 Del Monte Road","updatedValue":"90 Kent Ave"},{"key":"SSN","updatedValue":""},{"key":"companyName","previousValue":"Nutrics","updatedValue":"NutriTiger"}]'


# Loop over all custom objects and update the corresponding
# customer-data properties.
foreach($obj in $jsonData) {
   $propName = $obj.key
   $customer.$propName = $obj.UpdatedValue
}

No worries, @SuperKyllingen. Glad to hear the FAQ was helpful. Enjoy the experimenting.

Andrei Odegov · Accepted Answer · 2018-11-03 20:17:40Z

2

Try to get some ideas from the following snippet:

$reader = [System.IO.StringReader]::new(@'
ID;CHANGES
713422;[{"key":"zipCode","updatedValue":";"},{"key":"language","updatedValue":"EN"},{"key":"coaddress","updatedValue":""},{"key":"SSN","updatedValue":""},{"key":"msisdn","updatedValue":"1114455789"}]
114365;[{"key":"city","previousValue":"New York","updatedValue":"Palm City"},{"key":"zipcode","previousValue":"100012","updatedValue":"02118"},{"key":"coaddress","updatedValue":""},{"key":"streetaddress","previousValue":"9253 Del Monte Road","updatedValue":"90 Kent Ave"},{"key":"SSN","updatedValue":""},{"key":"companyName","previousValue":"Nutrics","updatedValue":"NutriTiger"}]
114365;[{"key":"zipCode","updatedValue":""},{"key":"coaddress","updatedValue":""},{"key":"SSN","updatedValue":""},{"key":"companyName","previousValue":"NutriTiger","updatedValue":"Nutri-Tiger"}]
713422;[{"key":"zipCode","updatedValue":""},{"key":"coaddress","updatedValue":"Roady Road"},{"key":"SSN","updatedValue":""},{"key":"msisdn","updatedValue":""}]
'@)
class CustomerData
{
  [int]$Id = 0

  [string]$companyName = ""
  [string]$ssn = ""
  [string]$msisdn = ""
  [string]$language = ""

  [string]$city = ""
  [string]$coaddress = ""
  [string]$streetaddress = ""
  [string]$zipCode = ""
}
$props = [CustomerData].GetProperties() | ForEach-Object Name
$counter = 0
while ( $line = $reader.ReadLine() )
{
  if( $counter -gt 0 )
  {
    $cd = [CustomerData]::new()
    $cd.Id, $ch = $line.Split(';')
    $ch = $ch -join ';'
    $jsondata = ConvertFrom-Json $ch
    $jsondata | Where-Object { $props -contains $_.key } |
      ForEach-Object { $cd."$($_.key)" = $_.updatedValue }
    $cd
  }
  $counter++
}
$reader.Close()

answered Nov 3, 2018 at 20:17

Andrei Odegov

3,5722 gold badges18 silver badges26 bronze badges

1 Comment

SuperKyllingen Over a year ago

This works! I've been playing around with it for a little bit and it does exactly what I want. If my idea to solve this is a good solution is a different matter, though hehe. I don't quite understand everything that goes on in the second to last line in the If statement (Where-object {$props -contains $_.key} etc). I understand the code, but I right now I won't be able to replicate this or use this later in a different case. Need to google and read up. :) Thank you!

Collectives™ on Stack Overflow

Powershell - Parsing inconsistent Json Data

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related