How to update key's value in json file by Python

Question

I want collect data from different pages, but when i get data and append to json file it is structurally bad. i have one dict num and i want to append all data in value as list. for example

num:[1,2,3]

after adding

num:[1,2,3,4,5]

this is my code -->

import requests
import json
from random import randint
from time import sleep
import os


num_list=[]
def req(i):
    url = 'Website_link{}'.format(i)
    x = requests.get(url, headers={
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36",
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate, br",
        "Accept-Language": "ka",
        "Cache-Control": "no-cache",
        "Connection": "keep-alive",
        "Content-Type": "application/json",
        "Host": "api2.myauto.ge",
        "Origin": "https://www.myauto.ge",
        "Pragma": "no-cache",
        "Referer": "https://www.myauto.ge/",
        "sec-ch-ua": "'Not A;Brand';v='99', 'Chromium';v='96', 'Google Chrome';v='96'",
        "sec-ch-ua-platform": "Windows",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "same-site"})

    dict_to_json = json.loads(x.content.decode("utf-8"))
    pretty_json = json.dumps(dict_to_json, indent=4)
    return dict_to_json


def list_append(parse):
    for i in parse["data"]["items"]:
        num_list.append(str(i["client_phone"]))


def write_to_json():
    num_dict={"num":num_list}
    if os.stat("data.json").st_size == 0:
        f = open("data.json", "w")
        json.dump(num_dict, f, indent=2)
        f.close()
    else:
        f = open("data.json","r+")
        data=json.load(f)
        f.close()

        f = open("data.json", "a")

        data["num"]+=([num_list])
        json.dump(data,f,indent=2)
        f.close()



for i in range(1,3):
    sleep(randint(3, 5))

    req_load=req(i)
    list_append(req_load)
    write_to_json()
    num_list=[]

I get such data

  "num": [
    "995579448820",
    "995595334438",
    "995555154416",
    "995577025245",
    "995599485252",
    "995597083412",
    "995598919090",
    "995599401516",
    "995597092860",
    "995551317007",
    "995577063439",
    "995514213366",
    "995592088897",
    "995577539231",
    "995596940606"
  ]
}{
  "num": [
    "995579448820",
    "995595334438",
    "995555154416",
    "995577025245",
    "995599485252",
    "995597083412",
    "995598919090",
    "995599401516",
    "995597092860",
    "995551317007",
    "995577063439",
    "995514213366",
    "995592088897",
    "995577539231",
    "995596940606",
    [
      "995597032700",
      "995595201008",
      "995593620467",
      "995555605884",
      "995555183888",
      "995597777422",
      "995577125074",
      "995595910006",
      "995595910006",
      "995577583288",
      "995599037070",
      "995558830303",
      "995591085255",
      "995597777608",
      "995555788888"
    ]
  ]
}```

Jorge Machado · Accepted Answer · 2022-01-06 18:01:39Z

1

Here is the issue:

data["num"]+=([num_list])

You are assuming that num_list is not an array and you are currently wrapping it inside another one. So you are adding not a list of values but another array to your currently stored inside the json data.

To solve this just change your code to:

data["num"]+=num_list

Besides that, you are writing in append mode and you are updating the json data on memory before writing to disk. Change the mode to write like this:

# change
f = open("data.json", "a")
# to
f = open("data.json", "w")

About the output you shared there is a weird format at the start. Is that right?

edited Jan 6, 2022 at 18:01

answered Jan 6, 2022 at 17:11

Jorge Machado

2021 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

kokiwebaaa Over a year ago

I think you are right, how do you think about fixing?

Jorge Machado Over a year ago

Just remove the brackets. You dont need them there.

kokiwebaaa Over a year ago

I did but same result

kokiwebaaa Over a year ago

no sorry, it is not same but wrong again. can u see? pastebin.com/6YueAyP5

Jorge Machado Over a year ago

Oh! Now I understand the meaning of the begining part of the output you initially shared... That happens because you are using append as the file mode, hence when you write it appends the new array to the end of the file.

|

Collectives™ on Stack Overflow

How to update key's value in json file by Python

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related