Python Scraping data

Question

have the following code. It is very easy to execute, only my problem is, I get it entered 11 times at once. So when I execute this code it copies the content 11 times into the database.

row 1
row 2
row 3 
...
row 11

This is what I try to output :

{
    'note': '',
    'money': 76682.1200000005,
    'main': True,
    'deleted': False,
    '_id': '5e3e2bc4667b8d5fc053e9a1',
    'ownerCharacter': 
        {
            '_id': '5e3ddb08667b8d5fc052903d',
            'name': 'Name'
        }, 
    'bank': 
        {
            '_id': '5b36b968614626df322e5b75',
            'imageUrl': 'urlToIMG',
            'name': 'Savings Bank Paleto Bay'
        }, 
    'bankAccountType': 
        {
            'type': 'private',
            'title': '',
            'interestRate': 0.01,
            '_id': '5b36b968614626df322e5b76',
            'name': 'Giro-Konto',
            'bank': '5b36b968614626df322e5b75',
            '__v': 0
        }, 
    'vban': '460907', 
    'pin': '1234', 
    '__v': 0
}

And this is the code:

def getData():
    databaseConn = dbConnect()
    cursor = databaseConn.cursor()

    delete = "TRUNCATE TABLE bank"
    try:
        cursor.execute(delete)
        databaseConn.commit()
    except:
        print("Delete error")

    money = json.loads(makeRequest("URL",  authToken, True).text)
    
    for amount in money:
        
        geld = str(money["money"])
        person = money["ownerCharacter"]["name"]

        sql = "INSERT INTO bank (menge,name) VALUES (%s,%s)"
        val = (geld,person)
        try:
            cursor.execute(sql, val)
            databaseConn.commit()
        except:
            print("Error Database")
    dbClose(databaseConn, cursor)

I just need it once the output not 11 times in a row. What am I doing wrong?

en_lorithai · Accepted Answer · 2020-08-27 13:15:42Z

1

You are inserting for each element in the "money" dictionary as it is in a loop. Do you need to loop over multiple accounts in the same json file? if not the loops shouldn't be needed

def getData():
    databaseConn = dbConnect()
    cursor = databaseConn.cursor()

    delete = "TRUNCATE TABLE bank"
    try:
        cursor.execute(delete)
        databaseConn.commit()
    except:
        print("Delete error")

    money = json.loads(makeRequest("URL",  authToken, True).text)
    
        
    geld = str(money["money"])
    person = money["ownerCharacter"]["name"]

    sql = "INSERT INTO bank (menge,name) VALUES (%s,%s)"
    val = (geld,person)
    try:
        cursor.execute(sql, val)
        databaseConn.commit()
    except:
        print("Error Database")
    dbClose(databaseConn, cursor)

edited Aug 27, 2020 at 13:15

answered Aug 27, 2020 at 12:53

en_lorithai

1,29010 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

mKeey Over a year ago

Thanks for the Answer, the problem with this code i'm getting an error ``` geld = str(money["money"]) ^ IndentationError: unexpected indent ``` And no I just need it once on one account.

en_lorithai Over a year ago

@mKeey Indentation error just means that there are some spaces in my code that i added. Just remove the for loop in your code and it should work.

mKeey Over a year ago

That's what I did, but Visual Studio Code keeps giving me this error I had already written. And here my knowledge ends.

en_lorithai Over a year ago

@mKeey i edited the answer to try and get the correct spaces, try and copy that

mKeey Over a year ago

I never saw this kind of error because of a space?! Anyways it works fine right now thanks for your Help!

kotolgifmemana · Accepted Answer · 2020-08-27 12:53:45Z

1

This is where the problem is

for amount in money:

You are iterating over the dictionary keys. And it looks like you have 11 keys in that dictionary. This means you are performing this action

geld = str(money["money"])

11 times. That is why you insert it 11 times in your database.

Dont do the for loop. Just insert it by accessing once

geld = str(money["money"])

answered Aug 27, 2020 at 12:53

kotolgifmemana

693 bronze badges

Collectives™ on Stack Overflow

Python Scraping data

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related