Using argparse to open a json file?

Question

I recently just started using argparse and I'm trying to use it to open a json file among other things like outputting into a csv file.

import argparse
import os
import json
import csv
import pandas as pd

parser = argparse.ArgumentParser(description='Json to Csv')
parser.add_argument('file', type=argparse.FileType('r'))
#parser.add_argument('file', type=str)
args = parser.parse_args()

with args.file as json_data:

print(json_data)
#argparse_dict = vars(args)
#argparse_dict.update(json_data)


baseline_dict = {}
try:
    baseline_dict = json.load(json_data)
except:
    print("JSON Baseline File {}: Unable to LOAD")

results_dict = baseline_dict["results"]

"""with open(args.file) as json_data:
baseline_dict = {}
try:
    baseline_dict = json.load(json_data)
except:
    print("JSON Baseline File {}: Unable to LOAD")"""

#Turns the new dictionary into a dataframe df = pd.DataFrame(results_dict)

When I try to open the json file using argparse in terminal I get this error. I know it's pulling the json file because I can get it to run before calling it as json_data.

with args.file as json_data:
JSON Baseline File {}: Unable to LOAD
Traceback (most recent call last):

Not part of your current problem, but your whole for loop makes no sense. If you want the value for a key named "results", just do results_dict = baseline_dict["results"] (or results_dict = baseline_dict.get("results", {}) if the "results" key might not exist and you want to silently use an empty dict in that case) and remove all the code from results_dict = {} to del (baseline_dict), which has multiple errors in it. — ShadowRanger
– ShadowRanger, Commented Jun 24, 2020 at 14:45
Drop the FileType use; let argparse give you a filename (string), which you can then with open and load. With the modern preference for with open, FileType is obsolete. — hpaulj
– hpaulj, Commented Jun 24, 2020 at 15:57
@hpaulj: I disagree; having argparse catch the exceptions and convert to a useful error message for you is super-convenient. Sure, you need to add with yourself, but for scripts (as opposed to libraries), you have top-level control so the issues are fairly limited. — ShadowRanger
– ShadowRanger, Commented Jun 24, 2020 at 16:57
Note that it's the object returned by open, not the call to open itself, that lets the with statement ensure it gets closed. — chepner
– chepner, Commented Jun 24, 2020 at 17:40
Trying both ways with open and without filetype or with filetype and just with returns: JSON Baseline File {}: Unable to LOAD Traceback (most recent call last): See edits above. — YellowJacket
– YellowJacket, Commented Jun 24, 2020 at 18:04

ShadowRanger · Accepted Answer · 2020-06-24 14:39:06Z

1

Because you used argparse.FileType as the argument type, args.file is already an open file handle, exactly as if it were returned by open. If you want to make sure it's closed deterministically (and to give it an unqualified name), all you need is:

with args.file as json_data:

or just:

with args.file:

if aliasing to a local name isn't needed.

answered Jun 24, 2020 at 14:39

ShadowRanger

158k12 gold badges222 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

YellowJacket Over a year ago

So when I originally opened the file without argparse I had no problem using json.load I did your suggestion which got rid of the previous error and used: with args.file as json_data: but json.load(json_data) doesn't work

ShadowRanger Over a year ago

@YellowJacket: Never use the phrase "doesn't work" here without explaining (with minimal reproducible example); I have literally no idea how to help you when I have no idea what's wrong beyond "it's wrong".

hpaulj · Accepted Answer · 2020-06-24 20:09:10Z

1

A script that can load a file two ways:

import argparse, json

parser = argparse.ArgumentParser()
parser.add_argument('--file', type=argparse.FileType('r'))
parser.add_argument('--name')
args = parser.parse_args()
print(args)

if args.file is not None:
    print(json.load(args.file))
    args.file.close()
    
if args.name is not None:
    with open(args.name) as f:
         print(json.load(f))

sample runs:

1307:~/mypy$ python3 stack62557562.py 
Namespace(file=None, name=None)
1307:~/mypy$ python3 stack62557562.py --file foo.json
Namespace(file=<_io.TextIOWrapper name='foo.json' mode='r' encoding='UTF-8'>, name=None)
{'foo': 12, 'bar': 'twelve'}
1307:~/mypy$ python3 stack62557562.py --name foo.json
Namespace(file=None, name='foo.json')
{'foo': 12, 'bar': 'twelve'}
1307:~/mypy$ python3 stack62557562.py --name foo.json --file foo.json
Namespace(file=<_io.TextIOWrapper name='foo.json' mode='r' encoding='UTF-8'>, name='foo.json')
{'foo': 12, 'bar': 'twelve'}
{'foo': 12, 'bar': 'twelve'}

answered Jun 24, 2020 at 20:09

hpaulj

233k14 gold badges260 silver badges392 bronze badges

2 Comments

YellowJacket Over a year ago

I tried your solution exactly, and get this for both solutions. raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Should I make a new question on stackoverflow for this?

hpaulj Over a year ago

I get that error if I try a second json.load on an open file. The first consumes the whole file, leaving it positioned at the end.

Collectives™ on Stack Overflow

Using argparse to open a json file?

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related