Python Imgur API JSON output to CSV

Question

I'm pretty new to Python and coding in general but I somehow figured out how to string together an the Imgur API to give me a JSON out put file.

My end goal is to be able to put the file into an Excel with some other already collected data so I'd like to be able to convert the API output to a CSV.

So far my only solution is to take the JSON output and to put it into an online converter (I've tried what other people said online but I never was able to get it working.)

Here's a sample output of the JSON (I'm pretty sure it doesn't have any nested parts):

{"status": 200, "data": {"in_gallery": false, "deletehash": "pfSgnqtf9eh4r2B", "layout": "blog", "description": null, "title": null, "cover_height": 177, "views": 0, "privacy": "public", "cover": "P1tTbZw", "images_count": 2, "datetime": 1468959627, "account_url": "JosephL32", "favorite": false, "cover_width": 222, "link": "http://imgur.com/a/3I3H7", "is_ad": false, "section": null, "images": [{"datetime": 1468959628, "bandwidth": 0, "nsfw": null, "vote": null, "id": "P1tTbZw", "account_id": null, "in_gallery": false, "title": null, "section": null, "width": 222, "size": 48248, "type": "image/png", "is_ad": false, "deletehash": "mGqP4DFgDtBZG8Y", "description": null, "views": 0, "link": "http://i.imgur.com/P1tTbZw.png", "height": 177, "name": "Screen Shot 2016-07-19 at 4.20.05 PM", "favorite": false, "account_url": null, "animated": false}, {"datetime": 1468959630, "bandwidth": 0, "nsfw": null, "vote": null, "id": "5zGa1go", "account_id": null, "in_gallery": false, "title": null, "section": null, "width": 221, "size": 74481, "type": "image/png", "is_ad": false, "deletehash": "LnJxl5rltxsIFl2", "description": null, "views": 0, "link": "http://i.imgur.com/5zGa1go.png", "height": 152, "name": "Screen Shot 2016-07-19 at 4.19.59 PM", "favorite": false, "account_url": null, "animated": false}], "nsfw": null, "id": "3I3H7", "account_id": 37918982}, "success": true}

To sum it up I'm looking for the python code I can insert after getting the JSON data to save it as a CSV file.

Simon Bilsky-Rollins · Accepted Answer · 2016-07-19 22:10:18Z

0

This shouldn't be too hard as both JSON and CSV file structures can be represented fairly easily in Python using dictionaries. First, however, I should point out that the JSON data is in fact nested, which can be seen if we format it nicely:

{
  "status": 200,
  "data":
  {
    "in_gallery": false,
    "deletehash": "pfSgnqtf9eh4r2B",
    "layout": "blog",
    "description": null,
    "title": null,
    "cover_height": 177,
    "views": 0,
    "privacy": "public",
    "cover": "P1tTbZw",
    "images_count": 2,
    "datetime": 1468959627,
    "account_url": "JosephL32",
    "favorite": false,
    "cover_width": 222,
    "link": "http://imgur.com/a/3I3H7",
    "is_ad": false,
    "section": null,
    "images":
    [
      {
        "datetime": 1468959628,
        "bandwidth": 0,
        "nsfw": null,
        "vote": null,
        "id": "P1tTbZw",
        "account_id": null,
        "in_gallery": false,
        "title": null,
        "section": null,
        "width": 222,
        "size": 48248,
        "type": "image/png",
        "is_ad": false,
        "deletehash": "mGqP4DFgDtBZG8Y",
        "description": null,
        "views": 0,
        "link": "http://i.imgur.com/P1tTbZw.png",
        "height": 177,
        "name": "Screen Shot 2016-07-19 at 4.20.05 PM",
        "favorite": false,
        "account_url": null,
        "animated": false
      },
      {
        "datetime": 1468959630,
        "bandwidth": 0,
        "nsfw": null,
        "vote": null,
        "id": "5zGa1go",
        "account_id": null,
        "in_gallery": false,
        "title": null,
        "section": null,
        "width": 221,
        "size": 74481,
        "type": "image/png",
        "is_ad": false,
        "deletehash": "LnJxl5rltxsIFl2",
        "description": null, "views": 0,
        "link": "http://i.imgur.com/5zGa1go.png",
        "height": 152,
        "name": "Screen Shot 2016-07-19 at 4.19.59 PM",
        "favorite": false,
        "account_url": null,
        "animated": false
      }
    ],
    "nsfw": null,
    "id": "3I3H7",
    "account_id": 37918982
  },
  "success": true
}

The nested structure of this JSON will make this problem a little more tricky since it contains two image dictionaries that each have the same set of properties. It's not clear exactly how you want your final data to be formatted, but I will approach this with a goal of outputting a CSV with one row for each image in the album.

The first thing we need to do is convert the JSON string to a Python dictionary:

import json

json_dict = json.loads(raw_json)

Now we can access all the important data in json_dict['data'] and a list of the images in the album (represented in Python as dictionaries) in json_dict['data']['images'].

Let's say you want to output a CSV with datetime, title, link, and name attributes for each image, as well as datetime, title, link, and images_count attributes for the album that each image appears in. We will first open a new CSV file and write a row of column headers at the top of it, then iterate through the images and write a row for each one.

import csv

# Open new CSV file
with open("output.csv", "w") as csv_file:
    writer = csv.writer(csv_file)

    # Write CSV headers
    writer.writerow(["datetime", "title", "link", "name", "album_datetime",
                     "album_title", "album_link", "album_images_count"])

    # Write data to CSV for each image
    album_data = json_dict['data']
    images_data = json_dict['data']['images']
    for image in images_data:
        writer.writerow([image['datetime'],
                           image['title'],
                           image['link'],
                           image['name'],
                           album_data['title'],
                           album_data['datetime'],
                           album_data['link'],
                           album_data['images_count']])

answered Jul 19, 2016 at 22:10

Simon Bilsky-Rollins

5455 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Joseph Levy Over a year ago

Thanks for the quick and detailed response (the way you set up the CSV output is exactly what I'm looking for)- still having some trouble though. I'm on python 2.7. I'm getting an error pointing to the json decoder when running it: ` File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())`

Simon Bilsky-Rollins Over a year ago

No problem! I tested my code on Python 2.7 on OS X, which looks like it's the same platform you're on, and I didn't get any errors. Are you sure that's the entire error message though? Usually there's a more informative message below the stack trace like TypeError: ....

Simon Bilsky-Rollins Over a year ago

Update: googling the error text yields this StackOverflow post. Make sure that what you're passing into json.loads() is actually a string and not something else.

Collectives™ on Stack Overflow

Python Imgur API JSON output to CSV

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related