github code search api returns html instead of json

Question

I want to get some data from github API through python:

import requests
headers = {'User-Agent': 'Awesome-Octocat-App', 'Accept': 'application/vnd.github.preview+json'}
link = "https://github.com/search?q=chembl+created:>=2000"
r = requests.get(link, headers=headers)

and it looks like everything went fine:

r.ok
>>> True

So I would expect to have json in response:

r.json()

But this throws an exception:

JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)

Unfortunately what I have is html:

r.content

<!DOCTYPE html>
<html>
  <head prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb# githubog: http://ogp.me/ns/fb/githubog#">
  <meta charset='utf-8'>
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
...

This html contains all repositories I'm looking for but I need json not html. Why?

Martijn Pieters · Accepted Answer · 2013-09-04 10:52:51Z

3

You need to use the actual API to get JSON content. http://github.com/search is the regular HTML frontend. You probably wanted to search for repositories:

import requests

headers = {'User-Agent': 'Awesome-Octocat-App', 'Accept': 'application/vnd.github.preview+json'}
link = "https://api.github.com/search/repositories"
query = {'q': 'chembl created:>=2000'}
r = requests.get(link, headers=headers, params=query)

This gives me:

>>> r = requests.get(link, headers=headers, params=query)
>>> r.ok
True
>>> r.json().keys()
[u'total_count', u'items']
>>> r.json()['total_count']
17

answered Sep 4, 2013 at 10:52

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

James Dennes · Accepted Answer · 2013-09-04 10:50:24Z

2

You are using the URL:

https://github.com/search?q=chembl+created:>=2000

You should be using the URL:

https://api.github.com/search/repositories?q=chembl+created:>=2000.

Here is the documentation:

http://developer.github.com/v3/search/#search-repositories

answered Sep 4, 2013 at 10:50

James Dennes

511 silver badge4 bronze badges

Collectives™ on Stack Overflow

github code search api returns html instead of json

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related