Python: How can I catch 404 errors with this JSON API?

Question

I'm writing a few scripts that use this API: http://www.robin-stocks.com/en/latest/functions.html

In one of my modules I am trying to run the following functions to grab a bunch of JSON responses and store them in a file.

import robin_stocks as r 

def getAllOptions(symbol):
        
        jsonAllOptions = r.options.find_tradable_options(symbol, expirationDate=None, strikePrice=None, optionType=None, info=None)
        
        with open('allOptions.json', 'w') as json_allop:
            json.dump(jsonAllOptions, json_allop)
        unfilteredOptions = pd.read_json (r'allOptions.json')
        allOptions = unfilteredOptions.loc[unfilteredOptions['rhs_tradability'] == 'untradable']
        return allOptions

and

def storeEachOption(allOptions):
    
    pathPrefix = "data/stockData/availableOptions/"
    pathSuffix = ".json"
    
    for sID in allOptions['id'].tolist():
        jsonCurrentOption = r.options.get_option_market_data_by_id(sID,info=None)
        
        fullPath = pathPrefix + sID + pathSuffix
        print('Writing....' + fullPath)
        with open(fullPath, 'w') as json_oplist:
            json.dump(jsonCurrentOption, json_oplist)
                
        pdCurrentOption = pd.DataFrame(jsonCurrentOption, index=[0])

Then I call them with:

storeEachOption(getAllOptions('TSLA'))

My other modules usually pull these JSON files and convert them into Pandas DataFrames to manipulate them. The trouble is that I am getting a bunch of 404s from Robin Hood's API. I thought this had something to do with my code, but aparently any time you try to get a list of available options through their API, it returns some invalid option IDs, hence the 404s.

I am looking for a way to catch these 404s and send them off to /dev/null. Preferably this would happen before it gets saved to a JSON file, but if not that is okay too. When I call this line of code in storeEachOption(), I start to get 404 errors in my console (Outputted by robin_stocks):

jsonCurrentOption = r.options.get_option_market_data_by_id(sID,info=None)

Console Output:

404 Client Error: Not Found for url: https://api.robinhood.com/marketdata/options/b81b5b6c-3f2b-45f2-a5cf-fdf44e1fed85/

Below I've listed the sample JSON reponses (and what is stored in each file) of a 404 and a normal response.

404 Response:

{"adjusted_mark_price": "", "ask_price": "", "ask_size": "", "bid_price": "", "bid_size": "", "break_even_price": "", "high_price": "", "instrument": "", "last_trade_price": "", "last_trade_size": "", "low_price": "", "mark_price": "", "open_interest": "", "previous_close_date": "", "previous_close_price": "", "volume": "", "chance_of_profit_long": "", "chance_of_profit_short": "", "delta": "", "gamma": "", "implied_volatility": "", "rho": "", "theta": "", "vega": "", "high_fill_rate_buy_price": "", "high_fill_rate_sell_price": "", "low_fill_rate_buy_price": "", "low_fill_rate_sell_price": ""}

Normal Response:

{"adjusted_mark_price": "0.010000", "ask_price": "0.620000", "ask_size": 110, "bid_price": "0.000000", "bid_size": 0, "break_even_price": "4.990000", "high_price": null, "instrument": "https://api.robinhood.com/options/instruments/00b70671-97d2-44cf-ad30-278f1c84ed1e/", "last_trade_price": null, "last_trade_size": null, "low_price": null, "mark_price": "0.310000", "open_interest": 0, "previous_close_date": "2020-07-01", "previous_close_price": "0.010000", "volume": 0, "chance_of_profit_long": "0.020993", "chance_of_profit_short": "0.979007", "delta": "-0.010636", "gamma": "0.011169", "implied_volatility": "0.806188", "rho": "-0.000126", "theta": "-0.000823", "vega": "0.000878", "high_fill_rate_buy_price": "0.450000", "high_fill_rate_sell_price": "0.020000", "low_fill_rate_buy_price": "0.210000", "low_fill_rate_sell_price": "0.270000"}

Any help would be greatly appreciated.

Kyle Dixon · Accepted Answer · 2020-07-06 15:13:25Z

1

Ideally, the API should return a 404. A workaround could be checking the fields of your response:

def storeEachOption(allOptions):
    
    pathPrefix = "data/stockData/availableOptions/"
    pathSuffix = ".json"
    
    for sID in allOptions['id'].tolist():
        jsonCurrentOption = r.options.get_option_market_data_by_id(sID,info=None)
        allEmpty = True
        for key, value in jsonCurrentOption.items():
            if(value!=""):
                allEmpty = False
        if(allEmpty is False):
            fullPath = pathPrefix + sID + pathSuffix
            print('Writing....' + fullPath)
            with open(fullPath, 'w') as json_oplist:
                json.dump(jsonCurrentOption, json_oplist)            
            pdCurrentOption = pd.DataFrame(jsonCurrentOption, index=[0])
        else: 
            print('404 ERROR! :(')
            pass

edited Jul 6, 2020 at 15:13

Kyle Dixon

4942 silver badges12 bronze badges

answered Jul 6, 2020 at 10:46

Parth Shah

1,51517 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kyle Dixon Over a year ago

I'm just now realizing that my 404s are coming from storeEachOption() rather than getAllOptions()...I think your code will still help here. I just need to find a way to implement it in storeEachOption() without saving the same file multiple times. Sorry for the confusion and thanks for the answer.

Kyle Dixon Over a year ago

Code worked just fine in storeEachOption(). Initially I forgot to replace jsonAllOptions with jsonCurrentOption and had an extra indent. Thanks for your help!

Collectives™ on Stack Overflow

Python: How can I catch 404 errors with this JSON API?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related