Unit Testing Replace remote API Server with predefined response

Question

So, I have a server running FastAPI which will make a API call to a remote API upon request. I am developping unit-testing for this application, but here comes the question: Can I, for the purpose of the test, replace a legit remote API server response by a predefined response ?

Example of the tests runned:

from fastapi.testclient import TestClient
from web_api import app

client = TestClient(app)


def test_get_root():
    response = client.get('/')
    assert response.status_code == 200
    assert response.json() == {"running": True}

And the my server

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def home():
    return {"running": True}

This is a simple example, but on other endpoints of my API I would call an external remote API

  def call_api(self, endpoint:str, params:dict):
        url = self.BASEURL + urllib.parse.quote(endpoint)
        try:
            response = requests.get(url, params=params)
            response.raise_for_status()
        except requests.exceptions.HTTPError as error:
            print(error)
        return response

Because I want to test the response of MY API, I would like to replace the remote API with a predefined response.

Also, one user request can end-up in multiple background API requests with transformed pieces of data.

Edit

Here are some more details on the structure of the application:

@app.get("/stuff/.......",  
         # lots of params
        )
def get_stuff_from_things(stuff:list, params):
    api = API(api_key=...)
    # Do some stuff with the params
    things = generate_things_list(params)
    api.search_things(params)
    # Check the result
    # do some other stuff
    return some_response

class API:
    BASE_URL = 'https://api.example.com/'

    def search_things(self, params):
        # Do some stuff
        # like putting stuff in the params
        for s in stuff:
            s.update(self.get_thing(params)) # -> get_thing()
        # Do some more stuff
        return stuff

    # get_thing <- search_things
    def get_thing(self, params...):
        #  Some stuff
        results = self.call_api('something', params) # -> call_api()
        json = results.json()
        #  Some more stuff
        things = []
        for thing in json['things']:
            t = Thing(thing)
            things.append(t)
        return things

    # call_api <- get_thing
    def call_api(self, endpoint:str, params:dict):
        url = self.BASEURL + urllib.parse.quote(endpoint)
        try:
            response = requests.get(url, params=params)
            response.raise_for_status()
        except requests.exceptions.HTTPError as error:
            print(error)
        self.last_response = response
        return response

Nb. That is pseudo-code, I simplified the functions by removing the parameters, etc.

I hope it is clear, thanks for your help.

In FastAPI you'll usually use Depends to get the relevant API client provided into your relevant locations (i.e. in other services or in the view itself). When using pytest you can use the dependency override support: fastapi.tiangolo.com/advanced/testing-dependencies to provide an alternative client (with pre-made responses for example). — MatsLindh
– MatsLindh, Commented Jul 12, 2022 at 20:53
Hum, can you explain a bit more please ? the only example is with common-parameters which just act as a common parameter for both endpoint, I don't really see how I can use Depends in my case (Where I get user input, transform it, and then proceed to different API Calls with pieces of data) — vinalti
– vinalti, Commented Jul 12, 2022 at 21:38
You use something like api_client: ApiClient = Depends(get_api_client) (it's hard to say since you haven't shown anything about how you intend to use the remove API from your views), then override get_api_client in your dependency overview. You can use the Depends system to compose services this way, for example data_service: DataService = Depends(get_data_service) with get_data_service(api_client: ApiClient = Depends(get_api_client): .. etc. That will let you override just parts of the hierarchy in your tests, or larger blocks if necessary (i.e. override get_data_service instead. — MatsLindh
– MatsLindh, Commented Jul 12, 2022 at 21:42

Peter K · Accepted Answer · 2022-07-13 07:03:37Z

1

A complex API method might look like this (please pay attention to the depends mechanism - it is crucial):

import urllib

import requests
from fastapi import FastAPI, Depends

app = FastAPI()


# this can be in a different file
class RemoteCallWrapper:
    def call_api(self, baseurl: str, endpoint: str, params: dict):
        url = baseurl + urllib.parse.quote(endpoint)
        try:
            response = requests.get(url, params=params)
            response.raise_for_status()
        except requests.exceptions.HTTPError as error:
            print(error)
        return response


@app.get("/complex_api")
def calls_other_api(remote_call_wrapper=Depends(RemoteCallWrapper)):
    response = remote_call_wrapper.call_api("https://jsonplaceholder.typicode.com",
                                            "/todos/1", None)
    return {"result": response.json()}

Now, we wish to replace the remote call class. I wrote a helper library that simplifies the replacement for tests - pytest-fastapi-deps:

from fastapi.testclient import TestClient
from mock.mock import Mock
from requests import Response

from web_api import app, RemoteCallWrapper

client = TestClient(app)


class MyRemoteCallWrapper:
    def call_api(self, baseurl: str, endpoint: str, params: dict):
        the_response = Mock(spec=Response)
        the_response.json.return_value = {"my": "response"}
        return the_response


def test_get_root(fastapi_dep):
    with fastapi_dep(app).override({RemoteCallWrapper: MyRemoteCallWrapper}):
        response = client.get('/complex_api')
        assert response.status_code == 200
        assert response.json() == {"result": {"my": "response"}}

You override the RemoteCallWrapper with your MyRemoteCallWrapper implementation for the test, which has the same spec.

As asserted - the response changed to our predefined response.

answered Jul 13, 2022 at 7:03

Peter K

2,5841 gold badge21 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

vinalti Over a year ago

Ah, I see, interesting. And can I use the Depends(... ) method in a sub module? So my get() endpoint, is calling a function which call another function (in a different file and class) which makes the API call. 😅 (I will try to provide more detail when I access a computer)

Peter K Over a year ago

The Depends is part of fastapi mechanism which can only be used for setting API endpoint arguments. So you can have a dependency that has the call logic inside it like in this code example and call it directly from the endpoint method, OR send the depends argument as a parameter to the function you are calling so it can use it. See gist.github.com/pksol/f942ca973a28d4af1c7ff71771acee9c for an alternative.

Paul P · Accepted Answer · 2022-07-15 17:28:26Z

1

It sounds like you'd want to mock your call_api() function.

With a small modification to call_api() (returning the result of .json()), you can easily mock the whole function while calling the endpoint in your tests.

I'll use two files, app.py and test_app.py, to demonstrate how I would do this:

# app.py

import requests
import urllib

from fastapi import FastAPI

app = FastAPI()

def call_api(self, endpoint: str, params: dict):
    url = self.BASEURL + urllib.parse.quote(endpoint)
    try:
        response = requests.get(url, params=params)
        response.raise_for_status()
    except requests.exceptions.HTTPError as error:
        print(error)
    return response.json()  # <-- This is the only change. Makes it easier to test things.

@app.get("/")
def home():
    return {"running": True}

@app.get("/call-api")
def make_call_to_external_api():
    # `endpoint` and `params` could be anything here and could be different
    # depending on the query parameters when calling this endpoint.
    response = call_api(endpoint="something", params={})

    # Do something with the response...
    result = response["some_parameter"]

    return result

# test_app.py

from unittest import mock

from fastapi import status
from fastapi.testclient import TestClient

import app as app_module
from app import app

def test_call_api_endpoint():

    test_response = {
        "some_parameter": "some_value",
        "another_parameter": "another_value",
    }
    # The line below will "replace" the result of `call_api()` with whatever
    # is given in `return_value`. The original function is never executed.
    with mock.patch.object(app_module, "call_api", return_value=test_response) as mock_call:
        with TestClient(app) as client:
            res = client.get("/call-api")

    assert res.status_code == status.HTTP_200_OK
    assert res.json() == "some_value"

    # Make sure the function has been called with the right parameters.
    # This could be dynamic based on how the endpoint has been called.
    mock_call.assert_called_once_with(endpoint="something", params={})

If app.py and test_app.py are in the same directory you can run the tests simply by running pytest inside that directory.

answered Jul 15, 2022 at 17:28

Paul P

4,0372 gold badges15 silver badges28 bronze badges

3 Comments

vinalti Over a year ago

Wow, that's sounds like a very elegant idea. Does it work if call_api is a method from a Class ?

Paul P Over a year ago

@vinalti - Yes, you should be able to use mock.patch.object() on a class method once the class has been imported (unless you are doing something very unusual). If you update your question to include the full class definition and how it's being called, I can update my answer accordingly.

vinalti Over a year ago

OK, so by looking the documentation I adpated myself, I could endup with @mock.patch("api.API.API.call_api", side_effect=call_api_mock) api being the directory containing all files, API.API the file API.py containing the API class. And that work. My new method replace the old one and I can generate a specific response for the endpoint.

Collectives™ on Stack Overflow

Unit Testing Replace remote API Server with predefined response

Edit

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Edit

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related