29

I'm looking for a possibility to use uvicorn.run() with a FastAPI app but without uvicorn.run() is blocking the thread. I already tried to use processes, subprocessesand threads but nothing worked. My problem is that I want to start the Server from another process that should go on with other tasks after starting the server. Additinally I have problems closing the server like this from another process.

Has anyone an idea how to use uvicorn.run() non blocking and how to stop it from another process?

Improve this question
1

5 Answers 5

Reset to default
47

Approach given by @HadiAlqattan will not work because uvicorn.run expects to be run in the main thread. Errors such as signal only works in main thread will be raised.

Correct approach is:

import contextlib
import time
import threading
import uvicorn

class Server(uvicorn.Server):
    def install_signal_handlers(self):
        pass

    @contextlib.contextmanager
    def run_in_thread(self):
        thread = threading.Thread(target=self.run)
        thread.start()
        try:
            while not self.started:
                time.sleep(1e-3)
            yield
        finally:
            self.should_exit = True
            thread.join()

config = uvicorn.Config("example:app", host="127.0.0.1", port=5000, log_level="info")
server = Server(config=config)

with server.run_in_thread():
    # Server is started.
    ...
    # Server will be stopped once code put here is completed
    ...

# Server stopped.

Very handy to run a live test server locally using a pytest fixture:

# conftest.py
import pytest

@pytest.fixture(scope="session")
def server():
    server = ...
    with server.run_in_thread():
        yield

Credits: uvicorn#742 by florimondmanca

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I'm getting that Config is undefined.. Should it be uvicorn.Config ?
@StealthRabbi Should be fixed now, thanks
Caution: subclassed uvicorn.Server seems to ignore "workers" config value and handles requests only in a single process.
actually uvicorn.Server is a single proces by design. It is uvicorn.run that actually utilizes the workers config and starts multiple server.run via uvicorn.supervisors.Multiprocess
5

This is an alternate version which works and was inspired by Aponace uvicorn#1103. The uvicorn maintainers want more community engagement with this issue, so if you are experiencing it, please join the conversation.

Example conftest.py file.

import pytest
from fastapi.testclient import TestClient
from app.main import app
import multiprocessing
from uvicorn import Config, Server


class UvicornServer(multiprocessing.Process):

    def __init__(self, config: Config):
        super().__init__()
        self.server = Server(config=config)
        self.config = config

    def stop(self):
        self.terminate()

    def run(self, *args, **kwargs):
        self.server.run()




@pytest.fixture(scope="session")
def server():
    config = Config("app.main:app", host="127.0.0.1", port=5000, log_level="debug")
    instance = UvicornServer(config=config)
    instance.start()
    yield instance
    instance.stop()

@pytest.fixture(scope="module")
def mock_app(server):
    client = TestClient(app)
    yield client

Example test_app.py file.

def test_root(mock_app):
    response = mock_app.get("")
    assert response.status_code == 200
Improve this answer

2 Comments

Hi @polka, this works but I have one question: How can I try/except a keyboard interrupt? I mean if I press ctrl + c the application exits which is fine (it recognizes SIGNALS). However, I want to add some logging to the exit process but I don't know where I can do this. There is no exception thrown. It just exits. Any recommendations?
Ok, I answered my question myself. You can just put logging messages below the server.run() line because ofc it is not async so the thread is blocked on that line. And if you ctrl + c the run gets cancelled and the following lines are executed.
2

According to Uvicorn documentation there is no programmatically way to stop the server. instead, you can stop the server only by pressing ctrl + c (officially).

But I have a trick to solve this problem programmatically using multiprocessing standard lib with these three simple functions :

  • A run function to run the server.
  • A start function to start a new process (start the server).
  • A stop function to join the process (stop the server).
from multiprocessing import Process
import uvicorn

# global process variable
proc = None


def run(): 
    """
    This function to run configured uvicorn server.
    """
    uvicorn.run(app=app, host=host, port=port)


def start():
    """
    This function to start a new process (start the server).
    """
    global proc
    # create process instance and set the target to run function.
    # use daemon mode to stop the process whenever the program stopped.
    proc = Process(target=run, args=(), daemon=True)
    proc.start()


def stop(): 
    """
    This function to join (stop) the process (stop the server).
    """
    global proc
    # check if the process is not None
    if proc: 
        # join (stop) the process with a timeout setten to 0.25 seconds.
        # using timeout (the optional arg) is too important in order to
        # enforce the server to stop.
        proc.join(0.25)


With the same idea you can :


Example of usage :

from time import sleep

if __name__ == "__main__":
    # to start the server call start function.
    start()
    # run some codes ....
    # to stop the server call stop function.
    stop()



You can read more about :

Improve this answer

4 Comments

Thanks for the answer but did you try your code above? I'm trying to run the code on Win10 with python 3.7 and I get errors either starting uvicorn in a Thread or starting it in a new process. The error using a thread looks like this: Traceback (most recent call last): File "C:\Python37\lib\site-packages\uvicorn\main.py", line 565, in install_signal_handlers loop.add_signal_handler(sig, self.handle_exit, sig, None) File "C:\Python37\lib\asyncio\events.py", line 540, in add_signal_handler raise NotImplementedError NotImplementedError and signal only works in main thread
Using a new process the following error occurs: cant pickle _thread.RLock objects. Any suggestions how I can solve this problem? Due to this post github.com/tiangolo/fastapi/issues/650 it is better to run it in a process but it's not working for me.
Ok found a solution on my own. First it is important to use a new process to start uvicorn in it. Then you can kill or terminate the process if you want to stop uvicorn. But this does not seem to work on windows, at least for me it is just working on linux. To avoid the error of "cant pickle _thread.RLock objects" it is important not to use a method with self. So for example run_server(self) is not working with a new Process but run_server() is.
@Leuko Posted an answer with a proper fix to the "main thread" errors
2

I think this kind of asynchronous startup can solve your problem.

import asyncio

import uvicorn
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello, World!"}


async def main():
    config = uvicorn.Config(app, port=5000, log_level="info")
    server = uvicorn.Server(config)
    await server.serve()


if __name__ == "__main__":
    asyncio.run(main())
Improve this answer

Comments

0

When I set reload to False, fastapi will start a multi-process web service. If it is true, there will only be one process for the web service

import uvicorn
from fastapi import FastAPI, APIRouter
from multiprocessing import cpu_count
import os

router = APIRouter()
app = FastAPI()


@router.post("/test")
async def detect_img():
    print("pid:{}".format(os.getpid()))
    return os.getpid

if __name__ == '__main__':
    app.include_router(router)
    print("cpu个数：{}".format(cpu_count()))
    workers = 2*cpu_count() + 1
    print("workers:{}".format(workers))
    reload = False
    #reload = True
    uvicorn.run("__main__:app", host="0.0.0.0", port=8082, reload=reload, workers=workers, timeout_keep_alive=5,
                limit_concurrency=100)
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.