5

I am creating a FastAPI Python application where a user uploads a file to be processed. I do not want the file to exceed size X (in bytes).

How do I limit the file upload size before the POST request stores the file in memory?

I am using uivcorn for testing but I am expecting to deploy this code with Google Cloud Platform (GCP). I am not sure if this can be done on the python code-side or server configuration-side.

Code Snippet:

from fastapi import (
     FastAPI, 
     Path, 
     File, 
     UploadFile, 
) 

app = FastAPI()

@app.post("/")
async def root(file: UploadFile  = File(...)):
     text = await file.read()
     text = text.decode("utf-8")
     return len(text)
2
  • usually this goes in the webserver (i.e. Apache/Nginx/...) configuration Commented May 22, 2022 at 16:09
  • Please have a look at this answer Commented Jun 7, 2024 at 2:00

4 Answers 4

5

I found a python library that takes care of this via FastAPI middleware. If the upload file is too large it will throw a 413 HTTP error; "Error: Request Entity Too Large"

from starlette_validation_uploadfile import ValidateUploadFileMiddleware

 from fastapi import (
      FastAPI, 
      Path, 
      File, 
      UploadFile, 
 ) 


app = FastAPI()

#add this after FastAPI app is declared 
app.add_middleware(
        ValidateUploadFileMiddleware,
        app_path="/",
        max_size=1048576, #1Mbyte
        file_type=["text/plain"]
)


 @app.post("/")
 async def root(file: UploadFile  = File(...)):
      #...do something with the file
      return {"status: upload successful"}
Sign up to request clarification or add additional context in comments.

1 Comment

Could you add a link to the library to your answer?
0

it usually controls by web server like nginx or Apache but if you want to control in the server side you can use this code:

from fastapi import (
    FastAPI,
    Path,
    File,
    UploadFile,
)

app = FastAPI()

@app.post("/")
async def root(file: UploadFile = File(...)):
    if len(await file.read()) >= 8388608:
        return {"Your file is more than 8MB"}

Comments

0

You can use middleware. I used this one and it works nice

Usage (I change the original code a bit):

middleware.py:

from typing import Optional
from fastapi import HTTPException

class ContentSizeLimitMiddleware:
    """Content size limiting middleware for ASGI applications

    Args:
      app (ASGI application): ASGI application
      max_content_size (optional): the maximum content size allowed in bytes, None for no limit
    """

    def __init__(
        self,
        app,
        max_content_size: Optional[int] = None,
    ):
        self.app = app
        self.max_content_size = max_content_size

    def receive_wrapper(self, receive):
        received = 0

        async def inner():
            nonlocal received
            message = await receive()
            if message["type"] != "http.request" or self.max_content_size is None:
                return message
            body_len = len(message.get("body", b""))
            received += body_len

            if received > self.max_content_size:
                raise HTTPException(detail="File is too big", status_code=413)

            return message

        return inner

    async def __call__(self, scope, receive, send) -> None:
        if scope["type"] != "http":
            await self.app(scope, receive, send)
            return

        wrapper = self.receive_wrapper(receive)
        await self.app(scope, wrapper, send)

main.py:

app.add_middleware(ContentSizeLimitMiddleware, max_content_size=5 * 1024 * 1024) # 5MB

This solution reads only the necessary part of the file and reject the request

Take this Starlette view for example:

@app.route("/documents/upload", methods=["POST"])
def upload_document(request):
    data = await request.body()
    if len(data) > Config.MAX_FILE_SIZE:
        return api_400(
            f"This file exceeds the maximum file size we support at this time ({Config.MAX_FILE_SIZE})",
            code=MAX_FILE_SIZE_EXCEEDED,
        )
    ...

If the maximum file size is 5MB, and the uploaded file was 50MB, then this implementation reads the entire 50MB into memory before rejecting the request.

Comments

0

I am using FastAPI. I used UploadFile in the endpoint.

@app.route("/documents/upload")
def upload_document(file: UploadFile = File(...),
    custom_fields_mapping: str = Form(None)):
    try:
        if file.size > 500*1024: 
           return {"success": True, "message": "BOD processing started in the background."}
    

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.