1

How should I modify the code below (taken from this post), so that the file is not saved to disk, but saved to memory instead? The code uses the streaming-form-data library for parsing multipart/form-data input chunks.

Unfortunately, the below is currently requiring a path, and I would like to prevent I/O operations like that.

from fastapi import FastAPI, Request, status
from fastapi.exceptions import HTTPException
from streaming_form_data import StreamingFormDataParser
from streaming_form_data.targets import FileTarget, ValueTarget
from streaming_form_data.validators import MaxSizeValidator
import streaming_form_data
from starlette.requests import ClientDisconnect
import os

MAX_FILE_SIZE = 1024 * 1024 * 1024 * 4  # = 4GB
MAX_REQUEST_BODY_SIZE = MAX_FILE_SIZE + 1024

app = FastAPI()

class MaxBodySizeException(Exception):
    def __init__(self, body_len: str):
        self.body_len = body_len

class MaxBodySizeValidator:
    def __init__(self, max_size: int):
        self.body_len = 0
        self.max_size = max_size

    def __call__(self, chunk: bytes):
        self.body_len += len(chunk)
        if self.body_len > self.max_size:
            raise MaxBodySizeException(body_len=self.body_len)
 
@app.post('/upload')
async def upload(request: Request):
    body_validator = MaxBodySizeValidator(MAX_REQUEST_BODY_SIZE)
    filename = request.headers.get('Filename')
    
    if not filename:
        raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, 
            detail='Filename header is missing')
    try:
        filepath = os.path.join('./', os.path.basename(filename)) 
        file_ = FileTarget(filepath, validator=MaxSizeValidator(MAX_FILE_SIZE))
        data = ValueTarget()
        parser = StreamingFormDataParser(headers=request.headers)
        parser.register('file', file_)
        parser.register('data', data)
        
        async for chunk in request.stream():
            body_validator(chunk)
            parser.data_received(chunk)

(...)
0

1 Answer 1

0

The solution is already given in the answer (demonstrating "How to Upload a large File (≥3GB) to FastAPI backend") from which you took the code. If you would like to hold the data in memory instead of writing them to disk, you simply need to use a ValueTarget instead of FileTarget object. For example:

file_ = ValueTarget(validator=MaxSizeValidator(MAX_FILE_SIZE))

To get the file contents afterwards, you can use the .value attribute, for instance:

print(file_.value)
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.