How to create a simple csv on the fly in python?

Question

I have an endpoint that takes a csv file.

Now, I want to write a test that makes a post request with this file.

I am trying to generate this csv file on the fly (rather than manually create and store it)

I tried this:

def csv_fixture(rows, type):
    headers = None
    if type == "merchant_upload":
        headers = MerchantCSV.ordered_columns()
    elif type == "invoice_upload":
        headers = InvoiceCSV.ordered_columns()
    assert headers is not None
    rows = [headers] + rows
    with open("file.csv", "w+") as f:
        writer = csv.writer(f)
        writer.writerows(rows)
        yield f

my_file = csv_fixture(merchants, type="merchant_upload")
request = rf.post("/invoice_admin/upload_organisations/",
                      {"onboarding_file": my_file})

My endpoint does something like this:

    if filename not in request.FILES:
        raise Exception("Upload Failed: No file submitted.")
    file = TextIOWrapper(
        request.FILES[filename].file, encoding=request.encoding)

    headers = peek_first_row(file)
    missing = required_cols - set(headers)
    if missing:
        raise Exception(f"Columns missing in csv: {str(missing)})")

    return csv.DictReader(file)

My endpoint works if I manually upload the file. However, if I try doing it programatically with the first snipper, I get an error:


    def peek_first_row(file):
        rows = csv.reader(file)
>       headers = next(rows)
E       StopIteration

app/invoice_admin/csv_parser.py:11: StopIteration

Please could someone guide me? I have looked at lots of tutorials, but I'm lost at this point.

Rakesh · Accepted Answer · 2019-07-25 12:48:29Z

2

This might help.

Ex:

def csv_fixture(rows, type):
    headers = None
    if type == "merchant_upload":
        headers = MerchantCSV.ordered_columns()
    elif type == "invoice_upload":
        headers = InvoiceCSV.ordered_columns()
    assert headers is not None
    rows = [headers] + rows
    with open("file.csv", "w+") as f:
        writer = csv.writer(f)
        writer.writerows(rows)
    return open("file.csv", "rb")

my_file = csv_fixture(merchants, type="merchant_upload")
request = rf.post("/invoice_admin/upload_organisations/",
                      files={"onboarding_file": my_file})

answered Jul 25, 2019 at 12:48

Rakesh

82.9k17 gold badges86 silver badges122 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

nz_21 Over a year ago

thank you! This worked! Do you know of a more pythonic way of writing the above code?

nz_21 Over a year ago

the receiver would have to manually close the file though, right? I think a combination of with() and yield would be better?

nz_21 Over a year ago

also I don't want to store the file, the current code stores it

Collectives™ on Stack Overflow

How to create a simple csv on the fly in python?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related