aws python lambda: reading csv file (iterator should return strings)

Question

I'm getting this message when I'm trying to test my python 3.8 lambda function:

Logs are:

soc-connect
contacts.csv
{'ResponseMetadata': {'RequestId': '9D7D7F0C5CB79984', 'HostId': 'wOd6HvIm+BpLOMKF2beRvqLiW0NQt5mK/kzjCjYxQ2kHQZY0MRCtGs3l/rqo4o0r4xAPuV1QpGM=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'wOd6HvIm+BpLOMKF2beRvqLiW0NQt5mK/kzjCjYxQ2kHQZY0MRCtGs3l/rqo4o0r4xAPuV1QpGM=', 'x-amz-request-id': '9D7D7F0C5CB79984', 'date': 'Thu, 26 Mar 2020 11:21:35 GMT', 'last-modified': 'Tue, 24 Mar 2020 16:07:30 GMT', 'etag': '"8a3785e750475af3ca25fa7eab159dab"', 'accept-ranges': 'bytes', 'content-type': 'text/csv', 'content-length': '52522', 'server': 'AmazonS3'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2020, 3, 24, 16, 7, 30, tzinfo=tzutc()), 'ContentLength': 52522, 'ETag': '"8a3785e750475af3ca25fa7eab159dab"', 'ContentType': 'text/csv', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x7f858dc1e6d0>}
1153
<_csv.reader object at 0x7f858ea76970>
[ERROR] Error: iterator should return strings, not bytes (did you open the file in text mode?)

The code snippet is:

import boto3
import csv

def digest_csv(bucket_name, key_name):
    # Let's use Amazon S3
    s3 = boto3.client('s3');

    print(bucket_name)
    print(key_name)

    s3_object = s3.get_object(Bucket=bucket_name, Key=key_name)
    print(s3_object)

    # read the contents of the file and split it into a list of lines
    lines = s3_object['Body'].read().splitlines(True)
    print(len(lines))

    contacts = csv.reader(lines, delimiter=';')
    print(contacts)

    # now iterate over those contacts
    for contact in contacts:
        # here you get a sequence of dicts
        # do whatever you want with each line here
       print('-*-'.join(contact))

I think the problem is on csv.reader.

I'm setting first parameter an array of lines... Should it be modified?

Any ideas?

csv.reader() is expecting a file. But it looks like your code is passing it what it got back from the splitlines() call, which is not a file but an iterator that returns a bunch of strings. — BoarGules
– BoarGules, Commented Mar 26, 2020 at 11:34

LeoC · Accepted Answer · 2021-01-08 12:21:37Z

1

Instead of using the csv.reader the following worked for me (adjusted for your delimiter and variables):

for line in lines:
    contact = ''.join(line.decode().split(';'))
    print(contact)

answered Jan 8, 2021 at 12:21

LeoC

9321 gold badge9 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

aws python lambda: reading csv file (iterator should return strings)

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related