AWS Redshift loading data from S3

Question

So I'm trying to load data into my Redshift database from an S3 bucket. I have a table 'Example' which has a field 'timestamp' in the format 'YY-MM-DD HH:MM:SS'.

Using the copy query to load the data, so I'm able to load for a specific pattern/prefix, but I want to load data after a certain timestamp, say, greater than '2014-07-09 10:00:00'. How do I approach this?

alexanderlz · Accepted Answer · 2014-07-10 23:30:46Z

1

You have two options:

either process the file before you load it to S3 (and upload only the data with timestamp greater than $SOME_TIMESTAMP)
use the COPY command to load the file into intermediate table (can be even temp table - as long as you stay within the same session) and then run:
```
insert into YOUR_ORIGINAL_TABLE (select * from YOUR_TEMP_TABLE where timestamp > WHATEVER_YOU_NEED)
```

answered Jul 10, 2014 at 23:30

alexanderlz

5893 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS Redshift loading data from S3

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related