How to bring through files after a certain date when using a SSIS Foreach Loop Container

Question

Currently created an SSIS package that goes through a folder and brings back the latest file depending on the file name. I've done this by using a Foreach Loop container

In testing it has worked fine, I've used about 20 files, all worked fine. However when then trying to put this into the live environment, where it going through approx 200+ files, it's failing as the older files alter slightly, something I didn't know when first creating the package

I need to bring files through really only from the date 01/03/2020 onward, but cannot remove the older files from it's location

Is there a way I can alter the package to only bring back files from this date onward at all? And if so how would I do this?

Cheers in advance

No just the date in SQL, removed to avoid any further confusion — Wil-Liam
– Wil-Liam, Commented Sep 1, 2020 at 14:18
I'm not following - if the date isn't in the file name and the format of the file has changed between the epoch date there, how can you tell what's in scope for processing versus out of scope? — billinkc
– billinkc, Commented Sep 1, 2020 at 16:02
It's ok, I sorted it in the end, I created a table for it to check against and manually inserted all the file names in there so it'll now skip past them all to the files I need Also apologies, I didn't read your comment properly, the date was in the file name so for example the file names were in the format of , 'File_Name_2017-01-01', so I knew which files I needed to skip, but just wasn't sure how to do it, but the table to check if the file has been loaded or not has worked — Wil-Liam
– Wil-Liam, Commented Sep 1, 2020 at 16:14

KeithL · Accepted Answer · 2020-09-01 18:27:22Z

0

you can use file properties to get the latest file to process:

var directory = new DirectoryInfo("C:\\MyDirectory");
var myFile = directory.GetFiles()
             .OrderByDescending(f => f.LastWriteTime)
             .First();

you can also add a where in there:

var myFile = directory.GetFiles()
             .Where(f=>f.LastWriteTime>=DateTime.Parse("1/3/2020"))
             .OrderByDescending(f => f.LastWriteTime)
             .First();

answered Sep 1, 2020 at 18:27

KeithL

5,6193 gold badges23 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Wil-Liam Over a year ago

Thank you @KeithL , what I have created works for now, however when I have more time I'll test this out! Thanks for posting

Collectives™ on Stack Overflow

How to bring through files after a certain date when using a SSIS Foreach Loop Container

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related