2

I don't have a ton of knowledge about Java, so I was wondering if someone could give me an idea about the best way to structure this application (or whether it even matters).

I have a situation where I am pulling data from a flat file and committing it to a SQL database. However, the particular prepared statement that I use depends on the file extension.

So essentially

  1. program checks the file extension.
  2. Opens file, creates buffered reader - reads string data.
  3. based on file extension, generates prepared statement.
  4. Adds prepared statement to batchlist.
  5. Repeats for each line in the file (there could be thousands of lines).
  6. Executes the prepared statements.

The way that I determine how to get from file extension to appropriate prepared statement is a Map (which calls the relevant method from a bunch that all implement the same interface).

There are two ways I could implement this.

  1. Pass the connection and the buffered reader to the appropriate method and it will do all the work (generate thousands of prepared statements and execute them. The method returns 'void').

  2. Pass the connection and the string for a line to the appropriate method. The method returns the prepared statement. The original application generates another string (the next line), passes it to the method which returns a prepared statement, etc...until the original application executes all of the prepared statements.

Essentially, option number 1 requires more code and option number 2 requires thousands of method calls.

Oh and it might be important to note that each method call is conditional based on using the ".equals()" method to check a string against the appropriate key in the map.

Thanks.

Improve this question

3 Answers 3

Reset to default
1

I would say option 2 comes closest. Make use of object oriented design to abstract only the functionality that differs between file extensions - try to get everything in common to happen in one place.

For example your Map could store objects that take a String and return a PreparedStatement, and there would be one of these for each file extension. Of course such an object would also need to reference a Connection in order to do so - maybe pass this into its constructor when it's first initialized and stored, assuming you're using the same database connection for everything.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I am using the same database connection. So your design pattern is completely an option. My only concern is whether conditionally checking a string with equals() and then calling a method (from another class) thousands of times would be a major performance hit. I don't know enough about Java to determine if this is a valid concern.
There should be no conditional string check - only the lookup of a statement-preparing object from the Map, based on file extension. Assuming the Map is properly calibrated, this should be nearly constant time. Although unless you have many, many different extensions it shouldn't matter much.
1

How about something like this:

PreparedStatement prepareStatement(String line, File f);
void executeBatchUpdate(List<PreparedStatement> ps);

main(){
    List<PreparedStatement> myStatements; //initialize as you like
    Collection<File> myFiles; //initialize as you like
    for(File f : myFiles){ 
        BufferedReader br = new BufferedReader(new FileInputStream(.....
        while(br.hasNextLine()){
            myStatements.add(prepareStatement(br.readLine(), f));
        }
    }
    executeBatchUpdate(myStatements, f);
}

The flow of the program is easily understood from looking at the main method, and the other parts do relatively simple tasks.

Edited to reflect that you should have file metadata when preparing statements, and to account for the possibility that you have multiple files to process.

It is not very expensive to use switch and handle each file type as you need. Or, if you have so many different file types that a linear search through the switch cases would be expensive, then you can initialize a HashMap which would allow constant-time lookups.

Improve this answer

2 Comments

How does this account for different file extensions needing differently prepared statements?
So this is close to the form I use. But it does not really answer my concern about calling a method based on a certain file extension thousands of times vs. calling a method based on a certain file extension just once (and having to write a lot of repeated code in each of the methods that might be called).
0

I suggest you write a class for each file type, and let the class handle the processing. Then if you have to add file types that don't follow strict line for insert statement semantics, you will have less risk of breaking your program. This is sort of like option #1, but it allows for more flexibility in the future.

Improve this answer

2 Comments

I need to specify that the file types are all the same: plain text, flat files. However, they have different extensions based on the file's purpose. However, they will always follow strict line for insert statement semantics.
If you can predict the future like that, then please cough up some lottery numbers for me. :-) Seriously, you never know what may come to you in the future. Encapsulating each type in a class protects you, and it doesn't mean that you have to duplicate a lot of code, just be smart about it. Good luck.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.