What is the best way for parsing multiple files from multiple directories?
I have a folder that contains 51 sub folders and each sub folder contains 100 files.
I know how to scan one single file using
File dataFile = new File("A.txt");
scan = new Scanner (dataFile);
while (scan.hasNext()){
System.out.print(scan.next() + "\t");
}
but how to generalize this to read from the different directories ?
You should use a FileVisitor (which is an interface you implement) and then put it to work using Files.walkFileTree(Path path, FileVisitor visitor). In your case you can just subclass the SimpleFileVisitor class:
import static java.nio.file.FileVisitResult.*;
public static class PrintFiles
extends SimpleFileVisitor<Path> {
@Override
public FileVisitResult visitFile(Path file,
BasicFileAttributes attr) {
if (attr.isSymbolicLink()) {
System.out.format("Symbolic link: %s ", file);
} else if (attr.isRegularFile()) {
System.out.format("Regular file: %s ", file);
} else {
System.out.format("Other: %s ", file);
}
System.out.println("(" + attr.size() + "bytes)");
return CONTINUE;
}
}
void recursiveParse(String directory){
for(File fileOrFolder : directory.listFiles()){
if(fileOrFolder.isDirectory()){
for (String singleFileOrFolder : fileOrFolder.listFiles()){
recursiveParse(singleFileOrFolder);
}
else{
// parse the file
}
}
}
It only works for a folder that contains subdirectories and each subdirectory doesn't contain subdirectories:
import java.io.File;
import java.io.IOException;
import java.util.Scanner;
class ScanFilesInSubFolders {
public static void main(String[] args) throws IOException {
System.out.println("Enter the path to search");
Scanner scanner = new Scanner(System.in);
String folderPath = scanner.next();
File [] files1 = null;
File folder = new File(folderPath);
if (folder.isDirectory()) {
File[] listOfFiles = folder.listFiles();
for (File file : listOfFiles) {
if(file.isDirectory()) {
files1 = file.listFiles();
for(File file2 : files1)
{
Scanner file3 = new Scanner (file2);
while (file3.hasNext()){
System.out.print(file3.next() + "\t");
}
System.out.print("\n");
}
}
}
}
}
}
Here you go :
public static void main (String[] args) throws Exception{
ReadThroughFolders rf = new ReadThroughFolders();
//C:\\Users\\username\\Desktop\\DOC\\COUNTRIES
File MainDirectory = new File("C:\\Users\\username\\Desktop\\DOC\\COUNTRIES");
rf.readDir(MainDirectory);
}
private void readDir(File f) throws Exception{
Set<String>set = new TreeSet<String>();
File []subdir = f.listFiles();
int count = 0;
for (File fi : subdir){
if(fi.isFile()){
BufferedReader br = new BufferedReader(new FileReader(fi));
String line;
while((line = br.readLine())!= null){
set.add(line);
}
br.close();
count++;
System.out.println("reding file ----------------");
System.out.println(fi.getName());
}
String dir = f.getName();
System.out.println("****** "+dir);
dir = dir+".txt";
if(subdir.length-(count) == 0){
System.out.println("end of directory"+dir);
count = 0;
print(set, dir);
set = new TreeSet<String>();
}
if(fi.isDirectory()){
//System.out.println("inside if check directory");
readDir(fi);
System.out.println("reading next directory");
}
}
}
The method will read the files in the different folders in your root folder.
java.io.Fileto represent the primary input, this would allow you to useFile#listFilesto list the contents of your directory and pass each match to a method. You could also look at Walking the File Tree and use it to process your files. Depending your needs, you might even consider using some kind ofExecutorServiceto allow you to process multiple files simultaneously