0

Have few thousand reports that have consistently formatted tabular data embedded within them that I need to extract.

Have a few ideas, but thought I'd post to see if there's a better way to do this than what I'm thinking; which is to extract the tabular data, create a new file for it, then parse that data as a tabular file.

Here's a sample input and output, where the output read and written row by row to a database.

INPUT_FILE

MiscText MiscText MiscText
MiscText MiscText MiscText
MiscText MiscText MiscText
SubHeader
PASS    1283019238  alksdjalskdjl
FAIL    102310928301    kajdlkajsldkaj
PASS    102930192830    aoisdajsdoiaj
PASS    192830192301    jiasdojoasi
MiscText MiscText MiscText
MiscText MiscText MiscText
MiscText MiscText MiscText

OUTPUT (read/write row-by-row from text-file to DB)

ROW-01{column01,column02,column03}
...
ROW-nth{column01,column02,column03}
Improve this question

3 Answers 3

Reset to default
2

Recognizing when to start processing tabular data is easy. You've got the marker line. The difficulty is recognizing when to stop processing data. You can apply the heuristics of stopping to process data when the split doesn't yield the expected result.

use strict;
use warnings;
my $tab_data;
my $num_cols;
while ( <> ) {
    $tab_data = 1, next if $_ eq "SubHeader\n";
    next unless $tab_data;
    chomp;
    my @cols = split /\t/;
    $num_cols ||= scalar @cols;
    last if $num_cols and $num_cols != scalar @cols;
    print join( "\t", @cols ), "\n";
}

Save as etd.pl (etd = extract tabular data, what did you think?), and call it like this from the command line:

perl etd.pl < your-mixed-input.txt
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

@Michael Ludwig: Thanks, look great -- though it appears I'm missing something. I've posted the code in the body of my question with $tab_data getting the sample data. When I run the code in a Perl debugger I use all the time (Ptkdb) the perl crashes/hanges at the while statement. Any idea what's going on, or what I'm missing? Again, thanks!
@blunders, the script expects the data streaming in on STDIN, which is standard input. Open up a command prompt and give it a try. - Ah, and please revert your edit to your original post - it is totally misleading, and not as intended at all. Thanks.
+2 @Michael Ludwig: Reverted the body of the question. Follow your edits, though heading out of the office for a few hours, but expect to get back to this within 24-hours. Again, thank you!
@TLP - $tab_data will only be set to one when $_ eq "SubHeader\n". Do perl -MO=Deparse,-p etd.pl or perl -d to verify.
@TLP - You're right, it's only for one pass. And I agree the solution is not perfect. However, given the spec, it's certainly not inappropriate. And yes, it's a little bit obfuscating.
|
1

If you know how to extract data, why create a new file instead of processing it immediately?

Improve this answer

1 Comment

+1 @zvrba: That's what I'd like to do, though I'd still have to figure out how to do it; all the code I've used so far OPEN<filehandle> a file, uses a WHILE<filehandle>, then CLOSE<filehandle> the file; no idea how to twist that into parse text by linefeeds. As for extracting the data, I just know it's possible; updated the sample data to give you a better idea of what I mean; that being the sub-header is always the same, and the tabular data goes on until there is not a (PASS or FAIL) on the next line.
0

In case this is a fixed width data, I would strongly suggest using unpack or plain old substr.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.