Need help parsing a file via UNIX commands

This is not exactly what you asked for, but no-one else is answering and it is pretty close for you to get started with!

awk -F'[&:]' '/^LINEID1:/{print $2,$3,$5,$6}' OFS='|' file

Output

FIELD1=ABCD,|FIELD2-0|FIELD4-0|FIELD9-0;
FIELD1=XYZ,|FIELD2-0|FIELD9-0|
FIELD1=PQRS,|FIELD3-1|FIELD9-0;|

The -F sets the Input Field Separator to colon or ampersand. Then it looks for lines starting LINEID1: and prints the fields you need. The OFS sets the Output Field Separator to the pipe symbol |.

To select rows from data with Unix command lines, use grep, awk, perl, python, or ruby (in increasing order of power & possible complexity).

To select columns from data, use cut, awk, or one of the previously mentioned scripting languages.

First, let's get only the lines with LINEID1 (assuming the input is in a file called input).

grep '^LINEID1' input

will output all the lines beginning with LINEID1.

Next, extract the columns we care about:

grep '^LINEID1' input |   # extract lines with LINEID1 in them
cut -d: -f2           |   # extract column 2 (after ':')
tr ',&' '\n\n'        |   # turn ',' and '&' into newlines
egrep 'FIELD[1249]'   |   # extract only fields FIELD1, FIELD2, FIELD4, FIELD9
tr '\n' '|'           |   # turn newlines into '|'
sed -e $'s/\\|\\(FIELD1\\)/\\\n\\1/g' -e 's/\|$//'

The last line inserts newlines in front of the FIELD1 lines, and removes any trailing '|'.

That last sed pattern is a little more challenging because sed doesn't like literal newlines in its replacement patterns. To put a literal newline, a bash escape needs to be used, which then requires escapes throughout that string.

Here's the output from the above command:

FIELD1=ABCD|FIELD2-0|FIELD4-0|FIELD9-0;
FIELD1=XYZ|FIELD2-0|FIELD9-0
FIELD1=PQRS|FIELD4-0|FIELD9-0;

This command took only a couple of minutes to cobble up.

Even so, it's bordering on the complexity threshold where I would shift to perl or ruby because of their excellent string processing.

The same script in ruby might look like:

#!/usr/bin/env ruby
#
while line = gets do
  if line.chomp =~ /^LINEID1:(.*)$/
    f1, others = $1.split(',')
    fields = others.split('&').map {|f| f if f =~ /FIELD[1249]/}.compact
    puts [f1, fields].flatten.join("|")
  end
end

Run this script on the same input file and the same output as above will occur:

$ ./parse-fields.rb < input
FIELD1=ABCD|FIELD2-0|FIELD4-0|FIELD9-0;
FIELD1=XYZ|FIELD2-0|FIELD9-0
FIELD1=PQRS|FIELD4-0|FIELD9-0;