remove line from file if more than one pattern appears in different line

Question

I have a file with a patter like this:

1 1 1 2 0 1 0.5
1 2 2 2 0 2 0.5
2 1 1 1 0 1 0.25
2 1 2 2 0 2 0.5
2 3 3 3 0 3 0.25

I want to remove a line if another line in the file has the same last two entries. In the example above this means that line 4 should be removed:

1 1 1 2 0 1 0.5
1 2 2 2 0 2 0.5
2 1 1 1 0 1 0.25
2 3 3 3 0 3 0.25

I can't get my head around how I can do this via the command line or with a simple awk/sed script. Any help is greatly appreciated!

123 · Accepted Answer · 2015-06-21 18:40:56Z

3

awk '!a[$NF,$(NF-1)]++' file

Make an array and check it hasn't already been populated with the last two fields.

answered Jun 21, 2015 at 18:40

123

11.3k2 gold badges25 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

pasaba por aqui · Accepted Answer · 2015-06-21 18:51:46Z

1

try:

 sort -k 6 f1.txt  | uniq -f 5

assuming the original order of the lines doesn't matters.

answered Jun 21, 2015 at 18:17

pasaba por aqui

3,5871 gold badge20 silver badges45 bronze badges

Thanks for the quick reply! I have only one file though. I want to remove the line in that file.

Sorry, misunderstood. Try new solution if order of lines doesn't matters.