I have a text file 2 fields separated by :
i3583063:b3587412
i3583064:b3587412
i3583065:b3587412
i3583076:b3587421
i3583077:b3587421
i3583787:b3587954
i3584458:b3588416
i3584459:b3588416
i3584460:b3588416
i3584461:b3588416
i3584462:b3588416
i3584463:b3588416
i3584464:b3588416
i3584465:b3588416
Field 1 is always uniq but not field 2 it can be repeated. How can I identify first, 2nd 3rd etc. occurrence of field 2? Can I use count?
Thanks
I don't know if I've ever heard of a standard Unix count utility, but you can do this with Awk. Here's an Awk script that adds the count as a third column:
awk -F: 'BEGIN {OFS=":"} {$3=++count[$2]; print}' input.txt
It should generate the output:
i3583063:b3587412:1
i3583064:b3587412:2
i3583065:b3587412:3
i3583076:b3587421:1
i3583077:b3587421:2
i3583787:b3587954:1
i3584458:b3588416:1
i3584459:b3588416:2
i3584460:b3588416:3
i3584461:b3588416:4
i3584462:b3588416:5
i3584463:b3588416:6
i3584464:b3588416:7
i3584465:b3588416:8
The heart of the script {$3=++count[$2]; print} simply increments a counter indexed by the value of the second field, stores it in a new third field, and then outputs the line with this new field. Awk is a great little language and still well worth learning.
You can use the sort command with the -u parameter. This way redundant lines are removed.
sort -u filename.txt
If you want to count occurrences
sort -u filename.txt | wc -l
