How to cut data that not have comma to separate variable with Shell script

Question

guys I'm quite new on coding world. I would like some advises here Here's example of my raw data. it's quite hard to look on I would like to cut these lines to seperate each data into better view on AIX server

0000031   C9999999999.999000CCN20001    121213.0000000000000.00099999999999999.99901    0000000121213.0000000000000.0000000000000.000000003     VP01003C000003         0.000
0000036   B9999999999.999050CC8200010000000000.0000000000000.00099999999999999.99901    000    315799.0410000000000.0000000090000.000000003     VP03003C000003         0.000

into this format with header column on it as a final result

char    CustID[10];                     /* xxxxxxxxxx */
char    AccType;                        /* C/A/M/B */
char    CreditLimit[14];                /* 9(10).999 */
char    MarginCode[3];                  /* 999 ; 00 for CASH */
char    CustType;                       /* C,F,M,P,I,O,U,S */
char    CommCustType;                   /* x */
char    CreditType;                     /* 1/2/3/4/8/9/H/N */
char    CreditLine;                     /* 1/2/3/4 */
char    TraderID[4];                    /* 9999 ; A/O */
char    BuyTotalCredit[14];             /* 9(10).999 */
char    SellCredit[14];                 /* 9(10).999 */
char    LimitPercentApprove[3];         /* 999 */
char    LimitValuePerOrder[15];         /* 9(8).999 */
char    SubAccCode[2];                  /* 99 ; 01 for fis broker */
char    SubBrokerID[2];                 /* xx ; [  ] */
char    MutualFundID[2];                /* xx ; [  ] */
char    BranchID[2];                    /* 99 ; [00] */
char    SubAccFlag;                     /* 0, 1 ; [0] */
char    CashBalance[14];                /* 9(10).999 */
char    Debt[14];                       /* 9(10).999 */
char    Collateral[14];                 /* 9(10).999 */
char    CustCode[10];                   /* xxxxxxxxxx ; [= CustID] */
char    Market;
char    CalCommMethod;

Is there's anyway to do with shell like..

input account number :
0000031
read $accno

then used grep $accno $filename.dat

then we get

0000031   C9999999999.999000CCN20001    121213.0000000000000.00099999999999999.99901    0000000121213.0000000000000.0000000000000.000000003     VP01003C000003         0.000

after this, I guess if it's java could we use array ? what about shell ?

while
 read $format.txt
10,1,14,3,1,1,1,1,4,14,14,,15,2,2,2,2,1,14,14,14,10,1,1

or it should be the range

 0-9,10,11-24,...

You can use the substr() function in awk to extract fields of the line by range of character positions. — Barmar
– Barmar, Commented Aug 31, 2018 at 15:56

Charles Duffy · Accepted Answer · 2018-08-31 20:37:02Z

1

You can use parameter expansion to pull out substrings. To do this in a loop:

#!/usr/bin/env bash
case $BASH_VERSION in ''|[123].*|4.[012].*) echo "ERROR: Bash 4.3 needed" >&2; exit 1;; esac

TradeFormat=(
  CustId=10              AccType=1              CreditLimit=14
  MarginCode=3           CustType=1             CommCustType=1
  CreditType=1           CreditLine=1           TraderID=4
  BuyTotalCredit=14      SellCredit=14          LimitPercentApprove=3
  LimitValuePerOrder=15  SubAccCode=2           SubBrokerID=2
  MutualFundID=2         BranchID=2             SubAccFlag=1
  Debt=14                Collateral=14          CustCode=10
  Market=1               CalCommMethod=1
)

parse_pieces() {
  local -n piece_defs="$1" || return
  local -A pieces
  local pos length line

  while IFS= read -r line; do
    pieces=( )
    pos=0
    for piece_def in "${piece_defs[@]}"; do
      name=${piece_def%=*}
      length=${piece_def##*=}
      pieces[$name]="${line:$pos:$length}"
      pos=$(( pos + length ))
    done
    declare -p pieces  # or invoke a callback that refers to "pieces" here
  done
}

parse_pieces TradeFormat <<EOF
0000031   C9999999999.999000CCN20001    121213.0000000000000.00099999999999999.99901    0000000121213.0000000000000.0000000000000.000000003     VP01003C000003         0.000
0000036   B9999999999.999050CC8200010000000000.0000000000000.00099999999999999.99901    000    315799.0410000000000.0000000090000.000000003     VP03003C000003         0.000
EOF

...emits as output:

declare -A pieces=([MutualFundID]="  " [CustId]="0000031   " [SubAccFlag]="0" [AccType]="C" [CreditLine]="2" [CalCommMethod]="0" [BranchID]="00" [CreditType]="N" [CustCode]="0000000000" [SubAccCode]="01" [BuyTotalCredit]="    121213.000" [LimitValuePerOrder]="99999999999.999" [LimitPercentApprove]="999" [SellCredit]="0000000000.000" [Market]="." [CommCustType]="C" [MarginCode]="000" [CustType]="C" [Collateral]="0000000000.000" [SubBrokerID]="  " [TraderID]="0001" [CreditLimit]="9999999999.999" [Debt]="0000121213.000" )
declare -A pieces=([MutualFundID]="  " [CustId]="0000036   " [SubAccFlag]="0" [AccType]="B" [CreditLine]="2" [CalCommMethod]="0" [BranchID]="00" [CreditType]="8" [CustCode]="0000090000" [SubAccCode]="01" [BuyTotalCredit]="0000000000.000" [LimitValuePerOrder]="99999999999.999" [LimitPercentApprove]="999" [SellCredit]="0000000000.000" [Market]="." [CommCustType]="C" [MarginCode]="050" [CustType]="C" [Collateral]="0000000000.000" [SubBrokerID]="  " [TraderID]="0001" [CreditLimit]="9999999999.999" [Debt]="    315799.041" )

...such that you can refer to ${pieces[Debt]} or ${pieces[SubAccCode]} for the specific values from any function called where the code instructs you to put a callback there.

edited Aug 31, 2018 at 20:37

answered Aug 31, 2018 at 16:04

Charles Duffy

299k43 gold badges442 silver badges498 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

TK45 Over a year ago

I still confused on method to execute this shell, as I would like shell to read $filename.dat and grep only prefered account and do parse_pieces loop-function. Or do I have to add parameter in function? Other curious question is, if I have 5 more files which have unique format. Is it way for user to choose specific class to match format file before start cutting data?

Charles Duffy Over a year ago

See the edit passing TradeFormat as an argument to parse_pieces.

Charles Duffy Over a year ago

...as an alternate approach if you had a format file with one line per array element, you could pass the filename and use readarray -t piece_defs <"$1" to populate the array from the file.

Collectives™ on Stack Overflow

How to cut data that not have comma to separate variable with Shell script

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related