bat file: parse from multiple files and write to csv

Question

I have been researching how to write this bat file I need - and I have come to a dead end. Not sure what is going wrong.

The problem/goal: I have huge txt files I must parse 5 key values from. The first 195 lines of each text file is essentially garbage I am not interested in.

There are 5 text files, 1 each resides in 5 sub-directories. The goal is to write those 5 key values (tokens 7, 11, 19, 21 and 25) into a csv file. (there are 1800 of each of the 5 key values, per text file - making a total of 9000 lines for the csv file I create)

The code I've written fails when entering the first for loop, stating "do( was unexpected at this time"

This is my first time writing functions into a bat file, or requiring more than 1 for loop. I have tweaked my syntax in multiple places, but still get the same error. Obviously I am missing something, but I am not sure what. There may also be additional errors I am not yet aware of, since I can't even get into my first loop.

Code:

@ECHO OFF
setlocal enabledelayedexpansion
set fileout="C:\ffmpeg\fit.csv"

for /R %%f in (*.txt) do (
set THEFILE=%%f
call :setTokens
goto TheEnd
)

:setTokens
for /F "skip=195 tokens=* delims=" %%A in ('%THEFILE%') do(
    set the_line=%%A
    call :process_line
)

:process_line
for /F "tokens=7,11,19,21,25 delims= =:" %%a in ('%the_line%') do (
    set qp=%%a
    set slice=%%b
    set skip=%%c
    set size=%%d
    set y=%%e
    set OUTLINE=%qp%,",",%slice%,",",%skip%,",",%size%,",",%y%  
    echo %OUTLINE%>>%fileout%
)

:TheEnd

Since you are changing the value of qp etc within the loop, you need to extract the value using !qp! (as you have invoked delayedexpansion) - %qp% will give you the variable's value at parse time. This is assuming you've invoked aschipfl's change, and note JosefZ's advice that since you are not manipulating the values that you assign to qp, then you can use the raw metavariables %%a..%%e — Magoo
– Magoo, Commented Dec 4, 2016 at 23:44

JosefZ · Accepted Answer · 2016-12-04 17:34:08Z

Next code snippet should do the job for you:

@ECHO OFF
SETLOCAL EnableExtensions DisableDelayedExpansion

set fileout="C:\ffmpeg\fit.csv"

>>"%fileout%" (
  for /R %%F in (*.txt) do (
    for /F "usebackq skip=195 delims=" %%A in ("%%~F") do (
      for /F "tokens=7,11,19,21,25 delims==: " %%a in ("%%~A") do (
        echo %%a,%%b,%%c,%%d,%%e
      )
    )
  )
)
:TheEnd

For the sake of completeness, next code snippet shows and repairs (most of) flagrant mistakes in the script (see all rem comments below):

@ECHO OFF
SETLOCAL EnableExtensions EnableDelayedExpansion

set fileout="C:\ffmpeg\fit.csv"

for /R %%f in (*.txt) do (
      rem  strip ↓ incidental surroundig doble quotes (be on the safe side)
  set "THEFILE=%%~f"
  call :setTokens
      rem `goto TheEnd` here would end the %%f loop just after the 1st iteration 
)
    rem next goto moved from inside `for /R` body
goto TheEnd


:setTokens
    rem ↓↓↓↓↓↓↓↓           ↓            quotes ↓         ↓    ↓ space
for /F "usebackq skip=195     delims=" %%A in ("%THEFILE%") do (
    rem removed `tokens=*` ↑  as `delims=` suffices
    set "the_line=%%~A"
    call :process_line
)
    rem return from subroutine
goto :eof

:process_line
    rem         order of delimiters ↓↓↓          ↓          ↓ quotes
for /F "tokens=7,11,19,21,25 delims==: " %%a in ("%the_line%") do (
    set qp=%%a
    set slice=%%b
    set skip=%%c
    set size=%%d
    set y=%%e
        rem note that all `,",",` denoted below are taken literally
        rem         ↓↓↓↓↓       ↓↓↓↓↓      ↓↓↓↓↓      ↓↓↓↓↓
    set OUTLINE=%qp%,",",%slice%,",",%skip%,",",%size%,",",%y%
    rem echo %OUTLINE%>>%fileout%
    rem you need to apply delayed expansion as follows: 
    set OUTLINE=!qp!,!slice!,!skip!,!size!,!y!
    echo !OUTLINE!>>%fileout%
        rem       ↑↑↑↑↑↑↑↑↑↑↑ redirection could be moved
        rem                ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ and lopp variables used directly 
    rem >>"%fileout%" echo %%a,%%b,%%c,%%d,%%e
)
    rem return from subroutine
goto :eof

:TheEnd

Resources (required reading, incomplete):

(command reference) An A-Z Index of the Windows CMD command line
(helpful particularities) Windows CMD Shell Command Line Syntax
(%~G etc. special page) Command Line arguments (Parameters)
(special page) EnableDelayedExpansion
(>>, 2>&1 etc. special page) Redirection

Thank you - this worked! Even better - thank you for the resources. I have been working from the first reference you posted - the others are much appreciated.

Collectives™ on Stack Overflow

bat file: parse from multiple files and write to csv

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related