Powershell regular expressions: get 'not matched' portion of string?

Question

I'm not very up on powershell and have a problem.

I have a powershell script that calls a batch file that mounts a backup image (shadowprotect), runs chkdsk against mounted image, dismounts image and returns.

All the output from the batch file processing is returned to my powershell script. I'm trying to parse this to remove all the 'progress' lines that get generated by the chkdsk command, chkdsk doesn't seem to have any way to suppress this output (on large disk images I can end up with hundreds of progress lines).

I can create regular expression to catch all the 'Progress' lines and put them out to my log file, but I can't figure out the syntax that says to give me everything that does NOT match my regular expression.

Very short example of returned value I'm trying to parse:

Try to mount d:\backups\colt\F_VOL-b001-i453-cd.spi
Mounting image chain "D:\Backups\colt\F_VOL-b001.spf|D:\Backups\colt\F_VOL-b001-i014-cd-cw.spi|D:\Backups\colt\F_VOL-b001-i018-cd.spi|D:\Backups\colt\F_VOL-b001-i022-cd.spi|D:\Backups\colt\F_VOL-b001-i026-cd.spi|D:\Backups\colt\F_VOL-b001-i030-cd.spi|D:\Backups\colt\F_VOL-b001-i445-cd.spi|D:\Backups\colt\F_VOL-b001-i449-cd.spi|D:\Backups\colt\F_VOL-b001-i453-cd.spi"

The type of the file system is NTFS.
Volume label is Local Disk - single.

WARNING!  F parameter not specified.
Running CHKDSK in read-only mode.

Stage 1: Examining basic file system structure ...
Progress: 0 of 4320 done; Stage:  0%; Total:  0%; ETA:   0:00:14    
Progress: 0 of 4320 done; Stage:  0%; Total:  0%; ETA:   0:00:21 .  
Progress: 6 of 4320 done; Stage:  0%; Total:  0%; ETA:   0:01:10 .. 
Progress: 257 of 4320 done; Stage:  5%; Total:  2%; ETA:   0:01:47 ...
Progress: 769 of 4320 done; Stage: 17%; Total:  6%; ETA:   0:01:20    
Progress: 2817 of 4320 done; Stage: 65%; Total: 19%; ETA:   0:00:23 .  
Progress: 4320 of 4320 done; Stage: 100%; Total: 29%; ETA:   0:00:14 .. 


  4320 file records processed.                                                        

File verification completed.
Progress: 0 of 0 done; Stage: 99%; Total: 46%; ETA:   0:00:13 ...


  0 large file records processed.                                   

Progress: 0 of 0 done; Stage: 99%; Total: 46%; ETA:   0:00:13    


  0 bad file records processed.                                     


Stage 2: Examining file name linkage ...
Progress: 4322 of 4360 done; Stage: 99%; Total: 92%; ETA:   0:00:01 .  
Progress: 4340 of 4360 done; Stage: 99%; Total: 93%; ETA:   0:00:01 .. 
Progress: 4344 of 4360 done; Stage: 99%; Total: 97%; ETA:   0:00:01 ...
Progress: 4360 of 4360 done; Stage: 100%; Total: 97%; ETA:   0:00:01    


  4360 index entries processed.                                                       

Index verification completed.
Progress: 0 of 0 done; Stage: 99%; Total: 97%; ETA:   0:00:01 .  


  0 unindexed files scanned.                                        

Progress: 0 of 0 done; Stage: 99%; Total: 97%; ETA:   0:00:01 .. 


  0 unindexed files recovered.                                      


Stage 3: Examining security descriptors ...
Security descriptor verification completed.
Progress: 0 of 0 done; Stage: 100%; Total: 99%; ETA:   0:00:00 ...


  20 data files processed.                                           


Windows has scanned the file system and found no problems.
No further action is required.

 145500673 KB total disk space.
  15814844 KB in 748 files.
       180 KB in 22 indexes.
         0 KB in bad sectors.
     74721 KB in use by the system.
     65536 KB occupied by the log file.
 129610928 KB available on disk.

      4096 bytes in each allocation unit.
  36375168 total allocation units on disk.
  32402732 allocation units available on disk.
OK -  0 

Return value is 0

My script fragment looks like this:

 # call batch file...
 $Out = c:\batch\Mount_and_Chkdsk_image_file.cmd $NewFilePath
 # append result of batch file to the log file, remove progress lines first...
 $Out -replace ("^Progress:.*$", "")
 Out-File -FilePath c:\batch\log\NewFileCheck.log  -Append -InputObject $Out

In above attempt, since I can match the progress lines I thought I'd just replace them with nothing but no substitution happened.

(for testing I substituted the batch file call with just a file read of the raw unfiltered log file:

   $FilePath = "c:\batch\test2.txt"
    $Out = [System.Io.File]::ReadAllText($filePath)

)

Anyway, since replacement didn't work, I used google and tried understanding the select-string as an option since it takes regular expressions:

$Regex = 'Pattern:.*'
$Out | select-string -pattern $Regex -notmatch

This just puts out all the lines, nothing filtered out, I had hoped 'notmatch' would mean everything that didn't match. Didn't seem to matter how I varied regex I couldn't get that to do what I needed.

Tried many many more variations on the theme like:

#$Regex = '^((Progress:).*$)'
$Regex = '([?<!Progress:].*)'

$Out | Select-String -Pattern $Regex -AllMatches | %{$_.Matches } | %{$_.value}

But I'm obviously missing something. I would have thought there would be an easy function that if you could select a certain string you could also choose to not have that selection in the output.

Can anyone please help, how do I capture all the lines that aren't a match?

Regards, Bryce S.

The question could be stripped down, so as it will be quicker to focus on the problematic subject. You could omit everything before $FilePath = .. portion, in my opinion. — basos
– basos, Commented Jan 29, 2016 at 10:25

TheMadTechnician · Accepted Answer · 2014-11-04 00:48:14Z

5

Sure, this can be done. By default when you load a text file with Get-Content it will load each line as a string, and the entire file will be an array of strings. You can run that through a Where statement and use the -notmatch operator to filter out things. Usage would be as such:

$ParsedData = Get-Content "c:\batch\test2.txt" | Where{$_ -notmatch "Progress:.*$"}

That would assist $ParsedData all of the lines in the file that did not match the regex "Progress:.*$".

Edit: Ok, what you're getting from your script is most likely a multi-line string. The easiest way that I know of is to simply break your string up on the new lines to make an array of strings. Something like this:

$Out.Split("`n") | Where{$_ -notmatch "Progress:.*$"}

edited Nov 4, 2014 at 0:48

answered Nov 3, 2014 at 22:37

TheMadTechnician

36.5k3 gold badges48 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user1840734 Over a year ago

Hi MadTechnician, I tried that and see that it works for files. I use a file for testing my regular expression and stuff. BUT, in the real script the variable I'm trying to parse is the returned output from a batch file (I have no idea if that is one enormous string or an array or what) - is there something equivalent of 'Get-Content' that can be used in this situation? if not I guess I can always write a temporary file and then read it in again.

TheMadTechnician Over a year ago

Ok, that should help you parse your output. I'm 99% sure that what you have is just a multi-line string. I think there might be a way to do it all with RegEx, but this seems simpler to me that trying some complicated RegEx solution.

user1840734 Over a year ago

Thanks MadTechnician, this solves the problem nicely. It is better than the '-replace' option that I tried (and solved by briantist below) because it does not leave me with empty lines in the output. I like that this solution is also short and straight forward, some of the regex stuff I was googling just left me completely confused. Cheers, Bryce

briantist · Accepted Answer · 2014-11-03 22:36:15Z

2

$Out = $Out -replace ("^Progress:.*$", "")
Out-File -FilePath c:\batch\log\NewFileCheck.log  -Append -InputObject $Out

The only problem you had was that -replace doesn't modify the left side value, it just returns the result, leaving $Out unchanged, so you have to make sure you assign the result back to $Out.

answered Nov 3, 2014 at 22:36

briantist

48.3k6 gold badges94 silver badges139 bronze badges

2 Comments

user1840734 Over a year ago

Thanks Briantist, that did the trick for me, I had got so close (I even think I had tried this at one point but must have got something else wrong). One of my main problems was changing to reading a file as a substitute for the returned output of a batch file -> completely different behaviours. (Tested this again by making my 'very short example' file a batch file that 'echo's every line,and running that back into the powershell script).

TheMadTechnician Over a year ago

Since the user's input turned out to be a multi-line string you would want to add (?m) to the beginning of your RegEx pattern so that ^ and $ match the beginning and end of lines rather than the beginning and end of the string. This would leave blank lines where that was, but will make it more or less functional at least. $Out = $Out -replace "(?m)^Progress:.*$" (you can exclude the enclosing ( ) and the ,"" from the statement)

Collectives™ on Stack Overflow

Powershell regular expressions: get 'not matched' portion of string?

2 Answers 2

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related