How to parse logs and mask specific characters using Powershell

Question

I have a problem that I really hope to get some help with. It's rather complex but I will try and keep my explanation as simple and objective as possible. In a nutshell, I have log files that contain thousands of lines. Each line consists of information like date/time, source, type and message. In this case the message contains a variable size ...999 password that I need to mask. Basically the message looks something like this (its an ISO message):

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

For each line I need to zero in on password length identifier (008) do a count on it and then proceed to mask the number of following characters, which would be PASSWORD in this case. I would change it to something like XXXXXXXX instead so once done the line would look like this:

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008XXXXXXXX6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

I honestly have no idea how to start doing this with PowerShell. I need to loop though each line in the log file, and identify the number of characters to mask.

I've kept this high level as a starting point, there are some other complexities that I hope to figure out at a later time, like the fact that there are different types of messages and depending on the type the password length starts at another character position. I might be able to build on my aforementioned question first but if anyone understands what I mean then I would appreciate some help or tips about that too.

Any help is appreciated.

Thanks!

Additional information to original post:

Firstly, thank you to everyone for your answers thus far, its been greatly appreciated. Now that I have a baseline for how your answers are being formulated based on my information I feel I need to provide some more details.

1) There was a question about whether or not the password starting position is fixed and the logic behind it.

The password position is not fixed. In an ISO message (which these are) the password, and all information in the message, is dependent on the data elements present in the message which are in turn are indicated by the bitmap. The bitmap is also part of the message. So in my case, I need to script additional logic above and beyond the answers provided to come full circle.

2) This is what I know and these are the steps I hope to accomplish with the script.

What I know: - There are 3 different msg types that contain passwords. I've figured out where the starting position of the password is for each msg type based on the bitmap and the data elements present. For example 0210 contains one in this case:

year-day-month 00:00:00,computername,source, info,rx 0210 22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff

What I need to do:

Pass the log file to the script
For each line in the log identify if the line has a msg type that contains a password
If the message type contains a password then determine length of password by reading the preceding 3 digits to the password ("ans ...999" which means alphanumeric - special with length max of 999 and 3 digit length info). Lets say the character position of the password would be 107 in this case for arguments sake, so we know to read the 3 numbers before it.
Starting at the character position of the password, mask the number of characters required with XXX. Loop through log until complete.

What is the logic to determine where the 3-digit number is in the line? Is it fixed character positions? — Alec Collier
– Alec Collier, Commented Oct 26, 2017 at 23:58
This masking isn't doing you too much good. You need to fix the root issue here which is you should never log passwords, let alone log them in plain text. — alroc
– alroc, Commented Oct 27, 2017 at 2:29
I understand the root cause and that it shouldn't be logged in the first place but that's outside the scope of my control. For reasons I won't get into on this forum I need to mask the passwords. I will be reviewing everyone's answers so far and providing additional information. I really appreciate the help, this is starting to lead me down the right path. — SADDAS16
– SADDAS16, Commented Oct 27, 2017 at 13:25

Mike Frank · Accepted Answer · 2017-10-27 04:58:01Z

1

It does seem as though you're indicating the position of the password and the length of the password will vary. As long as you have the '008' and something like '666' to indicate a starting and stopping point something like this should work.

$filePath = '.\YourFile.log'

(Get-Content $filePath) | ForEach-Object {
    $startIndex = $_.IndexOf('008') + 3
    $endIndex = $_.IndexOf('666', $startIndex)
    $passwordLength = $endIndex - $startIndex
    $passwordToReplace = $_.Substring($startIndex,$passwordLength)
    $obfuscation = New-Object 'string' -ArgumentList 'X', $passwordLength
    $_.Replace($passwordToReplace, $obfuscation)
} | Set-Content $filePath

If the file is too large to load into memory then you will have to StreamReader and StreamWriter to write the content to a new file and delete the old.

answered Oct 27, 2017 at 4:58

Mike Frank♦

3873 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mklement0 · Accepted Answer · 2017-10-27 01:00:08Z

Assuming a fixed position where the password-length field starts, based on your sample line (if that position is variable, as you've hinted at, you need to tell us more):

$line = '22222222222222333333333333333333444444444444444444444444455555008PASSWORD6666666666666666677777777777777777777777ccccdddddddddddffffffffffffff'

$posStart = 62 # fixed 0-based pos. where length-of-password field stats
$pwLenFieldLen = 3  # length of length-of-password field
$pwLen = [int] $line.SubString($posStart, $pwLenFieldLen) # extract password length
$pwSubstitute = 'X' * $pwLen # determine the password replacement string

# replace the password with all Xs
$line -replace "(?<=^.{$($posStart + $pwLenFieldLen)}).{$pwLen}(?=.*)", $pwSubstitute

Note: This is not the most efficient way to do it, but it is concise.

Collectives™ on Stack Overflow

How to parse logs and mask specific characters using Powershell

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related