0

I am trying to watch a log for certain messages within the last hour. The logs are formatted in this manner:

[1/18/19 9:59:13:791 CST] <Extra text here...>

I was having trouble with just doing a date comparison with awk, so my thought was to convert to epoch and compare. I am taking field 1 and cutting off the milliseconds from field 2, and removing [] (though I guess I could just do [ for my purposes).

while read -r line || [[ -n "$line" ]]
do
    log_date_str="$(awk '{gsub("\\[|\\]", "");print $1" "substr($2,1,length($2)-4)}' <<< "$line")"
    log_date="$(date -d "$log_date_str" +%s)"
    [[ $(($(date +%s)-$log_date)) -le 3600 ]] && echo "$line"
done < /path/to/file

When I try to run this against the log file though, I get this error:

date: invalid date `************ S'
-bash: 1547832909-: syntax error: operand expected (error token is "-")

Taking a single date, e.g. "1/18/19 9:59:13" works with the date conversion to epoch, but I'm not sure where to go with that error.

7
  • 2
    Running bash -x yourscript will let you see the commands as they're actually run, with the variables' real-world values; that makes it a lot easier to isolate what's going on, especially here where we don't know what your line contains, or what log_date_str or log_date values are in practice. Commented Jan 18, 2019 at 17:57
  • 1
    That said, I really don't know why you'd mix arithmetic and extended-test syntax this way. (( ( $(date +%s) - log_date ) <= 3600 )) && echo "$line" is less syntax. Commented Jan 18, 2019 at 17:58
  • 1
    Moreover, your code doesn't cause the stated error with only the line of input you provided; see it running correctly (and choosing not to print anything, but not throwing any error) at ideone.com/e57xWm. Please try to follow the minimal reproducible example definition in your code samples so others can actually see the problem you're asking about for themselves, and test their answers. Commented Jan 18, 2019 at 18:00
  • 1
    It looks like there's a log line that begins with like ************ S instead of the date and time. You need to filter out lines that don't follow the expected format. Commented Jan 18, 2019 at 18:08
  • That looks like the problem Barmar, I had been originally working with cat then grepping for a certain error, then trying to use awk. Didn't even cross my mind that I'm getting the entirety of the log using this method instead now. Commented Jan 18, 2019 at 18:14

1 Answer 1

0

As the comments pointed out, I was getting data that was not a date since it was parsing the entire log. Grepping the input file for my desired text solved my problems, and I've changed to Charles' suggestion.

while read -r line || [[ -n "$line" ]]
do
    log_date_str="$(awk '{gsub("\\[|\\]", "");print $1" "substr($2,1,length($2)-4)}' <<< "$line")"
    log_date="$(date -d "$log_date_str" +%s)"
    (( ( $(date +%s) - log_date ) <= 3600 )) && echo "$line"
done < <(grep ERROR_STRING /path/to/file.log)
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.