gnuplot: Plot data and use max of repeated x value

Question

I want to plot a curve of a huge data file with two columns:

datafile (example):

#dateYMD sum
2014-02-01 70
2014-02-01 85
2014-02-01 95
2014-02-02 116
2014-02-02 123
2014-02-09 130
2014-02-09 134
2014-02-11 145

if I use "plot 'data.txt' using 0:1 with lines" and set the date format etc right I'll get a plot which is nearly correct, but unfortunately it uses the "lower/upper" value if more than one line per date is given. I want to plot the maximum of the sum-column per day in my plot. I want to use the last/highest given value for each day.

You can filter your data with awk '{if (date && date != $1) {print date, val}; date = $1; val=$2}'. This will leave only last value for each date. — max taldykin
– max taldykin, Commented Feb 24, 2014 at 13:27
@hfs, you are right. I don't know how to fix this without making code twice as long. — max taldykin
– max taldykin, Commented Sep 3, 2014 at 20:28

Community · Accepted Answer · 2017-05-23 10:26:11Z

4

That is a bit tricky. My answer to Plotting different columns on the same file using boxes shows how that can be done for a numeric x-axis. If you have time data, you must use timecolumn(1) instead of $1, and thats it:

reset
xval = -1e10
max(x, y) = (x > y ? x : y)
maxval = 0

set timefmt '%Y-%m-%d'
set xdata time

plot 'data.txt' using (val = $2, timecolumn(1)):\
     (maxval_prev = (xval == timecolumn(1) ? maxval : 0), \
      maxval = (xval == timecolumn(1) ? max(maxval, val) : val),\
      xval = timecolumn(1), \
      (maxval > maxval_prev ? maxval-maxval_prev : 0)\
     ) \
     smooth frequency lw 3 with linespoints t 'maximum values'

That gives the result (with 4.6.3):

enter image description here

See the above linked answer for a more detailed explanation.

edited May 23, 2017 at 10:26

CommunityBot

11 silver badge

answered Feb 24, 2014 at 13:10

Christoph

48.7k8 gold badges94 silver badges198 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sehe Over a year ago

Wow this saved me a lot of time today

theozh · Accepted Answer · 2018-12-08 08:54:11Z

2

+125

Another solution might be the code below. It is assumed that the dates are strictly ascending or descending, but not random. The accepted solution is certainly the more efficient solution but not so easy to follow. The basic idea is to write the last date (of several identical dates) and its value into a new datablock and simply plot this new dataset.

### start code
reset session

$Data <<EOD
#dateYMD sum
2014-02-01 70
2014-02-01 85
2014-02-01 95
2014-02-02 116
2014-02-02 123
2014-02-09 130
2014-02-09 134
2014-02-11 145
EOD

stats $Data nooutput
set table $Dummy
set print $Data2
tmp = ""
do for [i=STATS_records-1:0:-1] {
   plot $Data u (a=stringcolumn(1),b=stringcolumn(2),$2) every ::i::i with table
   if (tmp ne a) { print sprintf("%s\t%s",a,b); tmp = a}
}
set print
unset table

set xdata time
set timefmt "%Y-%m-%d"
set yrange[0:160]
plot $Data2 u (timecolumn(1)):2 w lp lw 2 pt 7 t "maximum values"
### end code

edited Dec 8, 2018 at 8:54

answered Dec 7, 2018 at 17:36

theozh

27.1k6 gold badges42 silver badges92 bronze badges

2 Comments

Christoph Over a year ago

smooth frequency sorts the data, so this is assumption is always valid, no need to iterate over all rows again and again.

theozh Over a year ago

I agree, going through the loop with every is probably not very efficient. And yes, smooth frequency sorts the data, however, after "smoothing". If you test your script with dates which are not in strict ascending order you will not get the desired result. Although, the values on a same certain date can be random. So, at least you have to assume that the dates are strictly in ascending order or you first have to sort the dates with an external tool.

theozh · Accepted Answer · 2018-12-09 10:22:31Z

Ok, I guess here is a solution which

handles unsorted dates
returns maximum and minimum values
is gnuplot-only (no external tools)

A little drawback: I don't know how to prevent warnings from 'stats'.

### start code
reset session

# unsorted dates and values
$Data <<EOD
#dateYMD sum
2014-02-01 85
2014-02-09 134
2014-02-01 95
2014-02-02 116
2014-02-01 70
2014-02-02 123
2014-02-09 130
2014-02-11 145
EOD

# get unique dates in seconds
set xdata time
set format x "%s"
set table $DataInSeconds
    plot $Data u (timecolumn(1,"%Y-%m-%d")):2  smooth freq
unset table
set xdata  # stats will not work with 'xdata time'
stats $DataInSeconds nooutput
UniqueDates = STATS_records

# get the maximum and minimum value per date
set print $MinMaxData
set table $Dummy
do for [i=0:UniqueDates-1] {
    plot $DataInSeconds u (a=$1,$1):2 every ::i::i with table
    stats [a:a] $Data u (timecolumn(1,"%Y-%m-%d")):2 nooutput
    print sprintf("%s\t%g\t%g", strftime("%Y-%m-%d",a), STATS_min_y, STATS_max_y)
}
unset table
set print
print $MinMaxData

# plot the results
set key top left
set yrange[0:160]
set xdata time
set format x "%m/%d"
plot $MinMaxData u (timecolumn(1,"%Y-%m-%d")):3 \
        w lp lw 2 pt 7 ps 2 lc rgb "red" t "maximum values",\
     '' u (timecolumn(1,"%Y-%m-%d")):2 \
        w lp lw 2 pt 6 ps 2 lc rgb "web-green" t "minimum values",\
     $Data u (timecolumn(1,"%Y-%m-%d")):2 \
        w p lw 2 pt 2 ps 1 lc rgb "blue" t "all values"
### end code

will result in:

Collectives™ on Stack Overflow

gnuplot: Plot data and use max of repeated x value

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related