3

For sorting a file by column, Linux users have the utility sort. Windows users have to install, e.g. CoreUtils from GnuWin to get the same (or similar) functionality.

So, the minimal code for sorting the file first by column1 and then by column2, and then plotting the file would be something like this:

plot '<sort -k 1,2 "myFile.dat"' u 1:2

Now, however, I have a datablock $Data:

$Data <<EOD
1    6
4    8
3    7
2    5
1    4
2    3
EOD

The commands I tried so far which all end up in error messages:

plot '<sort -k 1,2' $Data u 1:2

#--> Bad data on line 1 of file <sort -k 1,2

plot '<sort -k 1,2  $Data' u 1:2
plot '<sort -k 1,2 <$Data' u 1:2 

#--> warning: Skipping data file with no valid points
#--> x range is invalid

plot '<sort -k 1,2 '<$Data u 1:2

#--> Column number or datablock line expected

I don't want to write the datablock to a file first and read it again from file. I currently don't see how I would redirect the content of $Data to the standard input of sort. Is there any solution for Windows as well as for Linux?

Update:

When using @Ethan's suggested code, I get the following result. Mind the lines 2 5 and 2 3 which I expected (and Ethan has it) the other way round.

# Curve 0 of 1, 6 points
# Curve title: "$Data_1 using 1:2:1"
# x y type
 1  4  i
 1  6  i
 2  5  i
 2  3  i
 3  7  i
 4  8  i

Any idea why this is? I'm running gnuplot 5.4.1 on Win10.

Improve this question

3 Answers 3

Reset to default
1

It wouldn't be gnuplot if there wasn't a workaround. Well, a bit cumbersome but it seems to work. Apparently, smooth zsort sorts each subblock separately. Hence, after the first sort you "simply" need to split your data into sub-blocks whenever the value in the first column changes.

  1. sort by the first column
  2. insert an empty line before the value in the first column changes
  3. sort by the second column
  4. plot it into a table to remove empty lines again

Code: (edit: with graphical representation it's easier to illustrate the undesired behaviour of zsort (under Windows only))

### sorting datablock, "bug": Windows zsort does not preserve order
reset session

# create some random test data 
set print $Data
    do for [i=1:100] {
        print sprintf("%g %g", int(rand(0)*10), int(rand(0)*10))
    }
set print

# order not preserving (only under Windows)
set table $Data1
    plot $Data  u 1:2:2 smooth zsort
set table $Data2
    plot $Data1  u 1:2:1 smooth zsort
unset table

# order preserving (even under Windows, but cumbersome)
set table $Data3
    plot $Data  u 1:2:1 smooth zsort
unset table
set print $Data4
    do for [i=1:|$Data3|] {
        print $Data3[i]
        if (i<|$Data3|) { if (word($Data3[i],1) ne word($Data3[i+1],1)) { print "" } }
    }
set print
set table $Data5
    plot $Data4 u 1:2:2 smooth zsort
set table $Data6
    plot $Data5 u 1:2 w table
unset table

set key out
set rmargin 20
set multiplot layout 3,1
    plot $Data  w lp pt 7 lc "black"     ti "Random"
    plot $Data2 w lp pt 7 lc "red"       ti "zsort"
    plot $Data6 w lp pt 7 lc "web-green" ti "Workaround"
unset multiplot
### end of code

Result:

enter image description here

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

should I have an additional data column (timestamp in my case, but that probably doesn't matter) how could I keep track of it in the resulting sorted file? (just track, no sorting). If I just do plot $Data u 1:2:2:3 smooth zsort where column no. 3 is my additional column, that column is absent in the output file
@ifffam If you don't mind using a temporary file on disk or using external tools or system (dependent) calls you could somehow modify Ethan's solution. Maybe there is even a gnuplot-only solution without an external file, but probably quite different from the solution above, I need to think about it. Could you please ask a new question with minimal data for testing?
'@theozh alright, between today and tomorrow I hope to do so. It will be for gnuplot 6, in which zsort works in a bit different way (it doubles the 1st column unlike previous versions) so I'll adapt the script accordingly
I've found a workaround and it has worked. Still there was something odd: havingset xrange [*:*],I tried to build a datablockset table $Data333 plot FILE u (gprintf("%.1f",$4)):2:1 smooth zsortand it returned me "x range is invalid". But if I write the format of the 1st column before,set format x "%.f" then plot FILE u 4:2:1 smooth zsort works perfectly. If I don't write the format before and I just plot FILE u 4:2:1 smooth zsort, it gives the 1st column with exponential format which makes me miss the last 3 or 4 digits. Isn't it possible to enforce an explicit gprintf with zsort?
@ifffam so, you want to sort only by one column and not by two (numeric) columns? gprintf("%.1f",$4) is a number formatted as string, but I guess smooth zsort expects a number. So, by set format x "%.1f" you will get your number with the desired digits.
1

I am going to back up and suggest that you reconsider your initial restriction against using a temporary file. The most straightforward solution is this:

   set print "| sort -k 1,2 > sorted.dat"
   print $Data
   unset print
   plot 'sorted.dat'

If you explain why you don't want to use a temp file, maybe there is an answer to that question independent of the sorting issue. If the concern is the name of the temp file, then perhaps something like this:

  tempfile = system("mktemp")
  set print "| sort -k 1,2 > ".tempfile
  print $Data
  unset print
  plot tempfile with points
Improve this answer

2 Comments

well, writing to a temporary file certainly works. I hope there will be no file opening, writing, closing and re-reading timing issues. If I have data in (fast) RAM, why should I write it to (slow) HDD just for sorting it? If you just sort once, the longer time might not be an issue. If you have to sort again and again, this would go towards data processing and analysis, and gnuplot doesn't want to be a tool for such tasks. My simple hope was that zsort (although is was not intended for sorting like that) still would work. However, now the Windows implementation of zsort thwarts this plan.
I know I can always use Python or other programming languages for preparing the data for gnuplot, although I would prefer a gnuplot-only solution. So, admittedly, besides platform independence, I do not have a good argument for making Windows zsort working the same way as Linux zsort does.
1

The syntax for sending $Data to stdin of the sort utility is set print "| sort"; print $Data. But that won't do what you want. Instead let's perform the double sort inside gnuplot.

$Data <<EOD
1    6
4    8
3    7
2    5
1    4
2    3
EOD

set table $Data_1
plot $Data using 1:2:2 smooth zsort with points
set table $Data_2
plot $Data_1 using 1:2:1 smooth zsort with points
unset table
print $Data_2

# Curve 0 of 1, 6 points
# Curve title: "$Data_1 using 1:2:1"
# x y type
 1  4  i
 1  6  i
 2  3  i
 2  5  i
 3  7  i
 4  8  i
Improve this answer

4 Comments

Thank you for your suggestion. That's a new feature since gnuplot 5.4.0. However, when I copy & paste your code I get a different result. I don't know why. I will edit my question.
gnuplot uses qsort(). In linux glibc, qsort is implemented as a merge sort that preserves the previous order of "equal" elements (I think this is not guaranteed by the API but is currently true in practice). This makes multi-pass sorting possible. I have no idea how qsort is implemented on Windows. Gnuplot's smooth zsort could be modified to guarantee a sub-sort order z/x/y (or z/y/x?) at some smallish cost in execution time. If you want to advocate for that, please raise the issue on the development mailing list.
Thank you for the explanation. Hmm, Windows again :-| It would be really good to have preserving order. I can file a "bug" report on the bugtracker on Sourceforge or would you prefer the development mailing list? What's the difference? The "audience"?
Bug report filed: sourceforge.net/p/gnuplot/bugs/2446 Might there eventually be the chance to include alphanumerical sort?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.