Strange strings comparison in Unix sort

Question

I was playing with sort in Ubuntu and saw a strange thing: Sorting (sort test.txt):

d
auto_design
tst
auto_tuning
autosport

Gives:

auto_design
autosport
auto_tuning
d
tst

But sorting

d
auto_design
auto_tuning
autoaport

Gives

autoaport
auto_design
auto_tuning
d
tst

If it sorts lexicographical why "a", "b", "c" are less than "_", but letters after "c" are "bigger" than "_"? In the first case it breaks auto_* words with autosport word which does not contain _ and it seems strange to me.

Thanks in advance.

Any answer about sort order requires knowing your current locale, LC_ALL and LC_COLLATE settings. — Charles Duffy
– Charles Duffy, Commented Apr 21, 2015 at 22:52
If you were to look up "auto_design" in a dictionary, wouldn't you look between "autobahn" and "autodial"? — that other guy
– that other guy, Commented Apr 21, 2015 at 23:02
Ergo, it does not sort lexicographically. Lexicographic sorts have limited utility. — Tom Blodget
– Tom Blodget, Commented Apr 22, 2015 at 0:43

Charles Duffy · Accepted Answer · 2015-04-21 22:59:31Z

1

The character order used by sort is provided by your current locale settings.

If you want a minimum of surprises, and don't need locale-specific character ordering, set LC_COLLATE=C in your environment. This can be scoped to a single command like so:

LC_COLLATE=C sort test.txt

See the glibc documentation on locales for more information on how locales can be configured.

To give an example of "locale-specific collation order" -- in Estonian, ö sorts after w, whereas it's more typical for ö to sort somewhere between n and p... but in a pure ASCII sort, such characters follow after the entire a-z set in full.

edited Apr 21, 2015 at 22:59

answered Apr 21, 2015 at 22:56

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

DuXeN0N Over a year ago

Thanks. Can you say how it sorts when this env variable is empty? Comparing by what?

Charles Duffy Over a year ago

If LC_COLLATE is empty, then LC_ALL is checked. If no collation order is set anywhere in the system, the default is C (or its synonym POSIX), but a modern operating system will almost always set it somewhere.

Charles Duffy Over a year ago

Setting LANG will also impact collation order, if LC_COLLATE and LC_ALL are not set.

Collectives™ on Stack Overflow

Strange strings comparison in Unix sort

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related