Non colliding hash algorithm for strings up to 255 characters

Question

I am looking for a hash-algorithm, to create as close to a unique hash of a string (max len = 255) as possible, that produces a long integer (DWORD).

I realize that 26^255 >> 2^32, but also know that the number of words in the English language is far less than 2^32.

The strings I need to 'hash' would be mostly single words or some simple construct using two or three words.

The answer:

One of the FNV variants should meet your requirements. They're fast, and produce fairly evenly distributed outputs. (Answered by Arachnid)

Community · Accepted Answer · 2017-05-23 12:13:34Z

2

See here for a previous iteration of this question (and the answer).

edited May 23, 2017 at 12:13

CommunityBot

11 silver badge

answered Sep 24, 2008 at 10:26

Nick Johnson

101k17 gold badges131 silver badges198 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

slashmais Over a year ago

Yes, that's what I'm looking for. I did search but the keywords I used does not appear in the link you gave. I added a comment there to add more relevant keywords.

Greg Hewgill · Accepted Answer · 2008-09-24 10:27:39Z

1

One technique is to use a well-known hash algorithm (say, MD5 or SHA-1) and use only the first 32 bits of the result.

Be aware that the risk of hash collisions increases faster than you might expect. For information on this, read about the Birthday Paradox.

answered Sep 24, 2008 at 10:27

Greg Hewgill

1.0m192 gold badges1.2k silver badges1.3k bronze badges

Comments

Armin Ronacher · Accepted Answer · 2008-09-24 10:28:26Z

1

Ronny Pfannschmidt did a test with common english words yesterday and hasn't encountered any collisions for the 10000 words he tested in the Python string hash function. I haven't tested it myself, but that algorithm is very simple and fast, and seems to be optimized for common words.

Here the implementation:

static long
string_hash(PyStringObject *a)
{
    register Py_ssize_t len;
    register unsigned char *p;
    register long x;

    if (a->ob_shash != -1)
        return a->ob_shash;
    len = Py_SIZE(a);
    p = (unsigned char *) a->ob_sval;
    x = *p << 7;
    while (--len >= 0)
        x = (1000003*x) ^ *p++;
    x ^= Py_SIZE(a);
    if (x == -1)
        x = -2;
    a->ob_shash = x;
    return x;
}

answered Sep 24, 2008 at 10:28

Armin Ronacher

32.6k14 gold badges68 silver badges69 bronze badges

2 Comments

Nick Johnson Over a year ago

Can we have a link to the article/post you're referencing?

Armin Ronacher Over a year ago

Was a discussion on IRC, no link. sorry

Catch22 · Accepted Answer · 2008-09-24 10:32:34Z

0

H(key) = [GetHash(key) + 1 + (((GetHash(key) >> 5) + 1) % (hashsize – 1))] % hashsize

MSDN article on HashCodes

answered Sep 24, 2008 at 10:32

Catch22

3,3611 gold badge30 silver badges34 bronze badges

Comments

Stephen Denne · Accepted Answer · 2008-09-24 10:33:20Z

0

Java's String.hash() can be easily viewed here, its algorithm is

s[0]*31^(n-1) + s[1]*31^(n-2) + ... + s[n-1]

answered Sep 24, 2008 at 10:33

Stephen Denne

37.2k10 gold badges49 silver badges61 bronze badges

Collectives™ on Stack Overflow

Non colliding hash algorithm for strings up to 255 characters

5 Answers 5

1 Comment

Comments

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related