Python: setting generator Polynomial in zlib.crc32

Question

I am trying to use zlib.crc32 to compute CRC in Python. I would like to be able to set the generator polynomial of the CRC, but I cannot find any documentation. So the question is: can this be done?

I know that there is crcmod library that can do this, but in my case I want to only use standard Python libraries and don't want to install any external library. Is there any standard Python library that can be used to calculate CRC and I can define its generator polynomial?

in crcmod it is done as:

import crcmod

crc32 = crcmod.Crc(poly=0x104c11db7, rev=True, initCrc=0, xorOut=0xFFFFFFFF)

crc32.update(bytearray(input_string))

in the above code the poly is the generator polynomial.

Unless it is there but not mentioned in the documentation, there is no direct way to use the zlib module for that. — John Coleman
– John Coleman, Commented Dec 1, 2021 at 18:52
Thanks @JohnColeman. Do you know any other standard Python library that can do this? — TJ1
– TJ1, Commented Dec 1, 2021 at 18:54
I don't think there is any in the standard library. You could implement it in raw python if needed. The Wikipedia article on CRC has a Python implementation (albeit one which seems designed for pedagogy rather than efficiency). — John Coleman
– John Coleman, Commented Dec 1, 2021 at 18:59
As I don't have a permission to install a package, can I copy all files of a library to a folder and direct python to look at that folder for the crc files? — TJ1
– TJ1, Commented Dec 1, 2021 at 19:06

Mark Adler · Accepted Answer · 2021-12-02 04:11:42Z

5

No, the polynomial and other CRC parameters are hard-coded in zlib.

However the CRC-32 generated by those parameters is the CRC that zlib implements. You can just use zlib.crc32.

>>> import crcmod
>>> crc32 = crcmod.mkCrcFun(poly=0x104c11db7, rev=True, initCrc=0, xorOut=0xFFFFFFFF)
>>> print(hex(crc32(b'123456789')))
0xcbf43926
>>> import zlib
>>> print(hex(zlib.crc32(b'123456789')))
0xcbf43926

Update:

From the comments, speed is not important, and they want a different CRC polynomial.

Here is a simple CRC-32 implementation, assuming rev=True, for which you can change or parameterize the polynomial, the initial value, and the final exclusive-or:

def crc32(msg):
    crc = 0xffffffff
    for b in msg:
        crc ^= b
        for _ in range(8):
            crc = (crc >> 1) ^ 0xedb88320 if crc & 1 else crc >> 1
    return crc ^ 0xffffffff

print(hex(crc32(b'123456789')))

Prints: 0xcbf43926.

If you need something faster, then you'd need to generate a table for a byte-wise table-driven CRC implementation.

Update:

As the original poster continues to trickle out tiny bits of information about their actual question in comments (as opposed to simply saying in total what they want to do in the question), we learn that they need to generate a forward CRC as well (rev=False), specifically a CRC-32/MPEG-2. So here is the simple code for that:

def crc32mpeg2(msg):
    crc = 0xffffffff
    for b in msg:
        crc ^= b << 24
        for _ in range(8):
            crc = (crc << 1) ^ 0x04c11db7 if crc & 0x80000000 else crc << 1
    return crc & 0xffffffff

print(hex(crc32mpeg2(b'123456789')))

Prints: 0x376e6e7.

edited Dec 2, 2021 at 4:11

answered Dec 1, 2021 at 19:26

Mark Adler

115k15 gold badges137 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

15 Comments

TJ1 Over a year ago

thanks for the answer, but that generator polynomial I used in crcmod was just an example, and I need to change it for my case. Is there any other standard library that can do that?

Mark Adler Over a year ago

Then you should put in the question what CRC you actually want, as opposed to the wrong CRC.

TJ1 Over a year ago

The CRC that I want will be different for my different cases, so it is not just a fixed polynomial. That is why I am asking if the polynomial can be defined.

Mark Adler Over a year ago

That's the same polynomial. When the CRC is reversed (rev=True), the polynomial is bit-flipped in the implementation. 0xedb88320 is 0x04c11db7 flipped.

Mark Adler Over a year ago

If you want a tutorial on CRCs and their calculation, I recommend this painless guide.

|

Collectives™ on Stack Overflow

Python: setting generator Polynomial in zlib.crc32

1 Answer 1

15 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

15 Comments

Your Answer

Sign up or log in

Post as a guest

Related