Compile error when compiling polars for python

Question

When compiling polars python package I get following error, any idea which version of rust should I use? Is there any workaround/fix for this?

I used following commands:

python -m pip install --no-binary :all: maturin
python -m pip install --no-binary :all: Polars==0.18.15

C:\Users\ashish.shirodkar>rustc --version
rustc 1.74.0-nightly (58eefc33a 2023-08-24)

Error:

   Compiling pyo3-built v0.4.7
         Compiling lz4 v1.24.0
         Compiling zstd v0.12.4
         Compiling parquet2 v0.17.2
         Compiling polars-error v0.32.0 (C:\Users\ashish.shirodkar\AppData\Local\Temp\pip-install-cor5irss\polars_180ec2d9ea9044ce86687698c1427599\local_dependencies\polars-error)
         Compiling polars-row v0.32.0 (C:\Users\ashish.shirodkar\AppData\Local\Temp\pip-install-cor5irss\polars_180ec2d9ea9044ce86687698c1427599\local_dependencies\polars-row)
      error: assigning to `&T` is undefined behavior, consider using an `UnsafeCell`
         --> local_dependencies\polars-core\src\chunked_array\ops\rolling_window.rs:120:29
          |
      96  |               let ptr = array_ptr.as_ref() as *const dyn Array as *mut dyn Array
          |  _______________________-
      97  | |                 as *mut PrimitiveArray<T::Native>;
          | |_________________________________________________- casting happend here
      ...
      120 |                               *ptr = arr_window;
          |                               ^^^^^^^^^^^^^^^^^
          |
          = note: for more information, visit <https://doc.rust-lang.org/book/ch15-05-interior-mutability.html>
          = note: `#[deny(invalid_reference_casting)]` on by default

I tried using older rust versions but no luck.

github.com/pola-rs/polars/issues/10604 but also this seems like straight UB so idk why they do this — drewtato
– drewtato, Commented Aug 25, 2023 at 2:51
You must use the exact compiler version we have defined in our rust-toolchain.toml. — ritchie46
– ritchie46, Commented Aug 25, 2023 at 7:54
@ritchie46 apparently you are a maintainer of polars (or at least acquainted with the code base). Do you know why polars would have this code which, as drewtato pointed out, directly produces UB? — jthulhu
– jthulhu, Commented Aug 25, 2023 at 9:56
@ritchie46 Using an older compiler version does not work; it "works" but silently gives UB and potentially miscompilations. — Chayim Friedman
– Chayim Friedman, Commented Aug 25, 2023 at 13:16
The code above has been fixed. The comment was more in general. If you compile python polars, you must follow the compiler version we use to compile it. — ritchie46
– ritchie46, Commented Aug 27, 2023 at 4:35

Ashish Shirodkar · Accepted Answer · 2023-08-27 18:06:20Z

0

I followed below steps as mentioned by @ritchie46 and it worked:

Get the version source code from GitHUB: https://github.com/pola-rs/polars/releases/tag/py-0.18.15
Install rust compiler from https://www.rust-lang.org/tools/install
Get the version of rust mentioned in rust-toolchain.toml in source downloaded from GitHub.

rustup install nightly-2023-07-27
rustup default nightly-2023-07-27-x86_64-pc-windows-msvc

C:\Users\ashish.shirodkar>rustc --version
rustc 1.73.0-nightly (0d95f9132 2023-07-26)
Execute following commands to compile and install Polars:

python -m pip install --no-binary :all: maturin
python -m pip install --no-binary :all: Polars==0.18.15

answered Aug 27, 2023 at 18:06

Ashish Shirodkar

3981 silver badge11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Compile error when compiling polars for python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related