1

I'm trying to write a Rust program which uses Polars to read a CSV. This particular CSV encodes an array of floats as a string.

In my program, I want to load the CSV into a DataFrame and then parse this column into an array of floats. In Python you might write code that looks like this:

df = pd.read_csv("...")
df["babbage_search"] = df.babbage_search.apply(eval).apply(np.array)

However in my Rust program it's not clear how to go about this. I could take an approach like this:

let mut df = CsvReader::from_path("...")?
    .has_header(true)
    .finish()?;

df.apply("babbage_search", parse_vector)?;

However the parse_vector function is not really clear. I might write something like this, but this won't compile:

fn parse_vector(series: &Series) -> Series {
    series
        .utf8()
        .unwrap()
        .into_iter()
        .map(|opt_v| match opt_v {
            Some(v) => {
                let vec: Vec<f64> = serde_json::from_str(v).unwrap();
                let series: Series = vec.iter().collect();
                series.f64().unwrap().to_owned() as ChunkedArray<Float64Type>
            }

            None => ChunkedArray::<Float64Type>::default(),
        })
        .collect()
}

I'd appreciate any help in figuring this out.

2
  • 2
    "this won't compile" how? You should provide a minimal reproducible example including the error messages. Commented Dec 31, 2022 at 20:13
  • 3
    Important note: Please don't use eval to parse json in python, that's awfully and needlessly insecure. Commented Dec 31, 2022 at 20:16

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.