Polars Rust API creating a dataframe from a string variable / reading csv with options from a string

Question

Using the Polars Rust API, is it possible to create a DataFrame directly from a CSV string / reader while specifying options such as the separator?

Currently, I'm working around this by saving the string to a temporary path and reading it using LazyCsvReader::new("tmp_path.csv").

In the eventual use case, (potentially large) CSV data is received via e.g. a request.

use anyhow::Result;
use polars::prelude::*;

fn main() -> Result<()> {
    let csv_str = "name|age|city
Alice|30|New York
Bob|25|London";

    // Writing it to a file, but I'd prefer to read the CSV data directly.
    std::fs::write("tmp.csv", csv_str)?;
    let df = LazyCsvReader::new("tmp.csv").with_separator(b'|').finish()?.collect()?;

    // Also tried `CsvReader`, though I couldn't figure out how to make it work with a custom delimiter.
    /* let cursor = std::io::Cursor::new(csv_str);
    let df = CsvReader::new(cursor).finish()?; */

    println!("{df}");

    Ok(())
}

What is your rust-polars version

user459872
– user459872

2024-11-30 23:04:42 +00:00
Commented Nov 30, 2024 at 23:04 — user459872
– user459872, Commented Nov 30, 2024 at 23:04
Current latest: 0.44

tenxsoydev
– tenxsoydev

2024-11-30 23:11:46 +00:00
Commented Nov 30, 2024 at 23:11 — tenxsoydev
– tenxsoydev, Commented Nov 30, 2024 at 23:11

user459872 · Accepted Answer · 2024-11-30 23:33:49Z

2

You can use the into_reader_with_file_handle method of the CsvReadOptions struct to creates a CSV reader using a file handle. Use the map_parse_options to set the custom separator.

use polars::prelude::*;

fn main() {
    let csv_str = "name|age|city
Alice|30|New York
Bob|25|London";
    let handle = std::io::Cursor::new(csv_str);

    let df = CsvReadOptions::default()
        .with_has_header(true)
        .map_parse_options(|parse_options| parse_options.with_separator(b'|'))
        .into_reader_with_file_handle(handle)
        .finish()
        .unwrap();
    println!("{:?}", df);
}

answered Nov 30, 2024 at 23:33

user459872

25.9k4 gold badges51 silver badges74 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Polars Rust API creating a dataframe from a string variable / reading csv with options from a string

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related