Rust appending unicode escape before creating new struct

Question

I am reading from a file and then creating a struct from those values.

let file = File::open(self.get_state_name().add(".sky")).unwrap();
let reader = BufReader::new(file);
for (_, line) in reader.lines().enumerate() {
    let line = line.unwrap();
    let key_value = line.split("`").collect::<Vec<&str>>();
    let key = key_value[0].to_string();
    let data = key_value[1].to_string();
    self.set(key, data);
}

Set function creates a new struct named model

let model = Model::new(key, data);

New function just returns a struct named model:

pub fn new(key: String, data: String) -> Model {
   Model { key, data }
}

Value of key is prefixed with unicode escapes like:

Model {
    key: "\u{f}\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}Test",
    data: "Test Data",
},

Update: Tried saving only ascii characters:

pub fn new(key: String, data: String) -> Model {
   let key = key.replace(|c: char| !c.is_ascii(), "");
   println!("Key: {}", key);
   Model { key, data }
}

Update: Saving file

let mut file = File::create(name.add(".sky")).unwrap();
for i in &data {
    let data = i.to_string();
    let bytes = bincode::serialize(&data).unwrap();
    file.write_all(&bytes).expect("Unable to write to file");
}

.to_string() methods on struct

pub(crate) fn to_string(&self) -> String {
    format!("{}`{}\n", self.key, self.data)
}

Here is key is without unicode escapes. It happens during Model { key, data } line.

Same doesn't happen when directly setting value not reading from file. How to remove this and why is this happening?

How did you save the file? It looks like you saved the value 15 as a 64-bit little endian value before the string itself. — CodesInChaos
– CodesInChaos, Commented Dec 24, 2021 at 21:30

Marcus Ilgner · Accepted Answer · 2021-12-24 22:12:41Z

3

You are writing the code using bincode::serialize but read back the data not with bincode::deserialize but using BufReader.

In order to properly serialize the string in a binary fashion, the encoder adds additional information about the data it stores.

If you know that only strings compatible with BufReader#lines will be processed, you can also use String#as_bytes when writing it to a file. Note that this will cause problems for some inputs, notably newline characters and others.

answered Dec 24, 2021 at 22:12

Marcus Ilgner

7,3093 gold badges34 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Rust appending unicode escape before creating new struct

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related