Deserialize a JSON string or array of strings into a Vec

Question

I'm writing a crate that interfaces with a JSON web API. One endpoint usually returns responses of the form { "key": ["value1", "value2"] }, but sometimes there's only one value for the key, and the endpoint returns { "key": "value" } instead of { "key": ["value"] }

I wanted to write something generic for this that I could use with #[serde(deserialize_with)] like so:

#[derive(Deserialize)]
struct SomeStruct {
    #[serde(deserialize_with = "deserialize_string_or_seq_string")]
    field1: Vec<SomeStringNewType>,

    #[serde(deserialize_with = "deserialize_string_or_seq_string")]
    field2: Vec<SomeTypeWithCustomDeserializeFromStr>,
}

#[derive(Deserialize)]
struct SomeStringNewType(String);

struct SomeTypeWithCustomDeserializeFromStr(String);
impl ::serde::de::Deserialize for SomeTypeWithCustomDeserializeFromStr {
    // Some custom implementation here
}

How can I write a deserialize_string_or_seq_string to be able to do this?

Pit · Accepted Answer · 2017-04-26 17:24:56Z

11

In case you want to deserialize a single string or a list of strings into the more general Vec<String> instead of a custom type, the following is a simpler solution for Serde 1.0:

extern crate serde;
#[macro_use] extern crate serde_derive;
extern crate serde_json;

use std::fmt;
use std::marker::PhantomData;

use serde::de;
use serde::de::{Deserialize, Deserializer};

#[derive(Deserialize, Debug, Clone)]
pub struct Parent {
    #[serde(deserialize_with = "string_or_seq_string")]
    pub strings: Vec<String>,
}

fn main() {
    let list_of_strings: Parent = serde_json::from_str(r#"{ "strings": ["value1", "value2"] }"#).unwrap();
    println!("list of strings: {:?}", list_of_strings);
    // Prints:
    //   list of strings: Parent { strings: ["value1", "value2"] }

    let single_string: Parent = serde_json::from_str(r#"{ "strings": "value" }"#).unwrap();
    println!("single string: {:?}", single_string);
    // Prints:
    //   single string: Parent { strings: ["value"] }
}

fn string_or_seq_string<'de, D>(deserializer: D) -> Result<Vec<String>, D::Error>
    where D: Deserializer<'de>
{
    struct StringOrVec(PhantomData<Vec<String>>);

    impl<'de> de::Visitor<'de> for StringOrVec {
        type Value = Vec<String>;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str("string or list of strings")
        }

        fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
            where E: de::Error
        {
            Ok(vec![value.to_owned()])
        }

        fn visit_seq<S>(self, visitor: S) -> Result<Self::Value, S::Error>
            where S: de::SeqAccess<'de>
        {
            Deserialize::deserialize(de::value::SeqAccessDeserializer::new(visitor))
        }
    }

    deserializer.deserialize_any(StringOrVec(PhantomData))
}

This solution also works under the 0.9 release of Serde with the following changes:

remove the lifetimes
SeqAccess -> SeqVisitor
SeqAccessDeserializer -> SeqVisitorDeserializer
MapAccess -> MapVisitor
MapAccessDeserializer -> MapVisitorDeserializer

edited Apr 26, 2017 at 17:24

answered Apr 26, 2017 at 7:14

Pit

4,1362 gold badges27 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Arnavion Over a year ago

This does not deserialize SomeStringNewType(String) or SomeTypeWithCustomDeserializeFromStr(String) as the question requires.

Pit Over a year ago

Sorry -- my understanding was that SomeStringNewType was some form of workaround because you couldn't get it to work with String. If you don't mind, I would still keep this answer around because at least I came across this question because I searched for its title. I have updated the answer to specify that this is for the general form of wanting a simple String.

Arnavion Over a year ago

Yes yes, your answer is good for the case where the user wants to deserialize String directly. No reason to delete it.

Peter Segerstedt · Accepted Answer · 2022-02-15 07:49:33Z

I found this pattern to work for me in a similar situation:

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
#[serde(untagged)]
enum ParameterValue {
    Primitive(String),
    List(Vec<String>),
}

#[derive(Debug, Serialize, Deserialize)]
struct Parameter {
    name: String,
    value: ParameterValue,
}

example primitive:

let primitive = Parameter {
    name: String::from("theKey"),
    value: ParameterValue::Primitive(String::from("theValue")),
};
let primitive_serialized = serde_json::to_string(&primitive).unwrap();
println!("{primitive_serialized}");
let primitive_again: Parameter = serde_json::from_str(&primitive_serialized).unwrap();
println!("{primitive_again:?}");

Prints:

{"name":"theKey","value":"theValue"}
Parameter { name: "theKey", value: Primitive("theValue") }

example array:

let list = Parameter {
    name: String::from("theKey"),
    value: ParameterValue::List(vec![String::from("v1"), String::from("v2")]),
};
let list_serialized = serde_json::to_string(&list).unwrap();
println!("{list_serialized}");
let list_again: Parameter = serde_json::from_str(&list_serialized).unwrap();
println!("{list_again:?}");

Prints:

{"name":"theKey","value":["v1","v2"]}
Parameter { name: "theKey", value: List(["v1", "v2"]) }

Shepmaster · Accepted Answer · 2017-04-26 12:16:51Z

This solution works for Serde 1.0.

The way I found also required me to write a custom deserializer, because I needed one that would call visitor.visit_newtype_struct to try deserializing newtypes, and there don't seem to be any in-built into serde that do so. (I was expecting something like the ValueDeserializer series of types.)

A self-contained example is below. The SomeStruct is deserialized correctly for both inputs, one where the values are JSON arrays of strings, and the other where they're just strings.

#[macro_use]
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;

fn main() {
    #[derive(Debug, Deserialize)]
    struct SomeStringNewType(String);

    #[derive(Debug)]
    struct SomeTypeWithCustomDeserializeFromStr(String);
    impl<'de> ::serde::Deserialize<'de> for SomeTypeWithCustomDeserializeFromStr {
        fn deserialize<D>(deserializer: D) -> Result<Self, D::Error> where D: ::serde::Deserializer<'de> {
            struct Visitor;

            impl<'de> ::serde::de::Visitor<'de> for Visitor {
                type Value = SomeTypeWithCustomDeserializeFromStr;

                fn expecting(&self, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result {
                    write!(f, "a string")
                }

                fn visit_str<E>(self, v: &str) -> Result<Self::Value, E> where E: ::serde::de::Error {
                    Ok(SomeTypeWithCustomDeserializeFromStr(v.to_string() + " custom"))
                }
            }

            deserializer.deserialize_any(Visitor)
        }
    }

    #[derive(Debug, Deserialize)]
    struct SomeStruct {
        #[serde(deserialize_with = "deserialize_string_or_seq_string")]
        field1: Vec<SomeStringNewType>,

        #[serde(deserialize_with = "deserialize_string_or_seq_string")]
        field2: Vec<SomeTypeWithCustomDeserializeFromStr>,
    }

    let x: SomeStruct = ::serde_json::from_str(r#"{ "field1": ["a"], "field2": ["b"] }"#).unwrap();
    println!("{:?}", x);
    assert_eq!(x.field1[0].0, "a");
    assert_eq!(x.field2[0].0, "b custom");

    let x: SomeStruct = ::serde_json::from_str(r#"{ "field1": "c", "field2": "d" }"#).unwrap();
    println!("{:?}", x);
    assert_eq!(x.field1[0].0, "c");
    assert_eq!(x.field2[0].0, "d custom");
}

/// Deserializes a string or a sequence of strings into a vector of the target type.
pub fn deserialize_string_or_seq_string<'de, T, D>(deserializer: D) -> Result<Vec<T>, D::Error>
    where T: ::serde::Deserialize<'de>, D: ::serde::Deserializer<'de> {

    struct Visitor<T>(::std::marker::PhantomData<T>);

    impl<'de, T> ::serde::de::Visitor<'de> for Visitor<T>
        where T: ::serde::Deserialize<'de> {

        type Value = Vec<T>;

        fn expecting(&self, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result {
            write!(f, "a string or sequence of strings")
        }

        fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
            where E: ::serde::de::Error {

            let value = {
                // Try parsing as a newtype
                let deserializer = StringNewTypeStructDeserializer(v, ::std::marker::PhantomData);
                ::serde::Deserialize::deserialize(deserializer)
            }.or_else(|_: E| {
                // Try parsing as a str
                let deserializer = ::serde::de::IntoDeserializer::into_deserializer(v);
                ::serde::Deserialize::deserialize(deserializer)
            })?;
            Ok(vec![value])
        }

        fn visit_seq<A>(self, visitor: A) -> Result<Self::Value, A::Error>
            where A: ::serde::de::SeqAccess<'de> {

            ::serde::Deserialize::deserialize(::serde::de::value::SeqAccessDeserializer::new(visitor))
        }
    }

    deserializer.deserialize_any(Visitor(::std::marker::PhantomData))
}

// Tries to deserialize the given string as a newtype
struct StringNewTypeStructDeserializer<'a, E>(&'a str, ::std::marker::PhantomData<E>);

impl<'de, 'a, E> ::serde::Deserializer<'de> for StringNewTypeStructDeserializer<'a, E> where E: ::serde::de::Error {
    type Error = E;

    fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error> where V: ::serde::de::Visitor<'de> {
        visitor.visit_newtype_struct(self)
    }

    fn deserialize_string<V>(self, visitor: V) -> Result<V::Value, Self::Error> where V: ::serde::de::Visitor<'de> {
        // Called by newtype visitor
        visitor.visit_str(self.0)
    }

    forward_to_deserialize_any! {
        bool i8 i16 i32 i64 u8 u16 u32 u64 f32 f64 char str bytes
        byte_buf option unit unit_struct newtype_struct seq tuple tuple_struct map
        struct enum identifier ignored_any
    }
}

atok · Accepted Answer · 2023-06-21 11:39:23Z

The shortest way of doing that is by introducing an Enum and using Serde from and into to translate both variants into Vec.

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Example {
    pub field: OneOrVec<String>,
}

With the following definitions:

#[derive(Serialize, Deserialize, Debug, Clone)]
#[serde(try_from = "OneOrVecEnum<T>", into = "OneOrVecEnum<T>")]
pub struct OneOrVec<T>
where
    T: Clone,
{
    pub items: Vec<T>,
}

impl<T> From<OneOrVec<T>> for OneOrVecEnum<T>
where
    T: Clone,
{
    fn from(value: OneOrVec<T>) -> Self {
        if value.items.len() == 1 {
            let mut items = value.items;
            OneOrVecEnum::One(items.remove(0))
        } else {
            OneOrVecEnum::Multiple(value.items)
        }
    }
}

impl<T> From<OneOrVecEnum<T>> for OneOrVec<T>
where
    T: Clone,
{
    fn from(value: OneOrVecEnum<T>) -> Self {
        match value {
            OneOrVecEnum::One(item) => OneOrVec { items: vec![item] },
            OneOrVecEnum::Multiple(vec) => OneOrVec { items: vec },
        }
    }
}

The Clone trait is optional but helpful to have.

ITChap · Accepted Answer · 2023-10-05 09:00:27Z

1

If somebody else is still struggling with this, serde_with crate as an annotation just for this: OneOrMany

answered Oct 5, 2023 at 9:00

ITChap

4,8421 gold badge32 silver badges56 bronze badges

1 Comment

Chayim Friedman Over a year ago

It'll be good if you'll add an code example of how to use it.

Collectives™ on Stack Overflow

Deserialize a JSON string or array of strings into a Vec

5 Answers 5

3 Comments

Comments

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related