Pydantic validation errors with None values

Question

I have a returned result from a webservice, whose values come as they are (see example). The keys are not Optional and must be included.

2 validation errors for Result:

cause - str type expected (type=type.error.str)
urls - str type expected (type=type.error.str)


{ 'code': 0, 'codetext': 'blablabla', 'status': None, 'cause': None, 'urls': None }


class Result(BaseModel):
  
  code: int
  codetext: str
  status: str = ""
  cause: str = ""
  urls: str = ""

  @validator("status", "cause", "urls")
  def replace_none(cls, v):
    return v or "None"

First questions is, what is the correct way to handle the None values coming as answer from the webservice and set to 'None' (with single-quotation marks and None as String) and secondly, why are there only 2 and not 3 errors?

Could you rephrase what do you expect to happen to those Nones? If you want to accept them as input, you should mark them optional.To your second question, at least pydantic 1.9.0 errors on all fields (with a different, clearer error: none is not an allowed value (type=type_error.none.not_allowed) — teprrr
– teprrr, Commented Mar 15, 2022 at 14:15
@teprrr entered an explanation, what I expect. Thank you for your advice. AFAIK Optional allows to have the property or not, but I got a constraint to definitive have it. — Semo
– Semo, Commented Mar 15, 2022 at 14:26
There's a difference between "The key must be present" and "The key must not have null as a value". (I'm assuming that the None comes from a JSON null value.) — chepner
– chepner, Commented Mar 15, 2022 at 15:00
@chepner Everything you said is true. In the end both conditions must be met at the same time, not None (in a sense of a void, when the Webservice returns nothing) AND the properties will have to be present. — Semo
– Semo, Commented Mar 15, 2022 at 15:33

teprrr · Accepted Answer · 2022-03-15 14:52:14Z

4

You can mark fields as required but optional by declaring the field optional and using ellipsis (...) as its value. Here is a complete, simplified example with tests to verify it works (execute with pytest <file>):

import pytest
from pydantic import BaseModel, ValidationError, validator


class Result(BaseModel):
    code: int
    status: str | None = ...

    @validator("status")
    def replace_none(cls, v):
        return v or "None"


def test_result():
    p = Result.parse_obj({"code": 1, "status": "test status"})
    assert p.code == 1
    assert p.status == "test status"


def test_required_optionals():
    """Test that missing fields raise an error."""
    with pytest.raises(ValidationError):
        Result.parse_obj({"code": 1})


def test_required_optional_accepts_none():
    """Test that none values are accepted."""
    p = Result.parse_obj(
        {
            "code": 0,
            "status": None,
        }
    )
    assert p.status == "None"

answered Mar 15, 2022 at 14:52

teprrr

4443 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Semo Over a year ago

The line status: str | None = ... wouldn't work, because of E TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'. I replaced it with status: Optional[str] = Field(...) and it works.

teprrr Over a year ago

Ah, PEP 604 allowing that form of optionals is indeed available first since python 3.10. If really wanted, there's a way to use that since 3.7 by adding the following to the top of the file: from __future__ import annotations but I'm not sure if it works with pydantic as I presume it expects concrete types.

Semo Over a year ago

Plus one. In the end you helped to get back on track. import from future was helpful.

user64204 Over a year ago

Nobody asked to make fields optional. Optional fields lead to a shitload of type-juggling shenanigans down the line. Is there a way to make it required, but replaced by validator if None? Seems like no.

ruslaniv Over a year ago

But how can a field be both required AND optional? Are not these mutually exclusive?

|

Collectives™ on Stack Overflow

Pydantic validation errors with None values

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related