Subclassing Sequence with proper type hints in Python

Question

I'm trying to implement a kind of custom sequence class in Python:

from typing import Sequence, TypeVar, List

T = TypeVar('T')

class MySequence(Sequence[T]):
    def __init__(self):
        self._container: Sequence[T] = []
    def __getitem__(self, idx):
        return self._container[idx]
    def __len__(self):
        return len(self._container)

Now I want to check that mypy is aware that elements of MySequence are items of type T:

foo: MySequence[str] = MySequence()
reveal_type(foo[0])
# Revealed type is 'Any'

So it fails: mypy knows nothing about items of foo. The same example for ordinary Sequence works:

bar: Sequence[str] = []
reveal_type(bar[0])
# Revealed type is 'builtins.str*'

If I'm trying to add type annotations to __getitem__ implementation, I have another error:

def __getitem__(self, idx) -> T:
# Signature of "__getitem__" incompatible with supertype "Sequence"

I also tried

def __getitem__(self, idx) -> Union[T, Sequence[T]]:

as idx can be a slice and in that case my code will return a sequence instead of one element. It fails either with the same message.

As discussed in my previous question, there is an open discussion on issues like that.

However, I still wonder, is it possible to create custom sequence types that allow mypy to extract information about the type of its items, like in my example?

psarka · Accepted Answer · 2021-10-07 17:44:24Z

10

In this case, the correct thing to do is to correctly override the exact signature for __getitem__, including the overloads.

from typing import Sequence, TypeVar, List, overload, Union

T = TypeVar('T', covariant=True)

class MySequence(Sequence[T]):
    def __init__(self):
        self._container: Sequence[T] = []

    @overload
    def __getitem__(self, idx: int) -> T: ...

    @overload
    def __getitem__(self, s: slice) -> Sequence[T]: ...

    def __getitem__(self, item):
        if isinstance(item, slice):
            raise Exception("Subclass disallows slicing")

        return self._container[item]

    def __len__(self) -> int:
        return len(self._container)

foo: MySequence[str] = MySequence()
reveal_type(foo[0])

(Note that I made the typevar covariant. This isn't, strictly-speaking, required, but if the container is effectively meant to represent a "read-only" kind of structure, we might as well for maximum flexibility.)

Note: the fact that mypy decides that the return type is Any in the first example is expected behavior. According to PEP 484, any method or signature without type annotations is treated as if the argument and return types are all Any.

This is a mechanism designed so that untyped Python code is treated as fully dynamic by default.

Mypy comes built-in with a variety of command line arguments you can use to try and coerce it to check the contents of untyped functions anyways (I believe it's --check-untyped-defs?), though it won't attempt to infer what the return type is.

edited Oct 7, 2021 at 17:44

psarka

1,8521 gold badge19 silver badges32 bronze badges

answered Oct 12, 2017 at 23:46

Michael0x2a

65.7k32 gold badges190 silver badges242 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Zachary Ryan Smith Over a year ago

Note that (at least on my machine) the order of @overload matters. If switch the order, then I get error: Signature of "__getitem__" incompatible with supertype "Sequence" on the first @overload.

jamesc Over a year ago

This strategy also helped me when inheriting from list and overriding __setitem__().

Collectives™ on Stack Overflow

Subclassing Sequence with proper type hints in Python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related