27

I want to create my own parameterized type in Python for use in type hinting:

class MaybeWrapped:
    # magic goes here

T = TypeVar('T')

assert MaybeWrapped[T] == Union[T, Tuple[T]]

Never mind the contrived example; how can I implement this? I looked at the source for Union and Optional, but it looks like some fairly low-level hackery that I'd like to avoid.

The only suggestion in the documentation comes from an example re-implementation of Mapping[KT,VT] that inherits from Generic. But that example is more about the __getitem__ method than about the class itself.

3 Answers 3

23

If you're just trying to create generic classes or functions, try taking a look at the documentation on mypy-lang.org about generic types -- it's fairly comprehensive, and more detailed then the standard library typing docs.

If you're trying to implement your specific example, it's worth pointing out that type aliases work with typevars -- you can simply do:

from typing import Union, TypeVar, Tuple

T = TypeVar('T')

MaybeWrapped = Union[T, Tuple[T]]

def foo(x: int) -> MaybeWrapped[str]:
    if x % 2 == 0:
        return "hi"
    else:
        return ("bye",)

# When running mypy, the output of this line is:
# test.py:13: error: Revealed type is 'Union[builtins.str, Tuple[builtins.str]]'
reveal_type(foo(3))

However, if you're trying to construct a generic type with genuinely new semantics, you're very likely out of luck. Your remaining options are to:

  1. Construct some kind of custom class/metaclass thing that PEP 484-compliant type checkers can understand and use that.
  2. Modify the type checker you're using somehow (mypy has an experimental "plugin" system, for example)
  3. Petition to modify PEP 484 to include your new, custom type (you can do this by opening an issue in the typing module repo).
Sign up to request clarification or add additional context in comments.

2 Comments

Thanks, using TypeVar here is what I was missing. Pretty amazing that generics "just work" like this.
Out of curiosity, do you happen to know if the "Construct some kind of custom class/metaclass thing that PEP 484-compliant type checkers can understand and use that" option is possible now?
10

It is exactly the __getitem__ method that does all the magic.

That is the method called in when you subscribe one name with [ and ] brackets.

So, you need an __getitem__ method in the class of your class - that is, its metaclass, that will get as parameters whatever is within the brackets. That method is responsible for dynamically creating (or retrieving a cached copy) of whatever you want to generate, and return it.

I just can't possibly imagin how you want this for type hinting, since the typing library seems to cover all reasonable cases (I can't think of an example they don't cover already). But let's suppose you want a class to return a copy of itself, but with the parameter anotated as its type_ attribute:

class MyMeta(type):
    def __getitem__(cls, key):
        new_cls = types.new_class(f"{cls.__name__}_{key.__name__}", (cls,), {}, lambda ns: ns.__setitem__("type", key))
        return new_cls

class Base(metaclass=MyMeta): pass

And on trying this in interactive mode, one can do:

In [27]: Base[int]
Out[27]: types.Base_int

update: As of Python 3.7, there is also the special method __class_getitem__ which is created just for this purpose: it acts as a classmethod and avoids the need or a metaclass just for this case. Whatever would be written in a metaclass.__getitem__ can be put in the cls.__class_getitem__ method directly. Defined in PEP 560

5 Comments

Thanks, although I'm still not sure how this all works. Can you speak to the example in my question? The example in your answer seems a little different from what I'm trying to do.
@jsbueno -- I don't think your answer works. The code you proposed is certainly a way to construct something that looks like a PEP 484 type, but since it isn't, PEP 484 compliant type checkers won't understand what to do with your MyMeta or Base classes.
Wow! Great! Could you please explain a bit lambda ns: ns.__setitem__("type", key)? if I got well ns is so-called "namespace", but what is meaning in this context and where to find the provided key "type"?
Got it! it's just class's dict itself, so it can be accessed via attribute of class or instance. b=Base[int] assert b.type is int
As a response to " the typing library seems to cover all reasonable cases": In your reference of PEP 560, it says "All generic types are instances of GenericMeta, so if a user uses a custom metaclass, then it is hard to make a corresponding class generic." I have exactly that case.
4

I'd like to propose improved solution, based on @jsbueno answer. Now our "generics" can be used in comparisons and identity checks, and they will behave like "true" generics from typing. Also we can forbid instantiation of non-typed class itself. Moreover! We got isinstance checking for free!

Also meet BaseMetaMixin class for perfect static type checking!

import types
from typing import Type, Optional, TypeVar, Union

T = TypeVar('T')


class BaseMetaMixin:
    type: Type


class BaseMeta(type):
    cache = {}

    def __getitem__(cls: T, key: Type) -> Union[T, Type[BaseMetaMixin]]:
        if key not in BaseMeta.cache:
            BaseMeta.cache[key] = types.new_class(
                f"{cls.__name__}_{key.__name__}",
                (cls,),
                {},
                lambda ns: ns.__setitem__("type", key)
            )

        return BaseMeta.cache[key]

    def __call__(cls, *args, **kwargs):
        assert getattr(cls, 'type', None) is not None, "Can not instantiate Base[] generic"
        return super().__call__(*args, **kwargs)


class Base(metaclass=BaseMeta):
    def __init__(self, some: int):
        self.some = some


# identity checking
assert Base[int] is Base[int]
assert Base[int] == Base[int]
assert Base[int].type is int
assert Optional[int] is Optional[int]

# instantiation
# noinspection PyCallByClass
b = Base[int](some=1)
assert b.type is int
assert b.some == 1

try:
    b = Base(1)
except AssertionError as e:
    assert str(e) == 'Can not instantiate Base[] generic'

# isinstance checking
assert isinstance(b, Base)
assert isinstance(b, Base[int])
assert not isinstance(b, Base[float])

exit(0)
# type hinting in IDE
assert b.type2 is not None # Cannot find reference 'type2' in 'Base | BaseMetaMixin'
b2 = Base[2]()  # Expected type 'type', got 'int' instead

2 Comments

btw now we have __class_getitem__(cls, item), defining it in base class we can to do the same without metaclass.
This is a really cool idea. You could simplify your __getitem__ logic by using dict.setdefault

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.