4

Given a metaclass, or simpler, type(), the last argument stands as the class dict, the one which is in charge with class variables. I was wondering, is there a way to set instance variables from within a metaclass?

The reason I am using a metaclass is that I need to collect some variables defined on class creation and process them. But then, the result of this processing needs to be attached to each instance of the class, not to the class itself, since the result will be a list, which will differ from one instance of the class to another.

I will provide an example so it's easier to follow me.

I have a Model class, defined as follows:

class Model(object):
    __metaclass__ = ModelType

    has_many = None

    def __init__(self, **kwargs):
        self._fields = kwargs

The has_many class variable will be filled in at class definition by whatever model someone feels the need. In the __init__(), I simply assign the keywords provided at model instantiation to instance variable _fields.

I then have the ModelType metaclass:

class ModelType(type):
    def __new__(cls, name, bases, dct):
        if 'has_many' in dct:
            dct['_restricted_fields'].append([has-many-model-name])

What I need to do here is pretty straight-forward. I check if someone defined the has_many class variable on his custom model class, then add the variable's content to a list of restricted fields.

And now comes the important part. When instantiating a model, I need _fields to be composed of both the keyword arguments used for instantiating the model and the restricted fields, as processed in the metaclass. This would be pretty easy if _fields were a class variable in the Model class, but since it is an instance variable, I don't know how can I also add the restricted fields to it.

These being said, is there any way of achieving this? Or is there a better way of handling this? (I thought of using only a metaclass and set a _restricted_fields class variable on the model, then use this variable within the class __init__() to add the restricted fields to the normal fields, but very soon the model class will be cluttered of code which, in my opinion, should rest in another part of the code).

Improve this question
2
  • Is this kind of what you are looking for: when you create a new instance of a class, some variables associated with existing instances change? Commented Feb 1, 2014 at 18:02
  • Actually, this is what I want to prevent. That's why I'm looking for a mechanism defined at class creation, that will assign default values for variables that will be created when instantiating the class. Commented Feb 1, 2014 at 18:20

5 Answers 5

Reset to default
10
+100

Using a metaclass for this is not the correct approach here. A metaclass modifies the class-creation behavior not the instance creation behavior. You should use the __init__or the __new__ function to modify instance creation behavior. Wanting to use a metaclass for such things is using a hammer instead of a screwdriver to put a screw in a wall. ;-)

I'd suggest you use __new__ to achieve what you want. From the Python docs:

__new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.

class MetaModel(type):
    def __new__(cls, name, bases, attrs):
        attrs['_restricted_fields'] = [attrs.get('has_many')]
        return type.__new__(cls, name, bases, attrs)

class Model(object):
    __metaclass__ = MetaModel
    has_many = None
    def __new__(cls, *args, **kwargs):
        instance = object.__new__(cls, *args, **kwargs)
        instance.instance_var = ['spam']
        return instance

class SubModel(Model):
    has_many = True
    def __init__(self):
        # No super call here.
        self.attr = 'eggs'

s = SubModel()
assert s._restricted_fields == [True]  # Added by MetaModel
assert s.instance_var == ['spam']  # Added by Model.__new__
assert s.attr == 'eggs'  # Added by __init__

# instance_var is added per instance.
assert SubModel().instance_var is not SubModel().instance_var

The MetaModel is responsible for creating Model classes. It adds a _restricted_fields class variable to any Model class created by it (the value is a list containing the has_many class variable).

The Model class defines a default has_many class variable. It also modifies the instance creation behavior, it adds an instance_var attribute to each created instance.

The SubModel is created by a user of your code. It defines an __init__ function to modify instance creation. Note that it does not call any super-class function, this is not a necessity. The __init__ adds an attr attribute to each SubClass instance.

Improve this answer
Sign up to request clarification or add additional context in comments.

14 Comments

I like your point. But the reason I use a metaclass is that there are many operations which should be done and putting all of them in Model's __new__() will make things look more complicated for someone who uses the Model class. Also, those restricted fields should be computed when a class is created, not every time that class is instanced, since the restricted fields are the same for each instance of that specific class. That's why I felt a metaclass is more suitable for processing the restricted fields, but then I had a difficulty in assigning them to each instance.
@AndreiHorak I'd say do the processing of the class variables in a metaclass, this is indeed suitable as it concerns class-creation. The assigning however should be done in either the __init__ or the __new__ function. I'd update my answer but I'm having trouble interpreting the question correctly (it'd not really a SSCCE).
That's what I was thinking, but how should I link the processed variables in the metaclass to the class' __init__() or __new__()? The only way I could think of is by creating some class variables on the Model class and use them at instantiation, but as I said, the class will look cluttered and complicated.
@AndreiHorak The Model class should gain a metaclass (MetaModel or something) that does the processing of the class' variables. The Model class provides the __new__ function to modify instance creation. Both will look kinda simple. I'll try to add to the example.
It sounds fair enough. But the thing is, those instance variables are exactly the variables processed by the metaclass. That is, _restricted_fields in your example. Those are the variables I need to have in each instance. And also judging by your example, I have those variables defined on the Model class after the metaclass does its job. But this is my concern, I don't like these variables to be stored on the Model class because it's "back-scene" stuff that shouldn't be visible to whoever uses the Model class.
|
3

The place to set up instance variables is in the class's __init__ method. But if you don't want to make each class that uses your metaclass include the same code, why not have the metaclass provide its own __init__ method for the class, wrapping around whatever existing __init__ method is there:

class MyMetaclass(type):
    def __new__(mcs, name, bases, dct):
        if 'has_many' in dct:
            dct['_restricted_fields'].append(["something"])

        orig_init = dct.get("__init__") # will be None if there was no __init__

        def init_wrapper(self, *args, **kwargs):
            if orig_init:
                orig_init(self, *args, **kwargs)          # call original __init__
            self._fields = getattr(self, "_fields", [])   # make sure _fields exists
            self._fields.extend(["whatever"])             # make our own additions

        dct["__init__"] = init_wrapper   # replace original __init__ with our wrapper

        return type.__new__(mcs, name, bases, dct)

I don't entirely understand what the actual logic you want to implement is, but you can replace self._fields.extend(["whatever"]) in the wrapper function with whatever it is you actually want to do. self in that context is the instance, not the class or metaclass! The wrapper function is a closure, so you do have access to the metaclass in mcs and the class dictionary in dct if you need them.

Improve this answer

5 Comments

Seems an interesting approach. But given that my Model class will be usable by anyone out there, the code will be kind of hard to understand. But since it is only adding some extra stuff preserving the original __init__(), I believe it is not that bad, right?
You could simplify the metaclass a bit if you could rely on the regular classes using it to behave appropriately (rather than trying to be foolproof). That is, you could just assume they all have an __init__ method that sets _fields to some list. Though, if you only need your Model class to be reusable (rather than the metaclass), you could just put the logic in Model.__init__ directly (this seems to be what you've done so far). Metaclasses are for deep magic. I wouldn't ever expect them to be easy to understand (if it was easy, you wouldn't need a metaclass).
Yes, just because I want my Model class to be reusable, I need to make it simple. And by this, I mean storing any related data in a separate class so field computing and other stuff wouldn't be visible to someone who uses the Model class. Also, I want to have only required public instance and class variables, so no variables which act as helpers for some kind of functionality. That's why I am trying to move most of the "back scene" logic to other entities, such as metaclasses or other classes. However, I find it quite difficult to separate this class vs. instance logic...
Perhaps you should put the "behind the scenes" stuff in a base class, that Model inherits from. That won't have the stuff not appear on the class or instance, but it will separate out the bits that are implementation details from the public interface.
This solution to wrapping __init__ has a problem if a class does not implement __init__ but its base class does. In that case the metaclass creates a totally new __init__ method for the class that does not call the base class __init__ method, thus leaving the class inappropriately (un)initialised.
1

If it needs to be set per-instance, the place to put the code to calculate this whatever it is, is in the __init__.

However, be aware that class variables are available through instances, unless shadowed by an instance variable.

Improve this answer

3 Comments

This is what I've thought of, but as I said, it will eventually add too much code in my class' __init__() method. That's why I was looking for a way of defining extra things around my class, using metaclasses or whatever else would be suitable. Is this the only way around?
@AndreiHorak -- You can have __init__ call any other methods on your class that you'd like.
@AndreiHorak You could use __new__, but there's no reason to do that. As @mgilson says, if you want to keep code out of your init, put it in a normal method.
1

You could do this all with a factory instead of a metaclass.

The factory can create types and then also create instances of those types - however doing so at initialization time rather than at class-creation time. That keeps the related knowledge in one place (the factory) but doesn't try to shoehorn two different kinds of work into one operation.

OTOH you could also dynamically compose a function to be run at init time in each instance using info you had at class-creation time via a closure

class ExMeta(type):

        def __new__(cls, name, parents, kwargs):
            kwargs['ADDED_DYNAMICALLY'] = True

            secret_knowledge = 42
            def fake_init(*args, **kwargs):
                print "hello world... my args are:",  args, kwargs
                print "the meaning of the universe is ", secret_knowledge
            kwargs['_post_init'] = fake_init
            return super(ExMeta, cls).__new__(cls, name, parents, kwargs)

class Example(object):
    SOME_ATTR = 1
    SOME_OTHER = 2
    __metaclass__ = ExMeta       

    def __init__(self):
        self._post_init()


bob = Example()
> hello world... my args are: (<__main__.Example object at 0x0000000002425BA8>,) {}
> the meaning of the universe is  42
Improve this answer

2 Comments

This was suggested above, but it doesn't keep the Model class clean. It adds the _post_init() class argument which clutters the model class which should be simple; also, someone who uses the Model class expects a simple API and this is not achieved since that _post_init() is added within the metaclass, overcomplicating things for a simple API.
The init wrapper solution would let you hide this from the users. There's no otherway out of the dilemma of getting runtime initialization guaranteed without one line of code, either a callback like mine or a super call. I prefer making explicit so user can decide when to call it, the wrapper version is probably the closest to a hands of solution
0

Update which might show what I kind of was thinking:

class MetaKlass(object):
    def __new__(cls, *args, **kwargs):
        class_methods = {'foo'}
        _ins = super(MetaKlass, cls).__new__(cls)
        for k, v in kwargs.iteritems():
            if k in class_methods:
                _do = _ins.__getattribute__(k)
                setattr(_ins, k, _do(v)) # self monkey patching:
                                        # classmethod is replaced by its output
            else:
                setattr(_ins, k, v)
        return _ins

    def __init__(self, arg1, arg2, *args, **kwargs):
        self.a = arg1
        self.b = arg2

    @classmethod
    def foo(cls, x):
        return 42

So we get:

>>> from overflow_responses import MetaKlass
>>> klass_dict = {'foo': 1, 'angel': 2}
>>> k = MetaKlass(3, 7, **klass_dict)
>>> k.foo, k.angel, k.a, k.b
(42, 2, 3, 7)

Are you looking for something like this:

>>> kwd = {'a': 2}  # the 'class variables dict'
>>> class MetaKlass(object):
...     def __new__(cls, kwd):
...             _ins = super(MetaKlass, cls).__new__(cls)
...             for k, v in kwd.iteritems():
...                     setattr(_ins, k, v)
...             return _ins
...
>>> mk = MetaKlass(kwd)
>>> mk.a
2
>>>

The code should be fairly straight forward. When I organize a collection of methods into a class, many of those methods rely on both a set of class values and specific instance values.

Improve this answer

5 Comments

It is not quite what I've asked for. The metaclass part of setting class variables is done, I was asking how to set instance variables -- or provide defaults for them -- at class creation time.
@AndreiHorak This actually does exactly that. The main problem with it is that it might not call __init__ (unless the super new does that).
@AndreiHorak I provide a more tangible example of what I was thinking. Does this address your comment about calling __init__?
Well, metaclasses inherit from type, not from object, and I use a metaclass in my case specifically to parse the class variables at creation time. You example simply defines a base class which provides values for instance variables, but at class instantiation time, not at class creation time. I was looking for a way to provide this kind of functionality at class creation time, since providing it at instantiation time will lead to duplication of code in my case and it will be much harder to understand it.
Downvoted. Metaclasses inherit from type as @AndreiHorak said. Needless to say, you're not using it as a metaclass and are not providing an explanation for an alternative.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.