Using `super()` within `__init_subclass__` doesn't find parent's classmethod - python

I try to access the classmethod of a parent from within __init_subclass__ however that doesn't seem to work.
Suppose the following example code:
class Foo:
def __init_subclass__(cls):
print('init', cls, cls.__mro__)
super(cls).foo()
#classmethod
def foo(cls):
print('foo')
class Bar(Foo):
pass
which produces the following exception:
AttributeError: 'super' object has no attribute 'foo'
The cls.__mro__ however shows that Foo is a part of it: (<class '__main__.Bar'>, <class '__main__.Foo'>, <class 'object'>).
So I don't understand why super(cls).foo() doesn't dispatch to Foo.foo. Can someone explain this?

A normal super object (what you normally get from calling super(MyType, self) or super() or super(MyType, myobj)) keeps track of both the type and the object it was created with. Whenever you look up an attribute on the super, it skips over MyType in the method resolution order, but if it finds a method it binds it to that self object.
An unbound super has no self object. So, super(cls) skips over cls in the MRO to find the method foo, and then binds it to… oops, it has nothing to call it on.
So, what things can you call a classmethod on? The class itself, or a subclass of it, or an instance of that class or subclass. So, any of those will work as the second argument to super here, the most obvious one being:
super(cls, cls)
This is somewhat similar to the difference between staticmethods (bound staticmethods are actually bound to nothing) and classmethods (bound classmethods are bound to the class instead of an instance), but it's not quite that simple.
If you want to know why an unbound super doesn't work, you have to understand what an unbound super really is. Unfortunately, the only explanation in the docs is:
If the second argument is omitted, the super object returned is unbound.
What does this mean? Well, you can try to work it out from first principles as a parallel to what it means for a method to be unbound (except, of course, that unbound methods aren't a thing in modern Python), or you can read the C source, or the original introduction to 2.2's class-type unification (including a pure-Python super clone).
A super object has a __self__ attribute, just like a method object. And super(cls) is missing its __self__, just like str.split is.1
You can't use an unbound super explicitly the way you can with an unbound method (e.g., str.split('123', '2') does the same as '123'.split('2'), but super(cls).foo(cls) doesn't work the same as super(cls, cls).foo()). But you can use them implicitly, the same way you do with unbound methods all the time without normally thinking about it.
If you don't know how methods work, the tl'dr is: when you evaluate myobj.mymeth, Python looks up mymeth, doesn't find it on myobj itself, but does find it on the type, so it checks whether it's a non-data descriptor, and, if so, calls its __get__ method to bind it to myobj.
So, unbound methods2 are non-data descriptors whose __get__ method returns a bound method. Unbound #classmethods are similar, but their __get__ ignores the object and returns a bound method bound to the class. And so on.
And unbound supers are non-data descriptors whose __get__ method returns a bound super.
Example (credit to wim for coming up with the closest thing to a use for unbound super that I've seen):
class A:
def f(self): print('A.f')
class B(A):
def f(self): print('B.f')
b = B()
bs = super(B)
B.bs = bs
b.bs.f()
We created an unbound super bs, stuck it on the type B, and then b.bs is a normal bound super, so b.bs.f is A.f, just like super().f would have been inside a B method.
Why would you want to do that? I'm not sure. I've written all kinds of ridiculously dynamic and reflective code in Python (e.g., for transparent proxies to other interpreters), and I can't remember ever needing an unbound super. But if you ever need it, it's there.
1. I'm cheating a bit here. First, unbound methods aren't a thing anymore in Python 3—but functions work the same way, so Python uses them where it used to use unbound methods. Second, str.split, being a C builtin, wasn't properly an unbound method even in 2.x—but it acts like one anyway, at least as far as we're concerned here.
2. Actually plain-old functions.

Related

Are there more than three types of methods in Python?

I understand there are at least 3 kinds of methods in Python having different first arguments:
instance method - instance, i.e. self
class method - class, i.e. cls
static method - nothing
These classic methods are implemented in the Test class below including an usual method:
class Test():
def __init__(self):
pass
def instance_mthd(self):
print("Instance method.")
#classmethod
def class_mthd(cls):
print("Class method.")
#staticmethod
def static_mthd():
print("Static method.")
def unknown_mthd():
# No decoration --> instance method, but
# No self (or cls) --> static method, so ... (?)
print("Unknown method.")
In Python 3, the unknown_mthd can be called safely, yet it raises an error in Python 2:
>>> t = Test()
>>> # Python 3
>>> t.instance_mthd()
>>> Test.class_mthd()
>>> t.static_mthd()
>>> Test.unknown_mthd()
Instance method.
Class method.
Static method.
Unknown method.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
This error suggests such a method was not intended in Python 2. Perhaps its allowance now is due to the elimination of unbound methods in Python 3 (REF 001). Moreover, unknown_mthd does not accept args, and it can be bound to called by a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator).
Questions
Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
Among the classic method types, what type of method is unknown_mthd?
Why can unknown_mthd be called by the class without passing an argument?
Some preliminary inspection yields inconclusive results:
>>> # Types
>>> print("i", type(t.instance_mthd))
>>> print("c", type(Test.class_mthd))
>>> print("s", type(t.static_mthd))
>>> print("u", type(Test.unknown_mthd))
>>> print()
>>> # __dict__ Types, REF 002
>>> print("i", type(t.__class__.__dict__["instance_mthd"]))
>>> print("c", type(t.__class__.__dict__["class_mthd"]))
>>> print("s", type(t.__class__.__dict__["static_mthd"]))
>>> print("u", type(t.__class__.__dict__["unknown_mthd"]))
>>> print()
i <class 'method'>
c <class 'method'>
s <class 'function'>
u <class 'function'>
i <class 'function'>
c <class 'classmethod'>
s <class 'staticmethod'>
u <class 'function'>
The first set of type inspections suggests unknown_mthd is something similar to a staticmethod. The second suggests it resembles an instance method. I'm not sure what this method is or why it should be used over the classic ones. I would appreciate some advice on how to inspect and understand it better. Thanks.
REF 001: What's New in Python 3: “unbound methods” has been removed
REF 002: How to distinguish an instance method, a class method, a static method or a function in Python 3?
REF 003: What's the point of #staticmethod in Python?
Some background: In Python 2, "regular" instance methods could give rise to two kinds of method objects, depending on whether you accessed them via an instance or the class. If you did inst.meth (where inst is an instance of the class), you got a bound method object, which keeps track of which instance it is attached to, and passes it as self. If you did Class.meth (where Class is the class), you got an unbound method object, which had no fixed value of self, but still did a check to make sure a self of the appropriate class was passed when you called it.
In Python 3, unbound methods were removed. Doing Class.meth now just gives you the "plain" function object, with no argument checking at all.
Was making a method this way intentional in Python 3's design?
If you mean, was removal of unbound methods intentional, the answer is yes. You can see discussion from Guido on the mailing list. Basically it was decided that unbound methods add complexity for little gain.
Among the classic method types, what type of method is unknown_mthd?
It is an instance method, but a broken one. When you access it, a bound method object is created, but since it accepts no arguments, it's unable to accept the self argument and can't be successfully called.
Why can unknown_mthd be called by the class without passing an argument?
In Python 3, unbound methods were removed, so Test.unkown_mthd is just a plain function. No wrapping takes place to handle the self argument, so you can call it as a plain function that accepts no arguments. In Python 2, Test.unknown_mthd is an unbound method object, which has a check that enforces passing a self argument of the appropriate class; since, again, the method accepts no arguments, this check fails.
#BrenBarn did a great job answering your question. This answer however, adds a plethora of details:
First of all, this change in bound and unbound method is version-specific, and it doesn't relate to new-style or classic classes:
2.X classic classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
>>> class A(object):
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
3.X new-style classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<function A.meth at 0x7efd07ea0a60>
You've already mentioned this in your question, it doesn't hurt to mention it twice as a reminder.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
Moreover, unknown_mthd does not accept args, and it can be bound to a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator)
unknown_meth doesn't accept args, normally because you've defined the function without so that it does not take any parameter. Be careful and cautious, static methods as well as your coded unknown_meth method will not be magically bound to a class when you reference them through the class name (e.g, Test.unknown_meth). Under Python 3.X Test.unknow_meth returns a simple function object in 3.X, not a method bound to a class.
1 - Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
I cannot speak for CPython developers nor do I claim to be their representative, but from my experience as a Python programmer, it seems like they wanted to get rid of a bad restriction, especially given the fact that Python is extremely dynamic, not a language of restrictions; why would you test the type of objects passed to class methods and hence restrict the method to specific instances of classes? Type testing eliminates polymorphism. It would be decent if you just return a simple function when a method is fetched through the class which functionally behaves like a static method, you can think of unknown_meth to be static method under 3.X so long as you're careful not to fetch it through an instance of Test you're good to go.
2- Among the classic method types, what type of method is unknown_mthd?
Under 3.X:
>>> from types import *
>>> class Test:
... def unknown_mthd(): pass
...
>>> type(Test.unknown_mthd) is FunctionType
True
It's simply a function in 3.X as you could see. Continuing the previous session under 2.X:
>>> type(Test.__dict__['unknown_mthd']) is FunctionType
True
>>> type(Test.unknown_mthd) is MethodType
True
unknown_mthd is a simple function that lives inside Test__dict__, really just a simple function which lives inside the namespace dictionary of Test. Then, when does it become an instance of MethodType? Well, it becomes an instance of MethodType when you fetch the method attribute either from the class itself which returns an unbound method or its instances which returns a bound method. In 3.X, Test.unknown_mthd is a simple function--instance of FunctionType, and Test().unknown_mthd is an instance of MethodType that retains the original instance of class Test and adds it as the first argument implicitly on function calls.
3- Why can unknown_mthd be called by the class without passing an argument?
Again, because Test.unknown_mthd is just a simple function under 3.X. Whereas in 2.X, unknown_mthd not a simple function and must be called be passed an instance of Test when called.
Are there more than three types of methods in Python?
Yes. There are the three built-in kinds that you mention (instance method, class method, static method), four if you count #property, and anyone can define new method types.
Once you understand the mechanism for doing this, it's easy to explain why unknown_mthd is callable from the class in Python 3.
A new kind of method
Suppose we wanted to create a new type of method, call it optionalselfmethod so that we could do something like this:
class Test(object):
#optionalselfmethod
def optionalself_mthd(self, *args):
print('Optional-Self Method:', self, *args)
The usage is like this:
In [3]: Test.optionalself_mthd(1, 2)
Optional-Self Method: None 1 2
In [4]: x = Test()
In [5]: x.optionalself_mthd(1, 2)
Optional-Self Method: <test.Test object at 0x7fe80049d748> 1 2
In [6]: Test.instance_mthd(1, 2)
Instance method: 1 2
optionalselfmethod works like a normal instance method when called on an instance, but when called on the class, it always receives None for the first parameter. If it were a normal instance method, you would always have to pass an explicit value for the self parameter in order for it to work.
So how does this work? How you can you create a new method type like this?
The Descriptor Protocol
When Python looks up a field of an instance, i.e. when you do x.whatever, it check in several places. It checks the instance's __dict__ of course, but it also checks the __dict__ of the object's class, and base classes thereof. In the instance dict, Python is just looking for the value, so if x.__dict__['whatever'] exists, that's the value. However, in the class dict, Python is looking for an object which implements the Descriptor Protocol.
The Descriptor Protocol is how all three built-in kinds of methods work, it's how #property works, and it's how our special optionalselfmethod will work.
Basically, if the class dict has a value with the correct name1, Python checks if it has an __get__ method, and calls it like type(x).whatever.__get__(x, type(x)) Then, the value returned from __get__ is used as the field value.
So for example, a trivial descriptor which always returns 3:
class GetExample:
def __get__(self, instance, cls):
print("__get__", instance, cls)
return 3
class Test:
get_test = GetExample()
Usage is like this:
In[22]: x = Test()
In[23]: x.get_test
__get__ <__main__.Test object at 0x7fe8003fc470> <class '__main__.Test'>
Out[23]: 3
Notice that the descriptor is called with both the instance and the class type. It can also be used on the class:
In [29]: Test.get_test
__get__ None <class '__main__.Test'>
Out[29]: 3
When a descriptor is used on a class rather than an instance, the __get__ method gets None for self, but still gets the class argument.
This allows a simple implementation of methods: functions simply implement the descriptor protocol. When you call __get__ on a function, it returns a bound method of instance. If the instance is None, it returns the original function. You can actually call __get__ yourself to see this:
In [30]: x = object()
In [31]: def test(self, *args):
...: print(f'Not really a method: self<{self}>, args: {args}')
...:
In [32]: test
Out[32]: <function __main__.test>
In [33]: test.__get__(None, object)
Out[33]: <function __main__.test>
In [34]: test.__get__(x, object)
Out[34]: <bound method test of <object object at 0x7fe7ff92d890>>
#classmethod and #staticmethod are similar. These decorators create proxy objects with __get__ methods which provide different binding. Class method's __get__ binds the method to the instance, and static method's __get__ doesn't bind to anything, even when called on an instance.
The Optional-Self Method Implementation
We can do something similar to create a new method which optionally binds to an instance.
import functools
class optionalselfmethod:
def __init__(self, function):
self.function = function
functools.update_wrapper(self, function)
def __get__(self, instance, cls):
return boundoptionalselfmethod(self.function, instance)
class boundoptionalselfmethod:
def __init__(self, function, instance):
self.function = function
self.instance = instance
functools.update_wrapper(self, function)
def __call__(self, *args, **kwargs):
return self.function(self.instance, *args, **kwargs)
def __repr__(self):
return f'<bound optionalselfmethod {self.__name__} of {self.instance}>'
When you decorate a function with optionalselfmethod, the function is replaced with our proxy. This proxy saves the original method and supplies a __get__ method which returns a boudnoptionalselfmethod. When we create a boundoptionalselfmethod, we tell it both the function to call and the value to pass as self. Finally, calling the boundoptionalselfmethod calls the original function, but with the instance or None inserted into the first parameter.
Specific Questions
Was making a method this way (without args while not explicitly
decorated as staticmethods) intentional in Python 3's design? UPDATED
I believe this was intentional; however the intent would have been to eliminate unbound methods. In both Python 2 and Python 3, def always creates a function (you can see this by checking a type's __dict__: even though Test.instance_mthd comes back as <unbound method Test.instance_mthd>, Test.__dict__['instance_mthd'] is still <function instance_mthd at 0x...>).
In Python 2, function's __get__ method always returns a instancemethod, even when accessed through the class. When accessed through an instance, the method is bound to that instance. When accessed through the class, the method is unbound, and includes a mechanism which checks that the first argument is an instance of the correct class.
In Python 3, function's __get__ method will return the original function unchanged when accessed through the class, and a method when accessed through the instance.
I don't know the exact rationale but I would guess that type-checking of the first argument to a class-level function was deemed unnecessary, maybe even harmful; Python allows duck-typing after all.
Among the classic method types, what type of method is unknown_mthd?
unknown_mthd is a plain function, just like any normal instance method. It only fails when called through the instance because when method.__call__ attempts to call the function unknown_mthd with the bound instance, it doesn't accept enough parameters to receive the instance argument.
Why can unknown_mthd be called by the class without passing an
argument?
Because it's just a plain function, same as any other function. I just doesn't take enough arguments to work correctly when used as an instance method.
You may note that both classmethod and staticmethod work the same whether they're called through an instance or a class, whereas unknown_mthd will only work correctly when when called through the class and fail when called through an instance.
1. If a particular name has both a value in the instance dict and a descriptor in the class dict, which one is used depends on what kind of descriptor it is. If the descriptor only defines __get__, the value in the instance dict is used. If the descriptor also defines __set__, then it's a data-descriptor and the descriptor always wins. This is why you can assign over top of a method but not a #property; method only define __get__, so you can put things in the same-named slot in the instance dict, while #properties define __set__, so even if they're read-only, you'll never get a value from the instance __dict__ even if you've previously bypassed property lookup and stuck a value in the dict with e.g. x.__dict__['whatever'] = 3.

What exactly does super() return in Python 3?

From Python3's documentation super() "returns a proxy object that delegates method calls to a parent or sibling class of type." What does that mean?
Suppose I have the following code:
class SuperClass():
def __init__(self):
print("__init__ from SuperClass.")
print("self object id from SuperClass: " + str(id(self)))
class SubClass(SuperClass):
def __init__(self):
print("__init__ from SubClass.")
print("self object id from SubClass: " + str(id(self)))
super().__init__()
sc = SubClass()
The output I get from this is:
__init__ from SubClass.
self object id from SubClass: 140690611849200
__init__ from SuperClass.
self object id from SuperClass: 140690611849200
This means that in the line super().__init__(), super() is returning the current object which is then implicitly passed to the superclass' __init__() method. Is this accurate or am I missing something here?
To put it simply, I want to understand the following:
When super().__init__() is run,
What exactly is being passed to __init__() and how? We are calling it on super() so whatever this is returning should be getting passed to the __init__() method from what I understand about Python so far.
Why don't we have to pass in self to super().__init__()?
returns a proxy object that delegates method calls to a parent or
sibling class of type.
This proxy is an object that acts as the method-calling portion of the parent class. It is not the class itself; rather, it's just enough information so that you can use it to call the parent class methods.
If you call __init__(), you get your own, local, sub-class __init__ function. When you call super(), you get that proxy object, which will redirect you to the parent-class methods. Thus, when you call super().__init__(), that proxy redirects the call to the parent-class __init__ method.
Similarly, if you were to call super().foo, you would get the foo method from the parent class -- again, re-routed by that proxy.
Is that clear to you?
RESPONSES TO OP COMMENTS
But that must mean that this proxy object is being passed to
init() when running super().init() right?
Wrong. The proxy object is like a package name, such as calling math.sqrt(). You're not passing math to sqrt, you're using it to denote which sqrt you're using. If you wanted to pass the proxy to init, the call would be init(super()). That call would be semantically ridiculous, of course.
When we have to actually pass in self which is the sc object in my example.
No, you are not passing in sc; that is the result of the object creation call (internal method __new__), which includes an invocation of init. For __init__, the self object is a new item created for you by the Python run-time system. For most class methods, that first argument (called self out of convention, this in other languages) is the object that invoked the method.
This means that in the line super().__init__(), super() is returning the current object which is then implicitly passed to the superclass' __init__() method. Is this accurate or am I missing something here?
>>> help(super)
super() -> same as super(__class__, <first argument>)
super call returns a proxy/wrapper object which remembers:
The instance invoking super()
The class of the calling object
The class that's invoking super()
This is perfectly sound. super always fetches the attribute of the next class in the hierarchy ( really the MRO) that has the attribute that you're looking for. So it's not returning the current object, but rather and more accurately, it returns an object that remembers enough information to search for attributes higher in the class hierarchy.
What exactly is being passed to __init__() and how? We are calling it on super() so whatever this is returning should be getting passed to the __init__() method from what I understand about Python so far.
You're almost right. But super loves to play tricks on us. super class defines __getattribute__, this method is responsible for attribute search. When you do something like: super().y(), super.__getattribute__ gets called searching for y. Once it finds y it passes the instance that's invoking the super call to y. Also, super has __get__ method, which makes it a descriptor, I'll omit the details of descriptors here, refer to the documentation to know more. This answers your second question as well, as to why self isn't passed explicitly.
*Note: super is a little bit different and relies on some magic. Almost for all other classes, the behavior is the same. That is:
a = A() # A is a class
a.y() # same as A.y(a), self is a
But super is different:
class A:
def y(self):
return self
class B(A):
def y(self)
return super().y() # equivalent to: A.y(self)
b = B()
b.y() is b # True: returns b not super(), self is b not super()
I wrote a simple test to investigate what CPython does for super:
class A:
pass
class B(A):
def f(self):
return super()
#classmethod
def g(cls):
return super()
def h(selfish):
selfish = B()
return super()
class C(B):
pass
c = C()
for method in 'fgh':
super_object = getattr(c, method)()
print(super_object, super_object.__self__, super_object.__self_class__, super_object.__thisclass__) # (These methods were found using dir.)
The zero-argument super call returns an object that stores three things:
__self__ stores the object whose name matches the first parameter of the method—even if that name has been reassigned.
__self_class__ stores its type, or itself in the case of a class method.
__thisclass__ stores the class in which the method is defined.
(It is unfortunate that __thisclass__ was implemented this way rather than fetching an attribute on the method because it makes it impossible to use the zero-argument form of super with meta-programming.)
The object returned by super implements getattribute, which forwards method calls to the type found in the __mro__ of __self_class__ one step after __thisclass__.

Unimplemented __init__()

I was looking over some code with a misspelled __init__() function (it had a total of 3 underbars instead of 4). And I realized I didn't really know what happens in a Python when an object cannot find a suitable __init__().
Is there a default initializer that is inserted automatically? Is there a default __new__ that gets called? Why does a constructor with arguments fail?
class TestClass(object):
def do_something(self):
print("Hello From TestClass!")
# What is called here?
instance = TestClass()
instance.do_something()
# Why does this fail?
argument = 100
instance = TestClass(argument)
Is there a default initializer that is inserted automatically?
Yes. There's a default dummy constructor on every class unless overwritten/inherited.
Is there a default __new__ that gets called?
Yes. Again by default it doesn't do much except for actually creating the object and setting its type.
Why does a constructor with arguments fail?
Because the default constructor takes no arguments and does nothing.
Everything inherits from object in Python 3 so if no base classes are supplied, defaults will be used for it.
#freakish answered your questions but I'll add a coded example to complement his explanation.
Let's create a new baseClass for which our new class myclass will inherit:
class baseClass:
def __init__(self):
print("Inherited __init__ got called")
def __new__(self, *args, **kwargs):
print("Inherited __new__ got called")
return object.__new__(self, *args, **kwargs)
This class overloads __init__ and __new__ and so any class which subclasses it will get these methods. Let's create a sample class which inherits from baseClass now:
class mycls(baseClass):
pass
Empty, no dunders defined. When we create an instance of this class though, we can see baseClass's inherited methods get called:
m = mycls()
Inherited __new__ got called
Inherited __init__ got called
So as you can see, we'll get everything our parents in the inheritance tree have defined. But, since classes with no explicit parent always inherit from object:
class f: pass
f.__bases__ # bases gives us the inherited classes
Out[218]: (object,)
We will always have these methods to our disposal with default behaviour defined. For the default __init__ this means no arguments, you need to redefine it in order to customize it!
You're asking about two frequently confused topics: the initializer __init__ and the constructor __new__. Also, I think you're confusing Python with languages like Java, where the constructor is a Big Deal and Not Like The Others, and where construction and initialization are mixed up in a single process (and thus a common misunderstanding for programmers new to Python).
In Python, both the constructor and the initializer are methods, no different from any other method. There's a naming convention that (some) methods with double underscores on both ends will be called by the interpreter without you having to do it explicitly, but you could call them yourself if you wanted, and you would get the same results.
In short: If you understand __str__, you understand __init__ and __new__.
(I think the weirdest thing about Python for some experienced programmers is how it doesn't ambush you with special cases at every turn.)
Method Resolution Order
Python 2.3's new-style classes look up method names by starting with the object's class, then working through that class's ancestors, until it either finds the name it's looking for or runs out of classes to search.
You can see the method resolution order of any class by looking at its __mro__ field:
>>> TestClass.__mro__
(<class '__main__.TestClass'>, <class 'object'>)
>>> float.__mro__
(<class 'float'>, <class 'object'>)
>>> object.__mro__
(<class 'object'>,)
(Python constructs the tuple using the C3 Method Resolution Order algorithm, but it's overkill for a simple case like yours. C3 is mostly concerned with linearizing a complex, multiple-inheritance graph into a tuple of classes that can be searched in a predictable order.)
Any method lookup, even __init__ or __new__, follows the Method Resolution Order. For each class in the tuple, Python looks for the string (like '__init__') in its class __dict__; if that string present, its associated value --- whatever it is --- will be returned as the result of the method lookup.
In your case, Python looks for TestClass.__new__, which doesn't exist, and then object.__new__, which does. It then executes object.__new__ (with some parameters filled in automatically) to construct your new TestClass object. Initialization with __init__ is identical, except that you might provide some additional arguments yourself.
Proof (in Python 3):
>>> class A: pass
...
>>> A.__init__ is object.__init__
True
>>> A.__new__ is object.__new__
True
>>> A.__repr__ is object.__repr__
True
>>> A.__str__ is object.__str__
True
Arguments to __init__
Your attempt to pass an argument to object.__init__ failed because that method doesn't take any parameters (aside from self, which was already filled in for you). It looks more or less like this:
class Object:
def __init__(self):
return
Conclusion
Python does not "create" a default constructor or initializer for your class like (IIRC) Java does. But it has one of each lying around in the object class, so if you don't write your own, you'll inherit them.

Can someone explain how the source code of staticmethod works in python

First of all, I understand how, in general, a decorator work. And I know #staticmethod strips off the instance argument in the signature, making
class C(object):
#staticmethod
def foo():
print 'foo'
C.foo //<function foo at 0x10efd4050>
C().foo //<function foo at 0x10efd4050>
valid.
However, I don't understand how the sourcec code of staticmethod make this happen.
It seems to me that when wrapping method foo in staticmethod, an instance of staticmethod is instantiated, then some magic happens, making C.foo() legit.
So.. what happen in those magic? what did staticmethod do?
I'm aware the enormous topics on SO regarding staticmethods but none of them addresses my doubts. But maybe I didn't hit the magic keyword. If so, please kindly let me know.
For whoever looking for staticmethod source code, please refer to https://hg.python.org/cpython/file/c6880edaf6f3/Objects/funcobject.c
A staticmethod object is a descriptor. The magic you are missing is that Python calls the __get__ method when accessing the object as an attribute on a class or instance.
So accessing the object as C.foo results in Python translating that to C.__dict__['foo'].__get__(None, C), while instance_of_C.foo becomes type(instace_of_C).__dict__['foo'].__get__(instance_of_C, type(instance_of_C)).
The staticmethod object is defined in C code, but an equivalent in Python would be:
class staticmethod(object):
def __init__(self, callable):
self.f = callable
def __get__(self, obj, type=None):
return self.f
#property
def __func__(self):
return self.f
where self.f is the original wrapped function.
All this is needed because functions are themselves descriptors too; it is the descriptor protocol that gives you method objects (see python bound and unbound method object for more details). Since they too have a __get__ method, without a staticmethod object wrapping the function, a functionobj.__get__ call produces a method object instead, passing in a self argument.
There is also a classmethod, which uses the second argument to descriptor.__get__ to bind a function to the class, and then there are property objects, which translate binding into a function call directly. See How does the #property decorator work?.

How does assignment of a function as a class attribute become a method in Python?

>>> class A(object): pass
>>> def func(cls): pass
>>> A.func = func
>>> A.func
<unbound method A.func>
How does this assignment create a method? It seems unintuitive that assignment does the following for classes:
Turn functions into unbound instance methods
Turn functions wrapped in classmethod() into class methods (actually, this is pretty intuitive)
Turn functions wrapped in staticmethod() into functions
It seems that for the first, there should be an instancemethod(), and for the last one, there shouldn't be a wrapper function at all. I understand that these are for uses within a class block, but why should they apply outside of it?
But more importantly, how exactly does assignment of the function into a class work? What magic happens that resolves those 3 things?
Even more confusing with this:
>>> A.func
<unbound method A.func>
>>> A.__dict__['func']
<function func at 0x...>
But I think this is something to do with descriptors, when retrieving attributes. I don't think it has much to do with the setting of attributes here.
You're right that this has something to do with descriptor protocol. Descriptors are how passing the receiver object as the first parameter of a method is implemented in Python. You can read more detail about Python attribute lookup from here. The following shows on a bit lower level, what is happening when you do A.func = func; A.func:
# A.func = func
A.__dict__['func'] = func # This just sets the attribute
# A.func
# The __getattribute__ method of a type object calls the __get__ method with
# None as the first parameter and the type as the second.
A.__dict__['func'].__get__(None, A) # The __get__ method of a function object
# returns an unbound method object if the
# first parameter is None.
a = A()
# a.func()
# The __getattribute__ method of object finds an attribute on the type object
# and calls the __get__ method of it with the instance as its first parameter.
a.__class__.__dict__['func'].__get__(a, a.__class__)
# This returns a bound method object that is actually just a proxy for
# inserting the object as the first parameter to the function call.
So it's the looking up of the function on a class or an instance that turns it into a method, not assigning it to a class attribute.
classmethod and staticmethod are just slightly different descriptors, classmethod returning a bound method object bound to a type object and staticmethod just returns the original function.
Descriptors are the magic1 that turns an ordinary function into a bound or unbound method when you retrieve it from an instance or class, since they’re all just functions that need different binding strategies. The classmethod and staticmethod decorators implement other binding strategies, and staticmethod actually just returns the raw function, which is the same behavior you get from a non-function callable object.
See “User-defined methods” for some gory details, but note this:
Also notice that this transformation only happens for user-defined functions; other callable objects (and all non-callable objects) are retrieved without transformation.
So if you wanted this transformation for your own callable object, you could just wrap it in a function, but you could also write a descriptor to implement your own binding strategy.
Here’s the staticmethod decorator in action, returning the underlying function when it’s accessed.
>>> #staticmethod
... def f(): pass
>>> class A(object): pass
>>> A.f = f
>>> A.f
<function f at 0x100479398>
>>> f
<staticmethod object at 0x100492750>
Whereas a normal object with a __call__ method doesn’t get transformed:
>>> class C(object):
... def __call__(self): pass
>>> c = C()
>>> A.c = c
>>> A.c
<__main__.C object at 0x10048b890>
>>> c
<__main__.C object at 0x10048b890>
1 The specific function is func_descr_get in Objects/funcobject.c.
What you have to consider is that in Python everything is an object. By establishing that it is easier to understand what is happening. If you have a function def foo(bar): print bar, you can do spam = foo and call spam(1), getting of course, 1.
Objects in Python keep their instance attributes in a dictionary called __dict__ with a "pointer" to other objects. As functions in Python are objects as well, they can be assigned and manipulated as simple variables, passed around to other functions, etc. Python's implementation of object orientation takes advantage of this, and treats methods as attributes, as functions that are in the __dict__ of the object.
Instance methods' first parameter is always the instance object itself, generally called self (but this could be called this or banana). When a method is called directly on the class, it is unbound to any instance, so you have to give it an instance object as the first parameter (A.func(A())). When you call a bound function (A().func()), the first parameter of the method, self, is implicit, but behind the curtains Python does exactly the same as calling directly on the unbound function and passing the instance object as the first parameter.
If this is understood, the fact that assigning A.func = func (which behind the curtains is doing A.__dict__["func"] = func) leaves you with an unbound method, is unsurprising.
In your example the cls in def func(cls): pass actually what will be passed on is the instance (self) of type A. When you apply the classmethod or staticmethod decorators do nothing more than take the first argument obtained during the call of the function/method, and transform it into something else, before calling the function.
classmethod takes the first argument, gets the class object of the instance, and passes that as the first argument to the function call, while staticmethod simply discards the first parameter and calls the function without it.
Point 1: The function func you defined exists as a First-Class Object in Python.
Point 2: Classes in Python store their attributes in their __dict__.
So what happens when you pass a function as the value of a class attribute in Python? That function is stored in the class' __dict__, making it a method of that class accessed by calling the attribute name you assigned it to.
Relating to MTsoul's comment to Gabriel Hurley's answer:
What is different is that func has a __call__() method, making it "callable", i.e. you can apply the () operator to it. Check out the Python docs (search for __call__ on that page).

Resources