diff options
| -rw-r--r-- | Doc/library/functions.rst | 10 | ||||
| -rw-r--r-- | Doc/library/types.rst | 67 | ||||
| -rw-r--r-- | Doc/reference/datamodel.rst | 136 | ||||
| -rw-r--r-- | Doc/whatsnew/3.3.rst | 4 | ||||
| -rw-r--r-- | Lib/test/test_types.py | 251 | ||||
| -rw-r--r-- | Lib/types.py | 58 | ||||
| -rw-r--r-- | Misc/NEWS | 11 |
7 files changed, 486 insertions, 51 deletions
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 9287bfb..35f05d4 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1324,10 +1324,12 @@ are always available. They are listed here in alphabetical order. Accordingly, :func:`super` is undefined for implicit lookups using statements or operators such as ``super()[name]``. - Also note that :func:`super` is not limited to use inside methods. The two - argument form specifies the arguments exactly and makes the appropriate - references. The zero argument form automatically searches the stack frame - for the class (``__class__``) and the first argument. + Also note that, aside from the zero argument form, :func:`super` is not + limited to use inside methods. The two argument form specifies the + arguments exactly and makes the appropriate references. The zero + argument form only works inside a class definition, as the compiler fills + in the necessary details to correctly retrieve the class being defined, + as well as accessing the current instance for ordinary methods. For practical suggestions on how to design cooperative classes using :func:`super`, see `guide to using super() diff --git a/Doc/library/types.rst b/Doc/library/types.rst index 0368177..bd728d0 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -1,5 +1,5 @@ -:mod:`types` --- Names for built-in types -========================================= +:mod:`types` --- Dynamic type creation and names for built-in types +=================================================================== .. module:: types :synopsis: Names for built-in types. @@ -8,20 +8,69 @@ -------------- -This module defines names for some object types that are used by the standard +This module defines utility function to assist in dynamic creation of +new types. + +It also defines names for some object types that are used by the standard Python interpreter, but not exposed as builtins like :class:`int` or -:class:`str` are. Also, it does not include some of the types that arise -transparently during processing such as the ``listiterator`` type. +:class:`str` are. + + +Dynamic Type Creation +--------------------- + +.. function:: new_class(name, bases=(), kwds=None, exec_body=None) + + Creates a class object dynamically using the appropriate metaclass. + + The arguments are the components that make up a class definition: the + class name, the base classes (in order), the keyword arguments (such as + ``metaclass``) and the callback function to populate the class namespace. + + The *exec_body* callback should accept the class namespace as its sole + argument and update the namespace directly with the class contents. + +.. function:: prepare_class(name, bases=(), kwds=None) + + Calculates the appropriate metaclass and creates the class namespace. + + The arguments are the components that make up a class definition: the + class name, the base classes (in order) and the keyword arguments (such as + ``metaclass``). + + The return value is a 3-tuple: ``metaclass, namespace, kwds`` + + *metaclass* is the appropriate metaclass + *namespace* is the prepared class namespace + *kwds* is an updated copy of the passed in *kwds* argument with any + ``'metaclass'`` entry removed. If no *kwds* argument is passed in, this + will be an empty dict. + + +.. seealso:: + + :pep:`3115` - Metaclasses in Python 3000 + Introduced the ``__prepare__`` namespace hook + + +Standard Interpreter Types +-------------------------- + +This module provides names for many of the types that are required to +implement a Python interpreter. It deliberately avoids including some of +the types that arise only incidentally during processing such as the +``listiterator`` type. -Typical use is for :func:`isinstance` or :func:`issubclass` checks. +Typical use is of these names is for :func:`isinstance` or +:func:`issubclass` checks. -The module defines the following names: +Standard names are defined for the following types: .. data:: FunctionType LambdaType - The type of user-defined functions and functions created by :keyword:`lambda` - expressions. + The type of user-defined functions and functions created by + :keyword:`lambda` expressions. .. data:: GeneratorType diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 1f1a660..0fd5e74 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1550,53 +1550,115 @@ Notes on using *__slots__* Customizing class creation -------------------------- -By default, classes are constructed using :func:`type`. A class definition is -read into a separate namespace and the value of class name is bound to the -result of ``type(name, bases, dict)``. +By default, classes are constructed using :func:`type`. The class body is +executed in a new namespace and the class name is bound locally to the +result of ``type(name, bases, namespace)``. -When the class definition is read, if a callable ``metaclass`` keyword argument -is passed after the bases in the class definition, the callable given will be -called instead of :func:`type`. If other keyword arguments are passed, they -will also be passed to the metaclass. This allows classes or functions to be -written which monitor or alter the class creation process: +The class creation process can be customised by passing the ``metaclass`` +keyword argument in the class definition line, or by inheriting from an +existing class that included such an argument. In the following example, +both ``MyClass`` and ``MySubclass`` are instances of ``Meta``:: -* Modifying the class dictionary prior to the class being created. + class Meta(type): + pass -* Returning an instance of another class -- essentially performing the role of a - factory function. + class MyClass(metaclass=Meta): + pass -These steps will have to be performed in the metaclass's :meth:`__new__` method --- :meth:`type.__new__` can then be called from this method to create a class -with different properties. This example adds a new element to the class -dictionary before creating the class:: + class MySubclass(MyClass): + pass - class metacls(type): - def __new__(mcs, name, bases, dict): - dict['foo'] = 'metacls was here' - return type.__new__(mcs, name, bases, dict) +Any other keyword arguments that are specified in the class definition are +passed through to all metaclass operations described below. -You can of course also override other class methods (or add new methods); for -example defining a custom :meth:`__call__` method in the metaclass allows custom -behavior when the class is called, e.g. not always creating a new instance. +When a class definition is executed, the following steps occur: -If the metaclass has a :meth:`__prepare__` attribute (usually implemented as a -class or static method), it is called before the class body is evaluated with -the name of the class and a tuple of its bases for arguments. It should return -an object that supports the mapping interface that will be used to store the -namespace of the class. The default is a plain dictionary. This could be used, -for example, to keep track of the order that class attributes are declared in by -returning an ordered dictionary. +* the appropriate metaclass is determined +* the class namespace is prepared +* the class body is executed +* the class object is created -The appropriate metaclass is determined by the following precedence rules: +Determining the appropriate metaclass +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -* If the ``metaclass`` keyword argument is passed with the bases, it is used. +The appropriate metaclass for a class definition is determined as follows: -* Otherwise, if there is at least one base class, its metaclass is used. +* if no bases and no explicit metaclass are given, then :func:`type` is used +* if an explicit metaclass is given and it is *not* an instance of + :func:`type`, then it is used directly as the metaclass +* if an instance of :func:`type` is given as the explicit metaclass, or + bases are defined, then the most derived metaclass is used -* Otherwise, the default metaclass (:class:`type`) is used. +The most derived metaclass is selected from the explicitly specified +metaclass (if any) and the metaclasses (i.e. ``type(cls)``) of all specified +base classes. The most derived metaclass is one which is a subtype of *all* +of these candidate metaclasses. If none of the candidate metaclasses meets +that criterion, then the class definition will fail with ``TypeError``. + + +Preparing the class namespace +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Once the appropriate metaclass has been identified, then the class namespace +is prepared. If the metaclass has a ``__prepare__`` attribute, it is called +as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the +additional keyword arguments, if any, come from the class definition). + +If the metaclass has no ``__prepare__`` attribute, then the class namespace +is initialised as an empty :func:`dict` instance. + +.. seealso:: + + :pep:`3115` - Metaclasses in Python 3000 + Introduced the ``__prepare__`` namespace hook + + +Executing the class body +^^^^^^^^^^^^^^^^^^^^^^^^ + +The class body is executed (approximately) as +``exec(body, globals(), namespace)``. The key difference from a normal +call to :func:`exec` is that lexical scoping allows the class body (including +any methods) to reference names from the current and outer scopes when the +class definition occurs inside a function. + +However, even when the class definition occurs inside the function, methods +defined inside the class still cannot see names defined at the class scope. +Class variables must be accessed through the first parameter of instance or +class methods, and cannot be accessed at all from static methods. + + +Creating the class object +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Once the class namespace has been populated by executing the class body, +the class object is created by calling +``metaclass(name, bases, namespace, **kwds)`` (the additional keywords +passed here are the same as those passed to ``__prepate__``). + +This class object is the one that will be referenced by the zero-argument +form of :func:`super`. ``__class__`` is an implicit closure reference +created by the compiler if any methods in a class body refer to either +``__class__`` or ``super``. This allows the zero argument form of +:func:`super` to correctly identify the class being defined based on +lexical scoping, while the class or instance that was used to make the +current call is identified based on the first argument passed to the method. + +After the class object is created, any class decorators included in the +function definition are invoked and the resulting object is bound in the +local namespace to the name of the class. + +.. seealso:: + + :pep:`3135` - New super + Describes the implicit ``__class__`` closure reference + + +Metaclass example +^^^^^^^^^^^^^^^^^ The potential uses for metaclasses are boundless. Some ideas that have been -explored including logging, interface checking, automatic delegation, automatic +explored include logging, interface checking, automatic delegation, automatic property creation, proxies, frameworks, and automatic resource locking/synchronization. @@ -1609,9 +1671,9 @@ to remember the order that class members were defined:: def __prepare__(metacls, name, bases, **kwds): return collections.OrderedDict() - def __new__(cls, name, bases, classdict): - result = type.__new__(cls, name, bases, dict(classdict)) - result.members = tuple(classdict) + def __new__(cls, name, bases, namespace, **kwds): + result = type.__new__(cls, name, bases, dict(namespace)) + result.members = tuple(namespace) return result class A(metaclass=OrderedClass): diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index fe1a84a..08823e0 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1239,6 +1239,10 @@ Add a new :class:`types.MappingProxyType` class: Read-only proxy of a mapping. (:issue:`14386`) +The new functions `types.new_class` and `types.prepare_class` provide support +for PEP 3115 compliant dynamic type creation. (:issue:`14588`) + + urllib ------ diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index 9a2e0d4..51b594c 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -747,8 +747,257 @@ class MappingProxyTests(unittest.TestCase): self.assertEqual(copy['key1'], 27) +class ClassCreationTests(unittest.TestCase): + + class Meta(type): + def __init__(cls, name, bases, ns, **kw): + super().__init__(name, bases, ns) + @staticmethod + def __new__(mcls, name, bases, ns, **kw): + return super().__new__(mcls, name, bases, ns) + @classmethod + def __prepare__(mcls, name, bases, **kw): + ns = super().__prepare__(name, bases) + ns["y"] = 1 + ns.update(kw) + return ns + + def test_new_class_basics(self): + C = types.new_class("C") + self.assertEqual(C.__name__, "C") + self.assertEqual(C.__bases__, (object,)) + + def test_new_class_subclass(self): + C = types.new_class("C", (int,)) + self.assertTrue(issubclass(C, int)) + + def test_new_class_meta(self): + Meta = self.Meta + settings = {"metaclass": Meta, "z": 2} + # We do this twice to make sure the passed in dict isn't mutated + for i in range(2): + C = types.new_class("C" + str(i), (), settings) + self.assertIsInstance(C, Meta) + self.assertEqual(C.y, 1) + self.assertEqual(C.z, 2) + + def test_new_class_exec_body(self): + Meta = self.Meta + def func(ns): + ns["x"] = 0 + C = types.new_class("C", (), {"metaclass": Meta, "z": 2}, func) + self.assertIsInstance(C, Meta) + self.assertEqual(C.x, 0) + self.assertEqual(C.y, 1) + self.assertEqual(C.z, 2) + + def test_new_class_exec_body(self): + #Test that keywords are passed to the metaclass: + def meta_func(name, bases, ns, **kw): + return name, bases, ns, kw + res = types.new_class("X", + (int, object), + dict(metaclass=meta_func, x=0)) + self.assertEqual(res, ("X", (int, object), {}, {"x": 0})) + + def test_new_class_defaults(self): + # Test defaults/keywords: + C = types.new_class("C", (), {}, None) + self.assertEqual(C.__name__, "C") + self.assertEqual(C.__bases__, (object,)) + + def test_new_class_meta_with_base(self): + Meta = self.Meta + def func(ns): + ns["x"] = 0 + C = types.new_class(name="C", + bases=(int,), + kwds=dict(metaclass=Meta, z=2), + exec_body=func) + self.assertTrue(issubclass(C, int)) + self.assertIsInstance(C, Meta) + self.assertEqual(C.x, 0) + self.assertEqual(C.y, 1) + self.assertEqual(C.z, 2) + + # Many of the following tests are derived from test_descr.py + def test_prepare_class(self): + # Basic test of metaclass derivation + expected_ns = {} + class A(type): + def __new__(*args, **kwargs): + return type.__new__(*args, **kwargs) + + def __prepare__(*args): + return expected_ns + + B = types.new_class("B", (object,)) + C = types.new_class("C", (object,), {"metaclass": A}) + + # The most derived metaclass of D is A rather than type. + meta, ns, kwds = types.prepare_class("D", (B, C), {"metaclass": type}) + self.assertIs(meta, A) + self.assertIs(ns, expected_ns) + self.assertEqual(len(kwds), 0) + + def test_metaclass_derivation(self): + # issue1294232: correct metaclass calculation + new_calls = [] # to check the order of __new__ calls + class AMeta(type): + def __new__(mcls, name, bases, ns): + new_calls.append('AMeta') + return super().__new__(mcls, name, bases, ns) + @classmethod + def __prepare__(mcls, name, bases): + return {} + + class BMeta(AMeta): + def __new__(mcls, name, bases, ns): + new_calls.append('BMeta') + return super().__new__(mcls, name, bases, ns) + @classmethod + def __prepare__(mcls, name, bases): + ns = super().__prepare__(name, bases) + ns['BMeta_was_here'] = True + return ns + + A = types.new_class("A", (), {"metaclass": AMeta}) + self.assertEqual(new_calls, ['AMeta']) + new_calls.clear() + + B = types.new_class("B", (), {"metaclass": BMeta}) + # BMeta.__new__ calls AMeta.__new__ with super: + self.assertEqual(new_calls, ['BMeta', 'AMeta']) + new_calls.clear() + + C = types.new_class("C", (A, B)) + # The most derived metaclass is BMeta: + self.assertEqual(new_calls, ['BMeta', 'AMeta']) + new_calls.clear() + # BMeta.__prepare__ should've been called: + self.assertIn('BMeta_was_here', C.__dict__) + + # The order of the bases shouldn't matter: + C2 = types.new_class("C2", (B, A)) + self.assertEqual(new_calls, ['BMeta', 'AMeta']) + new_calls.clear() + self.assertIn('BMeta_was_here', C2.__dict__) + + # Check correct metaclass calculation when a metaclass is declared: + D = types.new_class("D", (C,), {"metaclass": type}) + self.assertEqual(new_calls, ['BMeta', 'AMeta']) + new_calls.clear() + self.assertIn('BMeta_was_here', D.__dict__) + + E = types.new_class("E", (C,), {"metaclass": AMeta}) + self.assertEqual(new_calls, ['BMeta', 'AMeta']) + new_calls.clear() + self.assertIn('BMeta_was_here', E.__dict__) + + def test_metaclass_override_function(self): + # Special case: the given metaclass isn't a class, + # so there is no metaclass calculation. + class A(metaclass=self.Meta): + pass + + marker = object() + def func(*args, **kwargs): + return marker + + X = types.new_class("X", (), {"metaclass": func}) + Y = types.new_class("Y", (object,), {"metaclass": func}) + Z = types.new_class("Z", (A,), {"metaclass": func}) + self.assertIs(marker, X) + self.assertIs(marker, Y) + self.assertIs(marker, Z) + + def test_metaclass_override_callable(self): + # The given metaclass is a class, + # but not a descendant of type. + new_calls = [] # to check the order of __new__ calls + prepare_calls = [] # to track __prepare__ calls + class ANotMeta: + def __new__(mcls, *args, **kwargs): + new_calls.append('ANotMeta') + return super().__new__(mcls) + @classmethod + def __prepare__(mcls, name, bases): + prepare_calls.append('ANotMeta') + return {} + + class BNotMeta(ANotMeta): + def __new__(mcls, *args, **kwargs): + new_calls.append('BNotMeta') + return super().__new__(mcls) + @classmethod + def __prepare__(mcls, name, bases): + prepare_calls.append('BNotMeta') + return super().__prepare__(name, bases) + + A = types.new_class("A", (), {"metaclass": ANotMeta}) + self.assertIs(ANotMeta, type(A)) + self.assertEqual(prepare_calls, ['ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['ANotMeta']) + new_calls.clear() + + B = types.new_class("B", (), {"metaclass": BNotMeta}) + self.assertIs(BNotMeta, type(B)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + C = types.new_class("C", (A, B)) + self.assertIs(BNotMeta, type(C)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + C2 = types.new_class("C2", (B, A)) + self.assertIs(BNotMeta, type(C2)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + # This is a TypeError, because of a metaclass conflict: + # BNotMeta is neither a subclass, nor a superclass of type + with self.assertRaises(TypeError): + D = types.new_class("D", (C,), {"metaclass": type}) + + E = types.new_class("E", (C,), {"metaclass": ANotMeta}) + self.assertIs(BNotMeta, type(E)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + F = types.new_class("F", (object(), C)) + self.assertIs(BNotMeta, type(F)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + F2 = types.new_class("F2", (C, object())) + self.assertIs(BNotMeta, type(F2)) + self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta']) + prepare_calls.clear() + self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta']) + new_calls.clear() + + # TypeError: BNotMeta is neither a + # subclass, nor a superclass of int + with self.assertRaises(TypeError): + X = types.new_class("X", (C, int())) + with self.assertRaises(TypeError): + X = types.new_class("X", (int(), C)) + + def test_main(): - run_unittest(TypesTests, MappingProxyTests) + run_unittest(TypesTests, MappingProxyTests, ClassCreationTests) if __name__ == '__main__': test_main() diff --git a/Lib/types.py b/Lib/types.py index 08cbb83..2bfcd9b 100644 --- a/Lib/types.py +++ b/Lib/types.py @@ -40,3 +40,61 @@ GetSetDescriptorType = type(FunctionType.__code__) MemberDescriptorType = type(FunctionType.__globals__) del sys, _f, _g, _C, # Not for export + + +# Provide a PEP 3115 compliant mechanism for class creation +def new_class(name, bases=(), kwds=None, exec_body=None): + """Create a class object dynamically using the appropriate metaclass.""" + meta, ns, kwds = prepare_class(name, bases, kwds) + if exec_body is not None: + exec_body(ns) + return meta(name, bases, ns, **kwds) + +def prepare_class(name, bases=(), kwds=None): + """Call the __prepare__ method of the appropriate metaclass. + + Returns (metaclass, namespace, kwds) as a 3-tuple + + *metaclass* is the appropriate metaclass + *namespace* is the prepared class namespace + *kwds* is an updated copy of the passed in kwds argument with any + 'metaclass' entry removed. If no kwds argument is passed in, this will + be an empty dict. + """ + if kwds is None: + kwds = {} + else: + kwds = dict(kwds) # Don't alter the provided mapping + if 'metaclass' in kwds: + meta = kwds.pop('metaclass') + else: + if bases: + meta = type(bases[0]) + else: + meta = type + if isinstance(meta, type): + # when meta is a type, we first determine the most-derived metaclass + # instead of invoking the initial candidate directly + meta = _calculate_meta(meta, bases) + if hasattr(meta, '__prepare__'): + ns = meta.__prepare__(name, bases, **kwds) + else: + ns = {} + return meta, ns, kwds + +def _calculate_meta(meta, bases): + """Calculate the most derived metaclass.""" + winner = meta + for base in bases: + base_meta = type(base) + if issubclass(winner, base_meta): + continue + if issubclass(base_meta, winner): + winner = base_meta + continue + # else: + raise TypeError("metaclass conflict: " + "the metaclass of a derived class " + "must be a (non-strict) subclass " + "of the metaclasses of all its bases") + return winner @@ -42,6 +42,10 @@ Core and Builtins Library ------- +- Issue #14588: The types module now provide new_class() and prepare_class() + functions to support PEP 3115 compliant dynamic class creation. Patch by + Daniel Urban and Nick Coghlan. + - Issue #13152: Allow to specify a custom tabsize for expanding tabs in textwrap. Patch by John Feuerstein. @@ -166,6 +170,13 @@ Build - Issue #13210: Windows build now uses VS2010, ported from VS2008. +Documentation +------------- + +- Issue #14588: The language reference now accurately documents the Python 3 + class definition process. Patch by Nick Coghlan. + + What's New in Python 3.3.0 Alpha 3? =================================== |