Python Decorators Tutorial

Python Decorators Tutorial

I think it's safe to say that the goal of macros in a language is to provide a way to modify elements of the language. That's what decorators do in Python -- they modify functions, and in the case of class decorators, entire classes. This is why they usually provide a simpler alternative to metaclasses.

What Can You Do With Decorators?

Decorators allow you to inject or modify code in functions or classes. Sounds a bit like Aspect-Oriented Programming (AOP) in Java, doesn't it? Except that it's both much simpler and (as a result) much more powerful. For example, suppose you'd like to do something at the entry and exit points of a function (such as perform some kind of security, tracing, locking, etc. -- all the standard arguments for AOP). With decorators, it looks like this:

@entryExit

def func1():

    print "inside func1()"

@entryExit

def func2():

    print "inside func2()"

The @ indicates the application of the decorator.

Function Decorators

A function decorator is applied to a function definition by placing it on the line before that function definition begins. For example:

@myDecorator

def aFunction():

    print "inside aFunction"

When the compiler passes over this code, aFunction() is compiled and the resulting function object is passed to the myDecorator code, which does something to produce a function-like object that is then substituted for the original aFunction().

What does the myDecorator code look like? Well, most introductory examples show this as a function, but I've found that it's easier to start understanding decorators by using classes as decoration mechanisms instead of functions. In addition, it's more powerful.

The only constraint upon the object returned by the decorator is that it can be used as a function -- which basically means it must be callable. Thus, any classes we use as decorators must implement __call__.

What should the decorator do? Well, it can do anything but usually you expect the original function code to be used at some point. This is not required, however:

class myDecorator(object):

    def __init__(self, f):

        print "inside myDecorator.__init__()"

        f() # Prove that function definition has completed

    def __call__(self):

        print "inside myDecorator.__call__()"

@myDecorator

def aFunction():

    print "inside aFunction()"

print "Finished decorating aFunction()"

aFunction()

When you run this code, you see:

inside myDecorator.__init__()

inside aFunction()

Finished decorating aFunction()

inside myDecorator.__call__()

Notice that the constructor for myDecorator is executed at the point of decoration of the function. Since we can call f() inside __init__(), it shows that the creation of f() is complete before the decorator is called. Note also that the decorator constructor receives the function object being decorated. Typically, you'll capture the function object in the constructor and later use it in the __call__() method (the fact that decoration and calling are two clear phases when using classes is why I argue that it's easier and more powerful this way).

When aFunction() is called after it has been decorated, we get completely different behavior; the myDecorator.__call__() method is called instead of the original code. That's because the act of decoration replaces the original function object with the result of the decoration -- in our case, the myDecorator object replaces aFunction. Indeed, before decorators were added you had to do something much less elegant to achieve the same thing:

def foo(): pass

foo = staticmethod(foo)

With the addition of the @ decoration operator, you now get the same result by saying:

@staticmethod

def foo(): pass

This is the reason why people argued against decorators, because the @ is just a little syntax sugar meaning "pass a function object through another function and assign the result to the original function."

The reason I think decorators will have such a big impact is because this little bit of syntax sugar changes the way you think about programming. Indeed, it brings the idea of "applying code to other code" (i.e.: macros) into mainstream thinking by formalizing it as a language construct.

More Useful

Now let's go back and implement the first example. Here, we'll do the more typical thing and actually use the code in the decorated functions:

class entryExit(object):

    def __init__(self, f):

        self.f = f

    def __call__(self):

        print "Entering", self.f.__name__

        self.f()

        print "Exited", self.f.__name__

@entryExit

def func1():

    print "inside func1()"

@entryExit

def func2():

    print "inside func2()"

func1()

func2()

The output is:

Entering func1

inside func1()

Exited func1

Entering func2

inside func2()

Exited func2

You can see that the decorated functions now have the "Entering" and "Exited" trace statements around the call.

The constructor stores the argument, which is the function object. In the call, we use the __name__ attribute of the function to display that function's name, then call the function itself.

Using Functions as Decorators

The only constraint on the result of a decorator is that it be callable, so it can properly replace the decorated function. In the above examples, I've replaced the original function with an object of a class that has a __call__() method. But a function object is also callable, so we can rewrite the previous example using a function instead of a class, like this:

def entryExit(f):

    def new_f():

        print "Entering", f.__name__

        f()

        print "Exited", f.__name__

    return new_f

@entryExit

def func1():

    print "inside func1()"

@entryExit

def func2():

    print "inside func2()"

func1()

func2()

print func1.__name__

new_f() is defined within the body of entryExit(), so it is created and returned when entryExit() is called. Note that new_f() is a closure, because it captures the actual value of f.

Once new_f() has been defined, it is returned from entryExit() so that the decorator mechanism can assign the result as the decorated function.

The output of the line print func1.__name__ is new_f, because the new_f function has been substituted for the original function during decoration. If this is a problem you can change the name of the decorator function before you return it:

def entryExit(f):

    def new_f():

        print "Entering", f.__name__

        f()

        print "Exited", f.__name__

    new_f.__name__ = f.__name__

    return new_f

The information you can dynamically get about functions, and the modifications you can make to those functions, are quite powerful in Python.

More Examples

http://wiki.python.org/moin/PythonDecoratorLibrary

********************************************

So let’s start with the basic no-argument decorator. It doesn’t take any declarative arguments when decorating a function; you just type @some_deco without parenthesis when using it.

def some_deco(f):

    def _inner(*args, **kwargs):

        print "Decorated!"

        return f(*args, **kwargs)

    return _inner

@some_deco

def some_func(a, b):

    return a + b

print some_func(1, 2)

This will print output:

Decorated!

3

According to the PEP, if you wrap some_func with @some_deco, it is the equivalent of some_func = some_deco(some_func). What it fails to call out is that this happens at load time. What we're returning from the decorator function is a new function or callable that will be bound to the name some_func when this module is loaded. Subsequent run-time calls to some_func are actually calling the _inner function we defined on the fly when the decorator was called.

-------------------------------------------

def log_wrap(message):

    """Print `message` each time the decorated function is called."""

    def _second(f):

        def _inner(*args, **kwargs):

            print message

            return f(*args, **kwargs) 

        # This is called second, so return a callable that takes arguments

        # like the first example.

        return _inner

    # This is called first, so we want to pass back our inner function that accepts

    # a function as argument

    return _second

@log_wrap("Called it!")

def func(a, b):

    return a + b

print func(1, 2)

This will print output:

Decorated!

3

Ok, ok, that's not terribly useful, but it illustrates the point (and expands on the first example by being a parameterized version). It requires that we construct two callable items inside our decorator, and that each will be called in turn during load time. Let's quick dissect what happens at load time. When python gets to the declaration of func with the decorator sitting on top of it, it calls the decorator function with its argument (in this case, the string "Called!"). What is returned is a function reference to _second. Next, that just-returned reference to _second is invoked with a single argument: a function pointer to func, which is what we're actually trying to decorate. The return value of _second is a reference to _inner, which will be bound to the name func, ready for run-time invocation.

-------------------------------------------

********************************************

Basic Decoration

The basics: A python function takes arguments, performs a task, and returns a value. A decorator takes as its argument a function, and returns another function with different semantics. It has “decorated” the function with pre-call and post-call behaviors and conditions.

The biggest (and hardest) part for many beginning programmers to grasp is this: a function name is just a variable. A function is just an object; when you call a function, you’re (1) dereferencing the variable functionname, (2) checking that the referenced object is callable, (3) passing that object arguments and (4) invoking a new execution frame, which in turn builds a context and attempts to run the referenced object with the arguments. (That context, by the way, persists for all functions defined within it at each execution pass; this is how closure works, and decorators are highly closure-dependent.)

In some ways, what I’m about to do is go over the same ground as django.views.generic.list_detail.object_detail, but I think my way is somewhat more interesting, and is illustrative to the task at hand. What I’d like is to encapsulate my business logic in one function, and the details of rendering it could then be added in later via decoration. Here’s a very simple example:

@render('cards/home.html')

def home(request):

    return {'cards ':Card.objects.all()}

The business logic is simple: the home page for this application returns all the cards. And the decoration is equally simple: We’re literally going to “decorate” this data with the HTML for the home page.

So, we need a function that wraps Django’s render_to_response, decorating it with the template name and the RequestContext object (that provides all the miscellaneous information most templates in Django need). The outermost decorator ultimately needs to return a function that does this while preserving the template name in a closure. And we want to wrap our innermost function in the name of the function we pass in. Here’s the entirety of it:

from django.template import RequestContext

from functools import wraps

from django.shortcuts import render_to_response

def render(template_name):

    def inner_render(fn):

        def wrapped(request, *args, **kwargs):

            return render_to_response(template_name,

                                       fn(request, *args, **kwargs),

                                       context_instance = RequestContext(request))

        return wraps(fn)(wrapped)

    return inner_render

-------------------------------------------

*******************************