The Comprehensive Guide to Python Decorators#

Decorators are one of Python's most powerful metaprogramming tools, widely used in the industry to write cleaner, more maintainable code. They excel at solving "cross-cutting concerns"—logic that applies to many different parts of your application but doesn't belong to the core business logic of any single function.

Before diving into advanced implementation details, it's worth understanding why decorators are so prevalent in production codebases:

1. Reducing Code Duplication: Instead of repeating the same error-handling or logging logic in every function, you write it once in a decorator and apply it everywhere.
   • Example: A @retry decorator that automatically retries failed network requests, saving you from writing try...except loops in every API client method.
2. Separation of Concerns: Decorators allow you to keep your core business logic pure and focused, moving infrastructure concerns (like authentication or caching) to the outer layer.
   • Example: An @authenticated decorator that checks if a user is logged in before the main view function runs. The view function can then focus entirely on rendering the response, assuming the user is already validated.
3. Standardization: They enforce consistent behavior across a codebase.
   • Example: A @validate_schema decorator that ensures all API endpoints receive data in a strictly defined JSON format, preventing "garbage in, garbage out" errors across the entire backend.

In this guide, we will go beyond the basics and explore the advanced mechanics that allow you to build robust, production-grade decorators. You will learn how to:

• Optimize Performance: Use decorator scopes to perform expensive checks (like is_async) only once at definition time.
• Improve Developer Experience: Create dual-mode decorators that work with or without arguments (e.g., @timer vs @timer(unit="ms")).
• Ensure Type Safety: Use typing.overload to ensure static analysis tools understand your dynamic code.
• Leverage Descriptors: Implement the Descriptor Protocol (__get__) to create decorators that correctly handle method binding (accessing self or cls) when used on classes.
• Master Class-Based Decorators: Leverage __new__ and descriptors to create powerful, stateful decorators that work seamlessly on both functions and methods.

By the end of this post, you will have a toolkit of patterns to solve complex cross-cutting concerns in your Python applications.

Decorators with Parameters#

The standard decorator pattern involves a function wrapping another function. However, when you need to pass arguments to the decorator itself, you need an additional layer of nesting.

The outer function accepts the decorator's arguments, the middle function accepts the function to be decorated, and the inner function is the wrapper.

Understanding Decorator Scopes#

When writing complex decorators, it helps to think in terms of three distinct scopes, each offering unique opportunities to optimize and control behavior:

1. SCOPE 1 (Configuration): The outermost function where you capture arguments (e.g., prefix, max_retries).

   • Advantage: This is your "setup" phase. You can validate configuration, pre-calculate expensive values, or initialize shared resources (like a connection pool or cache) that will be shared across all decorated functions.
   • Flexibility: You can even choose to return entirely different decorators based on the configuration arguments passed here. For example, if enabled=False is passed, you could immediately return a no-op decorator that simply returns the original function, bypassing all overhead.

2. SCOPE 2 (Inspection): The middle function where you receive the function to be decorated (func).

   • Advantage: This runs once at definition time (when Python reads the @decorator line). You can inspect func to determine its properties (is it async? how many arguments does it have? what are its type hints?).
   • Optimization: Based on this inspection, you can choose to return different wrapper implementations. For example, you can return an async def wrapper for coroutines and a standard def wrapper for synchronous functions, avoiding the performance penalty of checking is_async every time the function runs.

3. SCOPE 3 (Runtime): The innermost wrapper function.

   • Advantage: This runs every time the decorated function is called. It has access to variables from both previous scopes (Configuration and Inspection). Because the heavy lifting (configuration validation and function inspection) was done in previous scopes, this runtime code can be kept lean and fast.

Here is a concrete example of inspecting func to create a universal timer that handles both synchronous and asynchronous functions seamlessly, explicitly labeled with these scopes.

import functools
import inspect
import time
import asyncio

def universal_timer(prefix="[Timer]"):
    # SCOPE 1: Configuration Scope
    # This outer function receives configuration arguments.
    # Variables defined here (like `prefix`) become closure variables 
    # accessible to all inner scopes (decorator and wrapper).
    
    def decorator(func):
        # SCOPE 2: Inspection Scope
        # This runs once per decorated function, at definition time.
        # We can inspect `func` here to decide WHICH wrapper to build.
        # This logic happens before the wrapper is ever called.
        
        is_async = inspect.iscoroutinefunction(func)
        
        if is_async:
            # If the target is async, we define an async wrapper.
            @functools.wraps(func)
            async def wrapper(*args, **kwargs):
                # SCOPE 3 (Async): Runtime Scope
                # IMPORTANT: We do NOT check `is_async` here.
                # Because we inspected `func` in SCOPE 2, we know for a fact this is an async function.
                # This avoids an `if is_async` check on every single function call, improving efficiency.
                start = time.perf_counter()
                result = await func(*args, **kwargs)
                end = time.perf_counter()
                print(f"{prefix} [Async] {func.__name__} took {end - start:.4f}s")
                return result
        else:
            # If the target is sync, we define a standard wrapper.
            @functools.wraps(func)
            def wrapper(*args, **kwargs):
                # SCOPE 3 (Sync): Runtime Scope
                # Similarly, we know this is a sync function.
                # We can call it directly without overhead checks.
                start = time.perf_counter()
                result = func(*args, **kwargs)
                end = time.perf_counter()
                print(f"{prefix} [Sync]  {func.__name__} took {end - start:.4f}s")
                return result
                
        # We return the specific wrapper we built for this function
        return wrapper
        
    return decorator

# Test it
@universal_timer(prefix=">>>")
def sync_task():
    time.sleep(0.1)

@universal_timer(prefix=">>>")
async def async_task():
    await asyncio.sleep(0.1)

sync_task()
await async_task()

State Management with nonlocal#

In the "Runtime Scope" (Scope 3), you often need to maintain state across calls, such as counting how many times a function has been called. Since integers and strings are immutable in Python, you cannot simply do count += 1 if count is defined in the outer scope (Scope 1 or 2). Python would treat it as a new local variable.

To modify a variable from an outer scope, you must use the nonlocal keyword.

def limit_calls(max_calls):
    # SCOPE 1: State initialization
    count = 0
    
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            # We need to modify 'count' from SCOPE 1
            nonlocal count
            
            if count >= max_calls:
                raise RuntimeError(f"Function {func.__name__} limit reached!")
            
            count += 1
            print(f"Call {count}/{max_calls}")
            return func(*args, **kwargs)
        return wrapper
    return decorator

@limit_calls(max_calls=2)
def demo():
    print("Running...")

demo() # Call 1/2
demo() # Call 2/2
# demo() # Raises RuntimeError

Creating Dual-Mode Decorators with / and *#

One of the most requested features for decorators is the ability to use them both with and without parentheses (e.g., @timer vs @timer(unit="s")).

By combining a default argument with the positional-only (/) and keyword-only (*) markers, you can create a single function that handles both cases elegantly without complex type checking or classes.

import functools
import time

def timer(func=None, /, *, unit="s"):
    """
    A timer decorator that works as both @timer and @timer(unit="ms").
    """
    if func is None:
        # Called as @timer(...) or @timer() -> return partial
        return functools.partial(timer, unit=unit)

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        end = time.perf_counter()
        
        duration = end - start
        if unit == "ms":
            duration *= 1000
            
        print(f"{func.__name__} took {duration:.2f}{unit}")
        return result
    
    return wrapper

@timer
def fast_function():
    time.sleep(0.1)

@timer(unit="ms")
def slow_function():
    time.sleep(0.1)

fast_function()
slow_function()

How it works:

1. func=None: Allows the function to be omitted (which happens when using brackets, i.e. @timer(unit="ms")).
2. / (Positional-Only): Ensures func can only be passed positionally. This is the key that prevents ambiguity if you have a config argument named func.
3. * (Keyword-Only): Ensures unit must be passed as a keyword.

When you use @timer, Python calls timer(fast_function). func is the function, so we return the wrapper.

When you use @timer(unit="ms"), Python calls timer(unit="ms"). func is None, so we return a partial of timer. This partial is then called with the function to be decorated.

Type Hinting with @overload#

When you write a decorator that supports multiple calling styles, static type checkers like MyPy or Pyright can get confused. You can use typing.overload to explicitly tell the type checker how the decorator behaves for each call form.

Here is the same @timer decorator with overloads so both @timer and @timer(...) are type checked correctly.

%% typing_extensions
from typing import Callable, TypeVar, overload
from typing_extensions import ParamSpec
import functools
import time

P = ParamSpec("P")
R = TypeVar("R")

@overload
def timer(func: Callable[P, R], /, *, unit: str = "s") -> Callable[P, R]: ...

@overload
def timer(func: None = None, /, *, unit: str = "s") -> Callable[[Callable[P, R]], Callable[P, R]]: ...

def timer(func=None, /, *, unit="s"):
    if func is None:
        return functools.partial(timer, unit=unit)

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        end = time.perf_counter()
        duration = end - start
        if unit == "ms":
            duration *= 1000
        print(f"{func.__name__} took {duration:.2f}{unit}")
        return result

    return wrapper

@timer
def fast_function() -> None:
    time.sleep(0.1)

@timer(unit="ms")
def slow_function() -> None:
    time.sleep(0.1)

fast_function()
slow_function()

Class-Based Decorators using __call__#

You can use a class as a decorator by implementing the __call__ method. This is often cleaner than nested functions when the decorator needs to maintain state.

import functools

class CountCalls:
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
        self.num_calls = 0

    def __call__(self, *args, **kwargs):
        self.num_calls += 1
        print(f"Call {self.num_calls} of {self.func.__name__}")
        return self.func(*args, **kwargs)

@CountCalls
def say_hello():
    print("Hello!")

say_hello()
say_hello()

However, a naive class decorator like above has a fatal flaw: it breaks when decorating class methods because it doesn't handle the self (or cls) argument correctly (it doesn't act as a descriptor). We will solve this in the "Descriptors" section.

Using __new__ for Flexible Arguments#

Sometimes you want a decorator (that is an instance of a user-defined class) that can be used both with and without arguments, like @my_decorator and @my_decorator(param=1). Using __new__ allows a class to intercept creation and decide whether to return an instance or a wrapper function.

Why __new__ and not __init__?

The key constraint in Python classes is that __init__ must always return None. It cannot return a new object or a different callable.

This behavior is enforced by the type metaclass (which is the default metaclass for all Python classes). When you instantiate a class (e.g., MyClass()), you are actually calling type.__call__(MyClass).

Conceptually, type.__call__ looks something like this:

def __call__(cls, *args, **kwargs):
    # 1. Call __new__ to create the instance
    obj = cls.__new__(cls, *args, **kwargs)
    
    # 2. Only call __init__ if obj is an instance of cls
    if isinstance(obj, cls):
        result = cls.__init__(obj, *args, **kwargs)
        # 3. Enforce that __init__ returns None
        if result is not None:
            raise TypeError(f"__init__() should return None, not '{type(result).__name__}'")
            
    # 4. Return the object created by __new__
    return obj

Because type.__call__ ignores the return value of __init__ (except to error if it's not None) and strictly returns the object from __new__, __init__ cannot be used to replace the decorated function with a wrapper. We must use __new__ to intercept the instantiation process and return our wrapper (lambda or partial) instead of the class instance.

Crucial Note on __init__ behavior: Python only automatically calls __init__ if __new__ returns an instance of the class being instantiated (cls).

• If __new__ returns a lambda (Case 2 below), __init__ is NOT called.
• If __new__ returns a FlexibleDecorator instance (Case 1), __init__ IS automatically called with the arguments passed to the constructor.

import functools

class FlexibleDecorator:
    def __new__(cls, func=None, *, config="default"):
        # Case 1: Called as @FlexibleDecorator (no parens)
        # func is the decorated function.
        if func is not None and callable(func):
            instance = super().__new__(cls)
            # Python automatically calls __init__ with the same arguments passed to the constructor.
            # In this case: __init__(instance, func)
            return instance
        
        # Case 2: Called as @FlexibleDecorator(config="custom")
        # func is None. We return a lambda (or partial) that will act as the actual decorator.
        # When called, it triggers Case 1 logic: cls(f, config=config)
        return lambda f: cls(f, config=config)

    def __init__(self, func, config="default"):
        # CRITICAL: We must provide a default for 'config' because in Case 1,
        # Python calls __init__ with only 'func' (since that's all we passed to the class).
        self.func = func
        self.config = config
        functools.update_wrapper(self, func)

    def __call__(self, *args, **kwargs):
        print(f"Running with config: {self.config}")
        return self.func(*args, **kwargs)

    def set_config(self, new_config):
        # A method to change state dynamically!
        self.config = new_config

@FlexibleDecorator
def simple():
    pass

@FlexibleDecorator(config="advanced")
def configured():
    pass

simple()
configured()

Class Decorators with Stateful Methods (e.g. @property)#

The @property decorator in Python itself is a classic example of a class-based decorator whose methods mutate its internal state, which is another advantage of using classes. When you define a property, Python creates a property object that stores fget, fset, and fdel. Calling .setter does not modify the function directly—it updates the decorator's internal state (fset) and returns a new property object.

This is a key advantage of class-based decorators: you can expose methods that change the decorator’s configuration after the initial decoration step, without re-wrapping the function.

class Account:
    def __init__(self, balance):
        self._balance = balance

    @property
    def balance(self):
        return self._balance

    @balance.setter
    def balance(self, value):
        if value < 0:
            raise ValueError("Balance cannot be negative")
        self._balance = value

acct = Account(100)
print(acct.balance)
acct.balance = 250
print(acct.balance)

Behind the scenes, balance is a property instance that stores fget and fset. The .setter call replaces fset in that instance (or returns a new property with updated fset). This is exactly the “state mutation” pattern of class-based decorators.

Descriptors as Decorators#

For most use cases, decorating standalone functions or even classes, functions returning closures (nested functions) are sufficient and often simpler. They handle state via closure variables and don't require understanding Python's object model deeply.

However, when decorating methods within a class, function decorators can sometimes fall short, especially if you need to manipulate how the method is bound to the instance or class. This is where Classes as Decorators shine, specifically because they can implement the Descriptor Protocol.

A perfect example of this is Python's built-in @classmethod. While it's a built-in type, we can implement a pure Python version to understand how descriptors allow us to change a method's behavior (receiving the class cls instead of the instance self).

import functools

class MyClassMethod:
    def __init__(self, func):
        self.func = func

    def __get__(self, instance, owner):
        # The descriptor protocol!
        # instance: The instance that the attribute was accessed through (or None if accessed via class)
        # owner: The class (type) the attribute belongs to
        
        # Example Scenario:
        # class Example:
        #     @MyClassMethod
        #     def foo(cls): ...
        
        # 1. Example.foo
        #    instance = None
        #    owner = Example
        #    -> We bind to 'owner' (Example)
        
        # 2. Example().foo
        #    instance = <Example object at 0x...>
        #    owner = Example
        #    -> We still bind to 'owner' (Example) because it's a classmethod!
        
        # For a classmethod, we ALWAYS want to bind to the owner (the class),
        # regardless of whether we accessed it via an instance or the class itself.
        
        # We delegate to the function's own descriptor protocol to bind it to 'owner'.
        # passing 'owner' as the instance argument binds the function to the class.
        return self.func.__get__(owner, owner)

class Example:
    def __init__(self):
        self.name = "Instance"

    @MyClassMethod
    def classic_method(cls):
        print(f"I am bound to: {cls.__name__}")

    @MyClassMethod
    def factory(cls):
        return cls()

# Access via Class
Example.classic_method()  # Output: I am bound to: Example

# Access via Instance
obj = Example()
obj.classic_method()      # Output: I am bound to: Example

# Verify it works as a factory
new_obj = Example.factory()
print(isinstance(new_obj, Example)) # Output: True

By implementing __get__, MyClassMethod intercepts the dot access (Example.classic_method). Instead of returning the raw function (which would expect an instance if it were a normal method), it returns a bound method where the first argument is permanently set to the class (owner). This is something a simple function closure cannot easily achieve without complex hacks.

Summary#

Mastering Python decorators involves understanding the layers of abstraction available to you:

• Decorator Scopes: Leverage the Configuration, Inspection, and Runtime scopes to optimize performance (e.g., checking for async once at definition time).
• Dual-Mode Decorators: Use func=None with positional-only (/) and keyword-only (*) arguments to create decorators that work with (@timer(...)) or without (@timer) parentheses.
• Type Hinting: Use @overload to help static type checkers understand how your decorator transforms function signatures.
• Class-Based Decorators:
  • Use __call__ to make instances callable.
  • Use __new__ instead of __init__ when you need flexible argument handling (dual-mode) because __init__ cannot return a replacement function.
• Descriptors: Implement __get__ when your class-based decorator needs to act as a method and correctly bind to self or cls.

This approach ensures your decorators are robust, efficient, and developer-friendly.