Functions, Decorators, Context Managers & Dataclasses

This is the chapter behind "do you actually know Python, or do you know pandas". Decorators, context managers and dataclasses are the three constructs that separate somebody who writes scripts from somebody who writes code other people maintain — and all three are asked about directly in almost every Python screen.

They are also the three that people use daily without being able to explain. `@retry` is copied from a colleague, `with open(...)` is muscle memory, `@dataclass` is "the thing that saves typing `__init__`". Each of those is true and none of them is the answer an interviewer is listening for.

The through-line is contracts. A decorator wraps a contract without changing it. A context manager guarantees a contract at the exit, however the block ends. A dataclass makes a record's contract explicit enough that equality, hashing and serialisation come for free.

first-class function

A function you can pass, store and return like any other object.

decorator

A callable that takes a function and returns a replacement for it.

context manager

An object with `__enter__` and `__exit__`, used by `with` to guarantee cleanup.

dataclass

A class decorator that generates `__init__`, `__repr__` and `__eq__` from annotated fields.

Protocol

A structural type: anything with the right methods satisfies it, with no inheritance.

A function in Python is an object like any other. It has a type, attributes, and a name it can be rebound away from. You can put one in a list, pass it as an argument, return it from another function, and store it in a dict.

That is the whole prerequisite for decorators. `@retry` is not special syntax with magic behind it — it is a function call whose result is rebound to the same name, and everything below is a consequence of that.

It is also how you build a registry: a dict from a name to a handler is the most common way pipeline code dispatches on record type without a fifteen-branch `if`.

higher-order function

A function that takes or returns another function.

registry

A dict from a key to a handler function, used to dispatch without a branch ladder.

def parse_amount(raw):
    return float(raw)

steps = [parse_amount]            # stored like any other value
print(steps[0]("12.5"))
print(parse_amount.__name__)      # functions carry their own metadata

A decorator is a callable that takes a function and returns a replacement for it. `@retry` above a `def` means exactly one thing: after the `def` runs, rebind the name to `retry(the_function)`.

That is the whole definition, and it is the answer to the interview question. Everything else — the inner `wrapper`, the `*args, **kwargs`, the `functools.wraps` — is the practical business of making the replacement behave like the original.

`functools.wraps` is the part people skip and it is not cosmetic. Without it the replacement carries the wrapper's name and docstring, so tracebacks, logs and `help()` all describe the decorator instead of the function that failed.

wrapper

The replacement function a decorator returns. It usually calls the original and adds behaviour around it.

functools.wraps

Copies `__name__`, `__doc__`, `__module__` and more from the wrapped function onto the wrapper.

import functools

def retry(fn):
    @functools.wraps(fn)               # copy name, docstring, signature metadata
    def wrapper(*args, **kwargs):      # accept whatever fn accepts
        for attempt in range(3):
            try:
                return fn(*args, **kwargs)
            except ValueError:
                if attempt == 2:
                    raise              # last attempt: let it out
    return wrapper                     # the replacement for fn

calls = []

@retry
def flaky(row):
    """Parse a row, badly."""
    calls.append(row)
    if len(calls) < 3:
        raise ValueError("transient")
    return row.upper()

print(flaky("r1"), len(calls))
print(flaky.__name__, flaky.__doc__)

def bare(fn):
    def wrapper(*a, **k):
        return fn(*a, **k)
    return wrapper

@bare
def load(rows):
    """Load rows."""
    return len(rows)

print(load.__name__, load.__doc__)

`@retry(times=3)` is one layer deeper than `@retry`, and the extra layer is why it confuses people. `retry(times=3)` is called first, and it must return a decorator — which is then applied to the function.

So a parameterised decorator is a function that returns a function that returns a function. Written out with names for each layer, it stops being confusing: the outer takes the settings, the middle takes the function, the inner does the work.

If you only ever need one form, prefer the simple one. A decorator that supports both `@retry` and `@retry(times=3)` is a well-known piece of Python trivia and a genuinely awkward thing to read.

decorator factory

A function that returns a decorator, so the decorator can be configured at the call site.

import functools

def retries(times):                       # 1. takes the setting
    def decorate(fn):                     # 2. takes the function
        @functools.wraps(fn)
        def wrapper(*a, **k):             # 3. takes the call
            for i in range(times):
                try:
                    return fn(*a, **k)
                except ValueError:
                    if i == times - 1:
                        raise
        return wrapper
    return decorate

seen = []

@retries(2)
def g():
    seen.append(1)
    if len(seen) < 2:
        raise ValueError("x")
    return "ok"

print(g(), len(seen))

A `with` block guarantees that the exit runs — normally, on `return`, on `break`, and on exception. That is the entire value proposition, and it is why file handles, database transactions, locks and timing scopes are all expressed this way.

The protocol is two methods. `__enter__` runs at the top and its return value is what `as` binds; `__exit__` runs at the bottom, receives the exception if there was one, and can suppress it by returning `True` — which it should almost never do.

You can also build one from a generator with `@contextmanager`: everything before `yield` is the enter, the `yield` value is what `as` binds, and everything after is the exit. Put the exit in a `finally` or it will not run when the block raises, which is the only case you wrote it for.

__enter__

Runs at the top of the block. Its return value is bound by `as`.

__exit__

Runs at the bottom, with the exception details if any. Returning `True` suppresses the exception.

@contextmanager

Builds a context manager from a generator: before `yield` is enter, after is exit.

from contextlib import contextmanager

log = []

@contextmanager
def batch(name):
    log.append(f"open {name}")
    try:
        yield {"name": name, "rows": []}   # what `as` binds
    finally:
        log.append(f"close {name}")        # runs however the block ends

try:
    with batch("2026-09-01") as b:
        b["rows"].append("r1")
        raise RuntimeError("sink down")
except RuntimeError as exc:
    print("raised:", exc)

print(log)

A record has four common shapes in Python and they are not interchangeable. A `dict` is flexible and unvalidated; a `tuple` is compact and positional; a `NamedTuple` is an immutable record with names; a `dataclass` is a mutable-by-default record with names, defaults and methods.

The honest default at an ingestion boundary is a `dict`, because that is what JSON and CSV give you and you do not yet know the shape is right. The honest default *after* validation is a typed record, because at that point the keys are a contract and a typo should be an error rather than a `None`.

The decision is not about elegance. It is about where you want a mistake to surface: `row["amuont"]` raises at the moment of the typo, `row.get("amuont")` returns `None` and surfaces three functions later, and `record.amuont` fails immediately with the attribute name in the message.

NamedTuple

A tuple subclass with named fields. Immutable, hashable, unpackable, and very cheap.

__slots__

Declares a fixed attribute set, dropping the per-instance dict. Saves real memory at millions of rows.

`@dataclass` reads the annotated class attributes and writes the boilerplate they imply: `__init__`, `__repr__` and `__eq__`, with the fields in declaration order. `eq=True` is the default, which is why two dataclass instances with the same field values compare equal without you writing anything.

The one rule you must know is the mutable-default rule, and it is the class-level version of the trap from chapter 16. A `list` as a field default would be shared by every instance, so dataclasses refuse it outright — you use `field(default_factory=list)` instead, which is called per instance.

`dataclasses.replace(record, field=value)` returns a new record with one field changed, which is the immutable-update idiom, and `asdict(record)` turns it back into a plain dict for serialisation.

default_factory

A zero-argument callable called per instance to build a field default. The fix for mutable defaults.

replace()

Returns a copy of a dataclass with named fields changed. The immutable-update idiom.

asdict()

Recursively converts a dataclass to a plain dict, for serialisation.

from dataclasses import dataclass, field

try:
    @dataclass
    class Bad:
        rows: list = []                        # refused at class creation
except ValueError as exc:
    print("ValueError:", exc)

@dataclass
class Good:
    rows: list = field(default_factory=list)   # called once per instance

a = Good()
b = Good()
a.rows.append("r1")
print(a.rows, b.rows)

from dataclasses import dataclass, asdict, replace

@dataclass(frozen=True)
class OrderKey:
    order_id: int
    sku: str

k1 = OrderKey(1042, "A1")
k3 = replace(k1, sku="B7")        # a NEW record, one field changed
print(k3, asdict(k3))

`@dataclass(frozen=True)` makes instances immutable — assignment to a field raises — and, as a consequence, generates a `__hash__` consistent with the generated `__eq__`. That is the exact contract chapter 16 said a key needs.

This is the clean answer to "deduplicate these records" and "index these by their business key". Instead of a tuple whose positions you have to remember, you get a named record that can go in a set or be a dict key, and prints readably when something goes wrong.

The cost is real and small: you cannot edit one in place, so updates go through `replace()`. In practice that is a benefit — a key that cannot be edited after insertion cannot get lost in its container.

frozen=True

Blocks attribute assignment and generates `__hash__`. Makes the record usable as a key.

eq=True

The default. Generates field-by-field equality — and, without `frozen`, sets `__hash__` to `None`.

from dataclasses import dataclass

@dataclass(frozen=True)
class OrderKey:
    order_id: int
    sku: str

k1 = OrderKey(1042, "A1")
k2 = OrderKey(1042, "A1")

print(k1 == k2, len({k1, k2}))   # equal by value, so the set holds one
print(k1)

Both express "this thing must have these methods". They differ in how the requirement is enforced and, more usefully, in who has to know about it.

An **ABC** is nominal: a class must inherit from it, and Python refuses to instantiate a subclass that has not implemented the abstract methods. That is a real runtime guarantee, and it requires you to own — or at least be able to edit — the implementing classes.

A **Protocol** is structural: anything with the right methods satisfies it, with no inheritance and no import. Type checkers verify it; at runtime nothing does. That is exactly right for describing something you do not own, like a third-party client, or for keeping a test double from having to inherit anything.

ABC

Abstract Base Class. Nominal typing: subclasses must inherit and implement, enforced at instantiation.

structural typing

Compatibility decided by shape rather than by declared inheritance. What Protocol provides.

from typing import Protocol

class Sink(Protocol):
    def write(self, rows: list) -> int: ...

class WarehouseSink:                 # no inheritance, no import of Sink
    def write(self, rows):
        return len(rows)

def flush(sink: Sink, rows):
    return sink.write(rows)

print(flush(WarehouseSink(), ["a", "b"]))

These are the ones that come up in code review, and each has a one-line fix once you can name it. The pattern across all of them is a construct used for its syntax rather than for its guarantee.

Four exercises where the shape of the contract is the exercise. Each one can be written with raw dicts and defensive `.get()` calls, and each is considerably shorter and more defensible when the record and the boundary are typed.

Where a construct in this chapter is unavailable in the sandbox — `contextlib`, for example — write the class form with `__enter__` and `__exit__` instead. The guarantee is the same.