Narcis Miclaus

7 min read

Python, from the ground up Lesson 6 / 60

Context managers and the with-statement, beyond opening files

The protocol behind `with`, three ways to write your own, and when context managers beat try/finally.

Every Python beginner learns context managers the same way. Day one: open a file with open(...). Day two: someone says “use with open(...) as f: instead, it closes the file automatically.” That’s the entire mental model most people carry around for years.

It’s a fine starting point. It’s also about 5% of what with does. The same protocol handles database transactions, lock acquisition, temporary directories that clean themselves up, redirected stdout, timing blocks, exception suppression, and dozens of patterns where you need to do X on the way in and Y on the way out — guaranteed, even if the body of the block raises.

Today we look at the protocol underneath, three ways to write your own, and the cases where with is dramatically nicer than try/finally.

The protocol

A context manager is any object with two methods:

  • __enter__(self) — called when execution enters the with block. Whatever it returns is bound to the variable after as.
  • __exit__(self, exc_type, exc_value, traceback) — called when execution leaves the block, whether normally or via an exception.

That’s the whole protocol. PEP 343 made it official; everything else is libraries built on top.

class Sandbox:
    def __enter__(self) -> "Sandbox":
        print("entering")
        return self

    def __exit__(self, exc_type, exc_value, traceback) -> None:
        print(f"exiting (exception: {exc_type})")


with Sandbox() as box:
    print("inside")
    raise ValueError("oops")
# entering
# inside
# exiting (exception: <class 'ValueError'>)
# Traceback (most recent call last):
#   ...
# ValueError: oops

Even though the body raised, __exit__ ran. That’s the guarantee with gives you. It’s the same guarantee try/finally gives, but with much less code on the page and a clearer signal of intent.

The __exit__ return value (don’t get this wrong)

__exit__ can return a value. If it returns a truthy value, the exception is suppressed — execution continues after the with block as if nothing happened. If it returns None or False, the exception propagates normally.

class SwallowEverything:
    def __enter__(self) -> "SwallowEverything":
        return self

    def __exit__(self, exc_type, exc_value, traceback) -> bool:
        return True   # suppress all exceptions


with SwallowEverything():
    raise ValueError("ignored silently")

print("we get here")

99% of the time, you do not want this. Silently swallowing exceptions hides bugs. The legitimate uses are narrow — contextlib.suppress, transaction rollback handlers that re-raise after cleanup, retry decorators built as context managers. If you find yourself writing return True from an __exit__, stop and ask whether you really want to. Usually, returning None (the default) is correct.

Way 1: a class with __enter__ / __exit__

The most explicit form. Best when the manager has state that lives across enter/exit, or when the user needs to call methods on it inside the block.

import time
from typing import Optional


class Timer:
    def __init__(self, label: str) -> None:
        self.label = label
        self.elapsed_ms: float = 0.0
        self._start: Optional[float] = None

    def __enter__(self) -> "Timer":
        self._start = time.perf_counter()
        return self

    def __exit__(self, exc_type, exc_value, traceback) -> None:
        assert self._start is not None
        self.elapsed_ms = (time.perf_counter() - self._start) * 1000
        print(f"{self.label}: {self.elapsed_ms:.2f} ms")


with Timer("import csv") as t:
    import csv
    # ... do work ...

print(f"that took {t.elapsed_ms:.2f} ms total")

The Timer instance survives past the with block — t.elapsed_ms is still readable. That’s the case for the class form: when you need the manager to be a real object you interact with afterwards.

Way 2: @contextlib.contextmanager on a generator

For the common case where __enter__ is short and __exit__ is short, the class form is overkill. contextlib gives you a decorator that turns a generator function into a context manager. Everything before yield is __enter__. Everything after yield is __exit__. The yielded value is what as binds to.

from contextlib import contextmanager
from typing import Iterator
import os


@contextmanager
def in_directory(path: str) -> Iterator[None]:
    previous = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(previous)


with in_directory("/tmp"):
    # ... work in /tmp ...
    pass
# back in the original directory, even if the block raised

The try/finally is mandatory here. If the body raises, control returns to the generator at the yield site as an exception — without finally, the os.chdir(previous) would be skipped, and you’d leak the working-directory change. This is the single most common bug in hand-written context managers; pytest will catch it the first time something throws.

A database transaction is the textbook example:

from contextlib import contextmanager
from typing import Iterator


@contextmanager
def transaction(conn) -> Iterator[None]:
    cursor = conn.cursor()
    try:
        cursor.execute("BEGIN")
        yield
        cursor.execute("COMMIT")
    except Exception:
        cursor.execute("ROLLBACK")
        raise
    finally:
        cursor.close()


with transaction(conn):
    cursor = conn.cursor()
    cursor.execute("UPDATE accounts SET balance = balance - 100 WHERE id = 1")
    cursor.execute("UPDATE accounts SET balance = balance + 100 WHERE id = 2")
# COMMIT on success, ROLLBACK on failure. Cursor always closed.

This is the pattern most ORMs and DB drivers expose under names like session.begin() or engine.begin(). The bones are always the same.

Way 3: contextlib.ExitStack for dynamic contexts

Sometimes you don’t know at write-time how many contexts you’ll need. You’re opening one file per row in a config. Or N database connections based on runtime arguments. Or a variable number of locks. You can’t write with a, b, c: because you don’t know how many a, b, c are.

ExitStack is the answer:

from contextlib import ExitStack
from typing import IO


def merge_files(paths: list[str], output: str) -> None:
    with ExitStack() as stack:
        inputs: list[IO[str]] = [
            stack.enter_context(open(p, encoding="utf-8"))
            for p in paths
        ]
        out = stack.enter_context(open(output, "w", encoding="utf-8"))

        for f in inputs:
            for line in f:
                out.write(line)
    # All files closed here, in reverse order, even on exception.

stack.enter_context(cm) is equivalent to entering cm and registering its __exit__ to be called when the stack unwinds. You can register arbitrary callbacks too:

import shutil
import tempfile
from contextlib import ExitStack


def process_with_workspace(input_path: str) -> None:
    with ExitStack() as stack:
        workspace = tempfile.mkdtemp()
        stack.callback(shutil.rmtree, workspace, ignore_errors=True)

        # ... work in `workspace` ...
    # workspace is cleaned up even if the body raised

ExitStack is also the right tool when you want some cleanup conditional on what happened earlier in the block — you can pop_all() to detach the registered cleanups and run them later (or never).

Useful built-ins from contextlib

from contextlib import suppress, redirect_stdout, nullcontext
import io


# 1. suppress: ignore a specific exception type
with suppress(FileNotFoundError):
    os.remove("maybe-doesnt-exist.txt")

# 2. redirect_stdout: capture print() output
buf = io.StringIO()
with redirect_stdout(buf):
    print("captured")
print(buf.getvalue())   # "captured\n"

# 3. nullcontext: a no-op context manager — useful when you conditionally
#    want to enter a real context or none
def maybe_lock(use_lock: bool, lock):
    cm = lock if use_lock else nullcontext()
    with cm:
        do_critical_work()

suppress is the legitimate use case for an __exit__ that returns True — for one specific, named exception type. Don’t use it for “ignore everything”; use it for “this exact exception is expected and means the operation is a no-op.”

Real-world examples I’ve used

A “log and re-raise” wrapper for the boundary of a service:

from contextlib import contextmanager
from typing import Iterator
import logging
import time


@contextmanager
def request_context(name: str, logger: logging.Logger) -> Iterator[None]:
    start = time.perf_counter()
    logger.info("start", extra={"op": name})
    try:
        yield
    except Exception:
        elapsed = (time.perf_counter() - start) * 1000
        logger.exception("failed", extra={"op": name, "ms": elapsed})
        raise
    else:
        elapsed = (time.perf_counter() - start) * 1000
        logger.info("ok", extra={"op": name, "ms": elapsed})


with request_context("export-orders", log):
    run_export()

A temporary feature flag override for testing:

from contextlib import contextmanager
from typing import Iterator


@contextmanager
def feature(flag: str, value: bool) -> Iterator[None]:
    previous = flags.get(flag)
    flags[flag] = value
    try:
        yield
    finally:
        if previous is None:
            del flags[flag]
        else:
            flags[flag] = previous


def test_new_export() -> None:
    with feature("new_export_pipeline", True):
        result = run_export()
    assert result.path.endswith(".parquet")

A scoped lock that records who held it:

from contextlib import contextmanager
from threading import RLock
from typing import Iterator


_lock = RLock()
_holder: str | None = None


@contextmanager
def held_by(name: str) -> Iterator[None]:
    global _holder
    _lock.acquire()
    _holder = name
    try:
        yield
    finally:
        _holder = None
        _lock.release()

(Side note: that global is a footgun in real concurrent code — fine for diagnostics, not for correctness.)

Async context managers (preview)

asyncio has its own version. The protocol is __aenter__ / __aexit__, both async, and the syntax is async with:

async with session.get(url) as response:
    body = await response.read()

Same idea, async all the way down. We’ll cover it properly in lesson 40 when we get to async I/O — the patterns transfer almost verbatim.

When with beats try/finally

Both are correct. The reasons to prefer with:

  • Intent. with transaction(conn): says exactly what’s happening. try: ... finally: says “something needs cleanup, scroll down to find out what.”
  • Reuse. A context manager you write once is reusable in twenty places. A try/finally block is copy-pasted in twenty places.
  • Composition. with a, b, c: enters three managers in order, exits them in reverse. ExitStack lets you do the same dynamically. try/finally doesn’t compose — you’d nest, and the cleanup ordering becomes a puzzle.
  • The __exit__ always runs. It’s hard to accidentally skip cleanup with with. With try/finally you can forget the finally, or put the wrong code outside it.

try/finally is still right when the cleanup is unique to one place and doesn’t deserve a name. Don’t make a context manager called _temp_close_thing that’s used in exactly one function.

That’s context managers — the with statement, the protocol, three ways to write your own, and a few patterns that show up in production. The next module shifts gears: data classes, typing, and the modern Python type system that makes all the code we’ve been writing actually safe to refactor.

Citations (retrieved 2026-05-01):

Search