Virtual Environments and Lockfiles

The two mechanisms behind “reproducible Python environments”: isolation (virtualenvs) and pinning (lockfiles). Different problems, both required.

Virtual environments — isolation

A virtualenv is a self-contained Python environment: its own site-packages, its own installed packages, independent of the system Python and of other projects.

The problem it solves: without it, pip install goes into the global/system Python. Project A needs Django 5.2 (LTS), project B needs Django 6.1 — they can’t coexist globally. And polluting the system Python (the one the OS depends on) is how you break your OS package manager.

python -m venv .venv          # create
source .venv/bin/activate     # activate (Unix);  .venv\Scripts\activate on Windows
pip install django            # now installs INTO .venv, not globally
deactivate

How it works (roughly): the venv has its own python (a symlink/copy), its own site-packages, and activation puts the venv’s bin/ first on PATH. There’s no magic — which python after activation points into .venv.

Tooling wraps this: uv venv (instant), Poetry creates/manages one automatically, pipenv, virtualenv (the older third-party tool venv was based on). The mechanism is the same.

Rule: one virtualenv per project, never install into system Python. Don’t commit the .venv/ directory — it’s machine-specific and rebuildable from the lockfile (.gitignore it).

Lockfiles — pinning / reproducibility

A virtualenv isolates, but doesn’t by itself make installs reproducible. That’s the lockfile’s job.

The distinction that matters:

You declare django>=5.2 in pyproject.toml. The tool resolves that to a consistent set and writes the lock (poetry.lock, uv.lock, or a fully-pinned requirements.txt from pip-compile). The lockfile is what makes “install the dependencies” produce the identical environment on your laptop, your teammate’s, and CI.

pyproject.toml   "django>=5.2"        ← you edit this (intent)
      │  resolve
      ▼
uv.lock          "django==4.2.11"     ← tool generates this (the exact graph)
                 + 30 transitive deps, all pinned, hashed
      │  sync
      ▼
.venv            exactly the locked set

Why pin transitive deps too

django>=5.2 is your direct dep. But Django pulls in asgiref, sqlparse, etc., and those have their own deps. If you only pin direct deps, a transitive dep can silently update between installs — and a transitive dep’s bad release can break your build with no change to your pyproject.toml. The lockfile pins the entire graph, so an install is byte-identical until you deliberately update.

Hashes

Good lockfiles include a hash per package (--hash=sha256:...). On install, pip/uv verifies the downloaded artifact matches the hash — defense against a compromised package index serving a tampered artifact. See 04_dependency_resolution_and_supply_chain.md.

Commit the lockfile (almost always)

• Applications / services — commit the lockfile. It’s the reproducibility guarantee. CI installs from the lock; production installs from the lock; every developer installs from the lock. “Works on my machine” disappears.
• Libraries — the nuance: a library publishes loose constraints (so consumers can resolve compatibly with their other deps) and shouldn’t force exact pins on consumers. But the library’s own repo still commits a lockfile for its own CI/dev environment to be reproducible. Declared constraints ship to PyPI; the lock stays in the repo.

Updating dependencies

The lock means nothing updates by accident. To update on purpose:

uv lock --upgrade-package django    # re-resolve just django (and what it forces)
uv lock --upgrade                   # re-resolve everything to latest allowed
poetry update django                # Poetry equivalent

This re-runs resolution, writes a new lock, and the diff on the lockfile shows exactly what changed — reviewable in a PR. Updating deps becomes a deliberate, visible, revertable action instead of ambient drift.

Reproducible install in CI / Docker

# copy only the dep files first → Docker layer cache stays valid
# unless the dependencies actually change
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev      # --frozen: fail if lock is stale; --no-dev: prod only
COPY . .

• --frozen (uv) / --locked (Poetry) — install exactly the lock, and error if pyproject.toml and the lock disagree. Use this in CI/prod — it catches “someone edited pyproject.toml but didn’t re-lock.”
• --no-dev — skip the dev group; production images shouldn’t ship pytest/mypy.
• Copy pyproject.toml + lock before the app code so the dependency-install layer is cached and only rebuilds when deps change.

Common gotchas

• No lockfile — installs aren’t reproducible; transitive deps drift; “works on my machine” returns.
• pip freeze treated as a lock — it’s a flat snapshot of what’s installed, not a clean resolution; doesn’t distinguish direct from transitive.
• Lockfile not committed — the reproducibility guarantee only works if everyone installs from the same lock.
• Committing .venv/ — machine-specific, huge, rebuildable. .gitignore it.
• Installing into system Python — version conflicts across projects, risk of breaking the OS.
• Editing pyproject.toml without re-locking — the lock goes stale; --frozen/--locked in CI is what catches this.
• pip install something into a Poetry/uv-managed venv — silently desyncs the environment from the lock.
• Pinning only direct deps — a transitive dep can still drift and break you.

Interview angle

• “Virtualenv vs lockfile — what does each solve?” — different problems. The virtualenv gives isolation (this project’s packages don’t collide with other projects or the system Python). The lockfile gives reproducibility (the exact same dependency graph installs everywhere). You need both.
• “What’s the difference between declared and locked dependencies?” — declared is the loose intent you write (django>=5.2); locked is the tool-generated exact resolution — every package and every transitive dependency pinned to an exact version, often hashed. You edit the declared spec; the tool writes the lock.
• “Why pin transitive dependencies?” — your direct dep django>=5.2 pulls in a tree of transitive deps; if those aren’t pinned, a transitive dep can silently update between installs and break your build with zero change to your own files. The lock pins the whole graph.
• “Do you commit the lockfile?” — for applications/services, yes — it’s the reproducibility guarantee; CI and prod install from it. For libraries, you publish loose constraints (so consumers can resolve compatibly) but still commit a lockfile in the repo for the library’s own reproducible CI/dev.
• “How do you do a reproducible install in CI/Docker?” — install from the lock with --frozen/--locked (errors if the lock is stale relative to pyproject.toml) and --no-dev for prod. Copy the dependency files before the app code so the install layer is Docker-cached.
• “How do you update a dependency safely?” — deliberately: uv lock --upgrade-package X (or poetry update X) re-resolves and rewrites the lock; the lockfile diff in the PR shows exactly what changed, making the update reviewable and revertable instead of ambient drift.