pathlib and logging

Two stdlib modules everyone uses but few use well.

pathlib — modern paths

Path replaces os.path string manipulation with object-oriented file operations. Cross-platform, more readable, and harder to misuse.

from pathlib import Path

p = Path("/var/log/app.log")
p.name           # 'app.log'
p.stem           # 'app'
p.suffix         # '.log'
p.parent         # PosixPath('/var/log')
p.parts          # ('/', 'var', 'log', 'app.log')
p.is_absolute()  # True

Building paths

home = Path.home()
config = home / ".config" / "myapp" / "settings.json"   # uses /
str(config)   # '/home/user/.config/myapp/settings.json'

The / operator joins paths and handles separators correctly on each OS.

File I/O

p = Path("notes.txt")

p.read_text()                    # whole file as str
p.read_bytes()                   # whole file as bytes
p.write_text("new content")
p.write_bytes(b"binary")

with p.open("r") as f:
    for line in f:
        ...

Inspection

p.exists()
p.is_file()
p.is_dir()
p.is_symlink()
p.stat()        # os.stat result
p.stat().st_size

Globbing

project = Path("/home/me/project")

list(project.glob("*.py"))           # top-level .py files
list(project.rglob("*.py"))          # recursive
list(project.glob("**/test_*.py"))   # recursive (alt syntax)

Mutating

p = Path("data.txt")
p.touch()              # create empty if missing
p.unlink(missing_ok=True)   # delete (3.8+ has missing_ok)
p.rename("newname.txt")
p.replace("newname.txt")    # like rename but overwrites if dest exists

Path("dir").mkdir(parents=True, exist_ok=True)   # equiv to `mkdir -p`

Common pattern: temporary work

import tempfile
from pathlib import Path

with tempfile.TemporaryDirectory() as tmp:
    workdir = Path(tmp)
    (workdir / "input.txt").write_text("...")
    # cleanup automatic on exit

Interview note

“Why use pathlib over os.path?” → cross-platform separators, OO methods chain better, type-safe (Path vs str), explicit semantics. Most modern stdlib accepts Path objects directly.

logging — structured logging done right

print for debugging is fine in scripts. For real applications, use logging. It supports levels, hierarchical loggers, configurable handlers, and structured output.

Levels

DEBUG   — verbose; for development
INFO    — normal operations
WARNING — something unusual but not failing
ERROR   — operation failed
CRITICAL — service-level failure

Higher levels include lower ones — setting INFO shows INFO, WARNING, ERROR, CRITICAL but suppresses DEBUG.

Module-level usage

import logging

logger = logging.getLogger(__name__)   # named logger per module

logger.debug("about to fetch %s", url)
logger.info("user logged in: %s", user_id)
logger.warning("retry %d for %s", n, url)
logger.error("failed to save: %s", err)
logger.exception("crashed during X")    # includes traceback automatically

logger.exception captures sys.exc_info() and logs at ERROR level with traceback. Always use it in except blocks.

Use lazy formatting

# correct — format string applied only if level enabled
logger.debug("user %s did %s", user, action)

# wrong — string built even when DEBUG is disabled
logger.debug(f"user {user} did {action}")

The %-style with deferred args is a small but real performance win, and integrates with structured logging tools.

Configuring handlers

import logging

# Set the root logger
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    handlers=[
        logging.StreamHandler(),                          # stderr
        logging.FileHandler("app.log"),                   # file
    ],
)

For larger apps, use logging.config.dictConfig to load configuration from a dict (or yaml/toml file).

Logger hierarchy

Logger names with dots form a hierarchy: myapp.api.auth is a child of myapp.api, which is a child of myapp, which is a child of root. Configuration propagates down the tree.

logging.getLogger("myapp").setLevel(logging.DEBUG)
# all of myapp.* now log at DEBUG

This is why logger = logging.getLogger(__name__) is the standard idiom — __name__ becomes the dotted module path, integrating with any hierarchy config.

Structured / JSON logging

For aggregators (Datadog, ELK, CloudWatch Insights), emit JSON:

import logging
import json

class JsonFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "ts": record.created,
            "level": record.levelname,
            "logger": record.name,
            "msg": record.getMessage(),
            "extra": getattr(record, "extra", {}),
        })

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logging.getLogger().addHandler(handler)

Or use python-json-logger, structlog, or loguru for less boilerplate.

Adding context: extra and LoggerAdapter

logger.info("payment received", extra={"user_id": 42, "amount": 100})

For per-request context (request_id, user_id), use LoggerAdapter or contextvars to flow context through async tasks.

Interview angle

• “What logging level should production use?” (INFO usually; DEBUG only in incidents.)
• “Why logger = getLogger(__name__) instead of logging.info(...) directly?” (Hierarchy — per-module config, namespaced filters/handlers.)
• “How would you implement request-scoped logging in async code?” (contextvars + LoggerAdapter.)