system_design / api integrations / 02_schema_and_models.md

Schema validation and the internal model

6 interview angles 5 min read source

Schema validation and the internal model

The other half of the integration exercise: three providers describe the same thing three different ways, and your system needs one coherent model. Where you put the translation decides how much pain a fourth provider costs.

Two model layers, not one

The single most important structural decision, and the one interviewers probe.

# API model - mirrors THEIR payload, warts included
class HotelApiResponse(BaseModel):
    hotel_id: str
    hotel_name: str
    price_cents: int
    currency_code: str
    avail: Literal["Y", "N"]                 # their encoding, preserved
    checkin: str                             # "2026-08-15", their format

# Domain model - what YOUR system means
class Offer(BaseModel):
    id: OfferId
    source: SourceName
    title: str
    price: Money                             # amount + currency, one object
    available: bool                          # a real boolean
    starts_on: date                          # a real date

Why not just parse straight into the domain model? Because then every provider quirk becomes a domain concern. When the hotel provider sends "Y" and the flight provider sends true and the car provider sends 1, a single model needs validators handling all three — and the domain model now encodes three vendors’ encoding choices.

With the split, each adapter owns its own translation and the domain model stays clean:

def _to_offer(self, raw: HotelApiResponse) -> Offer:
    return Offer(
        id=OfferId(f"hotel:{raw.hotel_id}"),          # namespaced - IDs collide across sources
        source="hotels",
        title=raw.hotel_name,
        price=Money(Decimal(raw.price_cents) / 100, raw.currency_code),
        available=raw.avail == "Y",
        starts_on=date.fromisoformat(raw.checkin),
    )

Namespace the IDs. Provider A’s 12345 and provider B’s 12345 are different things; a bare integer key will eventually collide, and the bug surfaces as one provider’s result mysteriously replacing another’s.

Validate at the boundary, trust inside

try:
    parsed = HotelApiResponse.model_validate(payload)
except ValidationError as e:
    logger.warning("contract drift", extra={"source": "hotels", "errors": e.errors()})
    raise UpstreamContractError("hotels") from e

Everything past that line is typed and trusted. This is the parse-don’t-validate principle: convert unknown data into a known type once, at the edge, then stop re-checking.

A validation failure is a signal, not just an error. It means the provider changed their contract. Log the field-level errors and alert on the rate — that’s your early warning that an upstream deployed a breaking change, and it’s often how you find out before they tell you.

Being tolerant without being blind

Robustness principle, applied carefully:

class HotelApiResponse(BaseModel):
    model_config = ConfigDict(extra="ignore")     # new fields: fine, ignore them

    hotel_id: str                                  # required - can't work without it
    hotel_name: str
    price_cents: int
    rating: float | None = None                    # optional - degrade gracefully
Field Setting Reason
Unknown extra fields extra="ignore" providers add fields; don’t break on it
Fields you need required fail loudly, don’t silently produce a broken Offer
Nice-to-have fields ` None = None`

extra="forbid" is right for your own internal APIs, where an unexpected field means a bug. It’s wrong for third-party payloads, where it means the provider shipped a feature.

Normalising the awkward types

Concept Store as Not as
Money Decimal + currency, or integer minor units float
Timestamps timezone-aware UTC datetime naive datetime, or a string
Dates date a string
Enums your own StrEnum their raw codes
IDs namespaced value object a bare int

Money as float is a real bug, not a style preference — 0.1 + 0.2 != 0.3 and rounding errors accumulate across a ledger. Store minor units as an integer, or Decimal, and keep the currency attached so you cannot accidentally add USD to EUR.

Timestamps: parse to aware UTC at the boundary, format to local only at the presentation edge. Naive datetimes from three providers in three timezones is a bug you will ship.

Mapping enums

class Availability(StrEnum):
    AVAILABLE = "available"
    SOLD_OUT = "sold_out"
    UNKNOWN = "unknown"

_HOTEL_MAP = {"Y": Availability.AVAILABLE, "N": Availability.SOLD_OUT}

def map_availability(code: str) -> Availability:
    mapped = _HOTEL_MAP.get(code)
    if mapped is None:
        logger.warning("unmapped availability code", extra={"code": code})
        return Availability.UNKNOWN            # don't crash on a new code
    return mapped

Explicit mapping tables per provider, with an UNKNOWN fallback that logs. A new status code from the provider degrades one field rather than failing the request — and the log tells you to add the mapping.

Versioning your own API

Once you expose the unified model, you own a contract:

  • Additive changes are safe — new optional fields don’t break consumers.
  • Removing or renaming a field is breaking, so version the endpoint (/v2/offers) or use an explicit deprecation window.
  • Response models on the way out too. In FastAPI, response_model= guarantees you don’t accidentally leak an internal field when the domain model grows. That’s both a contract and a security control.

See ../../backend/07_rest_apis/06_versioning_pagination.md.

Interview angle

  • “Three providers return the same concept differently. How do you model it?” — two layers: an API model per provider mirroring their payload exactly, and one internal domain model. Each adapter translates. Without the split, every provider quirk becomes a domain concern and the model accumulates vendor encodings.
  • “Where do you validate?” — once, at the boundary, into a typed model. Everything inside is then trusted. A validation failure means the provider’s contract drifted, so log field-level errors and alert on the rate.
  • “Strict or tolerant parsing of third-party payloads?”extra="ignore" for their responses, since providers add fields routinely; required only for fields you genuinely can’t work without; optional for the rest. Use extra="forbid" for your own internal APIs where an unexpected field is a bug.
  • “How do you handle money?”Decimal or integer minor units with the currency attached, never float. Float rounding accumulates across a ledger, and bundling the currency prevents adding two different ones.
  • “A provider sends a status code you’ve never seen. What happens?” — an explicit mapping table with an UNKNOWN fallback that logs. One field degrades instead of the request failing, and the log tells you to extend the mapping.
  • “Why namespace IDs across sources?” — provider A’s 12345 and provider B’s 12345 are unrelated. A bare key collides eventually, and the symptom is one source’s result silently replacing another’s.