AWS Bedrock and AgentCore

AWS’s managed LLM and agent-runtime offerings. Bedrock = managed model API (similar to OpenAI’s API but for Anthropic / Llama / Mistral / Titan / etc.). AgentCore (2024) = managed agent runtime — primitives for running agentic workflows on AWS.

Bedrock — managed LLM API

import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="anthropic.claude-sonnet-4-5-20251001-v2:0",
    messages=[{"role": "user", "content": [{"text": "Hello"}]}],
    inferenceConfig={"maxTokens": 1024, "temperature": 0.7},
)
print(response["output"]["message"]["content"][0]["text"])

Bedrock fronts multiple model providers:

• Anthropic Claude — most popular on Bedrock.
• Meta Llama.
• Mistral.
• AWS Titan — AWS’s own models.
• Cohere, AI21, Stability AI.

Same API, different modelId. The Converse API is unified across providers; the older InvokeModel API requires provider-specific request shapes.

Bedrock vs direct provider APIs

When to pick Bedrock:

• Already on AWS; want IAM auth (no API keys to rotate).
• Compliance / data residency requirements (don’t want data leaving your VPC).
• Multi-model strategy (switch between Claude / Llama in code).
• Bedrock-native features (Knowledge Bases, Agents, Guardrails).

When to skip Bedrock:

• Need bleeding-edge model versions immediately.
• Cost-optimized for a single provider’s direct rates.
• Streaming features sometimes lag (check current state).

Bedrock Knowledge Bases

Managed RAG: upload documents to S3, Bedrock chunks + embeds + indexes them, exposes a retrieve_and_generate API:

response = client.retrieve_and_generate(
    input={"text": "What's our refund policy?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": "KB123",
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-5-20251001-v2:0",
        }
    }
)

Underneath: OpenSearch Serverless (vector store), Bedrock embeddings, Bedrock generation. No infrastructure for you to manage.

Trade-off:

• Pros: zero-ops RAG; tight AWS integration; works.
• Cons: opinionated chunking / retrieval (limited tuning); per-token + per-document costs add up; less flexible than DIY (LangChain + Pinecone, etc.).

For prototypes / well-defined corpora: Knowledge Bases. For production with custom retrieval logic: roll your own.

Bedrock Agents (the original — pre-AgentCore)

Managed agent runtime that pre-dates AgentCore. Define an agent with:

• An “instruction” (system prompt).
• A foundation model.
• Action groups (tools — defined as OpenAPI schemas, backed by Lambdas).
• Optional Knowledge Base attachments.

response = client.invoke_agent(
    agentId="AGENT123",
    agentAliasId="ALIAS",
    sessionId="user-session-42",
    inputText="Find a hotel in Paris for next weekend",
)

The agent calls your Lambdas as tools, manages session memory, handles the ReAct loop. Comparable to LangChain agents but managed.

Limitations: opinionated framework; locked into Bedrock’s loop logic; less flexible than building your own agent on Bedrock’s raw Converse API.

AgentCore (2024)

AWS’s next-generation agent runtime, announced late 2024. Primitives:

Runtime

Serverless agent execution environment — your agent code runs in a managed runtime that handles:

• Long-running sessions (hours / days).
• Pausing and resuming on async tool waits.
• Scaling automatically.

Think Lambda but with first-class “wait for an external event / human input” semantics.

Memory

Managed conversation + long-term memory:

• Session memory (current conversation).
• Long-term memory (across sessions, per user).
• Tool result caching.

API similar to LangChain memory + Letta but managed.

Identity

Per-session identity propagation. The agent runs with the end-user’s IAM identity (or a delegated role), enabling per-user authorization on AWS resources without your code threading credentials.

Gateway

Managed API for exposing tools to agents. Define a tool schema; AgentCore routes calls to your Lambda / HTTP endpoint with auth and retry logic. Effectively MCP-like, AWS-flavored.

Observability

Built-in tracing, logging, evaluation hooks. Plugs into CloudWatch + X-Ray.

When to use AgentCore vs LangGraph vs custom

Pick AgentCore when:

• Heavily AWS-committed.
• Long-running agent sessions (multi-day workflows).
• Want managed memory + identity propagation.
• Don’t need to portability outside AWS.

Pick LangGraph or custom when:

• Multi-cloud or on-premise.
• Need full control over agent loop.
• Existing investment in LangGraph / similar frameworks.

For most senior backend roles in 2026: knowing that AgentCore exists and its high-level primitives is enough. Hands-on use is rare outside AWS-native shops.

Prompt caching on Bedrock

Bedrock supports prompt caching for Anthropic / Llama models (varies by provider):

response = client.converse(
    modelId="anthropic.claude-sonnet-4-5-20251001-v2:0",
    system=[{
        "text": LONG_STATIC_PROMPT,
        "cachePoint": {"type": "default"},
    }],
    messages=[...],
)

5-minute ephemeral cache. Subsequent calls within the window pay reduced rates for cached tokens (~10% of normal).

For agent loops with long system prompts (tool definitions, examples), this is a major cost lever.

Guardrails

Bedrock-native content filtering / topic restriction / PII redaction:

guardrail = client.create_guardrail(
    name="my-guardrail",
    blockedInputMessaging="Cannot answer that.",
    contentPolicyConfig={
        "filtersConfig": [
            {"type": "HATE", "inputStrength": "HIGH", "outputStrength": "HIGH"},
            {"type": "VIOLENCE", "inputStrength": "MEDIUM", "outputStrength": "HIGH"},
        ]
    },
    sensitiveInformationPolicyConfig={...},   # PII detection + redaction
    topicPolicyConfig={                          # block off-topic queries
        "topicsConfig": [
            {"name": "Medical Advice", "definition": "...", "type": "DENY"}
        ]
    },
)

response = client.converse(
    modelId="...",
    messages=[...],
    guardrailConfig={"guardrailIdentifier": guardrail["guardrailId"], "guardrailVersion": "1"},
)

Off-the-shelf safety controls. Less flexible than building your own classifier; convenient for getting started.

Pricing model

A single Bedrock token tends to cost slightly more than the same provider’s direct API. The Bedrock premium pays for IAM auth, VPC endpoints, AWS integration. Often worth it for AWS-heavy orgs; not for cost-only optimization.

Operational considerations

Quotas

Bedrock has per-model, per-region quotas (requests per minute, tokens per minute). Hit them → 429s. Request quota increases via Service Quotas console; can take days for non-trivial bumps.

Multi-region

Cross-region inference profiles route requests to the lowest-latency region with capacity. Reduces 429s; adds slight latency variance.

Streaming

converse_stream for token-by-token streaming. Same Converse API, response is an iterator of events.

response = client.converse_stream(
    modelId="...",
    messages=[...],
)
for event in response["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"]["text"], end="")

Tool use via Converse

response = client.converse(
    modelId="...",
    messages=[...],
    toolConfig={
        "tools": [{
            "toolSpec": {
                "name": "get_weather",
                "description": "Get current weather",
                "inputSchema": {"json": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}
            }
        }],
        "toolChoice": {"auto": {}},
    },
)

Same shape across providers; provider-specific quirks hidden by the Converse API. Cleaner than invoke_model with provider-specific request bodies.

When NOT to use Bedrock

• Need bleeding-edge model versions — direct provider APIs land new versions first.
• Cost-critical without AWS-flavored value-adds — direct rates often cheaper.
• Multi-cloud strategy — Bedrock locks the LLM layer to AWS.
• Heavy customization of inference (custom tokenizers, fine-grained logit access) — providers may expose more.

Interview angle

• “What is AWS Bedrock?” — AWS’s managed LLM API. Fronts multiple model providers (Anthropic, Meta, Mistral, AWS Titan) behind a unified API. IAM-based auth, VPC endpoints, tight AWS service integration. Bedrock vs direct provider: ease of integration on AWS vs day-one access to newest models.
• “Bedrock vs OpenAI / direct Anthropic API?” — Bedrock for AWS-committed orgs needing IAM auth, VPC routing, compliance / data residency. Direct APIs for bleeding-edge models, lower latency in some cases, sometimes lower cost. Different orgs land differently.
• “What’s Bedrock Knowledge Bases?” — managed RAG: upload docs to S3, Bedrock chunks/embeds/indexes them in OpenSearch Serverless, exposes retrieve_and_generate. Zero-ops but opinionated; suitable for prototypes and standard corpora.
• “What’s AgentCore?” — AWS’s next-gen agent runtime (2024+). Primitives: Runtime (serverless agent execution with long-running sessions), Memory (managed conv + long-term), Identity (per-user IAM propagation), Gateway (managed tools), Observability (CloudWatch + X-Ray). For AWS-heavy orgs running long-lived agents.
• “AgentCore vs LangGraph?” — AgentCore: managed runtime, AWS-locked, long-session-native, less customizable. LangGraph: self-hosted, portable, fully customizable, you operate the infra. Pick AgentCore if AWS-only and want managed; LangGraph for multi-cloud / portable / customizable.
• “How does Bedrock prompt caching work?” — mark prompt sections with cachePoint; provider caches them for ~5 min ephemeral. Subsequent calls pay ~10% of normal token rate for cached content. Huge win for agent loops with large fixed prompts (tool definitions, examples).
• “What are Bedrock Guardrails?” — managed content / topic / PII filters applied to inference. Configure once, attach to invocations. Less flexible than custom classifiers but plug-and-play for safety baselines.