Streamlit LLM Chat Apps - Interview Questions

1. What primitives does Streamlit ship for chat UIs?

Combined with st.session_state for chat history, that’s everything you need for a working LLM chat app.

2. What is the canonical chat-app skeleton?

import streamlit as st
from openai import OpenAI

client = OpenAI(api_key=st.secrets["OPENAI_API_KEY"])

if "messages" not in st.session_state:
    st.session_state.messages = []

# Replay history on every rerun
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

# Input box
if prompt := st.chat_input("Ask something"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        stream = client.chat.completions.create(
            model=MODEL,  # keep the id in config, not inline - model names churn fast
            messages=st.session_state.messages,
            stream=True,
        )
        response = st.write_stream(
            chunk.choices[0].delta.content or ""
            for chunk in stream
        )

    st.session_state.messages.append({"role": "assistant", "content": response})

Memorize this pattern — it’s the template ~90% of Streamlit chat questions are testing.

3. Why replay history on every rerun?

Because Streamlit re-runs the entire script top-to-bottom on every input. The DOM is rebuilt; st.chat_message blocks from the previous run are gone.

The pattern is:

1. Render all past messages from st.session_state.messages.
2. Read new user input from st.chat_input.
3. If non-empty: append user msg, call LLM, append assistant msg, let the next rerun replay them.

The history list in session state is the source of truth; the screen is a projection of it.

4. How do you stream tokens from the LLM?

st.write_stream takes any iterable of strings (or string-yielding generator), renders them incrementally, and returns the full concatenated string.

def token_gen():
    for chunk in openai_stream:
        delta = chunk.choices[0].delta.content
        if delta:
            yield delta

with st.chat_message("assistant"):
    full_response = st.write_stream(token_gen())

For Anthropic SDK:

with client.messages.stream(model=MODEL, messages=msgs) as stream:
    full_response = st.write_stream(stream.text_stream)

st.write_stream handles the typewriter effect; you handle saving full_response to session state.

5. How do you show tool calls / agent steps?

Wrap each step in st.status to give the user a collapsible “what the agent did” view:

with st.chat_message("assistant"):
    with st.status("Thinking...", expanded=False) as status:
        st.write("Looking up product...")
        product = lookup(query)
        st.write(f"Found: {product.name}")

        st.write("Checking inventory...")
        stock = check_stock(product.id)
        st.write(f"Stock: {stock}")

        status.update(label="Done", state="complete", expanded=False)

    st.write(f"In stock: {stock} units of {product.name}.")

Pattern works with LangChain / LangGraph callbacks — emit each tool start/end into a status block.

6. How do you handle file uploads in a chat app (RAG, vision)?

Two main patterns:

Persistent uploader at the top:

files = st.file_uploader("Upload context", accept_multiple_files=True)
if files:
    docs = [extract_text(f) for f in files]
    st.session_state["context_docs"] = docs

# In the chat handler:
if prompt := st.chat_input("Ask"):
    context = "\n\n".join(st.session_state.get("context_docs", []))
    response = llm(f"Context:\n{context}\n\nQuestion: {prompt}")

Inline file input (chat_input with files, 1.39+):

prompt = st.chat_input(
    "Ask",
    accept_file=True,
    file_type=["png", "jpg", "pdf"],
)
if prompt and prompt.text:
    if prompt["files"]:
        image = prompt["files"][0]
        # send to a vision model

For RAG, the common shape is: upload → chunk + embed (cached) → on each query, embed query + retrieve + stuff context into prompt.

7. How do you collect feedback on responses?

with st.chat_message("assistant"):
    st.markdown(response)
    feedback = st.feedback("thumbs", key=f"fb_{msg_id}")
    if feedback is not None:
        log_feedback(msg_id, "up" if feedback == 1 else "down")

st.feedback returns the rating (or None). Each message needs a unique key so feedback ratings are tracked per message in session state.

For free-text feedback, pair with st.text_area inside an expander — keeps the UI clean.

8. How do you scale a chat app — multi-user, persistence?

For anything user-facing past internal tooling, persist conversations to a DB (Postgres / DynamoDB / Firestore), key on user id, and load on app start.

9. How do you reset the conversation?

if st.sidebar.button("New chat"):
    st.session_state.messages = []
    st.rerun()

For multiple conversations (sidebar list), keep them as {conv_id: messages}:

if "conversations" not in st.session_state:
    st.session_state.conversations = {"default": []}
    st.session_state.active = "default"

# Sidebar: list + new chat button
for cid in st.session_state.conversations:
    if st.sidebar.button(cid):
        st.session_state.active = cid
        st.rerun()

if st.sidebar.button("+ New"):
    new_id = str(uuid.uuid4())[:8]
    st.session_state.conversations[new_id] = []
    st.session_state.active = new_id
    st.rerun()

messages = st.session_state.conversations[st.session_state.active]

10. Common LLM-chat pitfalls

• Forgetting to replay history — chat looks empty after every rerun. Always render st.session_state.messages before the input.
• Saving the partial stream instead of the final string — capture the return value of st.write_stream, not the generator itself.
• Calling the LLM at the top of the script (outside the if prompt: guard) — fires on every rerun, burning money.
• Mutating message dicts — append new dicts instead of editing existing ones, otherwise re-render produces stale displays.
• Not handling LLM errors — wrap calls in try/except and st.error(...) so the user sees what failed; otherwise the app silently breaks.
• Cache_data on the LLM call — usually wrong; chat answers should not be cached across users. If you must cache for cost (e.g., RAG retrieval), cache the retrieval, not the generation.
• Long blocking calls — Streamlit re-runs are blocking; if you don’t stream, the user stares at a blank space for 10s. Always stream.

11. Cross-links

• LLM JSON / structured output: ../../../ai_ml/10_agents_orchestration/09_llm_json_validation.md
• RAG and embeddings: ../../../ai_ml/09_rag_embeddings/
• Session state details: 02_session_state.md
• Practical case (LLM file pipeline): ../../../practical_cases/01_llm_file_processing_pipeline.md

Interview angle

• “Sketch the Streamlit chat skeleton.” - history in st.session_state, replay it on every rerun, st.chat_input for the prompt, st.chat_message containers, and st.write_stream for token streaming. The replay step is what people forget, and the chat appears empty without it.
• “How do you stream tokens?” - st.write_stream over a generator yielding deltas; it renders incrementally and returns the joined string, which is what you append to history.
• “What breaks at scale?” - each user holds a WebSocket and state is per-session, so persistence needs a database and horizontal scaling needs sticky sessions. Reruns also cancel in-flight streams, so avoid triggering them mid-response.