Simple Chat API

The Simple Chat API is the most basic GenAI architecture pattern. It consists of a single, stateless request-response cycle: a user sends a message, the system prepends a system prompt, calls an LLM, and returns the generated text. No memory, no retrieval, no tool use — just a direct conversation turn.

When to Use

• Single-turn Q&A applications (FAQ bots, helpdesk)
• Text transformation tasks (summarization, translation, reformatting)
• Code generation or completion from a single prompt
• Prototyping and validating prompt designs before adding complexity

Complexity Level

Low. This is the starting point for every GenAI project. If this pattern solves your problem, do not add additional complexity. Many production applications are just well-crafted system prompts with good error handling.

Basic Chat Completion

import anthropic

client = anthropic.Anthropic()

def chat(user_message: str, system: str = "You are a helpful assistant.") -> str:
    """Single-turn chat completion."""
    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        system=system,
        messages=[{"role": "user", "content": user_message}],
        temperature=0.3,
    )
    return response.content[0].text

Streaming Response

def chat_stream(user_message: str, system: str = "You are a helpful assistant."):
    """Stream tokens as they are generated."""
    with client.messages.stream(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        system=system,
        messages=[{"role": "user", "content": user_message}],
    ) as stream:
        for text in stream.text_stream:
            print(text, end="", flush=True)

Error Handling with Retry

import time
from anthropic import RateLimitError, APITimeoutError

def chat_with_retry(user_message, max_retries=3):
    for attempt in range(max_retries):
        try:
            return chat(user_message)
        except RateLimitError:
            wait = 2 ** attempt  # exponential backoff
            print(f"Rate limited. Retrying in {wait}s...")
            time.sleep(wait)
        except APITimeoutError:
            print("Timeout. Retrying...")
    raise Exception("Max retries exceeded")

Here is the step-by-step flow of a single request through the Simple Chat API architecture:

• 1. Client sends request — User message + optional parameters (temperature, max_tokens) via HTTP POST
• 2. API gateway validates — Check auth token, rate limits, input length, and content policy
• 3. System prompt prepended — Server-side system prompt is added to the messages array (never exposed to client)
• 4. LLM API called — Request forwarded to the model provider (Anthropic, OpenAI, etc.)
• 5. Tokens generated — Model generates output tokens autoregressively
• 6. Response returned — Complete text (or streamed chunks) sent back to client
• 7. Logging & metrics — Log latency, token counts, and errors for observability

• API key rotation and secrets management (e.g., AWS Secrets Manager, GCP Secret Manager)
• Rate limiting per user/API key to prevent abuse
• Input validation: max length, content filtering, injection detection
• Output validation: format checks, PII scanning, toxicity filtering
• Structured logging: request ID, latency, token usage, model version
• Retry logic with exponential backoff and circuit breaker
• Cost monitoring and alerting on token spend
• Prompt versioning and A/B testing framework
• Health check endpoint for load balancer
• Graceful degradation when LLM provider is down