Amazon Bedrock: models, Knowledge Bases and Guardrails

Lesson 2.2 · 5 min read · by Vinay C

We'll cover the following

Visual summary

Infographic summarizing Amazon Bedrock: models, Knowledge Bases and Guardrails: Amazon Bedrock is AWS's managed service for using foundation models from several providers through one API, inside…

Download imageFree to share, please link back to this lesson.

Amazon Bedrock is AWS's managed service for using foundation models from several providers through one API, inside your AWS account. For a platform team the attraction is that access is controlled by IAM, traffic can stay on PrivateLink, usage shows up on the AWS bill, and AWS states that your prompts and outputs are not used to train the models. Verify current data-handling terms for your regulatory needs.

Calling a model

Prefer the Converse API. It has one message format across models, and supports system prompts, tool use, streaming (ConverseStream) and guardrails, so changing models is mostly a change of modelId. InvokeModel takes each model's native request body and is for features Converse lacks.

import boto3

br = boto3.client("bedrock-runtime", region_name="us-east-1")

resp = br.converse(
    modelId=MODEL_OR_INFERENCE_PROFILE_ID,
    system=[{"text": "Answer only from the provided context."}],
    messages=[{"role": "user", "content": [{"text": question}]}],
    inferenceConfig={"maxTokens": 512, "temperature": 0},
    guardrailConfig={"guardrailIdentifier": GUARDRAIL_ID, "guardrailVersion": "1"},
)
text = resp["output"]["message"]["content"][0]["text"]
usage = resp["usage"]            # inputTokens, outputTokens
stop = resp["stopReason"]        # "end_turn", "max_tokens", "tool_use", "guardrail_intervened"

Capacity and quotas

  • Quotas limit requests and tokens per minute for each model and region. Exceeding them returns ThrottlingException, so retry with jittered backoff and have a fallback.
  • Cross-region inference profiles route a request across several regions to absorb bursts. The profile id (for example with a us. prefix) replaces the plain model id.
  • On-demand is pay per token. Provisioned throughput reserves capacity for a commitment, for steady high volume, custom models or latency guarantees.
  • Batch inference is cheaper for offline jobs.
  • Model access has to be enabled per account and region, and available models differ by region.

Knowledge Bases: managed RAG

A Knowledge Base takes documents from S3 (and some other connectors), chunks them, embeds them and stores vectors in a store such as OpenSearch Serverless or Aurora PostgreSQL with pgvector. You then call Retrieve to get chunks or RetrieveAndGenerate to get an answer with citations. It is quick to stand up. You trade control for convenience, with fewer chunking, hybrid-search and reranking options than a pipeline you build yourself, so measure retrieval quality before you rely on it.

Guardrails

PolicyWhat it does
Content filtersBlock hate, insults, sexual content, violence and misconduct at a strength you set
Prompt attack filterDetects jailbreak and prompt-injection attempts in input
Denied topicsBlocks subjects you define in plain language (for example investment advice)
Word filtersBlocks listed words or phrases
Sensitive informationDetects PII and custom patterns, then blocks or masks them
Contextual grounding checkScores whether an answer is supported by the source and relevant to the query, to catch hallucinations
  • Attach a guardrail to a Converse call, or call the standalone ApplyGuardrail API on any text, which is how a gateway can use it with models outside Bedrock.
  • Guardrails check input and output separately. Test both, and test false positives, since over-blocking hurts real users.
  • Version guardrails and reference a fixed version in production.
  • A guardrail reduces risk. It does not replace least-privilege tool access or output validation.

Running it in production

  • Model invocation logging to S3 or CloudWatch records prompts and responses. Treat it as sensitive data and encrypt it.
  • Metrics in CloudWatch: invocations, latency, throttles, input and output tokens. Alarm on throttles and spend.
  • Per-team cost: use application inference profiles with cost allocation tags, or attribute at your own gateway.
  • VPC endpoint for bedrock-runtime keeps traffic off the public internet.
Check yourselfYou see ThrottlingException during a traffic burst on a single region. What are your options?

Retry with jittered exponential backoff, use a cross-region inference profile to spread load, request a quota increase, buy provisioned throughput if volume is steady, and fall back to another model through your gateway. Queue non-interactive work.

Check yourselfWhen would you build RAG yourself instead of using a Bedrock Knowledge Base?

When you need control Knowledge Bases do not give: custom chunking for your document types, hybrid search, reranking, per-user permission filtering, specific freshness rules, or deeper evaluation hooks. Start with the managed option if it meets your measured retrieval quality.

Summary: say it in the interview

  • "I call Bedrock through Converse with a pinned guardrail version, retry throttles with jittered backoff and fall back across models, route over a VPC endpoint, and attribute cost with tags."