Amazon Bedrock: models, Knowledge Bases and Guardrails
We'll cover the following
Visual summary

Download imageFree to share, please link back to this lesson.
Amazon Bedrock is AWS's managed service for using foundation models from several providers through one API, inside your AWS account. For a platform team the attraction is that access is controlled by IAM, traffic can stay on PrivateLink, usage shows up on the AWS bill, and AWS states that your prompts and outputs are not used to train the models. Verify current data-handling terms for your regulatory needs.
Calling a model
Prefer the Converse API. It has one message format across models, and supports system prompts, tool use, streaming (ConverseStream) and guardrails, so changing models is mostly a change of modelId. InvokeModel takes each model's native request body and is for features Converse lacks.
import boto3
br = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = br.converse(
modelId=MODEL_OR_INFERENCE_PROFILE_ID,
system=[{"text": "Answer only from the provided context."}],
messages=[{"role": "user", "content": [{"text": question}]}],
inferenceConfig={"maxTokens": 512, "temperature": 0},
guardrailConfig={"guardrailIdentifier": GUARDRAIL_ID, "guardrailVersion": "1"},
)
text = resp["output"]["message"]["content"][0]["text"]
usage = resp["usage"] # inputTokens, outputTokens
stop = resp["stopReason"] # "end_turn", "max_tokens", "tool_use", "guardrail_intervened"Capacity and quotas
- Quotas limit requests and tokens per minute for each model and region. Exceeding them returns
ThrottlingException, so retry with jittered backoff and have a fallback. - Cross-region inference profiles route a request across several regions to absorb bursts. The profile id (for example with a
us.prefix) replaces the plain model id. - On-demand is pay per token. Provisioned throughput reserves capacity for a commitment, for steady high volume, custom models or latency guarantees.
- Batch inference is cheaper for offline jobs.
- Model access has to be enabled per account and region, and available models differ by region.
Knowledge Bases: managed RAG
A Knowledge Base takes documents from S3 (and some other connectors), chunks them, embeds them and stores vectors in a store such as OpenSearch Serverless or Aurora PostgreSQL with pgvector. You then call Retrieve to get chunks or RetrieveAndGenerate to get an answer with citations. It is quick to stand up. You trade control for convenience, with fewer chunking, hybrid-search and reranking options than a pipeline you build yourself, so measure retrieval quality before you rely on it.
Guardrails
| Policy | What it does |
|---|---|
| Content filters | Block hate, insults, sexual content, violence and misconduct at a strength you set |
| Prompt attack filter | Detects jailbreak and prompt-injection attempts in input |
| Denied topics | Blocks subjects you define in plain language (for example investment advice) |
| Word filters | Blocks listed words or phrases |
| Sensitive information | Detects PII and custom patterns, then blocks or masks them |
| Contextual grounding check | Scores whether an answer is supported by the source and relevant to the query, to catch hallucinations |
- Attach a guardrail to a Converse call, or call the standalone
ApplyGuardrailAPI on any text, which is how a gateway can use it with models outside Bedrock. - Guardrails check input and output separately. Test both, and test false positives, since over-blocking hurts real users.
- Version guardrails and reference a fixed version in production.
- A guardrail reduces risk. It does not replace least-privilege tool access or output validation.
Running it in production
- Model invocation logging to S3 or CloudWatch records prompts and responses. Treat it as sensitive data and encrypt it.
- Metrics in CloudWatch: invocations, latency, throttles, input and output tokens. Alarm on throttles and spend.
- Per-team cost: use application inference profiles with cost allocation tags, or attribute at your own gateway.
- VPC endpoint for
bedrock-runtimekeeps traffic off the public internet.
Check yourselfYou see ThrottlingException during a traffic burst on a single region. What are your options?
Retry with jittered exponential backoff, use a cross-region inference profile to spread load, request a quota increase, buy provisioned throughput if volume is steady, and fall back to another model through your gateway. Queue non-interactive work.
Check yourselfWhen would you build RAG yourself instead of using a Bedrock Knowledge Base?
When you need control Knowledge Bases do not give: custom chunking for your document types, hybrid search, reranking, per-user permission filtering, specific freshness rules, or deeper evaluation hooks. Start with the managed option if it meets your measured retrieval quality.
Summary: say it in the interview
- "I call Bedrock through Converse with a pinned guardrail version, retry throttles with jittered backoff and fall back across models, route over a VPC endpoint, and attribute cost with tags."