Home

Skill

Cloud Infrastructure

storing-and-querying-vectors

aws/agent-toolkit-for-aws

Store and query vector embeddings using Amazon S3 Vectors, a cost-effective long-term vector storage service with its own API namespace (s3vectors). Triggers on: create S3 vector bucket, vector index, store embeddings, semantic search, RAG vector storage, similarity search, vector database, migrate from other vector databases. Do NOT use for: querying tabular data (use querying-data-lake), S3 object storage, or hundreds/thousands of sustained QPS (use OpenSearch).

...Expand all

Updated time August 24, 2026

# rag # aws # Cloud Infrastructure # vector-database # semantic-search # embeddings

About storing-and-querying-vectors

storing-and-querying-vectors is a task guide for using Amazon S3 Vectors, a cost-effective AWS service for storing and querying vector embeddings at scale under its own s3vectors API namespace. It is optimized for long-term storage with subsecond latency on cold queries and as low as 100ms when warm, making it a fit for RAG and other workloads with infrequent queries. A decision guide steers users elsewhere when appropriate: OpenSearch for hundreds or thousands of sustained queries per second or for hybrid search and aggregations (optionally with S3 Vectors as the storage engine), and a tiered pattern pairing S3 Vectors with OpenSearch Serverless for real-time needs.

The workflow runs through creating a vector bucket, creating an index, optionally generating embeddings with Amazon Bedrock, writing vectors, and querying them, preferring AWS MCP server tools and falling back to the AWS CLI. A recurring theme is immutability: bucket encryption (SSE-S3 or SSE-KMS) and every index parameter, including dimension (1 to 4096), distance metric (cosine or euclidean), and up to ten non-filterable metadata keys, cannot be changed after creation, so they must be confirmed up front. The embedding dimension must match the index and the same model must be used for storing and querying.

Operational limits are called out: at most 500 vectors per put-vectors call, retry with backoff on 429 throttling, and metadata filtering via --filter or --return-metadata requiring both s3vectors:QueryVectors and s3vectors:GetVectors permissions. A troubleshooting table maps errors like DimensionMismatch, 403, and AccessDenied (the service uses the s3vectors namespace, not s3) to concrete fixes, and companion references cover limits, multi-tenant patterns, and metadata filtering.

FAQ

When is S3 Vectors the right choice versus OpenSearch?

S3 Vectors fits cost-effective long-term storage with infrequent queries and RAG; for hundreds or thousands of sustained QPS, hybrid search, or aggregations, the skill recommends OpenSearch, optionally with S3 Vectors as the storage engine.

Which index parameters are immutable?

Every index parameter is immutable after creation: dimension (1 to 4096), distance metric (cosine or euclidean), and up to ten non-filterable metadata keys, as is bucket encryption.

How many vectors can I write per call?

No more than 500 vectors per put-vectors call; batching is recommended and bulk operations should use an SDK rather than the CLI.

What IAM permissions do metadata filters require?

Using --filter or --return-metadata requires both s3vectors:QueryVectors and s3vectors:GetVectors; without GetVectors these options return 403. The service uses the s3vectors namespace, not s3.

How are embeddings generated and matched?

Via Bedrock invoke-model (for example Titan or Cohere); the embedding dimension must match the index, and the same model must be used for both storing and querying.