Skip to content
Research Agent Skillsby Kalaris Labs
Install
Knowledge and RAGv1.0.0AdaptedMIT

pinecone

Guides use of Pinecone, a managed serverless vector database, through its Python client and the LangChain and LlamaIndex integrations. Covers creating indexes, upserting and querying vectors, metadata filtering, namespaces, hybrid dense and sparse search, index management, and deleting vectors. Use when building a production RAG system on a hosted vector store, adding semantic search or recommendations without running infrastructure, isolating per-user or per-tenant data with namespaces, combining dense and sparse vectors in one query, or filtering results by metadata. Do not use for self-hosted or local stores (Chroma, Weaviate) or offline similarity search (FAISS).

Install this skill

Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.

skills.sh CLI (Recommended)
npx skills add KalarisLabs/research-agent-skills --skill pinecone
uvx (Python tool runner)
uvx research-agent-skills install pinecone

Documentation & Instructions

Pinecone - Managed Vector Database

The vector database for production AI applications.

When to use Pinecone

Use when:

  • Need managed, serverless vector database
  • Production RAG applications
  • Auto-scaling required
  • Low latency critical (<100ms)
  • Don't want to manage infrastructure
  • Need hybrid search (dense + sparse vectors)

Metrics:

  • Fully managed SaaS
  • Auto-scales to billions of vectors
  • p95 latency <100ms
  • 99.9% uptime SLA

Use alternatives instead:

  • Chroma: Self-hosted, open-source
  • FAISS: Offline, pure similarity search
  • Weaviate: Self-hosted with more features

Quick start

Installation

pip install pinecone-client

Basic usage

from pinecone import Pinecone, ServerlessSpec

# Initialize
pc = Pinecone(api_key="your-api-key")

# Create index
pc.create_index(
    name="my-index",
    dimension=1536,  # Must match embedding dimension
    metric="cosine",  # or "euclidean", "dotproduct"
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

# Connect to index
index = pc.Index("my-index")

# Upsert vectors
index.upsert(vectors=[
    {"id": "vec1", "values": [0.1, 0.2, ...], "metadata": {"category": "A"}},
    {"id": "vec2", "values": [0.3, 0.4, ...], "metadata": {"category": "B"}}
])

# Query
results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=5,
    include_metadata=True
)

print(results["matches"])

Core operations

Create index

# Serverless (recommended)
pc.create_index(
    name="my-index",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(
        cloud="aws",         # or "gcp", "azure"
        region="us-east-1"
    )
)

# Pod-based (for consistent performance)
from pinecone import PodSpec

pc.create_index(
    name="my-index",
    dimension=1536,
    metric="cosine",
    spec=PodSpec(
        environment="us-east1-gcp",
        pod_type="p1.x1"
    )
)

Upsert vectors

# Single upsert
index.upsert(vectors=[
    {
        "id": "doc1",
        "values": [0.1, 0.2, ...],  # 1536 dimensions
        "metadata": {
            "text": "Document content",
            "category": "tutorial",
            "timestamp": "2025-01-01"
        }
    }
])

# Batch upsert (recommended)
vectors = [
    {"id": f"vec{i}", "values": embedding, "metadata": metadata}
    for i, (embedding, metadata) in enumerate(zip(embeddings, metadatas))
]

index.upsert(vectors=vectors, batch_size=100)

Query vectors

# Basic query
results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=10,
    include_metadata=True,
    include_values=False
)

# With metadata filtering
results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=5,
    filter={"category": {"$eq": "tutorial"}}
)

# Namespace query
results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=5,
    namespace="production"
)

# Access results
for match in results["matches"]:
    print(f"ID: {match['id']}")
    print(f"Score: {match['score']}")
    print(f"Metadata: {match['metadata']}")

Metadata filtering

# Exact match
filter = {"category": "tutorial"}

# Comparison
filter = {"price": {"$gte": 100}}  # $gt, $gte, $lt, $lte, $ne

# Logical operators
filter = {
    "$and": [
        {"category": "tutorial"},
        {"difficulty": {"$lte": 3}}
    ]
}  # Also: $or

# In operator
filter = {"tags": {"$in": ["python", "ml"]}}

Namespaces

# Partition data by namespace
index.upsert(
    vectors=[{"id": "vec1", "values": [...]}],
    namespace="user-123"
)

# Query specific namespace
results = index.query(
    vector=[...],
    namespace="user-123",
    top_k=5
)

# List namespaces
stats = index.describe_index_stats()
print(stats['namespaces'])

Hybrid search (dense + sparse)

# Upsert with sparse vectors
index.upsert(vectors=[
    {
        "id": "doc1",
        "values": [0.1, 0.2, ...],  # Dense vector
        "sparse_values": {
            "indices": [10, 45, 123],  # Token IDs
            "values": [0.5, 0.3, 0.8]   # TF-IDF scores
        },
        "metadata": {"text": "..."}
    }
])

# Hybrid query
results = index.query(
    vector=[0.1, 0.2, ...],
    sparse_vector={
        "indices": [10, 45],
        "values": [0.5, 0.3]
    },
    top_k=5,
    alpha=0.5  # 0=sparse, 1=dense, 0.5=hybrid
)

LangChain integration

from langchain_pinecone import PineconeVectorStore
from langchain_openai import OpenAIEmbeddings

# Create vector store
vectorstore = PineconeVectorStore.from_documents(
    documents=docs,
    embedding=OpenAIEmbeddings(),
    index_name="my-index"
)

# Query
results = vectorstore.similarity_search("query", k=5)

# With metadata filter
results = vectorstore.similarity_search(
    "query",
    k=5,
    filter={"category": "tutorial"}
)

# As retriever
retriever = vectorstore.as_retriever(search_kwargs={"k": 10})

LlamaIndex integration

from llama_index.vector_stores.pinecone import PineconeVectorStore

# Connect to Pinecone
pc = Pinecone(api_key="your-key")
pinecone_index = pc.Index("my-index")

# Create vector store
vector_store = PineconeVectorStore(pinecone_index=pinecone_index)

# Use in LlamaIndex
from llama_index.core import StorageContext, VectorStoreIndex

storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

Index management

# List indices
indexes = pc.list_indexes()

# Describe index
index_info = pc.describe_index("my-index")
print(index_info)

# Get index stats
stats = index.describe_index_stats()
print(f"Total vectors: {stats['total_vector_count']}")
print(f"Namespaces: {stats['namespaces']}")

# Delete index
pc.delete_index("my-index")

Delete vectors

# Delete by ID
index.delete(ids=["vec1", "vec2"])

# Delete by filter
index.delete(filter={"category": "old"})

# Delete all in namespace
index.delete(delete_all=True, namespace="test")

# Delete entire index
index.delete(delete_all=True)

Best practices

  1. Use serverless - Auto-scaling, cost-effective
  2. Batch upserts - More efficient (100-200 per batch)
  3. Add metadata - Enable filtering
  4. Use namespaces - Isolate data by user/tenant
  5. Monitor usage - Check Pinecone dashboard
  6. Optimize filters - Index frequently filtered fields
  7. Test with free tier - 1 index, 100K vectors free
  8. Use hybrid search - Better quality
  9. Set appropriate dimensions - Match embedding model
  10. Regular backups - Export important data

Performance

Operation Latency Notes
Upsert ~50-100ms Per batch
Query (p50) ~50ms Depends on index size
Query (p95) ~100ms SLA target
Metadata filter ~+10-20ms Additional overhead

Pricing (as of 2025)

Serverless:

  • $0.096 per million read units
  • $0.06 per million write units
  • $0.06 per GB storage/month

Free tier:

  • 1 serverless index
  • 100K vectors (1536 dimensions)
  • Great for prototyping

Resources

Agent operating procedure

  1. Check the environment. Confirm the corpus, embedding model and vector store versions, and where the index will live.
  2. Pin down the inputs. Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
  3. Run a small version first. Index a small subset and test retrieval on known question-passage pairs.
  4. Execute the full task using the instructions and references above.
  5. Validate the result. Measure recall@k on a labeled set; answers cite retrieved passages only.
  6. Report. State what was run (versions, commands, parameters), what was checked, and what is still uncertain.
If this happens Do this
Retrieval returns irrelevant chunks Revisit chunking, add BM25/hybrid search, or re-rank before changing the generator.
A function, flag or endpoint in these instructions is missing in the installed version Check the installed version's own documentation (help(), --help, official docs), adapt, and tell the user. Never invent an API.
A required input, identifier or parameter is ambiguous Ask the user, or state the assumption explicitly before running.

Integrity rules

  • Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
  • Answers must come from retrieved text; say when the corpus does not contain the answer.
  • Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
  • Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.

Related skills

  • faiss: Facebook's library for efficient similarity search and clustering of dense vectors.
  • qdrant-vector-search: High-performance vector similarity search engine for RAG and semantic search.
  • chroma: Open-source embedding database for AI applications.

Trigger Prompts & Evaluation Corpus

These evaluation prompts test that the agent correctly selects pinecone for relevant tasks and leaves it out for near misses.

Should trigger (3)

  • should trigger

    “How do I upsert embeddings into a Pinecone index?”

    expects Shows creating or connecting to a Pinecone index with the Python client and upserting (id, values, metadata) tuples in batches of roughly 100-200. Notes that the dimension must match the embedding model, and suggests testing on a small subset first.

  • should trigger

    “I have 40k chunked abstracts from our lab's PubMed pull in abstracts.jsonl, embedded with text-embedding-3-small (1536 dims). I need to load them into Pinecone so each postdoc gets their own isolated space, and then query only records where year >= 2020 and journal == 'Cell'. Getting 'Vector dimension 768 does not match the dimension of the index 1536' on upsert. Can you fix and set it up?”

    expects Explains the error as a mismatch between the vectors being sent (768-dim) and the index (1536-dim), and says to check that the embedding model actually used matches the index dimension, or recreate the index with the correct dimension. Then sets up a serverless index, upserts in batches with metadata (year, journal), uses one namespace per postdoc, and queries with a metadata filter such as {'year': {'$gte': 2020}, 'journal': {'$eq': 'Cell'}}. Confirms the index name, region, and API key handling with the user before running anything that costs money.

  • should trigger

    “We're building a literature search tool for our group and want it hosted so we don't run any servers. Queries mix exact gene names like 'BRCA2 c.5946delT' with fuzzy conceptual phrasing, and keyword-only or embedding-only search each miss things. What's the best way to combine both in a single query against a managed vector store?”

    expects Recommends a managed serverless vector database such as Pinecone and hybrid search that combines dense and sparse (BM25/SPLADE) vectors in one query, with an alpha or weighting parameter to balance them. Mentions that exact identifiers benefit from the sparse component and suggests validating with recall@k on labeled query-passage pairs.

Near misses (2)

  • near miss

    “I want to run similarity search over 5M protein embeddings entirely offline on a GPU box with no cloud services. Which index type should I use, IVF or HNSW?”

    expects Recommends an offline library such as FAISS and compares IVF (with PQ for memory) against HNSW for a 5M-vector corpus, including GPU support. Does not suggest Pinecone or any hosted service.

  • near miss

    “I'm prototyping a RAG demo in a notebook and want a local, open-source embedding store with a simple API that persists to disk. How do I set up a collection and add documents with metadata?”

    expects Uses Chroma with a PersistentClient, creating a collection and calling add with documents, metadatas and ids. Does not use Pinecone or any hosted service.