Semantic Search

What is Semantic Search?

Semantic search understands meaning, not just keywords. Search for "what's the user's favorite color?" and find memories about color preferences—even if they don't contain the exact words.

Tip

v0.30.0: Facts now support native semantic search. Configure embedding in your SDK config and embeddings are auto-generated for recall() queries - no manual embedding code needed!

---

Quick Start

// Configure embedding once at SDK init (v0.30.0+)
const cortex = new Cortex({
  convexUrl: process.env.CONVEX_URL!,
  embedding: { provider: 'openai', apiKey: process.env.OPENAI_API_KEY },
});

// Search with semantic understanding - embeddings auto-generated!
const result = await cortex.memory.recall({
  memorySpaceId: "user-123-space",
  query: "what are the user's dietary preferences?",
  userId: "user-123",
  limit: 5,
});

---

recall() vs search()

Feature	recall()	search()
Primary Use	Unified orchestrated retrieval	Vector-focused search
Sources Queried	Vector + Facts + Graph	Vector only
Ranking	Multi-signal scoring	Vector similarity only
LLM Context	Built-in formatting	Manual formatting
Best For	RAG, context building	Simple searches, performance

Unified retrieval across all memory layers:

const result = await cortex.memory.recall({
  memorySpaceId: "user-123-space",
  query: "user preferences",
  embedding: await embed("user preferences"),
  userId: "user-123",
  limit: 10,
  sources: {
    vector: true,   // Search vector memories
    facts: true,    // Search structured facts
    graph: true,    // Query graph relationships
  },
  formatForLLM: true,
});

// Ready-to-use LLM context
console.log(result.context);

// Individual items with source info
result.items.forEach(item => {
  console.log(`${item.content} (${item.source}: ${item.score})`);
});

Direct vector search for performance-critical scenarios:

const memories = await cortex.memory.search(
  "user-123-space",
  "user preferences",
  {
    embedding: await embed("user preferences"),
    userId: "user-123",
    limit: 10,
  }
);

memories.forEach(memory => {
  console.log(`${memory.content} (score: ${memory.score})`);
});

---

Search Strategies

Uses vector embeddings for meaning-based search:

const results = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  strategy: "semantic",
});

Tip

Best for natural language queries and concept matching. "car" finds "automobile".

Traditional text matching (no embeddings required):

const results = await cortex.memory.search(spaceId, "dark mode", {
  strategy: "keyword",
});

Tip

Best for exact matches—names, IDs, specific terms. Works without embeddings.

Cortex chooses the best strategy automatically:

// With embedding → tries semantic first
const results = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  strategy: "auto",  // Default
});

// Without embedding → uses keyword search
const results = await cortex.memory.search(spaceId, query);

Fallback chain: Semantic → Keyword → Recent memories

---

Embedding Providers

Info

Cortex is embedding-agnostic. Use any provider that returns vectors.

import OpenAI from 'openai';

const openai = new OpenAI();

async function embed(text: string): Promise<number[]> {
  const response = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: text,
  });
  return response.data[0].embedding;
}

import { CohereClient } from 'cohere-ai';

const cohere = new CohereClient();

async function embed(text: string): Promise<number[]> {
  const response = await cohere.embed({
    texts: [text],
    model: "embed-english-v3.0",
  });
  return response.embeddings[0];
}

import { embed } from 'ai';
import { openai } from '@ai-sdk/openai';

const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: text,
});

Embedding Model Comparison

Feature	Dimensions	Speed	Accuracy	Cost
text-embedding-3-small (OpenAI)	1536	Fast	Good	$
text-embedding-3-large (OpenAI)	3072	Medium	Best	$$
embed-english-v3.0 (Cohere)	1024	Fast	Good	$
all-MiniLM-L6-v2 (Local)	384	Very Fast	Fair	Free

Dimension Matching

Query embeddings must match stored embeddings. Don't mix 1536-dim queries with 3072-dim stored vectors.

---

Search Options

Parameter	Type	Required	Default	Description
embedding	number[]	No	—	Query vector for semantic search
strategy	string	No	auto	'semantic', 'keyword', or 'auto'
limit	number	No	20	Maximum results to return
minScore	number	No	0	Minimum similarity score (0-1)
userId	string	No	—	Filter by user
tags	string[]	No	—	Filter by tags
tagMatch	string	No	any	'any' or 'all'
minImportance	number	No	—	Minimum importance (0-100)
createdAfter	Date	No	—	Filter by creation date
createdBefore	Date	No	—	Filter by creation date

---

Result Boosting

Boost Options

Enhance ranking beyond pure semantic similarity:

const results = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  boostImportance: true,  // Boost high-importance memories
  boostRecent: true,      // Boost recent memories
  boostPopular: true,     // Boost frequently accessed
});

---

Filtering Examples

// User-scoped search with tags
const memories = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  userId: "user-123",
  tags: ["preferences", "verified"],
  tagMatch: "all",  // Must have BOTH tags
});

// High-importance recent memories
const critical = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  minImportance: 80,
  createdAfter: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000),
  minScore: 0.75,
});

// Temporal search
const thisMonth = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  createdAfter: new Date("2026-01-01"),
  createdBefore: new Date("2026-01-31"),
});

---

Search Result Format

interface SearchResult {
  id: string;
  memorySpaceId: string;
  content: string;
  contentType: "raw" | "summarized";
  
  // Source info
  source: {
    type: "conversation" | "system" | "tool" | "a2a";
    userId?: string;
    userName?: string;
    timestamp: Date;
  };
  
  // Link to full conversation (ACID store)
  conversationRef?: {
    conversationId: string;
    messageIds: string[];
  };
  
  // Metadata
  metadata: {
    importance: number;  // 0-100
    tags: string[];
  };
  
  // Search-specific
  score: number;                              // 0-1
  strategy: "semantic" | "keyword" | "recent";
  highlights?: string[];
}

---

Best Practices

Always Provide Embeddings + User Scope

// Recommended: semantic search scoped to user
await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  userId: userId,
});

// Works but less accurate (keyword fallback)
await cortex.memory.search(spaceId, query, {
  userId: userId,
});

Use Descriptive Queries

// Vague - poor results
await cortex.memory.search(spaceId, "user");

// Specific - better results
await cortex.memory.search(spaceId, 
  "what are the user's dietary restrictions and food preferences?"
);

Handle Empty Results

let memories = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  userId,
});

if (memories.length === 0) {
  // Broaden search: remove user filter
  memories = await cortex.memory.search(spaceId, query, {
    embedding: await embed(query),
    limit: 5,
  });
}

if (memories.length === 0) {
  // Final fallback: recent memories
  memories = await cortex.memory.search(spaceId, "*", {
    userId,
    sortBy: "createdAt",
    sortOrder: "desc",
    limit: 10,
  });
}

---

Performance Tips

Batch Embeddings

// Slow: generate embedding per query
for (const query of queries) {
  const embedding = await embed(query);
  await cortex.memory.search(spaceId, query, { embedding });
}

// Fast: batch embeddings first
const embeddings = await embedBatch(queries);
for (let i = 0; i < queries.length; i++) {
  await cortex.memory.search(spaceId, queries[i], { 
    embedding: embeddings[i] 
  });
}

Filter During Search

// Slow: search everything, filter after
const all = await cortex.memory.search(spaceId, query, { limit: 100 });
const highPriority = all.filter(m => m.metadata.importance >= 80);

// Fast: filter during search
const highPriority = await cortex.memory.search(spaceId, query, {
  minImportance: 80,
  limit: 20,
});

---

Common Use Cases

Building LLM Context

async function buildContext(spaceId: string, userId: string, message: string) {
  const memories = await cortex.memory.search(spaceId, message, {
    embedding: await embed(message),
    userId,
    minScore: 0.7,
    minImportance: 40,
    limit: 5,
  });

  if (memories.length === 0) {
    return "No prior context found.";
  }

  return `Relevant context:\n${memories
    .map(m => `- ${m.content}`)
    .join("\n")}`;
}

Fact Retrieval

async function getFact(spaceId: string, userId: string, question: string) {
  const memories = await cortex.memory.search(spaceId, question, {
    embedding: await embed(question),
    userId,
    limit: 1,
    minScore: 0.8,
  });

  return memories.length > 0 
    ? { fact: memories[0].content, confidence: memories[0].score }
    : { fact: null, confidence: 0 };
}

---

Troubleshooting

Causes: Query/content embedding mismatch, low score threshold

Fix:

// Raise score threshold
const memories = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  minScore: 0.75,  // Raise from default
  tags: ["relevant-topic"],  // Narrow scope
});

Causes: No matching memories, query too specific, wrong space ID

Fix:

// Check if memories exist
const total = await cortex.memory.count(spaceId);
if (total === 0) {
  console.log("Memory space is empty");
}

// Try broader query
const broader = await cortex.memory.search(spaceId, "preferences", {
  embedding: await embed("preferences"),
  minScore: 0.5,
});

Causes: Large result sets, complex filters, no caching

Fix:

// Reduce scope with filters
const memories = await cortex.memory.search(spaceId, query, {
  embedding: await embed(query),
  createdAfter: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000),
  limit: 10,
});

// Cache frequent queries
const cacheKey = `search:${spaceId}:${query}`;
const cached = cache.get(cacheKey);
if (cached) return cached;

---

Integration with Facts

Search automatically includes facts extracted via remember():

const result = await cortex.memory.recall({
  memorySpaceId: spaceId,
  query: "user preferences",
  embedding: await embed("user preferences"),
});

result.items.forEach(item => {
  if (item.fact) {
    console.log(`Fact: ${item.fact.fact}`);
    console.log(`Confidence: ${item.fact.confidence}%`);
    console.log(`Current belief: ${item.fact.isCurrentBelief}`);
  }
});

See Fact Integration for details on automatic fact extraction.

---

Next Steps