Embedding Search Testing Tool

Embedding Search Testing Tool

Overview

The Embedding Search testing tool allows you to verify that your chosen embedding model produces good similarity scores for your queries before deploying to production.

Location: Document Processing tab → "Test Embedding Search" button

When to Use: After document processing is complete, before pipeline configuration

Accessing the Tool

Prerequisites

• Documents uploaded and processed
• Embedding model configured
• Chunking settings applied

Opening the Tool

1. Navigate to your RAG project
2. Open the Document Processing tab
3. Click the "Test Embedding Search" button

How to Use

Basic Search

1. Enter query in the search box
2. Press Enter or click Search
3. Review results with similarity scores
4. Note relevance of top results

Advanced Options

Document Filters:

• Filter by specific documents
• Exclude certain documents
• Filter by document metadata

Result Count:

• Select number of results to display (5, 10, 20)
• More results for comprehensive analysis

Score Threshold:

• Set minimum similarity score
• Filter out low-scoring results

Understanding Results

Result Format

Query: "return policy"

Results (Top 5):
─────────────────────────────────────────────────
1. [Score: 0.89] Returns accepted within 30 days...
   Source: policy.pdf, Chunk 3
   
2. [Score: 0.85] Return policy overview...
   Source: policy.pdf, Chunk 1
   
3. [Score: 0.82] Electronics returns...
   Source: electronics-faq.md, Chunk 2
   
4. [Score: 0.78] Refund processing timeline...
   Source: policy.pdf, Chunk 5
   
5. [Score: 0.75] Exception items: Software, DVDs...
   Source: returns.md, Chunk 4

Score Interpretation

Result Details

Each result shows:

• Similarity Score: How well the chunk matches your query
• Content Preview: First portion of the chunk
• Source Document: Where the chunk came from
• Chunk Location: Specific chunk identifier

Connection to API Output

The Embedding Search results directly mirror what your API will return:

API Response Comparison

Embedding Search Result:

1. [Score: 0.89] Returns accepted within 30 days...
   Source: policy.pdf

API Response:

{
  "results": [
    {
      "content": "Returns accepted within 30 days...",
      "similarity_score": 0.89,
      "source": "policy.pdf"
    }
  ]
}

Key Point: What you see in Embedding Search is exactly what your API returns.

Troubleshooting

No Results Returned

Possible Causes:

• Query too specific or unique
• Documents don't contain relevant information
• Embedding model mismatch

Solutions:

• Broaden query terms
• Verify documents contain expected content
• Try different embedding model

All Scores Below 0.5

Possible Causes:

• Wrong embedding model for domain
• Chunk size inappropriate
• Query-document mismatch

Solutions:

• Upgrade to larger embedding model
• Adjust chunk size (512-768 tokens recommended)
• Review document content relevance

Irrelevant Top Results

Possible Causes:

• Chunk size too large (diluted embeddings)
• Wrong embedding model
• Documents not properly processed

Solutions:

• Reduce chunk size
• Try different embedding model
• Verify document processing completed

Inconsistent Scores

Possible Causes:

• Mixed content types
• Varying document quality
• Embedding model limitations

Solutions:

• Enable BM25 hybrid search
• Review document quality
• Consider domain-specific embedding model

Best Practices

Test Query Selection

Include:

• Simple factual questions
• Multi-part questions
• Domain-specific terminology
• Edge cases

Example Set:

1. "What is the return policy?"
2. "How do I reset my password?"
3. "What products integrate with Slack?"
4. "Can I return opened software?"
5. "What is the enterprise SLA?"

Score Documentation

Record for each query:

• Average similarity score
• Number of relevant results
• Any anomalies observed

Track over time:

• Baseline scores for comparison
• Changes after configuration updates
• Improvement trends

Iteration Workflow

1. Run initial test with default settings
2. Document baseline scores
3. Adjust one setting (model, chunk size)
4. Re-run test with same queries
5. Compare results
6. Repeat until scores acceptable

Tips for Success

1. Use Real Queries: Test with actual user questions when available
2. Document Baseline: Record scores for future comparison
3. Test Edge Cases: Include unusual queries in your test set
4. Iterate Quickly: Don't hesitate to try different models
5. Trust the Scores: Low scores predict poor API performance