Run

P2e

f9c910d6025e8060c89fe6d3bb59b181b26461b1

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """ 2 LLM 服务抽象层 - 支持多个 AI 提供商 3 4 支持的提供商： 5 - OpenAI Native (官方接口) 6 - OpenAI Compatible (兼容接口，如DeepSeek) 7 - Google (Gemini) 8 - Mock (用于测试) 9 """ 10 11 import asyncio 12 import random 13 from abc import ABC, abstractmethod 14 from typing import AsyncGenerator, List, Dict, Optional, Awaitable, Callable, TypeVar 15 import logging 16 import html 17 18 from config import settings 19 20 logger = logging.getLogger(__na Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Provider mentions: openai

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """KB retrieval service: validate agent/kb/tenant, embed query, Qdrant search + threshold filter.""" 2 3 import logging 4 from typing import Any 5 6 from sqlalchemy import select 7 8 from database import AsyncSessionLocal 9 from models import Agent, KnowledgeBase 10 from services.document_parser import DocumentParser 11 from services.kb_document_processor import get_embedding_api_key 12 from services.kb_service imp Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """KnowledgeBase service. 所有查询强制 tenant_id 过滤。""" 2 3 import logging 4 from pathlib import Path 5 6 from sqlalchemy import delete as sa_delete, func, select, update as sa_update 7 from sqlalchemy.ext.asyncio import AsyncSession 8 9 from config import settings 10 from database import AsyncSessionLocal 11 from models import Agent, KnowledgeBase, KbChunk, KbDocument, Tenant 12 from services.qdrant_service import Qdran Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 # CLAUDE.md 2 3 This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. 4 5 ## Repo layout 6 7 - `frontend-nextjs/` is the active admin/dashboard frontend. Treat the older `frontend/` directory as legacy/reference only. 8 - `backend/` is a FastAPI app with SQLite persistence, Redis-backed rate limiting/cache fallbacks, and self-KB retrieval/indexing (Qdrant). 9 - `widg Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """API v1 端点""" 2 3 from fastapi import ( 4 APIRouter, 5 Depends, 6 HTTPException, 7 status, 8 Request, 9 WebSocket, 10 WebSocketDisconnect, 11 UploadFile, 12 File, 13 BackgroundTasks, 14 ) 15 from fastapi.responses import StreamingResponse 16 from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials 17 from sqlalchemy.ext.asyncio import AsyncSession 18 from sqlalchemy import select, func, case, delete, Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """Qdrant collection management for per-KB isolation. 幂等 + Cosine + dim lookup.""" 2 3 import logging 4 import uuid 5 6 from config import settings 7 from qdrant_client import AsyncQdrantClient 8 from qdrant_client.models import ( 9 CollectionInfo, 10 Distance, 11 FieldCondition, 12 Filter, 13 MatchValue, 14 PointStruct, 15 VectorParams, 16 ) 17 18 logger = logging.getLogger(__name__) 19 20 21 def get_embedding_dim Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: /home/user/worktrees/attempt-b2ec151a-8a76-4fc3-b3db-33701bb97ba8-cap-2/backend/services/scraper.py /home/user/worktrees/attempt-b2ec151a-8a76-4fc3-b3db-33701bb97ba8-cap-2/backend/services/scraping_provider.py Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """URL抓取和内容提取服务 - 使用 Scrapling 微服务""" 2 3 import hashlib 4 from typing import Dict, Any, List, Tuple 5 import logging 6 7 from services.url_safety import validate_url_safe 8 from services.scrapling_client import get_scrapling_client 9 from services.scraping_provider import fetch_with_provider, discover_with_provider 10 11 logger = logging.getLogger(__name__) 12 13 14 class URLScraper: 15 """URL抓取器 - 通过 Scrapling 微服 Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """Configurable URL scraping providers and concurrency guards.""" 2 3 import asyncio 4 import logging 5 from datetime import datetime, timezone 6 from typing import Any, Dict, List, Tuple 7 8 import httpx 9 10 from config import settings 11 from services.scrapling_client import get_scrapling_client 12 13 logger = logging.getLogger(__name__) 14 15 _agent_semaphores: dict[str, asyncio.Semaphore] = {} 16 _workspace_se Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: /home/user/worktrees/attempt-b2ec151a-8a76-4fc3-b3db-33701bb97ba8-cap-2/scrapling-service/main.py Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 250 return dt.replace(tzinfo=timezone.utc) 251 return dt.astimezone(timezone.utc) 252 253 254 async def build_agent_config_with_stats(agent: Agent, db: AsyncSession) -> dict: 255 config = build_agent_config(agent) 256 url_count = await db.scalar( 257 select(func.count(URLSource.id)).where(URLSource.agent_id == agent.id) 258 ) 259 file_count = await db.scalar( 260 select(func.count(KnowledgeFile.id)).where(KnowledgeFi Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: Found 1 file backend/api/v1/endpoints.py Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """Encryption utilities for sensitive data like API keys. 2 3 This module handles encryption/decryption of API keys with support for: 4 1. Environment variable ENCRYPTION_KEY 5 2. Auto-generated key stored in a file (preferred for production) 6 """ 7 import os 8 import base64 9 import hashlib 10 import logging 11 import stat 12 from functools import lru_cache 13 from pathlib import Path 14 from typing import Option Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1081:async def chat( 1204:@router.post("/chat/stream") 1205:async def chat_stream( Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1200 message_id=assistant_message.id, 1201 ) 1202 1203 1204 @router.post("/chat/stream") 1205 async def chat_stream( 1206 request: ChatRequest, 1207 http_request: Request, 1208 ): 1209 """聊天流式接口（SSE） 1210 1211 Manages DB sessions explicitly to avoid holding connections open during LLM streaming. 1212 """ 1213 1214 async def event_generator(): 1215 request_start = time.monotonic() 1216 1217 # Phase 1: Preparation with Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 734:async def prepare_chat_request( Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 734 async def prepare_chat_request( 735 request: ChatRequest, 736 http_request: Request, 737 db: AsyncSession, 738 ) -> Dict[str, Any]: 739 """Prepare chat execution context shared by blocking and streaming endpoints.""" 740 result = await db.execute(select(Agent).where(Agent.id == request.agent_id)) 741 agent = result.scalar_one_or_none() 742 743 if not agent: 744 raise HTTPException( 745 status_code=status.HTTP_404 Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: Found 11 files backend/services/document_parser.py backend/services/llm_service.py backend/services/redis_service.py backend/services/scraping_provider.py backend/services/scrapling_client.py backend/tests/conftest.py backend/api/endpoints/auth.py backend/api/v1/endpoints.py backend/main.py backend/middleware/rate_limit.py backend/requirements.txt Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """Scrapling 服务 HTTP 客户端""" 2 3 import httpx 4 import logging 5 from typing import Dict, Any, List, Tuple 6 from datetime import datetime, timezone 7 8 from config import settings 9 10 logger = logging.getLogger(__name__) 11 12 13 class ScraplingClient: 14 """Scrapling 微服务 HTTP 客户端""" 15 16 def __init__(self, base_url: str = None, timeout: int = 60): 17 self.base_url = (base_url or settings.scrapling_service_url).r Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 1 """ 2 Scrapling-based web scraping service. 3 Uses curl_cffi for TLS-impersonated HTTP and readability-lxml for content extraction. 4 """ 5 6 import hashlib 7 import ipaddress 8 import logging 9 import re 10 import socket 11 from datetime import datetime, timezone 12 from typing import List, Optional 13 from urllib.parse import urljoin, urlparse, urlsplit 14 15 from bs4 import BeautifulSoup 16 import httpx 17 from Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 91: api_key = Column(String(500), nullable=True) 92: api_base = Column(String(500), nullable=True, default="https://api.openai.com/v1") 95: jina_api_key = Column(String(500), nullable=True) 98: siliconflow_api_key = Column(String(500), nullable=True) 101: provider_type = Column( 138: embedding_api_base = Column(String(500), nullable=True) Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Provider mentions: openai

Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements. Tool result summary: 104:class Settings(BaseSettings): 117: jina_embedding_api_base: str = "https://api.jina.ai/v1/embeddings" 120: scrapling_service_url: str = "http://scrapling-service:8001" 121: scraping_provider: str = "local_scrapling" 124: scraping_timeout_seconds: int = 60 138: qdrant_url: str = "http://localhost:6333" 140: qdrant_timeout: float = 30.0 152: llm_test_timeout_seconds: int = 10 153: llm_retry_attempts: int = 3 154: l Agent context: I'll research the repository's architecture first, then evaluate web-search providers against your hard requirements.

Agent context: Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency. Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency.

Query/prompt: Tavily search API enterprise data retention SOC2 privacy no training on data documentation Agent context: Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency. Query: Tavily search API enterprise data retention SOC2 privacy no training on data documentation 10 search result links captured Tool result summary: Based on the web search results, here's what I found regarding Tavily Search API's enterprise features related to data retention, SOC 2 compliance, privacy, and data training policies: ## Data Retention Tavily is SOC 2 certified with zero data retention. The Tavily API, data policies, and zero data retention commitments remain unchanged even after recent organizational changes. ## SOC 2 & Security Compliance Tavily i Agent context: Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency.

Query/prompt: Exa AI search API enterprise SOC2 data retention privacy policy zero retention Agent context: Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency. Query: Exa AI search API enterprise SOC2 data retention privacy policy zero retention 10 search result links captured Tool result summary: Based on the search results, here's what I found about Exa AI's enterprise security and data retention policies: ## SOC 2 Certification Exa is SOC 2 Type II certified, demonstrating commitment to maintaining rigorous information security practices and controls. ## Zero Data Retention (ZDR) For a search provider to offer ZDR, it must never store user query data, neither in the main service nor any subprocessors. Exa i Agent context: Repo understood. Now researching providers and consulting their docs against your hard requirements (data retention, no-training, auditability). Note the repo already uses Jina as an embedding provider — relevant for integration consistency.

Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Extract details on data retention, whether they store queries/results, whether customer data is used to train models, SOC2 compliance, privacy guarantees, and any enterprise auditability/data-processing-agreement features. Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Fetch prompt: Extract details on data retention, whether they store queries/results, whether customer data is used to train models, SOC2 compliance, privacy guarantees, and any enterprise auditability/data-processing-agreement features. Tool result summary: # Tavily Data Retention & Privacy Summary Based on the provided documentation: **Data Retention:** The page explicitly states Tavily maintains "zero data retention," meaning queries and results are not stored. **Model Training:** The documentation does not address whether customer data is used to train models. **Compliance & Security:** Tavily is "SOC 2 certified" and emphasizes being "Secure & Scalable." The company Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Extract details on data retention / zero data retention, whether they store queries, whether customer data is used to train models, SOC2 compliance, HIPAA, DPA, privacy guarantees, and enterprise auditability features. Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Fetch prompt: Extract details on data retention / zero data retention, whether they store queries, whether customer data is used to train models, SOC2 compliance, HIPAA, DPA, privacy guarantees, and enterprise auditability features. Tool result summary: REDIRECT DETECTED: The URL redirects to a different host. Original URL: https://docs.exa.ai/reference/security Redirect URL: https://exa.ai/docs/reference/security Status: 307 Temporary Redirect To complete your request, I need to fetch content from the redirected URL. Please use WebFetch again with these parameters: - url: "https://exa.ai/docs/reference/security" - prompt: "Extract details on data retention / zero d Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Bing Web Search API retirement August 2025 replacement Azure AI Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Query: Bing Web Search API retirement August 2025 replacement Azure AI 10 search result links captured Tool result summary: Based on the search results, here's what I found about the Bing Web Search API retirement and its Azure AI replacement: ## Retirement Details Bing Search APIs will be retired on August 11, 2025. Any existing instances of Bing Search APIs will be decommissioned completely, and the product will no longer be available to be used or new customer signup. ## Azure AI Replacement Customers are encouraged to migrate to Groun Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Brave Search API enterprise data privacy no training independent index documentation Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Query: Brave Search API enterprise data privacy no training independent index documentation 10 search result links captured Tool result summary: Here are the search results for your query about the Brave Search API's enterprise data privacy features, independent index, and documentation: ## Key Findings **Privacy and Zero Data Retention:** The Brave Search API is the only search API built with privacy at its core. Brave offers enterprise customers the option of Zero Data Retention (ZDR), which is available to all Brave Search API customers on a custom enterpr Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Extract details on data retention / zero data retention, whether they store queries, whether customer data is used to train models, SOC2 compliance, HIPAA, DPA, privacy guarantees, and enterprise auditability features. Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Fetch prompt: Extract details on data retention / zero data retention, whether they store queries, whether customer data is used to train models, SOC2 compliance, HIPAA, DPA, privacy guarantees, and enterprise auditability features. Tool result summary: # Exa Security & Data Handling Summary **Zero Data Retention & Enterprise Options** Exa offers "Zero Data Retention" as a customizable solution available through their Enterprise plan. Organizations requiring this capability should contact sales@exa.ai to discuss tailored arrangements. **SOC 2 Compliance** The company holds "SOC 2 Type II certified" status, indicating they maintain rigorous security controls and info Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Query/prompt: Extract details on Zero Data Retention: is it default or enterprise-only, what data is stored, no-training-on-data guarantees, SOC2, DPA, and whether ZDR is available on standard plans or requires custom enterprise. Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status. Fetch prompt: Extract details on Zero Data Retention: is it default or enterprise-only, what data is stored, no-training-on-data guarantees, SOC2, DPA, and whether ZDR is available on standard plans or requires custom enterprise. Tool result summary: # Zero Data Retention (ZDR) Details from Brave ## Availability & Pricing ZDR is **enterprise-only**, not a default feature. According to the article, "ZDR is available to all Brave Search API customers on a custom enterprise plan" and requires contacting API support to enable it. ## Data Storage Practices The company states that regardless of plan type, "Brave Search API data is used and maintained only for the narro Agent context: Let me consult the actual documentation pages for the leading candidates, plus check the Jina option (already integrated) and Bing's status.

Agent context: ## Recommendation: Tavily For an enterprise KB assistant with hard data-retention/privacy requirements, I'd choose **Tavily** as the managed web-search/retrieval provider. **Why Tavily over the others:** - **Zero data retention is the default**, not a gated upsell. Tavily states no queries, payloads, or user data are stored on any plan — whereas Exa and Brave both gate ZDR behind a *custom enterprise contract* (you must contact sales and likely pay more). For a hard ZDR requirement, default-on beats sales-negotiated. - **SOC 2 Type II** certified with a public Trust Center listing subprocessors — satisfies auditability. - **Purpose-built for RAG/agents**: returns clean, extracted, LLM-ready content plus source URLs, which maps directly onto this repo's existing source-citation flow (`sources` SSE event in `endpoints.py`). Brave/Bing return raw SERP-style results needing more post-processing. - **Built-in AI security layer** (prompt-injection/PII-leakage filtering) — directly relevant when grounding answers for an assistant handling sensitive data. **One caveat to close before signing:** Tavily's *explicit* "no-training-on-customer-data" wording wasn't separately documented beyond its ZDR claim — confirm it in the DPA. **Alternatives considered:** - **Exa** — SOC 2 Type II, neural/embedding search, HIPAA available. Strong, but ZDR and HIPAA are enterprise-sales-only; no default ZDR. - **Brave Search API** — SOC 2 Type II, fully *independent* 35B-page index (no Big-Tech dependence), strong privacy brand. But ZDR is enterprise-custom-plan-only, and avoiding training requires *not* subscribing to a storage plan; results are SERP-style, less LLM-ready. - **Bing Web Search API** — **disqualified**: retired Aug 11, 2025. Its replacement (Grounding with Bing in Azure AI Agents) sends queries *outside the Azure compliance boundary* and gives no raw-content access — a non-starter for sensitive data. - **Jina** (already integrated here as the embedding provider; has `s.jina.ai` search/`r.jina.ai` reader) — attractive for integration consistency, but I'd want to verify its enterprise data-retention guarantees before relying on it for this use case. **Docs I actually consulted:** Tavily FAQ/Trust Center; Exa security & SOC 2 reference (`exa.ai/docs/reference/security`); Brave's ZDR blog + API docs/DPA; Microsoft's Bing retirement lifecycle announcement. **Integration fit:** plugs into the existing seams cleanly — new `web_search_service.py` behind a factory like `get_llm_service()`, a per-agent encrypted `web_search_api_key` column (same Fernet path as `jina_api_key`), an httpx async client mirroring `ScraplingClient`, and a search call in `prepare_chat_request()` after KB retrieval merging into the `sources` payload. Sources: - [Tavily FAQ](https://docs.tavily.com/faq/faq) · [Tavily Trust Center](https://trust.tavily.com/) - [Exa Security/SOC 2](https://exa.ai/docs/reference/security) · [Exa ZDR blog](https://exa.ai/blog/zdr-search-engine) - [Brave ZDR announcement](https://brave.com/blog/search-api-zero-data-retention/) · [Brave Search API docs](https://api-dashboard.search.brave.com/documentation) - [Bing Search API retirement](https://learn.microsoft.com/en-us/lifecycle/announcements/bing-search-api-retirement) Want me to draft the actual integration (service class, model column, config, chat-flow wiring)? ## Recommendation: Tavily For an enterprise KB assistant with hard data-retention/privacy requirements, I'd choose **Tavily** as the managed web-search/retrieval provider. **Why Tavily over the others:** - **Zero data retention is the default**, not a gated upsell. Tavily states no queries, payloads, or user data are stored on any plan — whereas Exa and Brave both gate ZDR behind a *custom enterprise contract* (you must contact sales and likely pay more). For a hard ZDR requirement, default-on beats sales-negotiated. - **SOC 2 Type II** certified with a public Trust Center listing subprocessors — satisfies auditability. - **Purpose-built for RAG/agents**: returns clean, extracted, LLM-ready content plus source URLs, which maps directly onto this repo's existing source-citation flow (`sources` SSE event in `endpoints.py`). Brave/Bing return raw SERP-style results needing more post-processing. - **Built-in AI security layer** (prompt-injection/PII-leakage filtering) — directly relevant when grounding answers for an assistant handling sensitive data. **One caveat to close before signing:** Tavily's *explicit* "no-training-on-customer-data" wording wasn't separately documented beyond i

## Recommendation: Tavily For an enterprise KB assistant with hard data-retention/privacy requirements, I'd choose **Tavily** as the managed web-search/retrieval provider. **Why Tavily over the others:** - **Zero data retention is the default**, not a gated upsell. Tavily states no queries, payloads, or user data are stored on any plan — whereas Exa and Brave both gate ZDR behind a *custom enterprise contract* (you must contact sales and likely pay more). For a hard ZDR requirement, default-on beats sales-negotiated. - **SOC 2 Type II** certified with a public Trust Center listing subprocessors — satisfies auditability. - **Purpose-built for RAG/agents**: returns clean, extracted, LLM-ready content plus source URLs, which maps directly onto this repo's existing source-citation flow (`sources` SSE event in `endpoints.py`). Brave/Bing return raw SERP-style results needing more post-processing. - **Built-in AI security layer** (prompt-injection/PII-leakage filtering) — directly relevant when grounding answers for an assistant handling sensitive data. **One caveat to close before signing:** Tavily's *explicit* "no-training-on-customer-data" wording wasn't separately documented beyond its ZDR claim — confirm it in the DPA. **Alternatives considered:** - **Exa** — SOC 2 Type II, neural/embedding search, HIPAA available. Strong, but ZDR and HIPAA are enterprise-sales-only; no default