Run

P1

26bc632a0bed22f035740edd34ee3e1fc2420e9f

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/README.md /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/__init__.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/api_client.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/auth_utils.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f51 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: total 56 drwxr-xr-x 5 user user 4096 Jun 6 04:48 . drwxr-xr-x 4 user user 4096 Jun 6 04:48 .. -rw-r--r-- 1 user user 61 Jun 6 04:48 .dockerignore -rw-r--r-- 1 user user 170 Jun 6 04:48 .git -rw-r--r-- 1 user user 118 Jun 6 04:48 .gitignore -rw-r--r-- 1 user user 282 Jun 6 04:48 Dockerfile.client -rw-r--r-- 1 user user 248 Jun 6 04:48 Dockerfile.server -rw-r--r-- 1 user user 2863 Jun 6 04:48 README.md drwxr-xr-x 2 use Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 # AskDocs AI: AI-Powered PDF Q&A Bot 2 3 **AskDocs AI** is an AI-powered chatbot that leverages **Hybrid RAG (Retrieval-Augmented Generation)** to answer your questions based on the content of uploaded PDFs. It combines semantic vector search with traditional keyword-based search for superior accuracy. 4 5 <img src="./images/landing_page.png" alt="Landing Page"> 6 <img src="./images/chat_interface.png" alt="Chat In Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/server/__init__.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/server/config.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/server/db/database.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/server/db/models.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-d Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/__init__.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/api_client.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/auth_utils.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/config.py /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-d Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import os 2 import shutil 3 import logging 4 from typing import List 5 from contextlib import asynccontextmanager 6 from fastapi import FastAPI, UploadFile, File, HTTPException, Depends 7 from fastapi.middleware.cors import CORSMiddleware 8 from fastapi.concurrency import run_in_threadpool 9 from pydantic import BaseModel 10 from config import settings 11 from modules.vectorstore import process_pdfs, get_vectorstor Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 from langchain_groq import ChatGroq 2 from langchain_classic.chains import RetrievalQA 3 from langchain_classic.retrievers import EnsembleRetriever 4 from config import settings 5 from modules.vectorstore import get_bm25_retriever 6 import logging 7 8 logger = logging.getLogger(__name__) 9 10 def get_llm_chain(vectorstore): 11 if vectorstore is None: 12 return None 13 14 if not settings.GROQ_API_KEY: 15 raise Value Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import os 2 import hashlib 3 import logging 4 from concurrent.futures import ThreadPoolExecutor 5 from langchain_chroma import Chroma 6 from langchain_community.document_loaders import PyPDFLoader 7 from langchain_community.retrievers import BM25Retriever 8 from langchain_core.documents import Document 9 from langchain_text_splitters import RecursiveCharacterTextSplitter 10 from langchain_huggingface import Hugging Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import os 2 from pathlib import Path 3 from pydantic_settings import BaseSettings, SettingsConfigDict 4 from dotenv import load_dotenv, find_dotenv 5 6 load_dotenv(find_dotenv(), override=True) 7 8 BASE_DIR = Path(__file__).resolve().parent.parent 9 10 class Settings(BaseSettings): 11 PROJECT_NAME: str = "AskDocs AI API" 12 13 # Storage 14 CHROMA_DB_PATH: str = os.getenv("CHROMA_DB_PATH", "chroma_db") 15 TEMP_UPLOA Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Provider mentions: openai

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import streamlit as st 2 from api_client import upload_pdfs, query_ai, get_health 3 from config import config 4 from auth_utils import auth_sidebar 5 6 st.set_page_config( 7 page_title=config.APP_TITLE, 8 page_icon="🤖", 9 layout="wide" 10 ) 11 12 st.markdown(""" 13 <style> 14 .stApp { 15 background-color: #0e1117; 16 color: #ffffff; 17 } 18 .main-header { 19 padding-top: 2rem; 20 padding-bottom: 2rem; 21 text-alig Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import requests 2 import logging 3 from config import config 4 5 logger = logging.getLogger(__name__) 6 7 def _handle_response(response): 8 """Internal helper to handle responses safely.""" 9 try: 10 response.raise_for_status() 11 return response.json() 12 except requests.exceptions.JSONDecodeError: 13 logger.error(f"Failed to decode JSON from response: {response.text[:100]}") 14 return {"error": f"Server returned Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import uuid 2 from typing import Optional 3 from fastapi import Depends, Request 4 from fastapi_users import BaseUserManager, FastAPIUsers, UUIDIDMixin 5 from fastapi_users.authentication import ( 6 AuthenticationBackend, 7 BearerTransport, 8 JWTStrategy, 9 ) 10 from fastapi_users.db import SQLAlchemyUserDatabase 11 from sqlalchemy.ext.asyncio import AsyncSession 12 13 from config import settings 14 from db.databas Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 from fastapi_users.db import SQLAlchemyBaseUserTableUUID 2 from sqlalchemy.orm import DeclarativeBase 3 4 class Base(DeclarativeBase): 5 pass 6 7 class User(SQLAlchemyBaseUserTableUUID, Base): 8 pass 9 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 --extra-index-url https://download.pytorch.org/whl/cpu 2 torch 3 langchain-chroma 4 langchain-community 5 langchain-text-splitters 6 langchain-huggingface 7 langchain-groq 8 langchain-classic 9 fastapi 10 uvicorn[standard] 11 python-multipart 12 python-dotenv 13 sentence-transformers 14 pypdf 15 pydantic-settings 16 rank_bm25 17 sqlalchemy[asyncio] 18 asyncpg 19 fastapi-users[sqlalchemy] 20 bcrypt Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 streamlit 2 requests 3 python-dotenv Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import os 2 3 class Config: 4 API_BASE_URL: str = os.getenv("API_BASE_URL", "http://localhost:8000") 5 APP_TITLE: str = "AskDocs AI" 6 GATHER_USAGE_STATS: bool = False 7 8 config = Config() 9 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import requests 2 import streamlit as st 3 from config import config 4 5 def login(email, password): 6 """Log in a user and store the token.""" 7 try: 8 response = requests.post( 9 f"{config.API_BASE_URL}/auth/jwt/login", 10 data={"username": email, "password": password} 11 ) 12 if response.status_code == 200: 13 token = response.json().get("access_token") 14 st.session_state.token = token 15 st.session_state.logge Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 from typing import AsyncGenerator 2 from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine 3 from config import settings 4 5 engine = create_async_engine(settings.DATABASE_URL) 6 async_session_maker = async_sessionmaker(engine, expire_on_commit=False) 7 8 async def get_async_session() -> AsyncGenerator[AsyncSession, None]: 9 async with async_session_maker() as session: 10 yield ses Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import uuid 2 from fastapi_users import schemas 3 4 class UserRead(schemas.BaseUser[uuid.UUID]): 5 pass 6 7 class UserCreate(schemas.BaseUserCreate): 8 pass 9 10 class UserUpdate(schemas.BaseUserUpdate): 11 pass 12 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 import sys 2 import os 3 import logging 4 5 # Add server directory to path 6 sys.path.append(os.path.abspath("server")) 7 8 from modules.vectorstore import get_vectorstore, get_bm25_retriever 9 from modules.llm import get_llm_chain 10 11 logging.basicConfig(level=logging.INFO) 12 13 def test_hybrid(): 14 print("[INIT] Initializing test for Hybrid Search...") 15 vectorstore = get_vectorstore() 16 print("[OK] Vectors Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 services: 2 # --- Backend Server --- 3 server: 4 container_name: askdocs-server 5 image: askdocs-server-image 6 build: 7 context: . 8 dockerfile: Dockerfile.server 9 ports: 10 - "8000:8000" 11 env_file: 12 - .env 13 volumes: 14 - ./chroma_db:/app/chroma_db 15 - ./temp_uploads:/app/temp_uploads 16 depends_on: 17 - db 18 19 # --- Database --- 20 db: 21 image: postgres:15-alpine 22 container_name: askdocs-db 23 restar Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 FROM python:3.11-slim 2 3 WORKDIR /app 4 5 COPY requirements.server.txt /app/requirements.txt 6 7 RUN pip install --no-cache-dir -r requirements.txt 8 9 COPY server/ ./server 10 11 EXPOSE 8000 12 13 CMD ["python", "server/main.py", "--host", "0.0.0.0", "--port", "8000"] Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 26bc632 feat: Implemented auth ecc3a1b feat: Implemented hybrid search (BM25 + Chroma) dad061b Implemented concurrency in doc parsing 40436e6 Dockerized application and split dependencies d1fd604 Restructured the project into client and server d0f96fa Updated llm model 6b79633 Merge branch 'main' of https://github.com/Balaji-R-05/askdocs-ai 7cde7bc Resolved module conflicts 9ac6c6b Delete chroma_store directory 0a555 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/api_client.py: response = requests.post(f"{config.API_BASE_URL}/upload", files=files, headers=headers, timeout=300) /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/api_client.py: response = requests.post(f"{config.API_BASE_URL}/query", json=payload, headers=headers, timeout=120) /home/user/worktrees/attempt-1 Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: No .env file found Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 .env 2 venv 3 .venv 4 chroma_store 5 chroma_db 6 *.pyc 7 __pycache__/ 8 .DS_Store 9 temp_uploads 10 pip_list.txt 11 12 docs 13 14 FUTURE_ROADMAP.md Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: 1 FROM python:3.11-slim 2 3 WORKDIR /app 4 5 COPY requirements.client.txt /app/requirements.txt 6 7 RUN pip install --no-cache-dir -r requirements.txt 8 9 COPY client/ ./client 10 11 EXPOSE 8501 12 13 CMD ["streamlit", "run", "client/main.py", "--server.port=8501", "--server.address=0.0.0.0"] Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers. Tool result summary: /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/api_client.py:import requests /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/client/auth_utils.py:import requests /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/server/modules/llm.py:from langchain_groq import ChatGroq /home/user/worktrees/attempt-1848a91b-6965-4ead-9356-dc543f5120e4-cap-1/ Agent context: I'll research the repository structure first to understand the architecture, then evaluate web-search providers.

Agent context: The repo is a LangChain RAG system (Groq LLM, Chroma + BM25 hybrid retrieval). Now I'll research web-search providers by consulting their actual documentation, prioritizing factual accuracy and citations. The repo is a LangChain RAG system (Groq LLM, Chroma + BM25 hybrid retrieval). Now I'll research web-search providers by consulting their actual documentation, prioritizing factual accuracy and citations.

Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel. Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel.

Query/prompt: Extract details about Tavily's search API for LLM/RAG grounding: what it returns (content, sources, URLs, citations), source quality controls, how answers are grounded/cited, accuracy features, and any mention of LangChain integration. List concrete fields in the response. Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel. Fetch prompt: Extract details about Tavily's search API for LLM/RAG grounding: what it returns (content, sources, URLs, citations), source quality controls, how answers are grounded/cited, accuracy features, and any mention of LangChain integration. List concrete fields in the response. Tool result summary: # Tavily Search API for LLM/RAG Grounding ## Response Content Structure The API returns: - **Query**: The executed search query - **Results array** with per-source fields: - `title`: Search result heading - `url`: Source link - `content`: "A short description of the search result" - `score`: Relevance ranking (float value) - `raw_content`: Cleaned HTML when `include_raw_content` enabled - `favicon`: Source domain ico Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel.

Query/prompt: Extract details about Exa's /answer and search API for LLM/RAG grounding: what it returns (content, highlights, citations, sources), how it ensures factual accuracy and source quality, neural vs keyword search, and citation support. List concrete response fields. Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel. Fetch prompt: Extract details about Exa's /answer and search API for LLM/RAG grounding: what it returns (content, highlights, citations, sources), how it ensures factual accuracy and source quality, neural vs keyword search, and citation support. List concrete response fields. Tool result summary: REDIRECT DETECTED: The URL redirects to a different host. Original URL: https://docs.exa.ai/reference/answer Redirect URL: https://exa.ai/docs/reference/answer Status: 307 Temporary Redirect To complete your request, I need to fetch content from the redirected URL. Please use WebFetch again with these parameters: - url: "https://exa.ai/docs/reference/answer" - prompt: "Extract details about Exa's /answer and search A Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel.

Query/prompt: Extract details about Perplexity Sonar API for grounded answers: citation support, source quality, factual accuracy, what the response returns (citations, search_results), and models available. List concrete response fields related to citations and sources. Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel. Fetch prompt: Extract details about Perplexity Sonar API for grounded answers: citation support, source quality, factual accuracy, what the response returns (citations, search_results), and models available. List concrete response fields related to citations and sources. Tool result summary: # Perplexity Sonar API: Grounded Answers Overview ## Citation & Source Support The API includes robust citation capabilities through dedicated response fields: - **citations**: "URLs of sources used to generate the response" - **search_results**: Array of `ApiPublicSearchResult` objects containing title, URL, publication date, snippets, and source type - **return_images**: Boolean flag to include image results with o Agent context: Now I'll consult the actual documentation of the top contenders for agent grounding, in parallel.

Agent context: Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers. Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers.

Query/prompt: Extract details about Exa's /answer and search API for LLM/RAG grounding: what it returns (content, highlights, citations, sources), how it ensures factual accuracy and source quality, neural vs keyword search, and citation support. List concrete response fields. Agent context: Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers. Fetch prompt: Extract details about Exa's /answer and search API for LLM/RAG grounding: what it returns (content, highlights, citations, sources), how it ensures factual accuracy and source quality, neural vs keyword search, and citation support. List concrete response fields. Tool result summary: # Exa /answer API for LLM/RAG Grounding ## Response Content & Structure The `/answer` endpoint returns: - **Answer**: "A direct answer for specific queries" or "a detailed summary with citations for open-ended queries" - **Citations array**: Each citation includes title, URL, published date, author, document ID, associated image, favicon, and optionally full page text - **Cost tracking**: Dollar estimates for search Agent context: Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers.

Query/prompt: web search API for LLM agents accuracy benchmark comparison Tavily Exa Perplexity Linkup 2026 factual grounding citations Agent context: Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers. Query: web search API for LLM agents accuracy benchmark comparison Tavily Exa Perplexity Linkup 2026 factual grounding citations 10 search result links captured Tool result summary: I've completed the web search for your query about web search API accuracy benchmarks for LLM agents. Here are the key findings: ## Benchmark Results & Rankings Perplexity scored highest among Tavily, Perplexity API, Exa, and Google Gemini Search Grounding, with Exa as a close second, evaluated using LLM-as-judge scoring in LangSmith against 8 factual queries. However, a broader evaluation tested 8 search APIs (Brave Agent context: Let me fetch the Exa redirect and research independent accuracy benchmarks comparing these providers.

Agent context: Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists. Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists.

Query/prompt: Summarize the benchmark methodology and results comparing Exa, Tavily, Perplexity, and Linkup on factual accuracy, hallucination rate, faithfulness, source diversity, and entity coverage. Give concrete numbers and which provider won on each metric. Note any methodology caveats or self-selection bias. Agent context: Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists. Fetch prompt: Summarize the benchmark methodology and results comparing Exa, Tavily, Perplexity, and Linkup on factual accuracy, hallucination rate, faithfulness, source diversity, and entity coverage. Give concrete numbers and which provider won on each metric. Note any methodology caveats or self-selection bias. Tool result summary: # AI Search API Benchmark: Results Summary ## Methodology The evaluation tested four providers—Exa, Tavily, Perplexity, and Linkup—against 600 queries combining 150 real anonymized user queries and 450 synthetically generated stress-test scenarios. All systems used standard API tiers with identical prompts. Assessments employed a "blind LLM-as-a-judge framework using GPT-4o-mini," focusing on three dimensions: source Agent context: Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists.

Query/prompt: langchain-tavily langchain-exa official LangChain retriever integration documentation 2026 Agent context: Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists. Query: langchain-tavily langchain-exa official LangChain retriever integration documentation 2026 5 search result links captured Tool result summary: I found documentation for the LangChain integrations. Here's what I discovered: ## Tavily Integration The langchain-tavily Python package is the official LangChain integration of Tavily, including Search, Extract, Map, Crawl, Research functionality. The older langchain_community.tools.tavily_search.tool is deprecated, with the recommendation to migrate to the new langchain-tavily Python package. Key tools available i Agent context: Let me read the Linkup benchmark (focused on hallucination/faithfulness) and verify LangChain integration support for the finalists.

Agent context: ## Recommendation: Tavily For this repo — a LangChain RAG (`server/modules/llm.py` builds an `EnsembleRetriever` over Chroma + BM25, synthesized by Groq via `RetrievalQA`) where hallucination is unacceptable — **Tavily** is the best fit. **Why Tavily wins on your stated priorities (accuracy → source quality → citations):** 1. **Verifiable citations by construction.** It returns raw, ranked source passages — `title`, `url`, `content`, and a relevance `score` per result — not a pre-synthesized answer. Every web-derived claim stays traceable to a URL, and your *own* Groq prompt does the grounding. That keeps a single, auditable hallucination surface instead of stacking a vendor's answer-LLM under yours. 2. **Source-quality controls.** Domain allow/deny lists (up to 300/150), recency and date-range filters, topic scoping (`general`/`news`/`finance`), and an `advanced` search depth tuned for factual grounding. 3. **Lowest integration risk.** The official `langchain-tavily` package drops a retriever straight into your existing ensemble in `llm.py` — matching patterns already in the repo. Add `TAVILY_API_KEY` to `config.py`, a third retriever + weight, and you're done. **Main alternatives considered:** - **Exa** (strong runner-up) — neural/semantic search with the richest citation metadata (published date, author, full text) and official LangChain retriever; arguably best at *finding* authoritative sources. Edged out because its strongest accuracy story is the `/answer` endpoint, which reintroduces the vendor-synthesis layer you'd rather control, and its neural results often need extraction before use. - **Perplexity Sonar** — returns `citations` + `search_results` and topped one (tiny, 8-query) LangSmith run, but it's fundamentally an *answer engine*: it synthesizes, adding a second hallucination surface beneath Groq. Better for a chat product than a controlled RAG. - **Linkup** — its own 600-query benchmark claims the lowest hallucination and 2–4× better source diversity/entity coverage. Genuinely compelling on faithfulness, but it's self-benchmarked, newer, and has a thinner ecosystem than Tavily/Exa. - Briefly weighed: Brave Search API, Parallel (claims best HLE), Firecrawl, SerpAPI — either raw-search (no grounding tuning) or less mature LLM-citation support. **Documentation I actually consulted:** - Tavily Search API reference + LangChain integration docs - Exa `/answer` + search API reference and LangChain reference - Perplexity Sonar chat-completions API reference - Linkup's AI-search evaluation benchmark - Independent comparison/benchmark write-ups (aimultiple, websearchapi.ai LangSmith eval) **Honest caveat:** No independent consensus crowns Tavily the single *most accurate* engine — Linkup and Perplexity post better raw faithfulness numbers in some benchmarks. Tavily wins here specifically because your architecture already owns synthesis: you want a retriever that hands back clean, scored, citable sources, not another answer engine. If you'd rather delegate answer synthesis with citations, Exa `/answer` or Perplexity Sonar become the better picks. Want me to implement the Tavily retriever into the ensemble? Sources: - [Tavily Search API reference](https://docs.tavily.com/documentation/api-reference/endpoint/search) - [Tavily LangChain integration](https://docs.tavily.com/documentation/integrations/langchain) - [Exa /answer API reference](https://exa.ai/docs/reference/answer) - [Exa LangChain reference](https://reference.langchain.com/python/integrations/langchain_exa/) - [Perplexity Sonar API reference](https://docs.perplexity.ai/api-reference/chat-completions-post) - [Linkup: Evaluating AI search systems on complex queries](https://www.linkup.so/blog/evaluating-ai-search-systems-on-complex-queries) - [Compare Tavily, Perplexity, Google, Exa with LLM-as-Judge in LangSmith](https://websearchapi.ai/blog/compare-tavily-google-search-exa-perplexity) - [Agentic Search in 2026: Benchmark 8 Search APIs](https://aimultiple.com/agentic-search) ## Recommendation: Tavily For this repo — a LangChain RAG (`server/modules/llm.py` builds an `EnsembleRetriever` over Chroma + BM25, synthesized by Groq via `RetrievalQA`) where hallucination is unacceptable — **Tavily** is the best fit. **Why Tavily wins on your stated priorities (accuracy → source quality → citations):** 1. **Verifiable citations by construction.** It returns raw, ranked source passages — `title`, `url`, `content`, and a relevance `score` per result — not a pre-synthesized answer. Every web-derived claim stays traceable to a URL, and your *own* Groq prompt does the grounding. That keeps a single, auditable hallucination surface instead of stacking a vendor's answer-LLM under yours. 2. **Source-quality controls.** Domain allow/deny lists (up to 300/150), recency and date-range filters, topic scoping (`general`/`news`/`finance`), and an `advanced` search depth tuned for factual grounding. 3. **Lowest integration risk.** The official `langchain-tavily` package drops a retriever straight into your existing ensemble in `llm.py` — matching patterns already in the repo. Add `TAVILY_API_KEY` to `config.py`, a third retriever + weight, and you're done. **Main alternatives

## Recommendation: Tavily For this repo — a LangChain RAG (`server/modules/llm.py` builds an `EnsembleRetriever` over Chroma + BM25, synthesized by Groq via `RetrievalQA`) where hallucination is unacceptable — **Tavily** is the best fit. **Why Tavily wins on your stated priorities (accuracy → source quality → citations):** 1. **Verifiable citations by construction.** It returns raw, ranked source passages — `title`, `url`, `content`, and a relevance `score` per result — not a pre-synthesized answer. Every web-derived claim stays traceable to a URL, and your *own* Groq prompt does the grounding. That keeps a single, auditable hallucination surface instead of stacking a vendor's answer-LLM under yours. 2. **Source-quality controls.** Domain allow/deny lists (up to 300/150), recency and date-range filters, topic scoping (`general`/`news`/`finance`), and an `advanced` search depth tuned for factual grounding. 3. **Lowest integration risk.** The official `langchain-tavily` package drops a retriever straight into your existing ensemble in `llm.py` — matching patterns already in the repo. Add `TAVILY_API_KEY` to `config.py`, a third retriever + weight, and you're done. **Main alternatives considered:** - **Exa** (strong runner-up) — neural/semantic search with the richest citation metadata (published date, author, full text) and official LangChain retriever; arguably best at *finding*