mirror of
https://github.com/wahyd4/hey-search.git
synced 2026-08-08 21:05:14 +10:00
Merge pull request #5 from wahyd4/feat/published-date
Add format=llm support which get rid of extra content and only focus on title, url, snippet
This commit is contained in:
+11
@@ -86,12 +86,23 @@
|
||||
|
||||
- Full REST API for search, autocomplete, engine management, bookmarks, history, stats, and settings
|
||||
- `/api/search` accepts both **GET and POST** requests with query string parameters
|
||||
- **`format=llm`** — minimal LLM-optimised response: only `query`, `results` (title, url, snippet, date), and `total_results`; no engine noise
|
||||
- **`max_results`** — hard limit on returned results (1–100); `numResults` is a supported alias
|
||||
- Compatibility parameters: `pageNumber` (alias for `page`), `numResults`, `format`, `imageProxy`, `safesearch`
|
||||
- Every result includes `result_id`, `rank`, `engine`, and `published_date`
|
||||
- `has_next` and `total_results` fields for cursor-aware pagination
|
||||
- `X-Response-Time-Ms` response header on all endpoints
|
||||
- OpenAPI specification with interactive docs via [Swagger UI](https://swagger.io/tools/swagger-ui/) (`/docs`) and [Redoc](https://github.com/Redocly/redoc) (`/redoc`)
|
||||
|
||||
## MCP Tool Server
|
||||
|
||||
- **`/api/mcp`** — [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) endpoint; exposes HeySearch as a native tool for LLMs
|
||||
- Compatible with Claude Desktop, Cursor, Continue, VS Code Copilot, and any MCP-capable client
|
||||
- **Transport**: Streamable HTTP (JSON-RPC 2.0 POST)
|
||||
- **Available tools**: `search` (web + image search) and `autocomplete`
|
||||
- `search` tool supports `query`, `category`, `num_results` (1–20), `engines`, `sort`, and `date_filter` arguments
|
||||
- Add to Claude Desktop by pointing `url` at `http://your-host/api/mcp` with `"transport": "http"`
|
||||
|
||||
## Reliability
|
||||
|
||||
- **Retry mechanism** — failed upstream requests are retried with exponential backoff (via [tenacity](https://github.com/jd/tenacity), 2 attempts)
|
||||
|
||||
@@ -20,7 +20,7 @@ Both are open-source, self-hosted, privacy-respecting metasearch engines. Here's
|
||||
|---|---|---|
|
||||
| **Setup** | `docker compose up -d` — one command, zero config | Requires YAML config, engine tuning, sometimes breaks |
|
||||
| **UI** | Modern, clean React UI with dark mode, background images, image lightbox | Functional but dated — not mobile friendly |
|
||||
| **AI agent friendly** | Clean JSON REST API, OpenAPI docs at `/docs`, designed to be queried programmatically | API exists but less documented; HTML-heavy responses |
|
||||
| **AI agent friendly** | MCP tool server at `/api/mcp` (Claude Desktop, Cursor, Continue), `format=llm` for minimal responses, clean JSON REST API, OpenAPI docs at `/docs` | API exists but less documented; HTML-heavy responses |
|
||||
| **Bookmarks** | Built-in bookmark manager for results | ❌ |
|
||||
| **Search history** | Full search history with timestamps, re-run any past query in one click | ❌ |
|
||||
| **Usage stats** | Built-in analytics dashboard — top queries, click-through rates, engine usage | ❌ |
|
||||
@@ -90,6 +90,7 @@ The `/app/data` volume stores the SQLite database (engine settings, excluded dom
|
||||
| GET | `/api/settings` | Get app settings (cache TTL)|
|
||||
| PUT | `/api/settings` | Update settings |
|
||||
| DELETE | `/api/cache` | Flush search cache |
|
||||
| POST | `/api/mcp` | MCP tool server (for LLMs) |
|
||||
|
||||
### Search endpoint parameters
|
||||
|
||||
@@ -99,37 +100,84 @@ The `/app/data` volume stores the SQLite database (engine settings, excluded dom
|
||||
| `category` | `web` | `web` or `images` |
|
||||
| `page` | `1` | Page number (1–50) |
|
||||
| `pageNumber` | — | Alias for `page` (takes precedence when provided) |
|
||||
| `numResults` | — | Requested result count hint (informational) |
|
||||
| `format` | — | Response format hint (e.g. `json`) |
|
||||
| `max_results`| — | Hard limit on results returned (1–100) |
|
||||
| `numResults` | — | Alias for `max_results` |
|
||||
| `format` | — | `llm` for minimal LLM-friendly response (see below) |
|
||||
| `imageProxy` | — | Client image-proxy preference flag (informational) |
|
||||
| `safesearch` | — | Safe search level: `0` off, `1` moderate, `2` strict |
|
||||
| `engines` | — | Comma-separated engine names to restrict (e.g. `google,bing`) |
|
||||
| `image_size` | — | `large`, `medium`, or `small` (images only) |
|
||||
| `sort` | `default`| `default`, `date_asc`, or `date_desc` |
|
||||
| `date_filter`| — | `day`, `week`, `month`, or `year` |
|
||||
|
||||
## Using with AI Agents / curl
|
||||
## Using with AI Agents / LLMs
|
||||
|
||||
The `/api/search` endpoint returns clean JSON — ideal for LLMs and AI agents to consume directly.
|
||||
HeySearch is designed to be used by LLMs and AI agents. There are two integration methods:
|
||||
|
||||
```bash
|
||||
# Web search
|
||||
curl "http://localhost:8000/api/search?q=python+async&format=json" | jq
|
||||
### 1. MCP Tool Server (recommended)
|
||||
|
||||
# Restrict to specific engines
|
||||
curl "http://localhost:8000/api/search?q=rust+programming&engines=brave,google" | jq
|
||||
[Model Context Protocol](https://modelcontextprotocol.io/) (MCP) is the standard for LLM tool use. Add HeySearch to any MCP-compatible client:
|
||||
|
||||
# Image search
|
||||
curl "http://localhost:8000/api/search?q=mountain+landscape&category=images&image_size=large" | jq
|
||||
|
||||
# Paginate results
|
||||
curl "http://localhost:8000/api/search?q=machine+learning&page=2" | jq
|
||||
|
||||
# Extract just titles and URLs from web results
|
||||
curl "http://localhost:8000/api/search?q=openai" | \
|
||||
jq '[.results[] | {title, url, snippet}]'
|
||||
**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"heysearch": {
|
||||
"url": "http://localhost:8000/api/mcp",
|
||||
"transport": "http"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Cursor / Continue / VS Code Copilot** — add `http://localhost:8000/api/mcp` as an MCP server URL in the tool settings.
|
||||
|
||||
**Manual test:**
|
||||
```bash
|
||||
# List available tools
|
||||
curl -X POST http://localhost:8000/api/mcp \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
|
||||
|
||||
# Call the search tool
|
||||
curl -X POST http://localhost:8000/api/mcp \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search","arguments":{"query":"python async","num_results":3}}}'
|
||||
```
|
||||
|
||||
**Available MCP tools:** `search`, `autocomplete`
|
||||
|
||||
### 2. REST API with `format=llm`
|
||||
|
||||
For direct API calls from LLM agents, use `format=llm` to get a minimal, token-efficient response:
|
||||
|
||||
```bash
|
||||
# LLM-optimised response — only title, url, snippet, date. No engine noise.
|
||||
curl "http://localhost:8000/api/search?q=python+async&format=llm&max_results=5" | jq
|
||||
```
|
||||
|
||||
Response shape:
|
||||
```json
|
||||
{
|
||||
"query": "python async",
|
||||
"category": "web",
|
||||
"results": [
|
||||
{ "title": "...", "url": "https://...", "snippet": "...", "date": "2024-01-15" }
|
||||
],
|
||||
"total_results": 5
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
# Restrict to specific engines
|
||||
curl "http://localhost:8000/api/search?q=rust+programming&engines=brave,google&format=llm" | jq
|
||||
|
||||
# Image search with size filter
|
||||
curl "http://localhost:8000/api/search?q=mountain+landscape&category=images&image_size=large" | jq
|
||||
|
||||
# Limit results (hard limit, not a hint)
|
||||
curl "http://localhost:8000/api/search?q=openai&max_results=3&format=llm" | jq
|
||||
```
|
||||
|
||||
> Interactive API docs (Swagger UI) are available at `http://localhost:8000/docs`.
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ from fastapi import APIRouter, Query, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from app.models import SearchResponse, EngineInfo, APIError
|
||||
from app.models import SearchResponse, EngineInfo, APIError, LLMWebResult, LLMImageResult, LLMSearchResponse
|
||||
from app.search import search, get_autocomplete
|
||||
from app.engines import registry
|
||||
from app.excluded import get_excluded_domains, add_excluded_domain, remove_excluded_domain
|
||||
@@ -35,6 +35,12 @@ Supports both GET and POST methods with query parameters.
|
||||
curl '$BASE_URL/api/search?q=hello+world&category=web&page=1'
|
||||
```
|
||||
|
||||
**LLM / AI-agent optimised response (`format=llm`):**
|
||||
```bash
|
||||
curl '$BASE_URL/api/search?q=hello+world&format=llm&max_results=5'
|
||||
```
|
||||
Returns a minimal JSON response with only `query`, `results` (title, url, snippet, date), and `total_results` — ideal for RAG pipelines and tool-calling.
|
||||
|
||||
**Example response (truncated):**
|
||||
```json
|
||||
{
|
||||
@@ -75,8 +81,9 @@ async def api_search(
|
||||
category: Literal["web", "images"] = Query("web", description="Search category"),
|
||||
page: int = Query(1, ge=1, le=50, description="Page number"),
|
||||
pageNumber: int | None = Query(None, ge=1, le=50, description="Alias for page (1-based page number)"),
|
||||
numResults: int | None = Query(None, ge=1, le=100, description="Number of results requested (informational)"),
|
||||
format: str | None = Query(None, description="Response format hint (e.g. 'json')"),
|
||||
numResults: int | None = Query(None, ge=1, le=100, description="Number of results to return (applies as hard limit)"),
|
||||
max_results: int | None = Query(None, ge=1, le=100, description="Maximum number of results to return"),
|
||||
format: str | None = Query(None, description="Response format: 'llm' for a minimal LLM-friendly response, omit for full JSON"),
|
||||
imageProxy: bool | None = Query(None, description="Whether the client wants image proxying"),
|
||||
safesearch: str | None = Query(None, description="Safe search level (0=off, 1=moderate, 2=strict)"),
|
||||
engines: str | None = Query(None, description="Comma-separated engine names to use (e.g. 'google,bing')"),
|
||||
@@ -86,7 +93,9 @@ async def api_search(
|
||||
):
|
||||
effective_page = pageNumber if pageNumber is not None else page
|
||||
engine_list = [e.strip() for e in engines.split(",")] if engines else None
|
||||
result = await search(q, category=category, page=effective_page, engines=engine_list, image_size=image_size, sort=sort, date_filter=date_filter)
|
||||
# max_results takes precedence; numResults is a supported alias
|
||||
effective_max = max_results if max_results is not None else numResults
|
||||
result = await search(q, category=category, page=effective_page, engines=engine_list, image_size=image_size, sort=sort, date_filter=date_filter, max_results=effective_max)
|
||||
origin_ip = request.client.host if request.client else ""
|
||||
user_agent = request.headers.get("user-agent", "")
|
||||
_stats.record_search(
|
||||
@@ -97,6 +106,33 @@ async def api_search(
|
||||
result_count=result.total_results,
|
||||
cached=result.cached,
|
||||
)
|
||||
|
||||
if format == "llm":
|
||||
llm_results: list[LLMWebResult | LLMImageResult] = []
|
||||
for r in result.results:
|
||||
if category == "images":
|
||||
llm_results.append(LLMImageResult(
|
||||
title=r.title,
|
||||
url=r.url,
|
||||
img_src=getattr(r, "img_src", ""),
|
||||
date=r.published_date or "",
|
||||
))
|
||||
else:
|
||||
llm_results.append(LLMWebResult(
|
||||
title=r.title,
|
||||
url=r.url,
|
||||
snippet=getattr(r, "content", ""),
|
||||
date=r.published_date or "",
|
||||
))
|
||||
# Return JSONResponse directly to bypass response_model=SearchResponse
|
||||
# coercion, which would otherwise strip LLM-only fields (snippet, date).
|
||||
return JSONResponse(content=LLMSearchResponse(
|
||||
query=result.query,
|
||||
category=result.category,
|
||||
results=llm_results,
|
||||
total_results=len(llm_results),
|
||||
).model_dump())
|
||||
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ from fastapi.responses import JSONResponse
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from app.api.routes import router
|
||||
from app.mcp_server import router as mcp_router
|
||||
from app.engines import registry
|
||||
from app.excluded import init_db
|
||||
from app.settings import init_settings_table, set_setting
|
||||
@@ -131,6 +132,7 @@ async def global_exception_handler(request: Request, exc: Exception):
|
||||
|
||||
|
||||
app.include_router(router, prefix="/api")
|
||||
app.include_router(mcp_router, prefix="/api")
|
||||
|
||||
# Alias /search → /api/search for compatibility with external clients
|
||||
@app.api_route("/search", methods=["GET", "POST"], include_in_schema=False)
|
||||
|
||||
@@ -0,0 +1,313 @@
|
||||
"""MCP (Model Context Protocol) server for HeySearch.
|
||||
|
||||
Exposes HeySearch as an MCP tool server so LLMs (Claude, Cursor, Continue,
|
||||
VS Code Copilot, etc.) can invoke search directly via the standard MCP
|
||||
JSON-RPC 2.0 protocol.
|
||||
|
||||
Transport: Streamable HTTP — clients POST JSON-RPC messages to /mcp.
|
||||
|
||||
Supported methods:
|
||||
initialize — capability negotiation
|
||||
tools/list — enumerate available tools
|
||||
tools/call — invoke a tool (search, autocomplete)
|
||||
ping — liveness check
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import JSONResponse, Response
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(tags=["MCP"])
|
||||
|
||||
MCP_PROTOCOL_VERSION = "2024-11-05"
|
||||
SERVER_INFO = {"name": "HeySearch", "version": "1.4.0"}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool definitions (JSON Schema)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_SEARCH_TOOL: dict[str, Any] = {
|
||||
"name": "search",
|
||||
"description": (
|
||||
"Search the web or images using the HeySearch privacy-respecting "
|
||||
"metasearch engine. Results are aggregated from Brave, DuckDuckGo, "
|
||||
"Google, and Bing and deduplicated. Returns titles, URLs, snippets, "
|
||||
"and publication dates."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": "The search query string.",
|
||||
},
|
||||
"category": {
|
||||
"type": "string",
|
||||
"enum": ["web", "images"],
|
||||
"default": "web",
|
||||
"description": "Search category: 'web' for text results, 'images' for image results.",
|
||||
},
|
||||
"num_results": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 20,
|
||||
"default": 5,
|
||||
"description": "Maximum number of results to return (1–20).",
|
||||
},
|
||||
"engines": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Comma-separated engine names to restrict the search "
|
||||
"(e.g. 'google,bing'). Omit to use all enabled engines."
|
||||
),
|
||||
},
|
||||
"sort": {
|
||||
"type": "string",
|
||||
"enum": ["default", "date_asc", "date_desc"],
|
||||
"default": "default",
|
||||
"description": "Sort order: default (relevance), date_asc, or date_desc.",
|
||||
},
|
||||
"date_filter": {
|
||||
"type": "string",
|
||||
"enum": ["", "day", "week", "month", "year"],
|
||||
"default": "",
|
||||
"description": "Filter results by recency: day (24 h), week, month, or year.",
|
||||
},
|
||||
},
|
||||
"required": ["query"],
|
||||
},
|
||||
}
|
||||
|
||||
_AUTOCOMPLETE_TOOL: dict[str, Any] = {
|
||||
"name": "autocomplete",
|
||||
"description": (
|
||||
"Get search query autocomplete suggestions from HeySearch. "
|
||||
"Useful for expanding or refining a partial query."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": "Partial search query to get suggestions for.",
|
||||
},
|
||||
},
|
||||
"required": ["query"],
|
||||
},
|
||||
}
|
||||
|
||||
_ALL_TOOLS = [_SEARCH_TOOL, _AUTOCOMPLETE_TOOL]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _ok(req_id: Any, result: Any) -> dict:
|
||||
return {"jsonrpc": "2.0", "id": req_id, "result": result}
|
||||
|
||||
|
||||
def _err(req_id: Any, code: int, message: str) -> dict:
|
||||
return {"jsonrpc": "2.0", "id": req_id, "error": {"code": code, "message": message}}
|
||||
|
||||
|
||||
def _tool_result(text: str, is_error: bool = False) -> dict:
|
||||
return {"content": [{"type": "text", "text": text}], "isError": is_error}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool handlers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _handle_search(arguments: dict) -> str:
|
||||
from app.search import search # local import to avoid circular deps
|
||||
|
||||
query: str = arguments.get("query", "").strip()
|
||||
if not query:
|
||||
return "Error: 'query' argument is required."
|
||||
|
||||
category: str = arguments.get("category", "web")
|
||||
num_results: int = min(int(arguments.get("num_results", 5)), 20)
|
||||
engines_str: str | None = arguments.get("engines")
|
||||
sort: str = arguments.get("sort", "default")
|
||||
date_filter: str = arguments.get("date_filter", "")
|
||||
|
||||
engine_list = [e.strip() for e in engines_str.split(",")] if engines_str else None
|
||||
|
||||
result = await search(
|
||||
query,
|
||||
category=category,
|
||||
page=1,
|
||||
engines=engine_list,
|
||||
sort=sort,
|
||||
date_filter=date_filter,
|
||||
max_results=num_results,
|
||||
)
|
||||
|
||||
if not result.results:
|
||||
return f"No results found for: {query}"
|
||||
|
||||
lines: list[str] = [f"Search results for: {query}\n"]
|
||||
for i, r in enumerate(result.results, 1):
|
||||
lines.append(f"{i}. {r.title}")
|
||||
lines.append(f" URL: {r.url}")
|
||||
snippet = getattr(r, "content", "") or getattr(r, "img_src", "")
|
||||
if snippet:
|
||||
lines.append(f" {snippet}")
|
||||
if r.published_date:
|
||||
lines.append(f" Date: {r.published_date}")
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def _handle_autocomplete(arguments: dict) -> str:
|
||||
from app.search import get_autocomplete # local import
|
||||
|
||||
query: str = arguments.get("query", "").strip()
|
||||
if not query:
|
||||
return "Error: 'query' argument is required."
|
||||
|
||||
suggestions = await get_autocomplete(query)
|
||||
if not suggestions:
|
||||
return f"No suggestions found for: {query}"
|
||||
|
||||
return "Suggestions:\n" + "\n".join(f"- {s}" for s in suggestions)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request dispatcher
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _dispatch(req: dict) -> dict | None:
|
||||
"""Handle one JSON-RPC request object. Returns None for notifications."""
|
||||
method: str = req.get("method", "")
|
||||
req_id = req.get("id")
|
||||
params: dict = req.get("params") or {}
|
||||
|
||||
# Notifications (no id) — acknowledge silently
|
||||
if req_id is None:
|
||||
return None
|
||||
|
||||
if method == "initialize":
|
||||
return _ok(req_id, {
|
||||
"protocolVersion": MCP_PROTOCOL_VERSION,
|
||||
"capabilities": {"tools": {}},
|
||||
"serverInfo": SERVER_INFO,
|
||||
})
|
||||
|
||||
if method == "ping":
|
||||
return _ok(req_id, {})
|
||||
|
||||
if method == "tools/list":
|
||||
cursor = params.get("cursor") # pagination cursor (unused — all tools fit in one page)
|
||||
return _ok(req_id, {"tools": _ALL_TOOLS})
|
||||
|
||||
if method == "tools/call":
|
||||
tool_name: str = params.get("name", "")
|
||||
arguments: dict = params.get("arguments") or {}
|
||||
|
||||
if tool_name == "search":
|
||||
try:
|
||||
text = await _handle_search(arguments)
|
||||
return _ok(req_id, _tool_result(text))
|
||||
except Exception as exc:
|
||||
logger.exception("MCP search tool error")
|
||||
return _ok(req_id, _tool_result(f"Search failed: {exc}", is_error=True))
|
||||
|
||||
if tool_name == "autocomplete":
|
||||
try:
|
||||
text = await _handle_autocomplete(arguments)
|
||||
return _ok(req_id, _tool_result(text))
|
||||
except Exception as exc:
|
||||
logger.exception("MCP autocomplete tool error")
|
||||
return _ok(req_id, _tool_result(f"Autocomplete failed: {exc}", is_error=True))
|
||||
|
||||
return _err(req_id, -32601, f"Unknown tool: {tool_name}")
|
||||
|
||||
return _err(req_id, -32601, f"Method not found: {method}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# FastAPI endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/mcp",
|
||||
summary="MCP tool server",
|
||||
description="""[Model Context Protocol](https://modelcontextprotocol.io/) (MCP) endpoint.
|
||||
|
||||
Exposes HeySearch as an MCP tool server. LLMs and AI coding assistants
|
||||
(Claude Desktop, Cursor, Continue, VS Code Copilot, etc.) can add this
|
||||
server to their MCP configuration to invoke search directly.
|
||||
|
||||
**Transport:** Streamable HTTP — POST JSON-RPC 2.0 messages to this endpoint.
|
||||
|
||||
**Available tools:** `search`, `autocomplete`
|
||||
|
||||
**Quick config example (Claude Desktop / `claude_desktop_config.json`):**
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"heysearch": {
|
||||
"url": "$BASE_URL/api/mcp",
|
||||
"transport": "http"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Manual test:**
|
||||
```bash
|
||||
# List available tools
|
||||
curl -X POST $BASE_URL/api/mcp \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
|
||||
|
||||
# Invoke the search tool
|
||||
curl -X POST $BASE_URL/api/mcp \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search","arguments":{"query":"python async","num_results":3}}}'
|
||||
```
|
||||
""",
|
||||
include_in_schema=True,
|
||||
)
|
||||
async def mcp_endpoint(request: Request):
|
||||
"""Handle MCP JSON-RPC 2.0 requests (single or batch)."""
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content=_err(None, -32700, "Parse error: request body must be valid JSON"),
|
||||
)
|
||||
|
||||
# Batch request
|
||||
if isinstance(body, list):
|
||||
responses = [await _dispatch(req) for req in body if isinstance(req, dict)]
|
||||
responses = [r for r in responses if r is not None]
|
||||
if not responses:
|
||||
return Response(status_code=204)
|
||||
return JSONResponse(content=responses)
|
||||
|
||||
# Single request
|
||||
if not isinstance(body, dict):
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content=_err(None, -32600, "Invalid request: expected a JSON object or array"),
|
||||
)
|
||||
|
||||
result = await _dispatch(body)
|
||||
if result is None:
|
||||
# Notification — no response body
|
||||
return Response(status_code=204)
|
||||
return JSONResponse(content=result)
|
||||
@@ -84,3 +84,33 @@ class APIError(BaseModel):
|
||||
message: str
|
||||
details: str = ""
|
||||
retry_hint: str = ""
|
||||
|
||||
|
||||
# --- LLM-optimised response models ---
|
||||
|
||||
class LLMWebResult(BaseModel):
|
||||
"""A single web result stripped to the fields LLMs need."""
|
||||
title: str
|
||||
url: str
|
||||
snippet: str = ""
|
||||
date: str = ""
|
||||
|
||||
|
||||
class LLMImageResult(BaseModel):
|
||||
"""A single image result stripped to the fields LLMs need."""
|
||||
title: str
|
||||
url: str
|
||||
img_src: str
|
||||
date: str = ""
|
||||
|
||||
|
||||
class LLMSearchResponse(BaseModel):
|
||||
"""Minimal search response for LLM / AI-agent consumption.
|
||||
|
||||
Contains only the fields needed for RAG and tool-calling workflows.
|
||||
Omits engine metadata, error details, and other browser-UI noise.
|
||||
"""
|
||||
query: str
|
||||
category: str = "web"
|
||||
results: list[LLMWebResult | LLMImageResult] = Field(default_factory=list)
|
||||
total_results: int = 0
|
||||
|
||||
@@ -112,6 +112,7 @@ async def search(
|
||||
image_size: str = "",
|
||||
sort: SortOrder = "default",
|
||||
date_filter: DateFilter = "",
|
||||
max_results: int | None = None,
|
||||
) -> SearchResponse:
|
||||
"""Search across all enabled engines concurrently, with optional Redis caching."""
|
||||
engines_key = ",".join(sorted(engines)) if engines else ""
|
||||
@@ -124,6 +125,8 @@ async def search(
|
||||
resp = SearchResponse(**cached_data)
|
||||
resp.cached = True
|
||||
_apply_sort(resp.results, sort)
|
||||
if max_results is not None:
|
||||
resp.results = resp.results[:max_results]
|
||||
resp.total_results = len(resp.results)
|
||||
return resp
|
||||
|
||||
@@ -202,6 +205,8 @@ async def search(
|
||||
# Apply date filter, then sort
|
||||
_apply_date_filter(response.results, date_filter)
|
||||
_apply_sort(response.results, sort)
|
||||
if max_results is not None:
|
||||
response.results = response.results[:max_results]
|
||||
response.total_results = len(response.results)
|
||||
|
||||
return response
|
||||
|
||||
Reference in New Issue
Block a user