Author SHA1 Message Date
junv e63ac6ee59 Load REDIS_URL and save it to db when it exists 2026-03-07 12:19:00 +11:00
junv 9c9c52bf8b Add docker-compose 2026-03-07 12:15:53 +11:00
junv 102f89a4bb Update ui for stats page 2026-03-07 11:49:54 +11:00
junv 91499c5133 Update readme about how to use curl to search 2026-03-07 11:47:20 +11:00
junv fe3b27ff86 Update menu style 2026-03-07 11:41:33 +11:00
junv d7a3c8549d Update readme 2026-03-07 11:28:26 +11:00
junvandGitHub 49e35a6b53 Merge pull request #3 from wahyd4/feat/update-guide
Showing app version at the bottom
2026-03-07 11:25:13 +11:00
7 changed files with 243 additions and 148 deletions
+77 -45
View File
@@ -3,13 +3,15 @@
## Search
- **Web search** — aggregates results from Brave, DuckDuckGo, Google, and Bing concurrently
- **Image search** — Pinterest-style masonry layout respecting natural image aspect ratios, with lightbox viewer and keyboard navigation (←/→ to browse, Escape to close)
- **Image size filter** — filter images by size (Large, Medium, Small) applied server-side across all engines
- **Autocomplete** — live search suggestions as you type (cascades Google → DuckDuckGo → Brave)
- **Image search** — masonry grid respecting natural aspect ratios, with lightbox viewer and keyboard navigation (←/→ to browse, Escape to close)
- **Image size filter** — filter images by size (Large, Medium, Small) applied server-side across all engines
- **Sort order** — sort results by engine rank (default), newest first, or oldest first
- **Autocomplete** — live search suggestions as you type (200ms debounce, ↑↓/Enter/Escape keyboard support)
- **Pagination** — navigate through result pages; URL reflects current state (`?q=...&page=2`)
- **URL deduplication** — duplicate results from multiple engines are merged automatically
- **Domain exclusion** — exclude specific websites from search results; settings persist in SQLite
- **Search stats** — per-engine result counts, status indicators (ok/error/timeout), collapsible sidebar
- **Domain exclusion** — exclude specific websites from all search results; settings persist in SQLite
- **Engine filter** — restrict a search to specific engines via the `engines` query parameter (e.g. `engines=google,bing`)
- **Per-engine stats** — result counts, status (ok / error / timeout), and error messages per engine shown in a collapsible bar
## Search Engines
@@ -18,80 +20,110 @@
- **DuckDuckGo** — HTML Lite scraping for web, API for images
- **Brave** — Brave Search API for web and images
## REST API
- Full REST API for search, autocomplete, and engine management
- `/api/search` accepts both **GET and POST** requests with query string parameters
- Compatibility parameters on `/api/search`: `pageNumber` (alias for `page`), `numResults`, `format`, `imageProxy`, `safesearch`
- OpenAPI specification with interactive docs via [Swagger UI](https://swagger.io/tools/swagger-ui/) (`/docs`) and [Redoc](https://github.com/Redocly/redoc) (`/redoc`)
## Engine Management
- **4 built-in engines**: Brave, DuckDuckGo, Google, Bing
- Enable or disable engines via the UI or API at runtime
- Enable or disable engines via the UI or API at runtime; settings persist in SQLite
- **Drag-and-drop reordering** — set engine priority order; affects result ordering and display badges
- Each engine supports both web and image search categories
- Google and Bing include automatic fallback mechanisms for resilience
## Reliability
## Search History
- **Retry mechanism** — failed upstream requests are retried with exponential backoff (via [tenacity](https://github.com/jd/tenacity))
- **Error reporting** — when an upstream engine fails or times out, the UI shows a toast notification identifying which engine had issues
- **Graceful degradation** — partial failures don't block results from other engines
- **Automatic recording** — every search query is saved with timestamp and category
- **History page** (`/history`) — paginated, date-grouped list of recent searches (Today, Yesterday, etc.)
- **Re-run any query** — click a history entry to instantly re-search it
- **Delete entries** — remove individual entries or clear all history at once
## Analytics Dashboard
- **Stats page** (`/stats`) — full analytics dashboard with configurable time range (7 / 30 / 90 days)
- **KPI cards** — total searches, total clicks, click-through rate, average clicked result position
- **Daily trend chart** — bar chart of search volume per day
- **Top queries** — ranked list of most-searched terms with counts
- **Clicks by engine** — bar chart showing which engines' results get clicked most
- **Click position distribution** — histogram of which result ranks users click on
- **Top clicked results** — ranked list of most-clicked URLs with title, engine, and count
- **Click tracking** — every result click is recorded server-side (fire-and-forget from the frontend)
## Caching
- **Redis cache** — identical searches served from Redis cache to reduce upstream load and latency
- **Redis cache** — identical searches served from Redis to reduce upstream load and latency
- **Configurable TTL** — cache duration adjustable from 0 (disabled) to 168 hours (1 week); default 6 hours
- **UI controls** — preset buttons + slider in Settings → Cache tab, with flush button
- **Cache status** — search response includes a `cached` flag; the stats bar indicates cache hits
- **UI controls** — TTL slider + preset buttons, Redis URL input, and flush button in Settings → Cache tab
- **Optional** — when `REDIS_URL` is not set or Redis is unreachable, caching is silently disabled
- **Deterministic keys** — cache key derived from query + category + page + image_size + engines
## Bookmarks
- **Bookmark any result** — click the bookmark icon on web or image search results to save them
- **Bookmarks page** — dedicated `/bookmarks` page to browse all saved items
- **Bookmark any result** — click the bookmark icon on web or image results to save them
- **Bookmarks page** (`/bookmarks`) — dedicated page to browse all saved items
- **Filter by type** — tabs to filter All / Web / Images
- **Masonry image grid** — saved images displayed in a masonry layout
- **Masonry image grid** — saved images displayed in a masonry layout with hover overlays
- **Remove bookmarks** — delete individual bookmarks from the bookmarks page
- **Persistent storage** — bookmarks stored in SQLite alongside other app data
## Background Images
- **Homepage backgrounds** — random background image displayed on the home page (sourced from Unsplash / Picsum)
- **Local caching** — images are downloaded and stored in `data/backgrounds/` to avoid re-fetching
- **Auto-refresh** — configurable refresh interval (11440 minutes, default 30) fetches new images automatically
- **Background gallery** (`/backgrounds`) — browse and manage all downloaded backgrounds; click to preview, view file size and source
- **Manual refresh** — refresh button in the gallery to fetch a new image immediately
- **Enable / disable** — toggle backgrounds on or off in Settings → Background tab
- **iOS color sampling** — samples edge pixels from the background to set the `theme-color` meta tag, preventing white/black bars in Safari
## Settings
- **Unified settings modal** — single ⚙ button opens a tabbed modal with four tabs:
1. **Engines** — enable/disable and drag-to-reorder search engines
2. **Excluded Sites** — add or remove domains from the search blocklist
3. **Cache** — view Redis status, set TTL, update Redis URL, flush cache
4. **Background** — enable/disable background images, set refresh interval
## REST API
- Full REST API for search, autocomplete, engine management, bookmarks, history, stats, and settings
- `/api/search` accepts both **GET and POST** requests with query string parameters
- Compatibility parameters: `pageNumber` (alias for `page`), `numResults`, `format`, `imageProxy`, `safesearch`
- Every result includes `result_id`, `rank`, `engine`, and `published_date`
- `has_next` and `total_results` fields for cursor-aware pagination
- `X-Response-Time-Ms` response header on all endpoints
- OpenAPI specification with interactive docs via [Swagger UI](https://swagger.io/tools/swagger-ui/) (`/docs`) and [Redoc](https://github.com/Redocly/redoc) (`/redoc`)
## Reliability
- **Retry mechanism** — failed upstream requests are retried with exponential backoff (via [tenacity](https://github.com/jd/tenacity), 2 attempts)
- **Error reporting** — when an upstream engine fails or times out, the UI shows a toast notification identifying which engine had issues
- **Graceful degradation** — partial failures don't block results from other engines
## UI / UX
- **Mobile-first** responsive design built with React, Tailwind CSS, and shadcn theming
- Clean search home page with branded gradient header
- Web results show favicons, engine badges, and content snippets
- Image results displayed in a masonry grid with hover previews and a full lightbox with arrow-key navigation
- **Visited link colors** — clicked links turn purple to distinguish from unvisited results
- Engine settings modal with toggle switches
- Excluded domains management modal with add/remove UI
- **Unified settings modal** — single Settings button opens a tabbed modal (Engines, Excluded Sites)
- Web results show favicons, engine badges, publish dates, and content snippets
- Image results in a masonry grid with hover previews and a full-screen lightbox
- **Visited link colours** — clicked links turn purple to distinguish from unvisited results
- **Dropdown navigation menu** — consistent ☰ menu across all pages (search, bookmarks, history, stats, gallery)
- Keyboard navigation for autocomplete suggestions (↑↓ arrows, Enter, Escape)
- Footer with link to interactive API documentation (Swagger UI)
- URL-based routing with browser history support — shareable search URLs
## Accessibility
- **Skip to main content** link for keyboard users
- Image cards are real `<a>` links (Cmd/Ctrl-click, right-click, open-in-new-tab)
- Image cards are real `<a>` links (Cmd/Ctrl-click, right-click, open-in-new-tab all work)
- `aria-label` on all icon-only buttons and interactive elements
- Search input with `<label>`, `name`, `type="search"`, and ARIA combobox pattern
- Visible `focus-visible` ring on all focusable elements
- `aria-live` regions for loading/results/error announcements
- `aria-live` regions for loading / results / error announcements
- Proper heading hierarchy (`h1`/`h2`) and landmark elements (`<nav>`, `<main>`, `<aside>`)
- Dialog semantics on modals and lightbox (`role="dialog"`, `aria-modal`)
## REST API
- Full REST API for search, autocomplete, and engine management
- `/api/search` accepts both **GET and POST** requests with query string parameters
- Compatibility parameters on `/api/search`: `pageNumber` (alias for `page`), `numResults`, `format`, `imageProxy`, `safesearch`
- Every result includes `result_id`, `rank`, `engine`, and `timestamp` for agent integration
- `has_next` and `total_results` fields for cursor-aware pagination
- Standardized error schema (`code`/`message`/`details`/`retry_hint`) on all error responses
- `X-Response-Time-Ms` and rate-limit headers on all responses
- Copy-paste `curl` examples in OpenAPI docs for every endpoint
- OpenAPI specification with interactive docs via [Swagger UI](https://swagger.io/tools/swagger-ui/) (`/docs`) and [Redoc](https://github.com/Redocly/redoc) (`/redoc`)
## Deployment
- **Docker** — multi-stage Dockerfile: Node.js frontend build → Python production image
- Backend uses [uv](https://docs.astral.sh/uv/) for fast, reproducible dependency management
- **GHCR image** — pre-built image published to GitHub Container Registry on every release
- **Data volume** — `/app/data` stores SQLite databases and background images; mount for persistence
- **Environment variables** — `DATA_DIR` (data directory path), `REDIS_URL` (optional Redis connection string)
- Backend uses [uv](https://docs.astral.sh/uv/) for fast, reproducible Python dependency management
+39 -5
View File
@@ -53,20 +53,28 @@ just dev
REDIS_URL=redis://192.168.1.2:6399 just dev
```
### 2. Docker (production)
### 2. Docker Compose
The easiest way to run HeySearch with Redis caching in one command:
```bash
docker build -t hey-search .
docker compose up -d
```
Then open [http://localhost:8000](http://localhost:8000). Data and Redis are persisted in named volumes automatically.
### 3. Docker (standalone)
```bash
# Basic — data stored in anonymous volume
docker run -p 8000:8000 hey-search
docker run -p 8000:8000 ghcr.io/wahyd4/hey-search
# Recommended — mount data directory for persistence
docker run -p 8000:8000 -v ./hey-search-data:/app/data hey-search
docker run -p 8000:8000 -v ./hey-search-data:/app/data ghcr.io/wahyd4/hey-search
# With Redis
docker run -p 8000:8000 -v ./hey-search-data:/app/data \
-e REDIS_URL=redis://your-redis:6379 hey-search
-e REDIS_URL=redis://your-redis:6379 ghcr.io/wahyd4/hey-search
```
The `/app/data` volume stores the SQLite database (engine settings, excluded domains, cache config). Mount it to preserve your settings across container restarts.
@@ -99,6 +107,32 @@ The `/app/data` volume stores the SQLite database (engine settings, excluded dom
| `image_size` | — | `large`, `medium`, or `small` (images only) |
| `sort` | `default`| `default`, `date_asc`, or `date_desc` |
## Using with AI Agents / curl
The `/api/search` endpoint returns clean JSON — ideal for LLMs and AI agents to consume directly.
```bash
# Web search
curl "http://localhost:8000/api/search?q=python+async&format=json" | jq
# Restrict to specific engines
curl "http://localhost:8000/api/search?q=rust+programming&engines=brave,google" | jq
# Image search
curl "http://localhost:8000/api/search?q=mountain+landscape&category=images&image_size=large" | jq
# Paginate results
curl "http://localhost:8000/api/search?q=machine+learning&page=2" | jq
# Extract just titles and URLs from web results
curl "http://localhost:8000/api/search?q=openai" | \
jq '[.results[] | {title, url, snippet}]'
```
> Interactive API docs (Swagger UI) are available at `http://localhost:8000/docs`.
## Features
See [FEATURES.md](FEATURES.md) for the full feature list.
+13 -1
View File
@@ -1,6 +1,7 @@
"""HeySearch - A metasearch engine."""
import json
import logging
import os
import time
from pathlib import Path
@@ -14,12 +15,14 @@ from fastapi.middleware.cors import CORSMiddleware
from app.api.routes import router
from app.engines import registry
from app.excluded import init_db
from app.settings import init_settings_table
from app.settings import init_settings_table, set_setting
from app.cache import init_redis, close_redis
from app.bookmarks import init_bookmarks_table
from app.stats import init_db as init_stats_db
from app.models import APIError
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(application: FastAPI):
@@ -27,6 +30,15 @@ async def lifespan(application: FastAPI):
init_settings_table()
init_bookmarks_table()
init_stats_db()
# If REDIS_URL is provided via environment, persist it to the DB so it
# shows up in the settings UI and takes effect even if the DB had a
# different (or empty) value.
env_redis_url = os.environ.get("REDIS_URL", "").strip()
if env_redis_url:
set_setting("redis_url", env_redis_url)
logger.info("REDIS_URL from environment saved to settings: %s", env_redis_url)
registry.load_default_engines()
await init_redis()
yield
+31
View File
@@ -0,0 +1,31 @@
services:
app:
image: ghcr.io/wahyd4/hey-search:latest
build: .
ports:
- "8000:8000"
environment:
- REDIS_URL=redis://redis:6379
- DATA_DIR=/app/data
volumes:
- hey-search-data:/app/data
depends_on:
redis:
condition: service_healthy
restart: unless-stopped
redis:
image: redis:7-alpine
volumes:
- hey-search-redis:/data
command: redis-server --save 60 1 --loglevel warning
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
restart: unless-stopped
volumes:
hey-search-data:
hey-search-redis:
+7 -1
View File
@@ -389,7 +389,13 @@ function App() {
// Stats page
if (showStats) {
return <>
<StatsPage onGoHome={handleGoHome} />
<StatsPage
onGoHome={handleGoHome}
onShowSettings={() => setShowSettings(true)}
onShowBookmarks={handleShowBookmarks}
onShowGallery={handleShowGallery}
onShowHistory={handleShowHistory}
/>
{settingsModal}
</>;
}
+56 -81
View File
@@ -17,8 +17,6 @@ interface AppHeaderProps {
children?: React.ReactNode;
}
// Items ordered left→right. Rightmost item (Settings) gets the shortest delay,
// so it appears first creating the right-to-left unfold effect.
const NAV_ITEMS = [
{ id: "docs", icon: ExternalLink, label: "API Docs", href: "/docs" },
{ id: "stats", icon: BarChart2, label: "Stats", href: null },
@@ -28,8 +26,6 @@ const NAV_ITEMS = [
{ id: "settings", icon: Settings, label: "Settings", href: null },
] as const;
const STAGGER_MS = 60;
export function AppHeader({
onGoHome,
onShowSettings,
@@ -47,11 +43,11 @@ export function AppHeader({
const close = () => setMenuOpen(false);
const actions: Record<string, () => void> = {
settings: () => { onShowSettings(); close(); },
bookmarks: () => { onShowBookmarks(); close(); },
gallery: () => { onShowGallery(); close(); },
stats: () => { onShowStats(); close(); },
history: () => { onShowHistory?.(); close(); },
settings: () => { onShowSettings(); close(); },
bookmarks: () => { onShowBookmarks(); close(); },
gallery: () => { onShowGallery(); close(); },
stats: () => { onShowStats(); close(); },
history: () => { onShowHistory?.(); close(); },
docs: close,
};
@@ -84,12 +80,10 @@ export function AppHeader({
: "text-muted-foreground hover:bg-accent"
);
const itemBase = cn(
"flex items-center gap-1.5 rounded-full px-3 py-1.5 text-sm font-medium whitespace-nowrap",
"transition-colors focus-visible:ring-2 focus-visible:ring-ring focus-visible:outline-none",
transparent
? "text-white/80 hover:text-white hover:bg-white/10"
: "text-muted-foreground hover:bg-accent"
const dropdownItemBase = cn(
"flex w-full items-center gap-3 px-4 py-3 text-sm font-medium transition-colors",
"focus-visible:ring-2 focus-visible:ring-ring focus-visible:outline-none",
"text-foreground hover:bg-accent"
);
return (
@@ -117,72 +111,8 @@ export function AppHeader({
: <div className="flex-1" />
}
{/* Right side: animated nav items + toggle button */}
<div ref={menuRef} className="flex items-center gap-1 shrink-0">
{/* Nav items — animate right→left on open */}
{NAV_ITEMS.map((item, i) => {
// Rightmost item (Settings, index 3) gets 0ms delay → appears first
const delay = menuOpen
? (NAV_ITEMS.length - 1 - i) * STAGGER_MS
: 0;
// Outer wrapper collapses to width 0 when closed so it doesn't
// squeeze sibling content (e.g. the search bar on the results page).
const wrapperStyle: React.CSSProperties = {
maxWidth: menuOpen ? "120px" : "0px",
overflow: "hidden",
transition: `max-width 180ms cubic-bezier(0.16, 1, 0.3, 1) ${delay}ms`,
};
const innerStyle: React.CSSProperties = {
transitionProperty: "opacity, transform",
transitionDuration: "180ms",
transitionTimingFunction: "cubic-bezier(0.16, 1, 0.3, 1)",
transitionDelay: `${delay}ms`,
opacity: menuOpen ? 1 : 0,
transform: menuOpen ? "translateX(0)" : "translateX(16px)",
pointerEvents: menuOpen ? "auto" : "none",
};
const Icon = item.icon;
if (item.href) {
return (
<div key={item.id} style={wrapperStyle}>
<a
href={item.href}
target="_blank"
rel="noopener noreferrer"
aria-label={item.label}
title={item.label}
className={itemBase}
style={innerStyle}
onClick={close}
>
<Icon className="h-4 w-4" aria-hidden="true" />
<span className="hidden sm:inline">{item.label}</span>
</a>
</div>
);
}
return (
<div key={item.id} style={wrapperStyle}>
<button
onClick={actions[item.id]}
aria-label={item.label}
title={item.label}
className={itemBase}
style={innerStyle}
>
<Icon className="h-4 w-4" aria-hidden="true" />
<span className="hidden sm:inline">{item.label}</span>
</button>
</div>
);
})}
{/* Toggle button */}
{/* Menu toggle */}
<div ref={menuRef} className="relative shrink-0">
<button
onClick={() => setMenuOpen((v) => !v)}
aria-label={menuOpen ? "Close menu" : "Open menu"}
@@ -200,6 +130,51 @@ export function AppHeader({
}
</span>
</button>
{/* Dropdown panel */}
<div
className={cn(
"absolute right-0 top-full mt-2 w-52 rounded-xl border shadow-lg",
"bg-background/95 backdrop-blur",
"overflow-hidden transition-all duration-200 origin-top-right",
menuOpen
? "opacity-100 scale-100 pointer-events-auto"
: "opacity-0 scale-95 pointer-events-none"
)}
role="menu"
aria-label="Navigation menu"
>
{NAV_ITEMS.map((item) => {
const Icon = item.icon;
if (item.href) {
return (
<a
key={item.id}
href={item.href}
target="_blank"
rel="noopener noreferrer"
role="menuitem"
className={dropdownItemBase}
onClick={close}
>
<Icon className="h-4 w-4 shrink-0 text-muted-foreground" aria-hidden="true" />
{item.label}
</a>
);
}
return (
<button
key={item.id}
role="menuitem"
onClick={actions[item.id]}
className={dropdownItemBase}
>
<Icon className="h-4 w-4 shrink-0 text-muted-foreground" aria-hidden="true" />
{item.label}
</button>
);
})}
</div>
</div>
</div>
</header>
+20 -15
View File
@@ -1,9 +1,14 @@
import { useEffect, useState } from "react";
import { fetchStats, type StatsSummary } from "@/lib/api";
import { BarChart2, TrendingUp, MousePointerClick, Search } from "lucide-react";
import { AppHeader } from "@/components/AppHeader";
interface StatsPageProps {
onGoHome: () => void;
onShowSettings: () => void;
onShowBookmarks: () => void;
onShowGallery: () => void;
onShowHistory?: () => void;
}
function Card({ title, value, sub }: { title: string; value: string | number; sub?: string }) {
@@ -16,7 +21,7 @@ function Card({ title, value, sub }: { title: string; value: string | number; su
);
}
export function StatsPage({ onGoHome }: StatsPageProps) {
export function StatsPage({ onGoHome, onShowSettings, onShowBookmarks, onShowGallery, onShowHistory }: StatsPageProps) {
const [days, setDays] = useState(7);
const [data, setData] = useState<StatsSummary | null>(null);
const [loading, setLoading] = useState(true);
@@ -33,20 +38,20 @@ export function StatsPage({ onGoHome }: StatsPageProps) {
return (
<div className="min-h-screen bg-background">
{/* Header */}
<header className="sticky top-0 z-40 border-b bg-background/95 backdrop-blur">
<div className="flex items-center gap-3 px-4 py-3">
<button
onClick={onGoHome}
className="text-xl font-bold bg-gradient-to-r from-blue-600 to-purple-600 bg-clip-text text-transparent focus-visible:ring-2 focus-visible:ring-ring focus-visible:outline-none rounded"
>
HS
</button>
<div className="flex items-center gap-2 text-sm font-medium">
<BarChart2 className="h-4 w-4" />
<AppHeader
onGoHome={onGoHome}
onShowSettings={onShowSettings}
onShowBookmarks={onShowBookmarks}
onShowGallery={onShowGallery}
onShowStats={() => {}}
onShowHistory={onShowHistory}
>
<div className="flex items-center gap-3 min-w-0">
<h1 className="flex items-center gap-2 text-sm font-semibold whitespace-nowrap">
<BarChart2 className="h-4 w-4 shrink-0" />
Search Analytics
</div>
<div className="ml-auto flex items-center gap-1">
</h1>
<div className="flex items-center gap-1">
{[7, 30, 90].map((d) => (
<button
key={d}
@@ -60,7 +65,7 @@ export function StatsPage({ onGoHome }: StatsPageProps) {
))}
</div>
</div>
</header>
</AppHeader>
<main className="mx-auto max-w-5xl px-4 py-8 space-y-8">
{loading && <p className="text-center text-muted-foreground py-16">Loading</p>}