search-tool-provider

What it is

A Python library that abstracts multiple web search engines (DuckDuckGo, Tavily, Brave, Serper, Bing, etc.) behind a single asynchronous API. It provides a registry pattern for swapping providers, built-in fallback chains, caching, and structured response models. It includes three interfaces: a programmatic Python API, an interactive Rich-based CLI, and a FastAPI-based Admin UI with a configuration wizard.

Features

• Unified async API for 6+ search backends (DuckDuckGo, Tavily, Brave, Serper, Bing, etc.)
• Automatic fallback chains to retry failed queries across configured providers
• Interactive CLI with Rich formatting for quick testing and comparison
• Admin UI with step-by-step wizard for API key setup and live search testing
• Built-in caching and rate limit handling with specific exception types
• MCP server support for integration with AI agent frameworks

Quickstart

pip install search-tool-provider[duckduckgo,cli]
search-provider-cli

Architecture

flowchart TD
    User[User/Agent] -->|Python API| App[Application]
    User -->|CLI| CLI_App[CLI Interface]
    User -->|Browser| Admin_UI[Admin UI FastAPI]
    
    subgraph Core_Library[search-tool-provider]
        Registry[Provider Registry]
        ABC[SearchProvider ABC]
        Models[Data Models]
        Env[Env Manager]
    end
    
    App --> Registry
    CLI_App --> Registry
    Admin_UI --> Registry
    
    Registry -->|Instantiates| Provider_Instance[Concrete Provider]
    
    subgraph Backends[External APIs]
        DDG[DuckDuckGo]
        Tav[Tavily]
        Brv[Brave Search]
        Oth[Other Providers]
    end
    
    Provider_Instance -->|HTTP Request| DDG
    Provider_Instance -->|HTTP Request| Tav
    Provider_Instance -->|HTTP Request| Brv
    Provider_Instance -->|HTTP Request| Oth

How it's built

Built on Python's asyncio for non-blocking I/O. Uses a Registry pattern to manage provider implementations of the SearchProvider ABC. Configuration is handled via environment variables or a safe .env merge utility. The Admin UI uses FastAPI and Jinja2 templates, while the CLI uses Rich for terminal rendering. It supports dependency injection for API keys and includes custom exception hierarchies for error handling.

How it runs

sequenceDiagram
    participant Client as Application/CLI
    participant Registry as get_provider()
    participant Provider as Concrete Provider
    participant Backend as Search Engine API
    
    Client->>Registry: get_provider("tavily", api_key="...")
    Registry->>Registry: Lookup registered class
    Registry-->>Client: Return Provider Instance
    
    Client->>Provider: await search(query, max_results=5)
    Provider->>Provider: Validate SearchQuery
    Provider->>Backend: HTTP POST /search
    Backend-->>Provider: JSON Response
    Provider->>Provider: Parse & Normalize Results
    Provider->>Provider: Map to SearchResult models
    Provider-->>Client: Return SearchResponse

How to apply & reuse

Install the package with desired backend extras (e.g., duckduckgo, tavily). Use get_provider() to instantiate a search engine by name. Call await provider.search() for standard results, search_news() for news, or get_answer() for direct answers. Use the FallbackProvider to automatically chain multiple providers based on available API keys. Integrate into LLM agents via the included MCP server or use the CLI for manual testing.

At a glance