GitHub Portfolio Search

What it is

A tool that indexes GitHub repositories using embeddings and TF-IDF to enable semantic search, faceted filtering, and capability clustering. It exposes data via a static web UI, a REST API, and an MCP server for AI agents, allowing developers to find reusable code patterns across their portfolio without loading entire codebases into context.

Features

• Semantic search using embeddings and TF-IDF across READMEs and code
• Auto-clustering of repositories into capability categories
• MCP server integration for AI agents to query portfolio data
• Faceted filtering by language, tech stack, and star count
• Static web UI with shareable URLs and localStorage collections
• Automated weekly reindexing via GitHub Actions

Quickstart

git clone https://github.com/davidbmar/github-portfolio-search.git
cd github-portfolio-search
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
cp .env.example .env
ghps index <your_github_username>
ghps export
cd web && python3 -m http.server 8000

Architecture

flowchart TD
    User[User Browser]
    Agent[AI Agent]
    CloudFront[CloudFront CDN]
    S3[S3 Bucket Static Assets]
    API[FastAPI Server]
    Indexer[Indexing Pipeline]
    DB[SQLite Vector DB]
    GitHub[GitHub API]
    LambdaOAuth[Lambda OAuth Handler]
    LambdaLog[Lambda Signin Log]
    User --> CloudFront
    Agent --> API
    CloudFront --> S3
    CloudFront --> API
    API --> DB
    Indexer --> GitHub
    Indexer --> DB
    LambdaOAuth --> GitHub
    LambdaLog --> S3Logs[S3 Log Bucket]

How it's built

Built with Python for the indexing pipeline (using sentence-transformers and SQLite-vec) and FastAPI for the backend. The frontend is a static HTML/JS application served from S3/CloudFront. AWS Lambda functions handle OAuth and logging. GitHub Actions automate weekly reindexing and deployment.

How it runs

sequenceDiagram
    participant Client as Web Client
    participant API as FastAPI Server
    participant DB as SQLite Vector DB
    participant GitHub as GitHub API
    Client->>API: GET /search?q=voice+processing
    API->>DB: Query embeddings and TF-IDF
    DB-->>API: Return ranked repo IDs
    API->>DB: Fetch repo metadata for IDs
    DB-->>API: Return repo details
    API-->>Client: Return JSON results
    Client->>GitHub: Fetch README content if needed
    GitHub-->>Client: Return README text

How to apply & reuse

Use it to build a searchable knowledge base of your own code. Integrate the MCP server into AI coding assistants like Claude Code to allow them to query your past solutions. Deploy the static site to showcase your portfolio with advanced filtering and semantic relatedness features.

At a glance