llm_gpt3.5-vs-gpt4-comparison

What it is

This project demonstrates a method to summarize long documents (such as an hour-long conversation or State of the Union address) at approximately 1/22nd the cost of GPT-4. It replaces fixed-character chunking with logic-based segmentation, breaking text into sentences, grouping them into coherent paragraphs based on topic changes, summarizing each paragraph individually, and then generating a final summary of those sub-summaries using GPT-3.5-16k.

Features

• Logical text segmentation based on sentence boundaries rather than fixed character counts
• Hierarchical summarization: paragraph-level summaries aggregated into a final document summary
• Topic change detection using cosine similarity of OpenAI embeddings
• Cost optimization targeting ~22.5x savings compared to GPT-4
• Visual heatmap generation for topic clustering and chunk analysis

Quickstart

bash llm_setup.sh
export OPENAI_API_KEY=yourkeyhere
python3 llm_summary.py

Architecture

flowchart TD
    A[Raw Text Input] --> B[Sentence Splitter]
    B --> C[Paragraph Grouping]
    C --> D[Embedding Generation]
    D --> E[Cosine Similarity Check]
    E -->|Topic Change| F[New Paragraph]
    E -->|Same Topic| C
    F --> G[GPT-3.5 Summarize Paragraph]
    G --> H[Collection of Sub-Summaries]
    H --> I[GPT-3.5-16k Final Summary]
    I --> J[Final Output]
    D --> K[Heatmap Visualization]

How it's built

The system is built in Python using LangChain for LLM orchestration and OpenAI embeddings. It processes raw text by splitting on punctuation to create sentence fragments, groups these into logical paragraphs, and uses cosine similarity on embeddings to detect topic shifts. Each paragraph is summarized and titled using GPT-3.5-turbo. Finally, the collection of paragraph summaries is fed into GPT-3.5-16k for a global summary. Visualization tools generate heatmaps of topic clusters.

How it runs

sequenceDiagram
    participant User
    participant Script as llm_summary.py
    participant OpenAI as OpenAI API
    
    User->>Script: Execute script with txt file
    Script->>Script: Split text into sentences
    Script->>Script: Group sentences into paragraphs
    loop For each paragraph
        Script->>OpenAI: Generate Embeddings
        OpenAI-->>Script: Vector embeddings
        Script->>Script: Calculate Cosine Similarity
        Script->>OpenAI: Summarize Paragraph (GPT-3.5)
        OpenAI-->>Script: Paragraph Summary & Title
    end
    Script->>OpenAI: Summarize All Sub-Summaries (GPT-3.5-16k)
    OpenAI-->>Script: Final Document Summary
    Script->>User: Display Summary & Save Heatmaps

How to apply & reuse

Use this approach when cost is a primary constraint and latency is secondary. It is ideal for batch processing large transcripts, articles, or logs where parallelization is not required. Replace the default 'stateoftheunion.txt' with your target document and ensure your OpenAI API key is configured.

At a glance