Liking cljdoc? Tell your friends :D

Archived Performance Documentation

This directory contains historical performance optimization documentation for liblevenshtein-rust.

Status: Archived - For historical reference only Current Documentation: See docs/PERFORMANCE.md for current performance information


Overview

These documents track the optimization journey from v0.1.0 through v0.2.0, showing the iterative process of improving performance through profiling, benchmarking, and targeted optimizations.

Key Results:

  • 40-60% overall performance improvements
  • 3.3x speedup for DAWG operations
  • 15-50% faster PathMap edge iteration
  • 5-18% improvements for filtering/prefix operations

Recommended Reading Order

1. Start Here: High-Level Overview

  • OPTIMIZATION_SUMMARY.md - Complete optimization overview
    • All optimization phases summarized
    • Key takeaways and lessons learned
    • Profiling-guided approach

2. Detailed Phase Reports

See ../optimization/ for phase-by-phase optimization journey:

3. Specific Optimizations

Core Engine

DAWG Backend

Serialization & Storage

Filtering & Code Completion

Other Optimizations

4. Comparisons & Validation

5. Early Phase Results


Document Organization

archive/performance/
├── README.md (this file)
│
├── High-Level Overviews
│   └── OPTIMIZATION_SUMMARY.md          # ⭐ Start here
│
├── Core Engine Optimizations
│   ├── ARC_OPTIMIZATION_RESULTS.md
│   ├── PATHNODE_OPTIMIZATION_RESULTS.md
│   └── QUERY_ARC_ANALYSIS.md
│
├── DAWG Backend
│   ├── DAWG_OPTIMIZATION_RESULTS.md
│   ├── DAWG_OPTIMIZATIONS_APPLIED.md
│   ├── DAWG_OPTIMIZATION_OPPORTUNITIES.md
│   └── INDEX_BASED_QUERY_RESULTS.md
│
├── Serialization
│   ├── SERIALIZATION_OPTIMIZATION_RESULTS.md
│   ├── SERIALIZATION_OPTIMIZATIONS_APPLIED.md
│   ├── SERIALIZATION_OPTIMIZATION_PLAN.md
│   └── SERIALIZATION_BENCHMARK_BASELINE.md
│
├── Filtering & Code Completion
│   ├── CODE_COMPLETION_PERFORMANCE.md
│   └── CONTEXTUAL_FILTERING_OPTIMIZATION.md
│
├── Comparisons & Validation
│   ├── DAWG_COMPARISON.md
│   ├── JAVA_COMPARISON.md
│   └── REAL_WORLD_VALIDATION.md
│
├── Tuning & Analysis
│   ├── THRESHOLD_TUNING_RESULTS.md
│   ├── PGO_IMPACT_ANALYSIS.md
│   └── PERFORMANCE_ANALYSIS.md
│
└── Early Phases
    ├── PHASE2_RESULTS.md
    ├── PHASE3_RESULTS.md
    ├── PROFILING_AND_PGO_RESULTS.md
    └── OPTIMIZATION_RESULTS.md

Key Lessons Learned

1. Profile Before Optimizing

  • Flamegraphs revealed true hotspots (often surprising)
  • Intuition about "slow" code was frequently wrong
  • Measure, don't guess

2. Object Pools Are Powerful

  • StatePool elimination of allocation overhead was exceptional
  • Significant impact in tight loops
  • Low implementation complexity for high reward

3. Arc Sharing Beats Cloning

  • Cheap reference counting vs expensive deep copies
  • Especially effective for immutable data
  • Path sharing was highly successful

4. Small Optimizations Add Up

  • SmallVec, inlining, lazy iteration individually modest
  • Combined effect: 5-18% improvements
  • Incremental approach works

5. Real-World Benchmarks Matter

  • Synthetic benchmarks can mislead
  • System dictionaries (/usr/share/dict/words) provided realistic validation
  • Performance varies with dictionary characteristics

Methodology

Tools Used

  • Criterion.rs - Statistical benchmarking
  • Flamegraph - CPU profiling and hotspot identification
  • cargo-llvm-cov - Code coverage analysis
  • PGO - Profile-Guided Optimization (tested, not currently used)

Process

  1. Baseline - Establish current performance
  2. Profile - Identify hotspots with flamegraphs
  3. Hypothesize - Target specific optimization
  4. Implement - Make targeted changes
  5. Benchmark - Measure impact with Criterion
  6. Validate - Test with real-world dictionaries
  7. Iterate - Repeat for next optimization

Current Status

This archived documentation reflects the state of optimizations through v0.2.0. For current performance information, see:


Navigation

Can you improve this documentation?Edit on GitHub

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close