optimization review

Guide for optimization review

Codebase Optimization & Bug Review

🚨 Critical Performance Issues

1. N+1 R2 Reads in kb-chat.ts

Location: src/kb-chat.ts -> getKBContext() Issue: The code lists all objects in the bucket and then sequentially awaits env.KB_BUCKET.get(file.key) for every single file. Impact:
  • Latency: Linear increase with file count. 10 files = ~2s, 50 files = ~10s+.
  • Cost: High Class A operations on R2.
  • Reliability: Higher chance of timeout.
Fix:
  • Parallelize: Use Promise.all to fetch files concurrently (capped limit).
  • Consolidation: Maintain a single _index.json or _all_context.txt file that is updated on write, rather than reading 50 files on read.

2. Full Context Loading (Context Window Overflow)

Location: src/kb-chat.ts Issue: It loads every line of every markdown file into the Gemini prompt. Impact:
  • Token Limits: Will crash as KB grows beyond ~30k tokens (Gemini Flash limit is high but latency/cost grows).
  • Cost: Input tokens are billed. Sending 1MB of text per chat is expensive.
Fix:
  • RAG (Retrieval Augmented Generation): Implement basic semantic search or keyword search to only include relevant files/chunks.

3. Synchronous DB Logging

Location: src/kb-chat.ts -> handleKBChat Issue: await env.DB.prepare(...).run() is called before returning response. Impact: Adds 100-300ms latency to every chat response unnecessarily. Fix: Use ctx.waitUntil(promise) to run logging in background.
Location: src/kb-files.ts -> searchKBFiles Issue: Downloads and reads ALL files to perform a regex search. Impact: Extremely slow and bandwidth heavy. Fix:
  • Maintain a Search Index (D1 or distinct R2 object).
  • Or use ctx.waitUntil to update a search index on file save.

🐛 Potential Bugs

1. better-sqlite3 in Edge Runtime

Location: lib/db.ts Issue: Uses require('better-sqlite3') which may crash in Next.js Edge Runtime if bundler doesn't tree-shake it out purely based on isEdge() check (dynamic require helps, but risky). Status: Likely fine due to process.env.NEXT_RUNTIME check, but fragile.

2. Race Condition in Context Cache

Location: src/kb-chat.ts Issue: getKBContext populates cache. If handleKBFileSave clears cache (kb_context), a concurrent getKBContext might re-cache stale data if R2 listing isn't immediately consistent (R2 is strongly consistent, so this is minor). Fix: Use a versioning system for cache keys.

Recommended Immediate Action Plan

  1. Parallelize R2 Reads: Change for (const file...) await get to await Promise.all(files.map(...)).
  2. Async Logging: Pass ctx to handleKBChat and use waitUntil.
  3. Context Limiting: Naively limit context to first 10 files or implement simple keyword matching for "relevant" files if RAG is out of scope.