LLM Enhancement

cuttledoc includes optional LLM-based post-processing to improve transcription quality. This page covers how to use it effectively.

Overview

Speech-to-text engines produce raw transcripts with typical errors:

• Missing or incorrect punctuation
• Word boundary issues ("gonna" vs "going to")
• Homophones ("their" vs "there")
• Filler words and repetitions

LLM enhancement corrects these issues while preserving the original meaning.

Quick Start

With CLI

# Correction is enabled by default
cuttledoc audio.mp3

# Disable correction
cuttledoc audio.mp3 --no-correct

# Full formatting (adds Markdown structure)
cuttledoc audio.mp3 --format

With API

import { transcribe } from 'cuttledoc'
import { enhanceTranscript } from '@cuttledoc/llm'

const result = await transcribe('audio.mp3')

// Correct errors (recommended)
const corrected = await enhanceTranscript(result.text, {
  mode: 'correct'
})

// Full formatting
const formatted = await enhanceTranscript(result.text, {
  mode: 'format'
})

Processing Modes

correct (Default)

Fixes transcription errors without changing structure:

• Punctuation and capitalization
• Word boundaries and contractions
• Obvious spelling errors
• Filler word removal

Does NOT:

• Add headings or sections
• Summarize content
• Change vocabulary or style

format

Full Markdown formatting:

• Everything from correct
• Section headings (##)
• Bullet lists where appropriate
• Bold for key terms
• Italic for emphasis

Available Models

Ollama (Recommended)

Easiest setup, best quality. Install Ollama and pull a model:

brew install ollama
ollama pull phi4:14b  # Best quality and Ollama default

GGUF Models (Embedded)

For applications without external dependencies. Models auto-download on first use.

import { enhanceTranscript } from '@cuttledoc/llm'

const result = await enhanceTranscript(text, {
  provider: 'local',
  model: 'gemma3n:e4b' // default for GGUF
})

Downloaded GGUF models are stored in a stable per-user cache:

• Linux: $XDG_CACHE_HOME/cuttledoc/models/llm, or ~/.cache/cuttledoc/models/llm
• macOS: $XDG_CACHE_HOME/cuttledoc/models/llm when XDG_CACHE_HOME is an absolute path, otherwise ~/Library/Caches/cuttledoc/models/llm
• Windows: %LOCALAPPDATA%\cuttledoc\models\llm

Set CUTTLEDOC_LLM_MODELS_DIR to an absolute path to use a custom location. Relative override paths are rejected. The old LOCAL_TRANSCRIBE_LLM_MODELS_DIR name remains supported for compatibility but is deprecated.

If an earlier cuttledoc release downloaded models into ./models/llm, move that directory into the user cache above or point CUTTLEDOC_LLM_MODELS_DIR at the existing absolute path to avoid another download.

OpenAI (Cloud)

For highest quality when latency isn't critical:

export OPENAI_API_KEY=sk-...

const result = await enhanceTranscript(text, {
  provider: 'openai',
  model: 'gpt-5-mini'
})

Benchmark results

On the documented correction test set, phi4:14b produced the strongest accuracy improvement, while mistral-nemo processed the samples fastest. Results varied by language, so validate a model against representative transcripts before deploying it.

See Benchmarks for the complete tables, per-language results, and methodology.

Best Practices

When to Use LLM Correction

✅ Recommended:

• Podcasts, interviews, meetings
• Content that will be read by humans
• When accuracy matters more than speed

⚠️ Consider disabling:

• Real-time/streaming transcription
• Very short clips (<30 seconds)
• Technical dictation with specialized vocabulary

Model Selection

// Best quality with Ollama
await enhanceTranscript(text, { provider: 'ollama', model: 'phi4:14b' })

// Fast processing
await enhanceTranscript(text, { model: 'mistral-nemo' })

// Low memory (<6GB RAM)
await enhanceTranscript(text, { model: 'gemma3n:e2b' })

Handling Long Transcripts

The library automatically processes long transcripts in bounded chunks for Ollama, OpenAI, and local GGUF models. It prefers sentence boundaries and hard-splits unusually long sentences when needed:

const result = await enhanceTranscript(longText, {
  // Automatically handled - no configuration needed
})

Troubleshooting

"Ollama not running"

# Start Ollama
ollama serve

# Or run in background
brew services start ollama

"Model not found"

# List available models
ollama list

# Pull the model
ollama pull phi4:14b

Slow performance

1. Use a smaller model: gemma3n:e2b or mistral-nemo
2. Disable LLM for short clips: --no-correct
3. Ensure GPU acceleration is enabled (check Ollama docs)

API Reference

See the API Reference for the public @cuttledoc/llm functions, options, result types, and exports.