WhisperX Transcription

Set up local, accurate speech-to-text for TomoriBot using the bundled WhisperX server. WhisperX provides fast audio transcription with word-level alignment.

Setup

Run these commands from the TomoriBot repository root:

Windows PowerShell

cd servers/stt
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install -r requirements.txt
python whisperx_server.py

Linux/macOS Bash

cd servers/stt
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python whisperx_server.py

Keep that terminal open while TomoriBot is using WhisperX. The default endpoint URL is http://127.0.0.1:8021. On Windows, use 127.0.0.1 instead of localhost.

Choose a model

Set WHISPERX_MODEL before starting the server.

`WHISPERX_MODEL`	Size	Use
`large-v3`	1.55B	Best multilingual accuracy; about 0.6 GiB more VRAM than turbo
`large-v3-turbo`	809M	Multilingual, faster decoder; slightly worse Japanese
`kotoba-tech/kotoba-whisper-v2.0-faster`	756M	Japanese only; more accurate than turbo on Japanese test clips, at the same speed

On short clips the encoder dominates, so all three run at similar speed. Pick by language and VRAM. small and base lose too much Japanese accuracy to recommend. Whisper models can still invent text on very short or near-silent clips.

GPU

The server uses an NVIDIA GPU automatically when one is available. Check http://127.0.0.1:8021/health: device should read cuda. On Windows, speech recognition needs the CUDA 12 cuBLAS and cuDNN DLLs on your PATH. Without them the server prints GPU found but unusable at startup and runs on the CPU, which is much slower: install them and restart the server. If you set WHISPERX_DEVICE=cuda, the server stops at startup instead.

Register in TomoriBot

Run /providers, choose Add New Custom Endpoint, and use the transcription API compatibility:

API Compatibility: openai-compatible-transcription
endpoint_url: http://127.0.0.1:8021

After saving the connection, select it and use its model dropdown to add the exact value of WHISPERX_MODEL (large-v3 by default, or the full repo ID such as kotoba-tech/kotoba-whisper-v2.0-faster) as a Transcription model.

Use /providers for endpoint registration and model setup. Then open /config > Models > Switch Models to select and activate the registered endpoint.

Use transcripts

After registration, TomoriBot transcribes audio attachments in the background and adds the text to chat context. Use /config > Behavior > Notice Behavior only if you also want transcripts posted visibly in chat.