Clarity Scribe is a desktop application that converts speech to text entirely on the machine it runs on. Press a key, speak, and the text lands in whatever application you were already using. No audio leaves the device, there is no account, and after a one-time model download there is no network dependency at all. It runs on Windows and Apple Silicon, using NVIDIA’s Parakeet TDT for English and 24 other European languages, and OpenAI’s Whisper Large V3 Turbo for the remaining hundred or so.

It began as a dictation tool. It is now the bottom layer of something larger.

The Dictation Layer Is Finished

The headline number is 1212x real-time: a 2 minute 53 second recording that appeared as text 143 milliseconds after I stopped speaking, paste into the target application included.

Version 3 transcribes speech while you are still talking instead of waiting for you to finish, so when you stop, only the final phrase remains to process. The consequence is that latency stopped scaling with recording length. Measured across 160 real dictations covering 63 minutes of speech, from 1.3 seconds to nearly three minutes, the median wait from key release to text on screen was 504 milliseconds and the worst case in the entire set was 0.9 seconds. Dictate a sentence or dictate three minutes of notes, and the wait is the same.

A system whose response time is flat and under a second is finished in the sense that matters to a user, because further optimization buys nothing they can perceive. Accuracy is at 6.05% word error rate on English, currently first on the HuggingFace ASR leaderboard, on a 600 million parameter model small enough to run on the Apple Neural Engine or a consumer GPU. For comparison, the version I wrote up in April processed audio in a batch after you finished and reached 53x real-time, and the common offline alternatives sit between 5x and 15x.

Speech to text, locally, is solved well enough to build on, so I started building on it.

What Sits On Top

Once transcription is reliable and instant, voice stops being a feature of one application and becomes an input channel available to everything else on the machine.

A local HTTP API, bound to loopback only and authenticated with a bearer token, exposes a server-sent event stream of live transcription alongside endpoints to start and stop recording. It ships disabled. Any script or process on the machine can drive dictation and consume transcripts as they are produced.

A Model Context Protocol server makes Clarity Scribe callable as a tool by Claude Desktop, Claude Code, or any other MCP host. An agent can request the microphone and receive back what was said, which turns a dictation app into infrastructure other software can depend on.

Command mode routes a spoken instruction through a local language model to a set of permitted actions instead of typing it out as text. The routing model runs offline through llama.cpp, so a spoken command never leaves the machine either. Commands that repeat, which in practice is most of them, bypass the model entirely through pattern matching and execute in roughly 90 to 190 milliseconds.

Beyond that sits an experimental screen agent that operates Windows applications the way a person would. It reads an application’s real controls through the Windows accessibility tree in about 100 to 200 milliseconds without touching the GPU, decides one action at a time, and activates controls programmatically rather than moving a cursor and clicking, which removes an entire class of failure. Computer vision handles only the applications that expose no accessibility information.

Because a voice agent that can act on your behalf is a materially different risk surface than one that types, an explicit rulebook governs what it may do. Benign operations run immediately, while consequential ones such as sending a message or launching an executable present a confirmation that cancels itself if unanswered. The rulebook refuses payments, credentials, and bulk deletion outright. It re-evaluates every action mid-task, because a button that was harmless when the agent started may not be by the time it gets there.

Where It Stands

The downloadable Windows installer is version 3.1.0. macOS runs from source today. Command mode and the screen agent are experimental, off by default, and Windows only. The repository is ahead of the shipped installer by several versions, and closing that gap is the current work.

The project is open source under AGPL-3.0 with a Commons Clause, which permits use, modification, and redistribution while reserving commercial sale.

Where It Goes

The end state I am building toward is an assistant that hears you, understands what you want, and operates your computer to do it, running entirely on your own hardware. Every layer of that depends on the layer beneath it being boring and reliable. Dictation had to be instant and accurate before commands were worth attempting, because a command mode that mishears is worse than no command mode at all. Commands had to be safely gated before an agent could be allowed to chain them.

An assistant of this kind hears your room and sees your screen, which makes it the single worst category of software to route through infrastructure you do not control. Running it locally means the capability and the privacy stop competing, and it removes per-request cost and rate limits along the way. The tradeoff is that everything has to run on hardware people already own, and that constraint is what forced the engineering.

The dictation layer is done, the command layer works and needs hardening, and the agent layer is early. The next milestone is shipping an installer that contains all three, and after that, the same treatment for macOS.

The project is at github.com/laloquidity/clarity-scribe.