If you’ve ever tried to build a transcription pipeline, you know the pain. You need one model to turn audio into text, and a completely separate tool to figure out who said what. Microsoft just made that a one-step problem. Their VibeVoice model does speech-to-text and speaker diarization (identifying which person is talking) in a single pass. It’s MIT licensed, meaning you can use it for anything, commercially, for free.
Simon Willison just put it through its paces on a real podcast episode, and the results are worth paying attention to if you run any kind of service business that touches audio.
What happened
- Microsoft released VibeVoice on January 21st, 2026. It’s a Whisper-style audio model that converts speech to text with speaker identification baked in.
- The model is MIT licensed, so you can run it locally, modify it, and use it in commercial products without restrictions.
- Simon Willison tested it on April 27th using a 4-bit quantized version (5.71GB) of the full 17.3GB model, running it on a Mac via mlx-audio.
- The output is structured JSON with text, timestamps, and a speaker ID for every segment. You get who said what, and exactly when they said it.
- It can handle up to an hour of audio in a single run. For longer files, you’d need to split the audio and stitch results together.
The numbers
- 8 minutes 45 seconds to transcribe a full hour of podcast audio.
- Ran on a 128GB M5 Max MacBook Pro.
- Prompt processing speed: 50.7 tokens per second. Generation speed: 38.6 tokens per second.
- Peak reported memory: 30.44GB, though Activity Monitor showed up to 61.5GB during the prefill stage.
- The quantized model weighs 5.71GB. The full model is 17.3GB.
- Default max tokens is 8,192 (enough for about 25 minutes of audio). Willison bumped it to 32,768 to cover the full hour.
5 reasons VibeVoice matters for service businesses
- It’s completely free to use. MIT license means no per-minute API fees. If you transcribe a lot of audio (client calls, depositions, patient intake recordings), the cost savings add up fast.
- Speaker diarization is built in. Most open source transcription models give you a wall of text. VibeVoice tells you which speaker said each line. That’s the difference between a transcript and something actually useful.
- It runs locally. No audio leaves your machine. For industries like legal, healthcare, and finance where confidentiality matters, this is a big deal. You’re not sending sensitive recordings to a third-party API.
- The output is structured JSON. Each segment comes with text, start time, end time, duration, and speaker ID. That’s ready to pipe into a database, a search index, or an AI agent without any extra parsing.
- It’s surprisingly accurate at speaker detection. In Willison’s test, it correctly identified two speakers in a podcast conversation, and even flagged the host’s different speaking style during intro and sponsor reads as a separate speaker. That’s a level of nuance most paid services struggle with.
The hot take
Speaker diarization has been the missing piece in open source transcription for years. Whisper gave us incredible speech-to-text, but knowing who said something is what makes a transcript actually useful for business. VibeVoice closes that gap, and it does it under an MIT license. Every paid transcription API just got a lot harder to justify. If you’re still sending client audio to a cloud service and paying per minute, you’re overpaying for something you could run on a laptop.
The Agency OS play
This week, pick one audio workflow in your business and test VibeVoice against it. If you do client intake calls, record one (with permission), run it through the model, and look at the JSON output. You’ll immediately see whether the speaker detection and transcript quality is good enough to replace whatever you’re using now. If you’re on a Mac with enough RAM, the mlx-audio setup Willison described is genuinely a one-liner to get started.
Think bigger than just transcripts, though. The structured JSON output (with speaker IDs and timestamps) is perfect for feeding into a second AI step. Imagine automatically pulling action items from a meeting, sorted by who’s responsible. Or flagging specific topics in client calls for compliance review. The speaker identification is what makes these workflows possible without a human manually tagging everything.
One practical note: VibeVoice caps at one hour per run. If your recordings are longer, you’ll need to split them with some overlap (a minute or so) and then reconcile the speaker IDs across segments. Build that into your pipeline from the start. Also keep in mind the memory requirements. The 4-bit model peaked at over 60GB of RAM in testing, so this isn’t running on a cheap office laptop. Plan your hardware accordingly, or look into running it on a GPU server if you need to process volume.