Speech recognition languages and models in Video by Zaweba
Video by Zaweba recognises 27 languages or detects one by itself; languages other than English need a multilingual whisper.cpp model added by you.
Captions and transcripts in Video by Zaweba are made from the speech on your own PC. That's a big part of why it works offline and why your footage never leaves your computer. It also means the app needs the right recognition model on hand for the language your footage is in. Here's how languages work and what you need to set up.
27 languages or auto-detect
Speech recognition covers 27 languages. You can choose the language of your footage, or let the app auto-detect it. Choosing is more reliable when you know the language; auto-detect is handy when you're working through a batch of clips in mixed languages and don't want to set each one.
English and the rest
English works straight away. For anything but English, you need a multilingual whisper.cpp model file. Whisper is a well known speech recognition system, and whisper.cpp is a version that runs efficiently on an ordinary computer. A multilingual model is one trained on many languages rather than English alone.
You add the model with Add model. Point it at the model file and it becomes available for recognition.
Nothing is downloaded
The app doesn't download models by itself. You obtain the model file and add it. That keeps the program fully offline and under your control: nothing is fetched behind your back, and no network connection is needed at any point once the file is on your machine.
It also suits anyone working with sensitive material, such as interviews, internal company videos or family footage, since the speech is never sent to a server to be turned into text.
The practical upshot is a small one-off setup step for anyone working in other languages. Get the multilingual model file, use Add model, and you're done. After that, recognition in your language works offline like English does.
Choosing a model
Whisper models come in different sizes. As a general rule, larger models are more accurate but slower and need more memory, while smaller ones are quicker but make more mistakes. If your computer is modest, start with a smaller multilingual model and see whether the accuracy is good enough for your footage. If you're working with difficult audio, such as accents, noise or fast speech, a larger model may be worth the wait.
Getting better recognition
Clean sound helps every language. Run noise reduction on noisy clips and let speech stand out from music before you make captions or a transcript. Recognition copes with a lot, but clear speech gives it the best chance.
Set the language when you know it. Auto-detect is convenient, but on short clips or ones with a lot of music it has less to go on.
Check names and specialist terms. Even good recognition can misspell unusual words. If you have a script, captions are spelled from it, which solves most of this.
Where it leads
With the right model added, everything built on speech recognition is available in your language: captions from speech, the transcript with text-based editing, and the tools that go with them. Video by Zaweba treats them all the same way, so your workflow doesn't change from one language to the next.
