TL;DR Polish is not supported as audio input for Gemma 4 12B

Google has released Gemma 4 12B, a new variant of its open-weight LLM family. It’s a medium-sized model advertised to run locally on laptop-sized devices using just 16 GB of VRAM. It also features native audio input.

Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

Source: Gemma 4 12B Announcement

I decided to dig deeper into the model’s automatic speech recognition capabilities. I wanted to know if it supports Polish, my native tongue.

First, I checked the model card:

Automatic speech recognition (ASR) and speech-to-translated-text translation across multiple languages.

Source: Gemma 4 model card

Multiple languages, aha! That’s not very revealing. Without that information, the only way forward is to check myself.

I started by upgrading to Ollama 0.30.5, the first version advertised to work with Gemma 4 12B.

ollama pull gemma4:12b
ollama ps
# Prints:
# NAME          ID              SIZE      PROCESSOR    CONTEXT    UNTIL
# gemma4:12b    4eb23ef187e2    8.2 GB    100% GPU     131072     4 minutes from now

# Let's do a text smoke test first
> curl http://127.0.0.1:11434/api/chat \
      -H 'Content-Type: application/json' \
      -d '{
    "model": "gemma4:12b",
    "stream": false,
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }' -s | jq .message.content
# > "Hello! How can I help you today?"

Text generation works; let’s move to voice.

For voice, I like to use sag, which is a CLI for ElevenLabs inspired by macOS’s say.

Before checking Polish, let’s smoke-test ASR with English audio.

export ELEVENLABS_API_KEY="..."
# Listen to the clip before generating a version for the ASR
sag speak "The quick brown fox jumps over the lazy dog."
# ... and let's get the version for the model:

sag speak --no-play --no-stream \
    -o /tmp/gemma4-12b-asr-polish/english-smoke.mp3 \
    "The quick brown fox jumps over the lazy dog."
# Convert it to a format Gemma4 expects
ffmpeg -y -hide_banner  \
    -i /tmp/gemma4-12b-asr-polish/english-smoke.mp3 \
    -ar 16000 -ac 1 \
    /tmp/gemma4-12b-asr-polish/english-smoke-16k-mono.wav

Time to send it to the model:

AUDIO_B64=$(base64 -i /tmp/gemma4-12b-asr-polish/english-smoke-16k-mono.wav | tr -d '\n')

jq -n --arg audio "$AUDIO_B64" '{
  model: "gemma4:12b",
  stream: false,
  messages: [{
    role: "user",
    content: "Listen to the attached audio and write the exact English words spoken. Do not translate. Return only the English transcript.",
    images: [$audio]
  }]
}' | curl -sS http://127.0.0.1:11434/api/chat -H 'Content-Type: application/json' -d @- | jq '.message.content'

It printed the expected The quick brown fox jumps over the lazy dog..

Notice we’re using the images field to send the base64-encoded audio. As of the time of writing this, this is the only way to use multimodal inputs with Ollama. Neither audio nor input_audio is supported.

Now it’s time for the final test. This time, the fixture is the famous opening lines by Adam Mickiewicz:

Litwo, Ojczyzno moja! ty jesteś jak zdrowie;
Ile cię trzeba cenić, ten tylko się dowie,
Kto cię stracił. Dziś piękność twą w całej ozdobie
Widzę i opisuję, bo tęsknię po tobie.

sag produces a ~15-second clip out of it.

I had to disable thinking for this clip because the model could not stop after several minutes on my M2 MacBook Air.

AUDIO_B64=$(base64 -i /tmp/gemma4-12b-asr-polish/litwo-ojczyzno-moja-16k-mono.wav | tr -d '\n')

jq -n --arg audio "$AUDIO_B64" '{
  model: "gemma4:12b",
  stream: false,
  think: false,
  options: {
    num_ctx: 8192,
    num_predict: 1024
  },
  messages: [{
    role: "user",
    content: "Listen to the attached audio and write the exact Polish words spoken. Do not translate. Preserve Polish spelling, diacritics, capitalization, and punctuation. Return only the Polish transcript.",
    images: [$audio]
  }]
}' | curl -sS http://127.0.0.1:11434/api/chat -H 'Content-Type: application/json' -d @- | jq -r .message.content

Prints:

Litfoj czs no ma, ty jests jak zdrowie, i lecz trzeba cenić, ten tylko se dowje kto cz straciu. Dżisz pęngnost dwonw czalej ozdobie widze i opisuje. Bo tenskne potens nie potolisz.

:sadpanda:

I’m not even going to compute standard ASR metrics like WER/CER here because the output is plainly unusable.

Let me just leave Whisper here for comparison:

> whisper /tmp/gemma4-12b-asr-polish/litwo-ojczyzno-moja-16k-mono.wav \
    --model large-v3-turbo \
    --language Polish \
    --task transcribe \
    --output_format txt \
    --output_dir /tmp/gemma4-12b-asr-polish \
    2>/dev/null

[00:00.000 --> 00:03.660]  Litwo, ojczyzno moja, Ty jesteś jak zdrowie.
[00:04.340 --> 00:08.240]  Ile Cię trzeba cenić, ten tylko się dowie, kto Cię stracił.
[00:09.480 --> 00:14.960]  Dziś piękność Twą w całej ozdobie widzę i opisuję, bo tęsknię po Tobie.

If you want to hear what it sounds like, there is a recording published on YouTube.

The baseline post is still Markdown. The blog still uses Hugo.

Math

KaTeX is now enabled for posts that need formulas embedded. A useful test is the scaled dot-product attention equation from the Transformer paper:

$$ \operatorname{Attention}(Q, K, V) = \operatorname{softmax}\left(\frac{QK^\mathsf{T}}{\sqrt{d_k}}\right)V $$

Diagrams

Mermaid is now enabled and rendered.

flowchart LR
  tokens[Tokens] --> embed[Embeddings]
  embed --> attention[Self-attention]
  attention --> norm1[Add + norm]
  norm1 --> ffn[Feed-forward]
  ffn --> norm2[Add + norm]
  norm2 --> output[Output states]

Charts

I will be using Chart.js to render charts; that should work better than screenshots because the charts can reflow across screen sizes and support tooltips.

The BirdNET detections are sourced from my BirdNET setup.

Sketches

p5.js can be used for small interactive sketches.

Typst to SVG

I have enabled Typst to SVG experimentally. I will figure out if I need it going forward.

Implementation details

The post opts into browser-side features from front matter:

[features]
math = true
mermaid = true
chartjs = true
p5 = true

The theme partials read those flags and only load the libraries the page asks for. The body uses shortcodes for the parts that need structure. Ordinary posts do not load KaTeX, Mermaid, Chart.js, or p5.js by default.

Typst is handled differently. I keep the .typ source in assets/typst/, compile it to SVG, and reference the generated file from static/images/. That makes the figure version-controlled without adding runtime JavaScript to the page.

The idea was inspired by Maxime Guerreiro’s post about his agent workflow: How I use agents to write this blog.

The software stack I use is based on BirdNET-Go, which is a bird vocalization classifier and dashboard. Under the hood it uses BirdNET-Analyzer and the BirdNET 2.4 model. In broad strokes, BirdNET splits 48 kHz audio into three-second segments, converts them into log-mel spectrograms, and runs a convolutional neural network over those images to estimate species probabilities, with location and date used as extra context for filtering the results.

My setup is split across two Raspberry Pis. The outdoor Raspberry Pi 3 A+ captures audio from a Clippy EM272Z1 microphone through a Ugreen US205 USB sound card and publishes a 48 kHz mono LPCM stream over RTSP. An indoor Raspberry Pi 4 runs BirdNET-Go, pulls that stream over RTSP/TCP, exports clips, and serves the web dashboard.

Hardware

Before building a BirdNET station yourself, I strongly recommend reading the BirdNET-Go hardware recommendations from the project authors. I followed that page when choosing the microphone, and it is a much better starting point than guessing your way through Raspberry Pi models, sound cards, and microphones.

The outdoor half lives in an IP54 weatherproof box on the balcony, powered through an IP44 extension lead. It is still a practical build rather than a polished one: the Raspberry Pi, power, sensor, and cable routing all share the same enclosure, because getting a reliable microphone outdoors mattered more to me than making the first version elegant.

The microphone itself is mounted under the balcony ceiling and connected back to the enclosure with a 3.5 mm extension cable. This is still a temporary setup, with electrical tape standing in for a more elegant fixture, because part of the exercise was simply to test microphone placement before committing to something cleaner. There is still work to be done there.

The windshield matters a lot. It improves outdoor audio quality tremendously. Pro tip: use double-sided adhesive tape to attach it to the microphone. The first time I skipped that, the wind carried it off.

Inside the box, besides the Raspberry Pi and audio hardware, I also keep a Govee H5179 thermo-hygrometer. I already had it at home and it had very little use before, so it became enclosure telemetry. One of the obvious risks of keeping electronics outdoors is condensation, so I export the Govee readings into a local Prometheus and watch the relative humidity metrics in my local Grafana instead of guessing.

I also keep about 300 g of silica gel sachets inside. In practice they pretty much saturate after one heavier rainfall, so I treat them as temporary help rather than a long-term answer. A more permanent fix would be a small heating element, but I decided against that because I did not want to run extra wiring into the enclosure.

Another upgrade I want to make before summer is better cooling for the outdoor Raspberry Pi. Right now it only has the small heatsink that came in the Botland kit, but my early estimates suggest I may not be able to keep the CPU below 70 °C in summer without switching to a larger heatsink or adding a fan. On the Raspberry Pi 3 A+, that is also the maximum configurable temp_soft_limit, beyond which the CPU is throttled down to 1.2 GHz.

Bird Detections

The variety is what it is. This is still a microphone on a ninth-floor balcony in a densely populated area, so I am not expecting spectacular species diversity. Even so, I think the results are great for this location, especially around dawn.

Bill of Materials

This is the BOM for the outdoor side. The Raspberry Pi 4 that runs BirdNET-Go also runs all my home automation.

• Raspberry Pi 3 A+ kit: Set of Raspberry Pi 3A+, original case and 5 V / 2.5 A power supply
• Outdoor extension lead: Brennenstuhl outdoor extension lead
• Enclosure / outdoor cable box: Outdoor cable box, IP54
• 3.5 mm audio extension cable: UGREEN 3.5 mm extension cable
• USB sound card: Ugreen US205
• Microphone: Clippy EM272Z1 Mono Microphone
• Silica gel sachets: Wisedry silica gel sachets
• Thermo-hygrometer: Govee Wi-Fi Thermo-Hygrometer, model H5179

At current prices, that outdoor-side BOM comes to roughly 739 PLN before shipping.

• Andres Freund’s email to Openwall mail list, that publicly disclosed the vulnerability
• Bash Obfuscation Explained from gynvael.coldwind.pl
• two pieces by Russ Cox: Timeline of the xz open source attack and The xz attack shell script
• Filippo Valsorda’s thread on Bluesky

I found Russ Cox’s Timeline of the xz open source attack to be particularly informative on the social / human aspects of the operation. Here is an excerpt:

2022-05-27: Jigar Kumar sends pressure email to patch thread. “Over 1 month and no closer to being merged. Not a surprise.”

2022-06-07: Jigar Kumar sends pressure email to Java thread. “Progress will not happen until there is new maintainer. XZ for C has sparse commit log too. Dennis you are better off waiting until new maintainer happens or fork yourself. Submitting patches here has no purpose these days. The current maintainer lost interest or doesn’t care to maintain anymore. It is sad to see for a repo like this.”

There were also pieces published explaining the problem to the general audience like this one by The Economist.