Building a voice companion that runs on your Mac

Parley is my voice-first companion project for Apple silicon Macs. Its published design brings speech recognition, a local language model, speech synthesis, and an avatar together on the same machine, with optional cloud use.

That makes a simple question useful: what should count as a better conversation?

Start with the whole interaction

For this kind of project, I would judge an improvement from the moment someone finishes speaking to the moment they hear a useful response. I would also check what happens when they interrupt, change direction, or choose to use a cloud model. Those are evaluation criteria, not claims that every part of Parley has already passed them.

A language model is one part of that experience. Optimizing it is worthwhile, but I would be careful about borrowing a performance number from a different workload and putting it on the product page.

Ollama makes that distinction in its June discussion of multi-token prediction: the benefit depends on the workload, and benchmark selection can strongly influence the result. That is a useful constraint when reading any proposed speedup for a conversational application. Ollama’s explanation.

Treat engine updates as something to test

Ollama’s September 14 release notes list an improvement to MLX memory handling on Apple silicon. The notes do not establish a particular improvement for Parley, or say that this project has adopted that release. Ollama v0.34.1 release notes.

My takeaway is to make engine changes measurable at the application level. A useful comparison would keep the machine, model, and prompts consistent, then record the first audible response and the behavior of the rest of the companion. A promising release note can justify that comparison; it cannot supply its result.

Make the choices understandable

For Parley, local-first describes the design direction. It should not become shorthand for a guarantee that every possible integration works offline or that no action can send data elsewhere.

I want the distinction between local work and optional cloud work to be clear enough that someone can make an informed choice. The same standard applies to progress reports: explain what was designed, what was observed, and what still needs a real test.

That is the standard I want this journal to meet: small engineering questions, retained evidence, and claims that fit what the evidence actually says. Follow the Parley journal for the next note.