Apple silicon · Local by default

The companion is already there.

A voice-first companion that lives on your Mac.

A little company. Right where you work.

Character concept

Your voice stays on your Mac. So does Parley.

Wake word, speech recognition, the language model, text-to-speech, lip sync, vision and the renderer run together on Apple silicon.

The cloud is optional and explicit. Turn it on for a bigger model. Leave it off and audio, faces, transcripts and replies stay local.

How it works

Six stages between hey and an answer, streamed so you hear the first sentence before the model finishes the last.

Wake

A custom openWakeWord model trained on the phrase itself. Two syllables, plosive onset, rare in ordinary speech.

hey_parlay.onnx · ~200 KB

Hear

Silero VAD finds the end of your thought. NVIDIA Parakeet on MLX transcribes it on the Mac GPU.

Parakeet MLX · Apple silicon

Think

A local model through Ollama, with 16 turns of history and an emotion tag per reply. Sentences stream out as they are generated.

qwen3 · llama3.1 · optional cloud

Speak

Kokoro TTS starts on the first clause. Echo-cancelled input means you can interrupt mid-sentence and it stops and listens.

Kokoro ONNX · VPIO AEC barge-in

Move

Audio drives 52 ARKit blendshapes at 60 Hz through LAM Audio2Expression. Emotion shapes the brows; audio shapes the mouth.

LAM · 52 shapes · 60 Hz

Act

Requests that touch your data go through an authenticated broker, get previewed aloud, and wait for a spoken yes.

approve · always · deny · stop

Measured, not promised

Numbers from the running stack.

Every figure below comes from a live run on a MacBook Pro, logged and kept as evidence in the repo.

60fps: Real-time PBR renderGoogle Filament on Metal, composited into a transparent desktop overlay.
315×540: Native desktop presenceA borderless, click-through window stays above ordinary Mac apps without taking focus.
1.8s: To the first spoken wordWarm turn on a 24 GB MacBook Pro, wake word to audio out.
52: ARKit blendshapesFull facial rig, driven live. Twenty baked full-body motion clips cycle underneath it.
15Hz: Face-aware presenceYOLOv8n-face plus emotion recognition on CoreML. The avatar looks toward you and reads the room.
0bytes: Sent to the cloud by defaultAudio, transcripts, faces and replies stay on the device unless you switch a cloud model on.

Designed for the Mac you already own

Nearby while you work. Yours when you want it.

Full-body Parley character concept standing beside a Mac workspace — Companion concept · Illustrated workspace

Nearby while you work

One DMGDeveloper ID signed, notarized, stapled. Drag to Applications, answer the system prompts, done.

Does not steal focusThe borderless overlay is click-through, transparent and always on top.

Metal rendererThe C++ Filament renderer draws the live avatar directly on your desktop.

Local voice loopWake word, speech, reasoning and motion run together on Apple silicon.

Make it yours

Drop in an avatarAdd a GLB or VRM from Finder. Parley inspects it before switching.

Honest capability badgesFULL, LIMITED and BODY tell you exactly what each avatar can do.

Rollback safeA bad import leaves your current avatar running.

One place to control itReadiness, action status, privacy choices and Quit stay in the menu bar.

It sees you, and it knows when to stop talking.

A camera feed runs face detection, tracking and emotion recognition. Walk up and it turns attentive. Talk and its gaze follows you. Frown and the expression shifts.

Playback is echo-cancelled at the OS level, so it can hear you over its own voice. Interrupt it and the sentence stops, your words become the next turn.

PresenceIdle to attentive on a face in frame. Greeting is opt-in.
GazeTracks the speaker from bounding boxes on the vision channel.
Barge-inConfident speech interrupts. Its own echo and backchannels are rejected.

It asks before it acts.

Anything that writes to your life goes through an authenticated action broker with one run at a time, a durable journal, and a spoken preview. Nothing personal executes without a yes.

You“Remind me to call Mom at three.”

Parley“I’ll add ‘call Mom’ to your default Reminders list. No due date is set. Approve?”

You“Always.”

Parley“Done, your reminder is set.”

StopA reserved lane. “Stop” reaches a live run from any state, never gated behind the queue.
Fail closedUnsure means no. A broker that cannot answer never dispatches.
Grants“Always” is remembered per exact tool name, never per family.

Under the hood

Three languages, one wire.

Every service is a process on a Redis pub/sub bus speaking one typed envelope. Python orchestrates, C++ renders, Swift owns the Mac shell. The contract is swept across all three before anything ships.

Orchestrator	Python · FastAPI · asyncio state machine	Idle → Attentive → Listening → Transcribing → Thinking → Speaking, with per-state timeouts and a priority lane for stop.
Transport	Redis pub/sub · typed EventEnvelope	orchestrator:state, avatar:command, avatar:blendshapes, vision:faces, orchestrator:tts and friends.
Renderer	C++ · Google Filament · Metal	glTF with morph targets, transparent desktop composition, blink and breathing controllers, crossfaded skeletal clips.
Lip sync	LAM Audio2Expression · 52 ARKit shapes	Facial motion at 60 Hz. Emotion presets merge on top of the mouth.
Speech	openWakeWord · Silero VAD · Parakeet MLX · Kokoro	Custom wake model, Apple silicon ASR and ONNX TTS with first-clause streaming.
Reasoning	Ollama · OpenAI-compatible API	Local by default. Capped history, farewell-clears, emotion tags parsed from the reply.
Vision	YOLOv8n-face · enet_b0 emotion · CoreML	Faces, tracks, gaze targets and moods on a 15 Hz channel.
Actions	Authenticated broker · HMAC-signed v1/v2 wire · journal	Two client roles, one-run lock, approval tier by category, standing grants by exact name.
Mac shell	Swift · SwiftUI · SMAppService	Menu-bar app, drag-drop avatar ingest, live approval panel, signed bundle with helpers inside.
Ops	launchd · signed app bundle · Prometheus	Health board, retention CLI, sanitizer gate for the renderer, live-proof evidence per ticket.

A separate Jetson edge-runtime path remains available for specialized deployments. Parley for Apple silicon is the primary product and the experience shown here.