← prabs.xyz

I Built a Fully On-Device AI Coach on Apple Watch

August 13, 2026

Most AI experiences on the Apple Watch are just thin clients. You speak, the request goes to the phone or the cloud, and a response comes back.

I wanted to try something different: a small agent that actually runs on the Watch itself.

What I Built

An interactive readiness coach.

You raise your wrist or say “How am I?”
The app pulls data from HealthKit (resting heart rate, HRV, sleep, recent activity), runs a tiny language model on-device, and returns a structured assessment:

  • Readiness level and score
  • Key signals
  • A short recommendation
  • Suggested follow-up questions

You can then ask follow-ups like “Why?” or “What should I do instead?” and get another structured response — still fully on-device.

The whole interaction stays short, glanceable, and private. Health data never leaves the Watch.

Demo

How It Works

Here’s the simple flow:

User (Voice / Raise wrist)
        │
        ▼
┌───────────────────┐
│   Watch App UI    │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│  Cactus Needle 2  │  ← on-device model
│  (Tool Calling)   │
└─────────┬─────────┘
          │
          │  tool calls
          ▼
┌───────────────────┐
│     HealthKit     │
│  • Resting HR     │
│  • HRV            │
│  • Sleep          │
│  • Activity Load  │
└─────────┬─────────┘
          │
          │  results
          ▼
┌───────────────────┐
│  Cactus Needle 2  │
│  Structured JSON  │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│   Watch App UI    │
│  (Card + Follow-ups)
└───────────────────┘

The model decides which HealthKit tools to call, receives the data, and must return a strict JSON response. The UI only renders that structured output — it never has to parse free-form text.

The Model: Cactus Needle 2

I used Cactus Needle 2, a 45M-parameter model designed specifically for tool calling, device use, and structured extraction.

Key numbers:

14 MB binary ~28 MB session RAM Trained for tool use and structured output

It is small enough to realistically run on an Apple Watch while still being reliable at selecting tools and returning clean structured data.

Why This Approach

Running even a small agent on the Watch comes with tight constraints: memory, battery, screen size, and interaction patterns. Using a model built for tool calling + structured output turned out to be the right fit. It doesn’t need to be a great general conversationalist. It just needs to:

Decide which health signals to fetch Interpret them simply Return clean, predictable output Support a couple of follow-up turns

That combination feels much more useful on the wrist than a generic chatbot.

The plot twist:

the textbook “let the model call 4 tools then emit structured output” failed on-device. Logs showed Needle calling one tool then dropping to prose, and refusing to produce a score at conf=0.000000, because its spec says it only emits values evidenced by the input. It’s an extraction model, not a reasoning model; a recovery score is a judgment, so it correctly won’t invent one.

The fix:

split by strength - score computed by a transparent deterministic heuristic; Needle does the one thing it’s great at (grammar-constrained structured extraction + short summaries); a fallback guarantees a complete card. The generalizable lesson: with tiny on-device models, do the logic in code and use the model for the narrow thing it’s good at.

Gotchas:

arm64_32 vs arm64 linking, needing -lc++ for the C++ lib, bundling the .cact as a file (no download), and the joys of wireless-debugging a watch through office firewalls.

Main Takeaways

Tiny specialized models can be more practical than larger general ones for constrained devices. Structured output is essential when the screen is small and the interaction must stay fast. Short multi-turn sessions work better than trying to maintain long conversations. Keeping everything on-device makes the privacy story straightforward and removes network dependency.

This started as an experiment to see whether a real agent loop could feel native on the Apple Watch. The result is limited but surprisingly usable — and it runs entirely on the wrist.