The Problem I Wanted to Solve
I take a lot of voice notes. Quick thoughts while walking, ideas during commutes, fragments of research I want to remember. The problem is they pile up as raw audio files, unsearchable and unorganised. I wanted something that would transcribe them on-device, figure out what they were about, and surface them back to me before I forgot them entirely. No cloud services, no subscriptions, no data leaving my phone.
What I Built
Field Notes (internally called Concept Capture) is a local-first iOS app built with SwiftUI and SwiftData. You record a voice note, it transcribes on-device using Apple's SFSpeechRecognizer, automatically categorises the note into a concept using NaturalLanguage framework embeddings, and surfaces it back through a daily review feed. The entire thing runs on-device with zero backend dependencies.
The app has four main pieces: a capture pipeline (record, transcribe, persist), a smart organisation layer (auto-categorisation into concepts, dynamic collections with rules), an active recall system (Today's Review with a trailing 7-day window), and a full navigation shell with concept browsing, search, and settings.
Decisions That Shaped the Build
I chose SwiftUI and SwiftData over UIKit and Core Data because I wanted the modern declarative approach, and targeting iOS 17+ was an acceptable trade-off for a personal tool. Free provisioning over a paid Developer Account was sufficient since I only need it on my own phone, even with the 7-day re-signing requirement.
The most consequential decision was going fully on-device for intelligence. SFSpeechRecognizer handles transcription without sending audio to Apple's servers. The NaturalLanguage framework provides sentence embeddings for semantic matching, which powers both auto-categorisation and the related content engine. I considered using a Whisper API or Claude for deeper concept matching, but eliminated those to keep the MVP simple and offline-capable. They remain in the post-MVP backlog.
Dynamic collections were a deliberate MVP inclusion, not polish. The idea came from mem.ai -- organisation that maintains itself without manual effort. Collections can use tag matching, recency rules, or semantic similarity rules, and they re-evaluate automatically when new content arrives.
How the Build Went
The project ran across six phases over about a week. Phase 0 was environment setup, wiring Claude Code into the Xcode build toolchain via XcodeBuildMCP. Phases 1 through 3 were the core product: capture pipeline, smart organisation, and active recall. Phase 4 built out full navigation (TabView, concept browsing, settings). Phase 5 was polish, testing, and device deployment verification.
I resolved 17 bugs along the way, several of which were instructive. BUG-010 was a critical one where notes were not persisting across app restarts because the capture pipeline was not wired to a SwiftData ModelContext. BUG-011 was similar -- the CategorisationService was never actually injected into the production pipeline, so auto-categorisation silently did nothing. Both were cases where the service layer worked perfectly in tests but the dependency injection into the real app was incomplete.
BUG-001 was a performance issue: the note embedding vector was being recomputed for every concept in the matching loop instead of being computed once upfront. BUG-002 was a related concern where the embedding cache grew unboundedly. I added LRU eviction at 500 entries.
BUG-012 caught duplicated logic -- a buildConceptText helper was copied across three services with divergence risk. I consolidated it into a computed property on the Concept model itself.
What I Learned
The NaturalLanguage framework is surprisingly capable for on-device work. Sentence embeddings give you usable semantic similarity without any network calls. The categorisation threshold of 0.7 cosine similarity works well in practice -- below that and unrelated notes start clustering together, above it and the system creates too many single-note concepts.
SwiftData is pleasant to work with but unforgiving about threading. Three separate bugs traced back to missing @MainActor annotations on view models or services that interact with the UI. The compiler does not always catch these, so they manifest as silent data races or UI that never updates.
The active recall loop is the feature that makes the app actually useful. Without Today's Review, the app is just a fancy voice recorder. With it, knowledge resurfaces before it fades, which is the entire point.
Final State
The MVP shipped with 27 tasks complete across all six phases, 192 tests passing, and 17 bugs resolved. The app runs on a physical iPhone via free provisioning. Post-MVP, I want to add a conversational interface over the notes using Claude, Notion sync for concept export, and Action Button integration via AppIntents. But the core loop -- record, transcribe, categorise, review -- works reliably today.