Foundation Models against a rule-based parser

What the on-device model does to messy OCR text, the four API edges I cut myself on, and a benchmark you can copy. 24 September 2026.

I have an app that reads wine labels. Vision recognises the text on the device, and about three hundred lines of rules turn that text into fields: producer, name, vintage, region, grapes, alcohol. Regular expressions for the numbers, catalog lookups for the rest. Unfashionable code.

iOS 26 ships a language model on the phone, free and offline, so the obvious question was whether the unfashionable code should go. This is what I found, starting with the parts of the API that cost me time.

Four things worth knowing before you start

1. Check availability, and handle all three refusals

SystemLanguageModel can be unavailable for reasons you cannot fix from code, so every path needs a fallback that is not the model:

let model = SystemLanguageModel(useCase: .general,
                                guardrails: .permissiveContentTransformations)
switch model.availability {
case .available: break
case .unavailable(let reason):
    // .deviceNotEligible — no Apple Intelligence on this device
    // .appleIntelligenceNotEnabled — the user has not turned it on
    // .modelNotReady — downloading, or the device is busy
    return nil
}

2. The default guardrails may refuse your domain

Mine is wine. The default guardrails treat alcohol as sensitive, and a prompt full of it can come back as guardrailViolation. .permissiveContentTransformations, shown above, is the documented escape hatch for transforming user-supplied content, and it is the right one here: the text comes from the user's own photograph.

3. Nothing is reproducible until you ask for greedy sampling

My first runs gave different answers to the same input on consecutive calls, which makes a regression test impossible. Sampling is probabilistic by default:

let response = try await session.respond(
    to: prompt,
    generating: LabelReading.self,
    options: GenerationOptions(sampling: .greedy))

With .greedy the same eight labels produced byte-identical output across runs.

4. representNilExplicitlyInGeneratedContent crashed the request

I wanted optional fields the model could decline to fill. The iOS 26.4 overload looks made for it:

@Generable(representNilExplicitlyInGeneratedContent: true)
struct LabelReading { var producer: String? /* ... */ }

Every request then failed inside the framework, before the model ran:

DecodingError.keyNotFound: Key 'title' not found in keyed decoding container.
Path: properties.producer

Plain @Generable with ordinary optionals works, so that is what I shipped. (Seen on macOS 26.6 with the iOS 26.5 SDK; if it works for you, I would like to know.)

The schema and the prompt

@Generable
struct LabelReading {
  @Guide(description: "The winery or producer. nil unless a line clearly names one.")
  var producer: String?
  @Guide(description: "The wine's own name, with camera misreadings corrected.")
  var name: String?
  @Guide(description: "Grape varieties, in standard spelling, joining words split over two lines.")
  var grapes: [String]
  @Guide(description: "Appellation or sub-region such as Lodi or Cornas. nil if none.")
  var appellation: String?
  @Guide(description: "State or wider region such as California. nil if none.")
  var region: String?
  @Guide(description: "Country in English. nil if it cannot be told.")
  var country: String?
  @Guide(description: "The vintage year ONLY if four digits of a year appear. Otherwise nil.")
  var vintage: Int?
}

let session = LanguageModelSession(model: model, instructions: """
You correct what a camera read from a wine label. Letters are often misread
(BREAT is GREAT, CALIFORNIP is CALIFORNIA) and a word may be split over two lines
(ZIN then FANDEL is ZINFANDEL). Shop stickers and codes belong to no field.
Only report what the text supports. When unsure, answer nil. Never guess a year.
""")

Note the last instruction. The first thing the model did was guess a year — 2022 on one run, 2024 on the next — for a label that carries no vintage at all. Greedy sampling and an optional field stopped that particular invention, but it set the tone for what follows.

The input is worse than you think

Benchmarks on clean strings flatter everyone. Here is what Vision actually returns for one bottle, with its own confidence per line:

0.30  GRLBSE          <- a shop sticker
0.50  Voini Wi        <- the same sticker
1.00  THINK BIG!
1.00  BREAT           <- GREAT
0.50  ZIN        ℿ
1.00  FANDEL     ⅀    <- one word, set across two lines
1.00  ODI-CALIFORNE   <- LODI-CALIFORNIA, misread at both ends
1.00  UAT.66          <- VAT.66

That per-line confidence is the most underused thing Vision gives you, and I will come back to it.

The benchmark

Eight photographs of my own bottles, each recognised by Vision once, then handed to both readers so they saw identical input. The model ran on macOS 26.6, which carries the same on-device model as the phone. One to three and a half seconds per label.

Four to two, one shared failure, and one refusal: the French label came back as GenerationError.unsupportedLanguageOrLocale, "Unsupported language". If your input can be in any language your users photograph, that error is not an edge case.

The errors are the point, not the score

The rules fail by leaving a field empty. You see the gap and you fill it in. The model fails with confidence: "Rabago" and "Citran" are a village and a château, and nothing on either label suggests a grape variety — it supplied one because the schema had a field for it. Spätburgunder is the German name for Pinot Noir, and the "burgunder" inside it was enough to move the bottle to Burgundy, France.

For a journal, a plausible wrong value is worse than a blank one, because the user will not notice it. That asymmetry, not the four-to-two, is why the rules stayed.

What the model was right about was data, not reasoning

Both of its wins were gaps in my catalogs: Susumaniello was not in my grape list, Pays d'Oc not in my places. The model knew them because it has read the internet. That is a fair criticism, and the fix is a bigger list rather than an inference engine — mine went from 127 grape varieties to 206 with local synonyms, and from 108 wine places to 434.

The three failures on the Zinfandel label were fixable in the rules, and the fixes are generic:

• Join adjacent lines. Two single-word lines are tried concatenated, so ZIN + FANDEL resolves. Large type wraps; recognisers report the halves.
• Split on dashes and resolve each part. ODI-CALIFORNE yields California from its second half.
• Use the confidence. A line the recogniser scored below 0.5 is no longer eligible to be a name. That one rule drops shop stickers — the exact trap the model fell into twice.

Lookups are tolerant now: bounded edit distance, one edit for five to eight characters, two beyond, nothing below five, and a tie refused rather than guessed. "Sauvignen Blanc" finds Sauvignon Blanc; "Gamza" does not become Gamay.

What I would take from this

It does not show that models are bad at this. A large multimodal model given the photograph would likely beat both readers — it would see the gold foil Vision missed entirely. It shows something narrower: the free, offline model on the device today, fed the text a phone camera produced, was not better than rules and a good list, and it failed in the more expensive direction.

If you try it anyway — and it is worth trying, the integration is an afternoon — then: build a fixture set from real captures before you compare anything, keep the recogniser's confidence values, set .greedy so your tests mean something, and treat every field the model fills as a claim to be checked against the input rather than an answer.

All eight labels are now regression fixtures in my test suite, stored as Vision's raw output with confidences and boxes, so the next model gets the same exam.

The app is Decanted, a wine journal that keeps everything on the device. If a label defeats it, I would like the photograph — agnevstsikov@gmail.com.