Blog

One mistyped field hid a third of the games

By Vivek Tanna ·

In August, baseball disappeared from The Lineup Is. Not the scores, the whole sport. The widgets still showed baseball games. Live Activities still showed them. The app itself acted like baseball didn't exist, and the sport filter menu only listed the sports that had survived.

There was no error screen, no empty state, no crash. About a third of the day's games were simply gone. Basketball was gone too, and roughly half of soccer.

The cause: one field, two types

Each game carries a small nested record about the two teams' recent meetings. One field in it describes a run of results, the kind of thing you'd display as "won the last three."

On the server, that value was computed and stored as a signed integer, because server-side code did arithmetic with it. On the iPhone, the Swift model declared it as a non-optional String. Two halves of the same product disagreed about one field's type, and nothing checked.

The decode used decodeIfPresent(String.self, forKey:). That looks safe, and it is safe against a missing key or a null. It is not safe against the wrong type. When the value is a number, it throws DecodingError.typeMismatch.

That throw propagated up through the nested record and failed the entire game. My list decode was already tolerant at the row level: it wrapped each row, kept the ones that decoded, and logged the rest as a quiet breadcrumb. That tolerance is what made the bug invisible. Instead of a failed request I could see, I got a successful response with a third of the rows missing.

Why whole sports? Because the field was only populated for some of them. Sports where it was null decoded fine. Sports where every game had a number lost every game.

Why widgets were fine

The widget fetches a narrow, explicit list of fields and never asks for that nested record. It never decoded the bad value, so it never dropped a row. "Widgets show it, the app doesn't" turned out to be the fingerprint of a decode failure. When two surfaces disagree, compare what each one requests before anything else.

How I found it

Reading the struct and the JSON side by side didn't find it. I looked at both more than once and missed it. What worked was compiling the real model files against a real captured payload in a small command-line program and printing DecodingError.codingPath. That points at the exact key, nested several levels deep, that killed each row.

The fix, in three parts

1. Accept both shapes on the client

The field now decodes as a string or an integer, and converts the integer into the display string. An unexpected shape becomes an empty value instead of a throw.

static func lenientString(
    _ c: KeyedDecodingContainer<Keys>, _ key: Keys
) -> String {
    if let s = try? c.decodeIfPresent(String.self, forKey: key) {
        return s
    }
    if let n = try? c.decodeIfPresent(Int.self, forKey: key) {
        return String(n)
    }
    return ""
}

2. Fix the server, so shipped apps recover

A client fix only helps people after they update. The server half is what repaired the app already in the App Store. The server now writes the value as a string, and every server-side reader accepts both shapes. I found one of those readers only by searching the codebase for the field name, not by trusting my earlier trace.

Fixing the writer wasn't enough either. The server carries an existing value forward on each update, so stored rows kept their integers. I had to normalize the value at the points where it gets copied forward, and explicitly mark finished games as dirty, because a finished game stops changing and would otherwise never be rewritten.

3. Make most fields unable to delete a row

Then I measured how bad this could get. I fuzzed every field in the game payload across every sport, swapping in wrong types, and counted which ones deleted the whole row. The large majority could. Most of them were enrichment: nice-to-have details where a bad value should degrade one label, not erase the game.

So I split fields into tiers. Identity fields, the ones without which a row means nothing, stay strict and still drop the row. Fields the card needs to render stay strict. Everything else decodes through one helper that returns a fallback on missing, null or wrong type:

static func tolerant<T: Decodable>(
    _ type: T.Type, _ key: Keys,
    from c: KeyedDecodingContainer<Keys>, fallback: T
) -> T {
    (try? c.decodeIfPresent(type, forKey: key)) ?? fallback
}

// let score = Item.tolerant(Int.self, .score, from: c, fallback: 0)

After the migration, no nested field and no enrichment field can drop a row. The only ones left that can are the identity and render fields that should. A test locks those counts in, and another test makes sure the identity fields stay fatal, so nobody "fixes" the next bug with a blanket try?.

I also stopped dropping rows quietly. A mass drop now sends a real error event that names the failing key path, and a sport going to zero while rows for it exist is escalated and named. Replayed against the original payload, the alert says exactly which sports were wiped and which key did it.

Rules for decoding data you don't control

One more lesson, about process rather than Swift. I diagnosed this, wrote the fix, verified it, and then didn't commit the source. Only the tests landed. My notes said "fixed" for six days while the bug was still live, until two soccer season openers were missing from the app. Now a fix isn't fixed until it's merged, deployed and checked against production.

I build The Lineup Is, an iPhone app that tells you which games are worth watching.

Get The Lineup Is on iOS — free — or see tonight's best games ranked right now. More from the blog.