,

Marginally Better S01E24: The Wrong Metric

Marginally Better

Every business has a number it stares at — average handle time in the call center, table turns in the restaurant, door-to-doctor minutes in the ER. Put it on a dashboard and watch it every Monday, and over time a substitution sneaks in: the number stops being a way to see the customer and starts being the customer.

In this episode of Marginally Better, Joe Taylor, Jr. looks at what happens when the metric on the dashboard isn’t the one customers are grading you on: the emergency rooms that spent decades timing themselves while patients were counting explanations, the Ritz-Carlton executive who spent three months paralyzed in a hospital bed and then got handed a mandate to fix how twenty-one hospitals feel, and why your flight home is officially more on time than it was in 1990 — even though it takes longer.

Episode Links:

The Stopwatch

The Feature: Sven Gierlinger and Northwell Health

The Counterweight: The On-Time Machine

Background Reading

Products Mentioned


Transcript:

[00:00:00] In 2000, a rising executive with the Ritz-Carlton hotel company — a man whose whole career was teaching people how to deliver five-star service — felt a tingling in his fingers and toes.

Within days, he was paralyzed.

He spent the next three months in a hospital bed. Intubated. Fed through a tube. For ninety days and nights, by his own count, he was one hundred percent dependent on the doctors, nurses, and therapists around him. The man who wrote service standards for a living was now on the receiving end — and from that bed, he couldn’t help but take notes.

What he noticed would eventually change how one of the largest health systems in America does business. Because it turns out the thing patients were grading their hospitals on was not the thing the hospitals were measuring.

Today: the number on your dashboard that your customers do not care about. Also on the show — the emergency room, the most stopwatch-obsessed room in America. And why your flight home is officially more on time than it was in 1990, even though it takes longer.

That’s all coming up, on Marginally Better.

Every business has a number it stares at. Average handle time in the call center. Table turns in the restaurant. Pick the number, put it on a dashboard, watch it every Monday — and over time, a substitution sneaks in. The number stops being a way to see the customer, and starts being the customer.

Nobody has lived this harder than the American emergency room. So that’s where we’ll start.

For decades, the ER has been measured in minutes. The Centers for Medicare and Medicaid Services — Medicare’s parent agency — publicly reports how long patients spend in each hospital’s emergency department. Anyone can look it up. The Joint Commission, the body that accredits most American hospitals, tracks a measure called ED-1: the median time from the moment you arrive in the emergency department to the moment you leave for an inpatient bed. There are companion measures for how long you wait to see a provider, and for the percentage of people who give up and leave without being seen at all.

And let’s be fair about why. Those clocks exist because crowded emergency rooms are dangerous. When an ER backs up, treatment gets delayed, sick people walk out the door untreated, and patients get parked in hallways — the industry calls that boarding — sometimes for days. Time in an emergency room is not a trivial thing to watch.

So hospitals watched it. They hired consultants for it. They built patient-flow war rooms around it. Door-to-provider. Door-to-doc. Arrival-to-departure. An entire vocabulary of minutes.

Which raises an uncomfortable question that almost nobody thought to ask: when a patient goes home and decides how they feel about your hospital — are minutes what they’re counting?

Somebody did ask. In December 1996, a team led by Dr. David Thompson published a study in the Annals of Emergency Medicine. They called over sixteen hundred patients who’d been through a suburban community hospital’s emergency department and asked about the visit. How long did you wait? How long did it feel like you waited? Did anyone explain what was happening? Were people courteous? And overall — how satisfied were you?

Then they lined the answers up against the clock. And the actual waiting time — the real, measurable minutes those patients sat there — had no meaningful relationship to how satisfied they were.

What did predict satisfaction?

Whether the wait felt shorter than expected. Whether somebody explained the delays and the procedures. Whether the staff came across as caring. The researchers’ conclusion, nearly thirty years ago now: managing perceptions, information, and expectations may do more for patient satisfaction than cutting the actual wait.

And that finding wasn’t a fluke. In 2017, a team at Massachusetts General Hospital published a systematic review in the Journal of Patient Experience — 107 studies of what drives emergency department patient experience. The theme that came up most often, in 78 of those 107 studies, was communication between staff and patients. Wait times came second, at 56. Empathy and compassion showed up in 45.

The stopwatch wasn’t wrong, exactly. It was just answering a different question than the one patients were asking. Rick Evans, the chief experience officer at NewYork-Presbyterian, put it flatly in a Becker’s Hospital Review column just this past May: “At its heart, improving the patient experience is all about communication.”

So how does a whole industry end up staring at the wrong number for decades?

Turns out there’s a name for this. In 1975, a British economist named Charles Goodhart was writing about monetary policy — nothing to do with hospitals — and he observed that any statistical pattern tends to fall apart the moment you start using it as a target. An anthropologist named Marilyn Strathern later boiled that down to the version people quote today: when a measure becomes a target, it ceases to be a good measure.

Goodhart’s Law, they call it.

And once you know it, you see it everywhere. A 2021 editorial in the Journal of Graduate Medical Education walked through the medical versions: residency programs coaching trainees on how to answer the survey that’s supposed to catch overwork. Med students cramming for one licensing exam score because program directors screen on it — at the expense of the actual skills the score was supposed to represent. The number gets better. The thing the number stood for doesn’t.

Hospitals could hit the time targets and still send home patients who felt ignored and forgotten — because nobody was measuring whether anyone explained anything.

Which sets up the question at the heart of today’s show: if the clock isn’t the metric that matters — what happens when a major health system bets on the one that is?

When we come back: a luxury hotel executive loses the use of his entire body, spends three months as the most demanding kind of customer there is — a helpless one — and then gets handed twenty-one hospitals and a mandate to fix how they feel. That’s next. It’s Marginally Better.

Quick word about something our team built.

Today’s show is about the gap between what your dashboard says and what your customers actually feel. If you’ve ever suspected that gap exists on your own website — checkout numbers that look fine while the inbox fills with confused customers — that’s exactly the kind of question Experience Helpdesk was made for.

It’s our subscription helpdesk for user experience. You send us the question — why do people abandon this form, what should this page say, which of these two designs works — and a working UX specialist sends back a real answer, fast. No discovery calls. No six-figure engagement. Just expert eyes when you need them, plus a library of guided playbooks covering the most common customer experience fixes.

Come see it at experiencehelpdesk.com. That’s experiencehelpdesk.com.

It’s Marginally Better. I’m Joe Taylor Jr.

Sven Gierlinger spent the first act of his career with the Ritz-Carlton Hotel Company, helping open hotels in Germany, Japan, Indonesia, and the United States. His job was the famous Ritz service culture itself — training staff, installing standards, teaching teams how to anticipate what a guest needs before the guest says it. Years later he’d tell Becker’s Hospital Review what that time taught him: what guests really want is, quote, “much more than crystal chandeliers, marble floors and fancy ocean views. It is the consistency of service delivered through highly engaged and empowered people that make the difference.”

In 2000, Gierlinger was a father of two young children, just starting a new Ritz-Carlton assignment, when the tingling started. Fingers first. Then toes. Within days, he was paralyzed.

The diagnosis was Guillain-Barré syndrome — a rare condition where your own immune system attacks the wiring of your nervous system. It’s usually temporary. But the recovery is slow, and in the meantime, your body simply stops taking instructions.

He spent three months hospitalized. He was intubated — a machine breathing for him — then extubated, then fed through a tube, then slowly worked his way back. Liquids. Soft foods. Solids. “Suddenly, and for 90 days and nights,” he said later, “I was 100% dependent on my doctors, nurses, and therapists.”

Think about who was lying in that bed. A man professionally trained to notice every detail of a service experience, now completely unable to leave one. And the details piled up. The food, by his own account, was atrocious — he lived on protein drinks he could barely stand, while his wife, already raising their two kids alone, hauled meals in from home and from restaurants. This, at what was supposed to be the most caring institution a person ever encounters.

Years later, though, he called the whole ordeal a gift. “Believe it or not, I’m actually grateful for this experience,” he told Becker’s, because it showed him that a patient is trying to heal in three ways at once — physically, mentally, and emotionally — and that service and empathy, embedded into the clinical routine, can make or break all three.

The hotel man had seen the product from the wrong side of the counter. Fourteen years later, somebody handed him the keys.

In 2014, Northwell Health — the largest health system in New York State, more than twenty hospitals across the New York metro area — created a brand-new job: chief experience officer. The first person to hold it was Sven Gierlinger.

One of the first things he did was read what patients were actually writing on their surveys. Not the scores. The words. And on the subject of food, the words were brutal. “Every meal brought to me was unappetizing.” “People need good food to get better.” And the one that became a headline on Northwell’s own website: “Food was atrocious. Not fit to feed my dog.” The system was serving more than ten million meals a year, its food scores sat at the ninth percentile nationally — meaning roughly ninety percent of American hospitals scored better — and the CEO, Michael Dowling, was getting complaint letters about the meatloaf.

For a Guillain-Barré survivor who had lived on hospital protein drinks, this was personal. Gierlinger recruited Bruno Tison, a Michelin-starred chef who had never set foot in a hospital, and rebuilt the kitchens — fryers out, can openers out, fresh ingredients and made-to-order menus in. And because fresh ingredients turned out to cost less than processed ones once you stop throwing half of them away, the whole overhaul stayed inside the existing food budget. Food scores went from the ninth percentile to the eighty-fourth.

But the food was the easy problem. It was visible, it was concrete, you could taste the fix. The harder problem was hiding in a different survey line.

Since 2006, every hospital in America has been graded by its own patients through a standard survey — HCAHPS, it’s called. And three of its questions ask about the same thing the ER researchers kept finding: did your doctor treat you with respect, listen carefully, explain things in a way you could understand. In 2015, on that doctor-communication measure, Northwell ranked in the 29th percentile compared with peer hospitals nationally. Put plainly: on the question of whether doctors talked to patients like human beings, about seven in ten American hospitals were doing better.

Now — the standard corporate playbook here writes itself. Blast out a memo. Tie scores to bonuses. Put the number on a dashboard and squeeze. That’s the Goodhart playbook, and we know how it ends: the number moves, the experience doesn’t.

Northwell did something slower. Working with its own medical school and a group called the Academy of Communication in Healthcare, Gierlinger’s office built a course — Relationship-Centered Communication. A day of small groups, live demonstrations, and role play, teaching doctors a structure for the medical conversation: build the relationship, understand what’s actually going on with this person, and manage the plan together. Communication with the patient instead of at them.

Two details tell you why it worked. First — the course was voluntary. No mandate, no penalty for skipping it. Second — the faculty were forty-six of Northwell’s own physicians and clinicians, teaching on top of their day jobs, unpaid. When COVID hit in 2020, they moved the whole thing to Zoom rather than pause it, and added telehealth scenarios, because suddenly every bedside conversation was happening through a screen.

Here’s what the scoreboard looked like after six years.

Between the course’s launch in June 2017 and June 2023, more than thirty-three hundred Northwell physicians and providers went through it. In that same window, the system’s doctor-communication ranking climbed from the 36th percentile to the 58th — from the bottom third of American hospitals to above the middle, against peers who were all trying to improve too. The single biggest gain was on the question “did your doctor listen carefully” — up 24 percentile points. And the measure closest to a business’s bottom line, “would you recommend this hospital,” rose from the 42nd percentile to the 64th. Before the program, four of Northwell’s adult hospitals scored at or above the national median on doctor communication. By late 2023, it was eight.

Now, the honest asterisk — and I want you to notice who supplies it. Northwell’s own team, writing this up in the Journal of Patient Experience, cautions that the course, quote, “may not be the only factor” in those gains. The system overhauled access and care models and its medical records in those same years. That’s a before-and-after story, not a controlled experiment. The people who ran it say so out loud — which, frankly, is why I trust the rest of their numbers.

And there’s a bigger reason to take communication seriously as an intervention, not a nicety. In 2018, the British Medical Journal published results from seven North American hospitals that restructured how care teams talk with families during morning rounds — plain language, families invited to speak first, a read-back at the end to confirm everyone understood. Harmful medical errors fell by about 38 percent. Families understood the plan — and the care itself got safer.

One of Northwell’s volunteer faculty members put the whole philosophy in a single line: “Patients and families don’t experience our caring and compassion until we give voice to it. Our long hours, our sacrifice isn’t interpreted as compassion. What is — are the words we use and the way we use them.”

And in this whole story, nobody ever promised a shorter visit. The fix was never speed. A man who spent ninety days unable to move — with plenty of time to think about what actually reaches a person lying in that bed — bet twenty-plus hospitals on the idea that the metric worth chasing was whether anybody explained anything. And the dashboard followed the conversation. Not the other way around.

So — the question for your business. Somewhere in your operation there’s a stopwatch: the number that’s easy to track, easy to benchmark, easy to report at the Monday meeting. What’s your version of the conversation — the thing your customers are actually grading you on while you’re busy timing yourself?

I want to end at thirty-five thousand feet, with the purest specimen of Goodhart’s Law in American life.

Since the late 1980s, the U.S. Department of Transportation has required airlines to report on-time performance, and it defines “on time” as arriving within fifteen minutes of schedule. Those rankings get published. Airlines advertise them. Careers ride on them.

But there’s a flaw built into the design. The government measures you against the schedule — and you write the schedule.

In 2018, economists Silke Forbes, Mara Lederman, and Zhe Yuan published a study in the Review of Industrial Organization with a title that gives away the ending: “Do Airlines Pad Their Schedules?” They analyzed U.S. flights from 1990 through 2016, and found that scheduled flight times have grown steadily — with the biggest jumps after 2008. Actual flight times grew too. But the schedules grew faster than the flights. So on paper, delays went down — while the same trips, wheels to wheels, take longer than they did in 1990. The Wall Street Journal captured it in a headline back in 2010: why a six-hour flight now takes seven.

And the fifteen-minute line itself warps behavior in the air. Earlier research by Forbes and Lederman found airlines systematically speed up the flights that are projected to land right around that fifteen-minute threshold. Fourteen minutes late counts as on time. Sixteen doesn’t. So the flights that get the extra push are the ones near the line — not necessarily the ones carrying the most stranded passengers.

None of this required anyone to lie. Every departure time, every arrival, every schedule is accurately reported. The airlines simply did what every organization does when a measure becomes a target: they optimized the measure. Your flight lands, the pilot announces you’ve arrived twenty minutes early, and the cabin feels a little glow of winning — a victory over a schedule that was padded precisely so you could beat it.

The metric has never looked better. Your Tuesday has never been longer.

That’s our show. If you take one thing with you this week, take this: your customers are not grading you on your dashboard. The ER patients were counting explanations, not minutes. Northwell’s patients just wanted to know whether anybody listened. The number that’s easiest to measure has a way of replacing the experience it was supposed to represent — and nobody notices the swap until the customers stop coming back. So this week, ask the Goodhart question about your own business: what would your customers say you should be measuring instead?

If the honest answer is “I don’t know what my customers are confused about” — that’s a fixable problem, and it’s exactly what our Experience Helpdesk was built for. Real UX specialists, answering your specific questions about where customers get stuck. Find it at experiencehelpdesk.com.

Thanks for listening to Marginally Better. If something here was useful, send this episode to one person who runs a business — that’s how the show grows, one forwarded episode at a time. And a review wherever you listen helps one more business owner find us.

For show notes and the research behind today’s episode, head to marginallybettershow.com, or follow the link wherever you’re listening.

Marginally Better is a Calufrax production. Our producer is Nicole Hubbard, with research by Connie Evans.

I’m Joe Taylor Jr.