Menu Data is Like a Box of Chocolates

Something you may not expect as you drive up to a beloved nationwide drive-thru restaurant chain: the data that sits behind that light-up menu board is as unstructured and chaotic as this kid’s dream.

This problem obviously doesn’t flow through to the food – have you ever seen the Krispy Kreme drive-thru lines when the Hot Light is on?

The data comes in as a JSON or XML. You’ll get duplicate items with different names. Inconsistent combo structures. Modifier trees that make no sense. Item IDs that read like OpenClaw agents spoke to each other a few too many turns. And because a POS menu has to serve the customer-facing website, the staff-facing handheld, the back-of-house display, and now an AI that needs to actually understand what a "KK OG GLZD 12CT" is, there's no clean version of this data at the source.

This matters because when a customer at the drive-thru says "can I get a dozen of the original glazed," the AI needs to map that to a system that was never designed to talk to an AI. And if Krispy Kreme drops a limited-time Artemis II donut, the AI needs to know what it looks like, what it's called, and that it exists at all when someone drives up and orders “the new blue donut.” That context is crucial, and is part of what makes voice AI ordering an especially tricky problem.

The menu data problem became apparent immediately, and we realized it was going to be a major deterrent to our ability to scale as a company. Without hesitation, a few engineers started writing code. Our engineers are given the trust and freedom to step back and solve the important problems – even if it means using resources differently in the short term.

What came out the other side is an agentic menu integration pipeline. It ingests raw POS data (line items like "POP CRSPY CHKN SNDWCH CLS"), runs it through an LLM to translate it into natural language, and maps that back to the ordering system when a customer says "I'll take a crispy chicken sandwich."

But translation is only half the problem. POS menus were designed for cashiers who already know what the buttons mean, not for AI. A 2-piece tender and a 3-piece tender show up as completely separate items when they should be one item with a quantity modifier. The pipeline restructures all of that too, turning menus into something an LLM can actually reason about.

Importantly, the work they did will allow us to scale voice AI operations at any brand, regardless of the state of their menu data.

It's on track to become our primary menu integration method, cutting weeks off the restaurant onboarding process. All the stuff that used to require days of manual data cleaning (“whack-a-mole” style), we can get to a pre-pilot-ready product in one day. When you're onboarding enterprise brands with thousands of locations, or platforming a long tail of smaller chains, you simply cannot have humans hand-translating every menu.

The fact that this system exists at all is because we have an engineering culture where people are encouraged to just build the thing that needs to exist. An engineer who loves improving data quality and saw a need for it in our business said, "this is important," opened a repo, and started shipping. Another moonshot for the books.

If you’re an engineer who wants to ship more moonshots, we’re hiring.

If you're a platform, kiosk provider, or restaurant tech company and you're tired of hand-translating POS data for every new integration, reach out. We built this so you don't have to.

This is Part 2 of our series on engineering culture at Deepgram for Restaurants. [Read Part 1 here].