The hard part of running several agents isn’t starting them. It’s knowing which one needs your judgment.

During an agent-driven research session, one of my background agents turned red in its status line and kept retrying. I checked on it, saw it was stuck, and told the orchestrator to launch a plain agent instead. The work went on. Without that signal, I’d have waited on a result that wasn’t coming.

I didn’t need to follow every step of the background work. I needed a signal that something was wrong, access to the details, and a way to intervene. That’s what I mean by driver mode.

My setup uses Pi and Herdr. Below, I’ll walk through what I load into each session, which work I keep visible, and how I check results and keep the workspace usable. The operating principles apply beyond those tools.

This is for people already using agents to write code, do research, or explore data, who want to run several without losing track of the work. Autonomous agents in production need different guardrails and aren’t what I’m covering here.

Move between the overview and the decision

Donald Knuth called it “jumping levels of abstraction”: seeing things at many levels of detail, from high levels down to the lowest. With agents, I need both a view of where the work is headed and access to the specific decision that needs my context.

Think of driving with a map app. In the city, you zoom in to see the next turn. On a five-hour drive, you zoom out to check the route. You might reroute because you know about road construction the map doesn’t. Agents have similar gaps: a product’s cultural nuances, a codebase’s history, or what a research field already knows. An overview helps me notice where that context might be missing; the detailed view lets me supply it.

Two scarce budgets

The model’s context is limited. Tool definitions and instructions loaded up front take space alongside the task. Research shows that longer inputs can degrade a model’s reasoning, even before the context limit. This isn’t an argument against useful context. It’s a reason to avoid loading instructions for capabilities the current task doesn’t use.

My attention is limited too. If every agent’s tool calls demand a read, running another agent gives me another conversation to follow. I need enough status to decide where to look, then enough detail to understand what happened.

A carpenter can leave most of the shop’s tools within reach instead of carrying all of them on a belt. I want the same arrangement for capabilities and detail: less loaded by default, available when needed. Here’s how I apply that to the agents and to the view in front of me.

Start with a small core and load what the task needs

Pi is Mario Zechner’s open-source coding agent. Its default tools are read, write, edit, and bash. Its README puts it simply: “Adapt Pi to your workflows, not the other way around.” A small starting point gives me fewer places to investigate when the agent does something odd.

Installed isn’t the same as loaded. A skill, a set of Markdown instructions, normally appears in the prompt as a short description. The agent reads the full instructions when they’re relevant, or I can make the skill manual-only. Tools can be deferred until the agent searches for one it needs. Project settings let a research repo and a coding repo load different resources.

This lets me keep a capability available without putting all of its instructions into every session. I can ask Pi to make a skill manual-only or configure deferred tools, then review the changes. The agent handles the bookkeeping; I still decide what should be present from the start.

Add capabilities for a problem you actually have. Ask Pi to write an extension and run /reload, or browse the packages on pi.dev. Before installing one, read what it does. Pi’s docs warn that extensions run with your permissions and can access prompts, files, and credentials.

Then apply the keeper test: would you miss it if it were gone? My background-agent extension, goal extension, and cleanup skill each replace work I’d otherwise do inefficiently. That is the reason to keep them, rather than the fact that they are available. Less by default doesn’t mean refusing useful capabilities.

Choose what to observe, then choose your zoom level

Pi deliberately ships without sub-agents; Mario Zechner suggested running another Pi session in tmux instead. I use visible sessions for work I want to follow, and background agents for work I can leave unattended until a result or exception arrives.

Macro: find where your attention is needed. Herdr organizes visible sessions into workspaces, tabs, and panes, and shows whether an agent is working, blocked on a question, or done. An orchestrating agent can start, prompt, wait for, and read other agents through its command-line interface. I can check that view instead of reading each terminal to infer its state.

In the Kolmogorov–Arnold Networks (KAN) research session below, the four agents were named for origins, variants, efficiency, and applications. The orchestrator read their output and consolidated it into one summary, while their sessions stayed available for follow-up questions.

The consolidated output also flagged claims drawn from abstracts or search summaries rather than full-text reading. That gives me a specific place to inspect next. A finished status tells me the agent has returned its work; it doesn’t tell me those claims have been verified.

Micro: inspect what an agent actually did. When a decision needs my judgment, I step into that agent’s pane and inspect its conversation, tool calls, and output. In the research session, that means I can move from the consolidated summary to the searches and source checks behind it. Each visible sub-agent is another Pi session, so I can steer it directly.

In the background. pi-subagents, a community extension, gives me a status line and the results of each run. Code review and evidence checks fit here because I care more about the verdict and its supporting evidence than watching every step. Those tasks can still be consequential. Background is a visibility choice, not a judgment about risk.

That status line caught the stuck agent at the start of this post. It sent my attention to the one run that needed an intervention, while the other work could continue.

Macro and micro are zoom levels for visible work. Background work stays outside that view until I need to inspect it. Either way, the signal tells me where to look, not whether the result is right.

A pane organizes work but doesn’t isolate it. Give parallel writers separate git worktrees so their edits don’t collide in the same checkout.

Check completion and keep the overview usable

Long runs need an agreed definition of done. I use pi-goal-x to keep an agreed goal and completion requirements in view; its /sisyphus mode follows an ordered plan. A separate model audits the evidence against those requirements before the goal can close. This gives me a check that isn’t performed by the agent doing the work. It is still a model, so I keep tests and my own review. The run has its own Herdr tab, where I can inspect it when needed.

The overview also needs upkeep. Finished agents sit idle, panes pile up, and worktrees and notes go stale. If I leave them all there, the status view gets harder to read. I keep a Janitor tab with a cleanup skill that knows my working directories and Herdr setup. It proposes what to close, merge, or archive, and I approve before it acts.

The goal extension gives me a completion check. The Janitor keeps finished work from obscuring active work. They solve different problems, and each passes the keeper test for my workflow.

Own the setup and the output

Updates can break extensions, and no vendor tests my exact workflow. Pi can often help diagnose and repair a broken extension, but the fix still needs checking. Adding a capability also means taking on its maintenance.

The same applies to the work the agents produce. A consolidated research summary still needs source checks. A code review is a verdict to assess, not permission to ship. When I share results with colleagues or use them in something that matters, I am responsible for them. You can delegate the work, but not the accountability.

Try the smallest useful version

A batteries-included setup may suit you better. You don’t need to adopt every piece of mine to test whether this approach helps.

Start with a task you understand well. Run one Pi session inside Herdr, then ask it to start a second beside it. Check two things:

• Can you tell which agent needs your attention without reading both conversations?
• When you step into one session, can you inspect its work and return to the overview without losing track of the other?

If either is difficult, that’s a concrete problem to solve before adding more agents. Add the first capability you actually miss, and nothing more.

There will be more agents making more decisions, faster. Our attention won’t grow with them. Keep capabilities within reach, keep the default view focused, and zoom in where your judgment matters. You’re still the driver.

If this kind of work gets you excited and you want to tackle applied research problems, come build with us at Whatnot. We’re hiring!

Driver Mode: Jumping levels of abstraction with agents was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

On June 8, 2026, newly loaded web clients began showing black screens instead of livestream video. For our platform, that’s a serious problem: it’s pretty hard to sell Pokémon cards that no one can see.

It turned out that a third-party script we use for client-side security monitoring wasn’t bundled with our release, but was rather fetched live from the provider’s CDN. When the provider updated the script all new web clients fetched it, and it included an additional URL validation which our video provider didn’t pass.

The result? Black screens for users of the affected web clients, with most internal dashboards staying green. 3,000 users were impacted in the first 30 minutes of the incident.

One system did catch it. Our end-to-end service-level objective (E2E SLO) monitoring was already in production and paged the owning teams within five minutes. Here’s what it saw.

The real problem

Most of our dashboards stayed green because they monitor component-level health: a server, an endpoint, a specific function. These are all useful, but can all be healthy while the actual user experience is completely broken. This problem gets worse the more external dependencies there are, or the more sophisticated an experience you want to deliver.

In complex, integrated product experiences like ours, a “small” problem can have an outsize impact on the user experience. Measuring this requires a different approach to observability, namely, to model the user journey across multiple surfaces that must be true for a customer to have a good experience.

So how do we measure this in a complex distributed application?

Joining a livestream is not one thing

Joining a livestream sounds like one action, but the user expects at least three things:

• Video to be playing
• Auction details to be shown
• Chat to be visible and up to date

Each of those can succeed or fail completely independently of the other two. Our video depends on third-party providers and CDN network, auctions are handled by one backend service, while chat depends on a different one, both pushing messages asynchronously when auctions start and new messages are sent.

Users need all three parts to render. We set off to make our monitoring measure the same.

Watching the whole trip

Each user scenario follows the same pattern: there’s a trigger, usually an action a user takes, then maybe a few intermediate steps, and finally a set of expected outcomes. In the livestream join scenario, a user tapping a livestream tile is a trigger, server authorizing the user is a step, and finally the user seeing chat messages, running auction, and video playing are outcomes.

We treat each user action as part of a story. A story starts with a trigger and succeeds only if all steps and required outcomes arrive within a fixed time. We decided to encode such “stories” as protobuf messages, each containing a correlation key, allowing us to decide if user attempt is successful or not.

Whatnot already had an infrastructure that handled event reporting from our backend services and clients, which we decided to use here. Now every step of the user journey emits events, which are correlated by the nodes’ key.

All of the emitted events are handled by our events ingestion service and forwarded to Kafka, from where they go to ClickHouse. Using randomized keys with every story allows not only for performant correlation of events related to each individual attempt, but also allows us to deduplicate stories easily, which matters as our event pipeline delivers events at least once.

Every minute, a background worker then finds triggers that have reached a 60-second deadline. It then collects all events with the same story key and checks, based on the schema, whether every required outcome arrived.

Below you can see an example tick, where a period [t-2, t-1) was processed, where we search for all triggers and correlate events forward. Notice how starting with triggers avoids reprocessing stories that were already processed in previous ticks, even if their events overlap with future tick windows.

The background worker then emits story metrics to Datadog. Each minute, every client with at least one trigger contributes one result. That result is successful only if all required outcomes arrive. The SLI is the percentage of successful client results.

Proven in production

Thanks to this system not all dashboards remained green during the June 8 web incident. Because web applications suddenly stopped emitting outcome events related to the video stream, all stories in the incident window come back incomplete.

Because each of our SLOs has a matching burn rate alert, within 5 minutes the story-based SLO turned red and paged owning teams. The responsible teams started the investigation immediately. If it wasn’t for this new capability, we would most likely only learn about the outage from users, which is the worst kind of detection.

And that’s only one example of incidents caught by this technology. At the time of writing E2E SLOs guard 30 core features across many teams, and the adoption is with 1–2 new SLOs added each month.

What production taught us

Building the first version was only part of the work. Production traffic changed how we defined success, counted results, and processed events:

• Measure users, not attempts. A user with a poor connection can retry an action 100 times. If we count every attempt, one user can dominate the SLI. We now produce one result for each client in each minute and use that client’s worst result. The SLI therefore shows the percentage of affected users without hiding poor experiences.
• Treat client telemetry as production infrastructure. The system depends on clients delivering outcome events. We now use local storage, compression, and robust retries in our client SDKs. We also replaced a third-party ingestion path, which provided P95 latency below five minutes, with an internal system that provides P99 latency of one second. One limitation remains: a fully offline client cannot appear in a live graph.
• Use product rules to define success. Some journeys end early for valid reasons. For example, a seller can block a user from joining a livestream. We added additional event roles for these expected failures so that they do not reduce the SLI. We also use a 60-second deadline. If a story takes longer, we count it as a failure because the experience has already failed for the user.
• Plan for real devices and production scale. Well-instrumented stories reach about 99.5% SLI, not the 99.99% that we expect from many backend systems. Devices and client networks add failure modes that we do not fully control. The system now processes hundreds of thousands of story events each minute. This volume is also pushing our minute-based ClickHouse queries toward their limit, so stream processing is our next step.

Back to black screen

The dreaded black screen exposed a gap in component-level monitoring. Async messaging, client rendering, and third-party scripts can all break a journey while individual services remain healthy.

Not anymore. E2E SLOs watch each person’s trip instead of any one system, and nothing stays hidden. Our reliability is no longer defined by infrastructure boundaries, but rather by user and product outcomes, capturing our users experience in real-time.

If building reliable, real-time systems that power live commerce and bring communities together sounds interesting to you, we’re hiring!

All Dashboards Green, All Screens Black was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

There’s a bug we never shipped, and it kept me up anyway. Here’s how it goes: a designer renames a property from purchaseCount to purchase_count inside an animation file. Every place our Swift code references that property is a string literal. The build stays green. Code review passes; nothing in the diff even mentions animations. And the celebration animation at the heart of our live auctions silently stops firing, in production, in front of a live audience.

Nothing stood between us and that outcome except everyone remembering to be careful. And the strings were only the most visible tripwire. Driving our animation runtime correctly also meant knowing things the compiler never checks: call order, load timing, which properties were designer-internal and untouchable, how not to leak memory. Every engineer touching an animation had to know all of it, and every new animation was another chance to get it wrong.

If you’ve done animation work on iOS, some version of this unease is familiar. This post is the story of how we defused it: by generating the entire animation binding layer from the asset files themselves, so the compiler enforces the contract and the generated code encodes the nuance. The methodology isn’t specific to our stack, our vendor, or our app. If your code talks to any designer-owned asset through string literals and tribal knowledge, you can build this too.

Why hand-rolled animation code collapses

Our first instinct, like every iOS team’s, was to build the animations ourselves. SwiftUI has withAnimation. Core Animation has CAKeyframeAnimation. We’d shipped complex UI before. How hard could it be?

The animation we were building wasn’t a loop. It was state-driven: it needed a user’s avatar, their name, a running count that ramps up the visuals, and different behavior depending on who’s watching. Each input changes the intro, the escalation, the outro. In native Swift, that decomposed into:

• State explosion. A @State for the image, a @Binding for the count, a publisher for the mode, a synchronization tangle that grew every time the design changed.
• No iteration loop for designers. Every easing tweak was a code change, a review, and a build. The people responsible for how the animation feels had no way to adjust it.
• One-off everything. Code written for one animation couldn’t be reused for the next. Every animation was hand-rolled from scratch.

The conclusion we kept circling back to: the behavior of an animation should live in a designer-owned tool, and engineering’s job should shrink to binding data to it.

There’s a healthy ecosystem of runtimes built on exactly that idea: Lottie for After Effects playback, Rive for state-machine-driven interactive graphics, and others. We use Rive, so it’s the concrete example in this post, but everything that follows applies to any runtime where a designer-authored asset drives behavior. What sold us was the state-machine model: you don’t play the animation, you drive it. You set inputs, fire triggers, and the state machine decides what happens.

That choice solved the design problem. It’s also where the engineering problem started.

The latent enemy: nuance the compiler can’t see

Out of the box, driving an animation through the runtime’s SDK looks like this:

let instance = riveViewModel.riveDataBindingViewModel?.instance
let count = instance?.numberProperty(fromPath: "purchaseCount")
count?.value = 3

Look at what that code trusts. It trusts that “purchaseCount” exists in the asset, that it’s a number and not a bool, that no designer will ever rename it. Break any of those and you don’t get an error. You get an optional that’s silently nil, or a no-op, at runtime.

But the strings are just the visible half. The invisible half is sequencing and lifecycle: set that value before the asset file finishes loading asynchronously and it’s dropped on the floor. Fire a trigger before the state machine is bound and nothing happens. Touch a property the designer added for internal state-machine bookkeeping and you break transitions with no indication why. None of this is in the type system. All of it lived in heads, Slack threads, and code review comments.

That’s the actual failure mode: correctness lived in people’s memory, and forgetting failed silently. And it isn’t a problem with any one runtime; it’s the same contract gap you hit with localization keys, analytics events, or any SDK where correctness depends on strings plus folklore.

It compounds with scale. With two animation files and one engineer, you keep it all in your head. With thirty files and a whole team, every new feature starts with someone opening the design tool, hand-transcribing property names and types, and hoping they also remembered the ordering rules. There’s no spec, no contract, just string literals, tribal knowledge, and hope.

The insight: the asset already knows everything

The turning point was realizing the asset file isn’t opaque. It’s a structured document that knows every artboard, every state machine, every property name and type it contains. All the information an engineer was transcribing by hand was sitting in the binary, machine-readable.

So: stop writing the binding layer. Generate it from the asset.

The methodology, in one pass:

1. Introspect each asset file and extract its full structure: artboards, state machines, properties, types, defaults.
2. Generate a typed API from that structure: a Swift class per file, a typed property per binding, an enum per artboard.
3. Generate the proof too. The generator knows every property and artboard, so it also emits unit tests for the bindings and snapshot tests for the rendering. Code you generate is code you can test automatically.
4. Hide the SDK behind a small protocol so feature code depends only on generated types, never on the runtime directly.
5. Encode the nuance in the generated layer: load-order handling, lifecycle management, memory hygiene, logging live in the templates once, not in every engineer’s head.
6. Regenerate on every asset change, so the generated code can never drift from the asset.

The compiler becomes the contract enforcer between designers and engineers: rename a property in the design tool and the next build fails at every call site that uses it. And the generated code becomes the keeper of the folklore: an engineer who has never read the runtime’s docs can’t set a property too early, can’t fire an unbound trigger, can’t leak a display link. The generated layer gives them no way to.

Here’s the same code from above, against the generated layer:

viewModel.properties.purchaseCount = 3

Typed, non-optional, autocompleted. If it compiles, the property exists.

Building it: the pipeline

Our runtime’s vendor ships an open-source code generator that reads the asset files and emits stubs. Its stock templating couldn’t express what we needed: conditional output, so that a file with data bindings generates a rich API while a simple confetti loop gets a minimal shell. We forked the generator and built our templates on Inja, a Jinja2-style C++ template engine with native {% if %} blocks and partials:

{% if file.has_view_model %}{% include "_properties.inja" %}{% endif %}
{% if file.has_metadata %}{% include "_metadata.inja" %}{% endif %}

The whole pipeline runs in CI on a dedicated animation repo. Every asset file lives there, and a change to one is a pull request like any other:

Nobody runs anything, and nobody can skip anything. The app doesn’t contain animation files or binding code at all; it consumes a versioned package, and the only way an asset becomes part of that package is by passing lint, codegen, and the generated tests. That property turned out to be load-bearing: any generator someone has to remember to run is eventually out of date, and any check that’s optional eventually gets skipped.

A few details in the pipeline earned their keep:

Lint before code exists, fail the PR. The generator validates assets before emitting code: embedded images must be webp and under 200 KB. A 2 MB PNG inside an animation file is now a failed check with an exact file:line: error: message, not a binary-size mystery three releases later.

Generated code means generated tests. Because the generator knows every property name, type, artboard, and state machine, it emits a companion test suite alongside each binding: unit tests that verify the bindings round-trip correctly, and snapshot tests that render each artboard and diff it against a reference. A designer’s visual regression or an SDK behavior change gets caught on the PR, before a package ever ships.

Enforced instrumentation. Logging isn’t left to feature engineers to remember. The generated layer logs state machine activity uniformly, so every animation ships with the same structured instrumentation and debugging a misbehaving animation doesn’t depend on who wired it up.

A privacy convention for designers. Designers use internal properties to coordinate state machines: scratch counters, transition flags. Anything prefixed with _ in the editor is excluded from generation:

purchaseCount → generated → viewModel.properties.purchaseCount
_sequenceIndex → excluded → (not visible in Swift)

Without this, every internal scratchpad shows up in autocomplete, and someday an engineer sets one and breaks a state machine with no idea why.

Absorb the async timing problem once. Asset files load asynchronously, and callers set properties before the load finishes, values that a naive implementation silently drops. The generated Properties class stores plain values and flushes them all to the SDK when the file binds. Every animation gets correct pre-load behavior for free.

Hiding the SDK entirely

The generated code conforms to one small protocol, and that protocol is the only thing feature code ever sees:

@MainActor
public protocol AnimationViewModelProtocol: AnyObject {
    func play(onCompletion: (() -> Void)?)
    func pause()
    func stop()
    func reset()
    var isPlaying: Bool { get }
    func createSwiftUIView(isUserInteractionEnabled: Bool) -> AnyView
}

Feature engineers never import the runtime. That boundary bought more than cleanliness:

• We control the API surface. New runtime capabilities get exposed deliberately, through a template change, not discovered ad hoc in feature code.
• SDK upgrades are invisible. The runtime version is an implementation detail of one module.
• Cross-cutting fixes land everywhere at once. When we found the runtime’s display link kept ticking for offscreen animations, burning CPU on views nobody could see, we fixed it with one view modifier that pauses on scenePhase changes and disappearance, applied automatically to every generated view.
• Hard things become one-liners. Image properties generate named async methods, like loadAndSetAvatar(from:), that internally handle download, decode, caching, and cancellation. The call site is a single try await.

The before/after, compressed:

The proof

This system now backs every animation in the Whatnot iOS app: 30+ production animation files and counting, each with generated bindings, generated unit tests, and generated snapshot tests. The most demanding is the Jumbotron: a real-time celebration that fires per-buyer during live auctions, driven by purchase count, username, a remotely-loaded avatar, and whether you’re the buyer or a spectator. The entire integration is typed property assignments and one trigger call against generated code; the engineer who built the feature never touched the animation SDK, never learned the load-order rules, and never had a string literal to get wrong. The rename-breaks-a-live-show timebomb we opened with never went off, and now it can’t. The whole class of bug is unrepresentable.

What we’d tell you before you build one

• Generate from the asset, or drift is guaranteed. Any binding layer maintained by hand is a second source of truth.
• Make generation impossible to skip. A shared repo where CI gates every asset change beats any step someone has to remember. If an asset can reach the app without passing lint, codegen, and tests, eventually one will.
• Put lint where the code is born. The generator sees every asset before code exists; it’s the cheapest possible place to enforce format and size budgets.
• Generate the tests with the code. The generator already knows everything the tests need to assert. Emitting the test suite alongside the bindings costs a template, and it turns every asset change into a regression-checked change.
• Budget real time for memory and lifecycle. Display links, retain cycles, image decoding off the main thread: this is the part that bites, and none of it shows up in the demo. (We generate a per-type leak tracker as a canary.)
• The pattern is portable. The same repo generates Kotlin for Android today: same introspection, same lint, same generated tests, different templates. This post focused on iOS; the multi-platform story is the next one.

We’re not done: fallback views for failed loads and disk caching with a real TTL are still on the list, and both are template or pipeline changes, the quiet superpower of this architecture: fix it once, and all animations on all platforms get it in the next package.

Every animation was one rename away from breaking a live show was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

This post covers what we talked about, why we built it, and future direction.

Snowflake recently published a recap of our sessions, which you can read here: Observability at Scale: Whatnot at Snowflake Summit.

Data at Whatnot

Every live auction, bid, chat message, purchase, moderation action and recommendation produces data. That translates into billions of events each day powering recommendations, seller analytics, and customer support. For us, data isn’t just for internal analysis. We use it to power the product while millions of people are buying and selling in real time. That creates an unique set of engineering challenges.

As Whatnot has grown, we’ve had to evolve our platform to support thousands of datasets, hundreds of developers, and an increasingly diverse set of workloads while maintaining reliability, governance, and developer velocity. At Summit, we talked about the investments we have made to get engineers to vetted data faster

Accelerating Builders with AI

The Whatnot data platform is a living system. Schemas evolve, pipelines are deployed continuously, business definitions change, and new data products are published every day. Keeping up with that pace of change has become one of the biggest challenges for developers. In our first session, we shared how we’re using Snowflake Cortex AI and Semantic Views to make that knowledge more accessible.

By grounding AI in governed business definitions and trusted metadata (see this earlier post for more details), builders can ask natural language questions about their data, quickly identify the right datasets and metrics, and investigate issues without manually searching documentation or relying on the Data team. Whatnot’s full-time employees now use AI weekly, and much of that usage runs through the data platform.

The transition from clunky, manual data pulls to agentic AI completely transformed the company’s internal culture:

• Widespread adoption: Within 90 days of launch, over 80% of Whatnot’s 1,000+ employees were actively using the agentic solution.
• Universal access: 17 different company departments reached 100% active utilization. Teams such as performance marketing, talent acquisition and business operations became entirely self-sufficient.
• Deeper strategic analysis: Rather than just pulling static lists, teams are using conversational AI for complex work, such as tracking international weekly trends, matching messy data fields and building seller churn prediction models.

What previously required ad hoc data requests planned weeks in advance has been compressed into data available at the speed of typing. This drastically reduces technical friction, allowing product teams to run SQL execution checks, test feed-generation logic and rapidly iterate on predictive models (such as seller churn) without waiting on engineering queues.

Observability by Default

Developer velocity only matters if engineers can trust the data products they’re building on. Our second session explored why observability has become a foundational capability for Whatnot’s data platform. Every day, we orchestrate thousands of data assets across Snowflake, dbt, and Dagster, where small regressions in freshness, quality, or performance can quickly cascade into downstream systems.

Historically, tracking platform health in real time was a massive headache. Standard utilization logs were delayed by three to four hours, which is far too slow to catch a live pipeline error. The alternative was running massive diagnostic scans every 15 minutes, which burned through the compute budget and required strict admin security clearances just to view the logs.

Setting up high-quality system alerts used to require data engineers to write over a hundred lines of intricate SQL code. This technical barrier effectively locked business analysts and operational managers out of the loop. To address this, we adopted Snowflake’s overhauled event tables, which improved event ingestion speeds by 10x and made comprehensive logging significantly more efficient.

All Whatnot’s data products are now monitored by default with freshness and error rate alerts directed to the relevant product owner using CI-enforced metadata. Developers can opt in to define their own custom data quality checks and asset-level SLAs. Moving forward, we are focused on using these insights to provide a company-wide view of platform health and accelerate time to repair when problems occur.

Looking ahead

The rest of 2026 is straightforward. It’s a company-wide view of platform health, faster fixes, and better context for the AI tools our developers already rely on. By making trustworthy data easier to discover and the health of that data easier to understand, we can help teams move faster without negatively impacting reliability.

And if solving problems like these sounds interesting, we’re hiring.

Whatnot at Snowflake Summit 2026 was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

We solved this with one line of regex; a filter that cut token consumption by up to 30% and, more importantly, stopped the agent from flailing. It’s not just a cost-saving trick; it makes the agent’s behavior more predictable, spending its budget on the fix instead of the hunt.

To grok this, imagine a chef tasked with making a soufflé with ingredients but no recipe.

An expert chef, using school-honed skills, might succeed immediately or through a few refined attempts. Success is inevitable because they possess the competence to iterate. A mediocre chef eventually succeeds too, but wastes far more food in the process.

AI coding agents operate similarly when facing unknowns. They iterate, try, and fail before reaching an answer. They are capable enough to succeed, but they waste significant compute and tokens during the process. The solution isn’t rocket science, but just really good filtering that is specific to tasks that can be token-heavy. Here’s an example of that below.

For the rest of the post we will do a deep dive into what we did, how it works, and you can ultimately use the recipe for your coding agents.

The asymmetry

Coding agents with capable LLMs are good at knowing which tool to call. They pick the right command as they work towards their objective. But they are bad at predicting what the output will look like and where in that output to focus.

The agent fires off a build command. Hundreds of lines of build output land in its context window. The relevant compiler error is buried somewhere in the middle.

Reading through a log is second nature to you; you scan the text without a second thought. You scroll. Your eye snags on red text, stack traces, the word FAILED. You find what matters in seconds, and the rest of the output costs you nothing. It sits in your scrollback, and you forget it was ever there.

For an agent, there is no such thing as free output. Every line lands in the context window. Every turn after that, the harness sends the whole context back to the model. The noise adds up and burns tokens on every turn. In a bash-heavy environment (like Android development), where a single task can fire multiple commands, you get the compounding effect of wasted tokens. Context fills up. Token consumption grows on every turn. The lines you would have ignored, the agent goes on paying for, again and again, for the rest of the session.

It will improvise. tail -20. Then tail -50. Then head -30. Then a grep with a pattern it invented on the spot. Then it dumps the output to a file and starts reading it in chunks from the end.

The smarter the agent gets, the more confidently it improvises, and the more it can waste doing it. A weaker agent will just keep cycling. tail, head, grep, different tail, different grep, a fresh approach every minute, none of them working, until you notice and stop it.

Just like the chef without the recipe, you don’t know in advance how many tries the agent will take to tackle its task. If you provide a clear recipe for the domain space it is working in, then you get two things: (1) More deterministic behaviors for each run and (2) reduced token burn!

The recipe

The solution rests on a single core principle: eliminating output ambiguity. By predefining the expected command output and highlighting the critical data points, you eliminate the need for agent improvisation. Instead of forcing the agent to devise its own filtering logic on the fly, you provide a pre-built filter to guide it.

In our implementation, we achieved this by documenting the “recipe” within AGENTS.md (or your preferred config file). This includes a specific named filter and concise guidelines on when to apply it. To strengthen this further, a pre-tool hook could be used to block any build requests that lack a filter, making errors significantly less likely. For our purposes, however, maintaining these instructions in the markdown files has proven effective.

The concept is straightforward: route the command output through a filter designed to retain only the essential details of progress (such as errors, failures, and final results) while discarding the surrounding noise.

<build-or-test command> 2>&1 | rg "<the lines that signal success or failure in your stack>"

Real-world example

In our Android repo, Gradle was the worst offender. Android builds spit out logs hundreds (if not thousands) of lines long, with the relevant lines being buried somewhere along the way. Our filter, defined once in the docs and referenced everywhere by name, looks like this:

2>&1 | rg "^e: |FAILED|FAILURE:|Cannot locate|tests completed|BUILD (SUCCESSFUL|FAILED)|problems-report|AssertionFailedError|AssertionError|Exception( at|:)"

The output is distilled from hundreds of lines down to a mere handful … twenty at most.

By removing the surrounding noise, such as JVM warm-up sequences, dependency resolutions, configuration phases, and plugin chatter, the agent is left only with what truly matters: compiler errors, test failures, exception classes, and the final build outcome.

Each token included in the filter was selected to address a particular type of failure:

• ^e: is anchored because the unanchored version was matching e: fragments that appear in normal Gradle task output. The agent treated those matches as compile errors and burned cycles analysing a build that had compiled fine.
• Cannot locate got added because misnamed tasks amplified the noise problem. The agent would typo a task, see FAILED at the tail of the output, assume compile errors, and start fixing code that wasn't broken. Gradle had already said the task didn't exist. That line just sat higher up than tail -10 reaches.
• AssertionFailedError is there to surface why a test failed, not just that it failed. Without it, the agent would see FAILED next to a test name and have to guess at the fix from the name alone.
• … and so on for every other token. Each one earned by a specific failure and protecting the agent from a specific kind of confusion.

You’ll notice the filter mixes test failure tokens with compile output ones. That’s deliberate. We use one filter for everything, not one for compiles and another for tests, because conditional rules are exactly the kind of thing agents quietly forget. The test tokens just don’t match anything when the filter runs on compile output.

The following comparison illustrates the impact of applying this recipe. While the specific implementation focuses on Android and Gradle, the underlying principle is universal: any tool generating excessive output for an agent to process efficiently can be improved by the same strategy. By defining expected output and highlighting critical segments beforehand, you empower the agent to identify which parts of the stream truly require its attention.

The regex is the recipe. The chef is the agent. Hand them the recipe, and even a mediocre one ships a soufflé.

The proof is in the pudding

Does the filter actually work, or is it just theater? A shorter log is obviously cheaper, but if the agent loses the plot because it lacks context, the savings are a trap. We ran the numbers to find out.

Across sixty-plus runs on two constructed Android fix tasks (seeded errors, not observed production sessions), the filtered agent and the unfiltered agent reached the same finish line every single time. Accuracy didn’t budge. But the path they took to get there was radically different.

The real win isn’t just shrinking the payload; it’s stopping the thrash. When an agent gets a thousand lines of noise, it panics. It starts guessing. It fires off a tail, then a grep, then reads the file in chunks. Each new tool call drags the old noise along and burns tokens on it again. That is the compounding effect of wasted tokens: you do not pay for the junk once, you re-pay for it every turn after.

By handing the agent the signal on a silver platter, we cut transcript size by 20% on the simpler fix and ~30% on the cascade-style failure. The more complex the build failure, the more the unfiltered agent flails, and the more the recipe saves. You’re not just saving pennies on tokens; you’re buying back the rounds that never should have happened in the first place.

Four lessons

Months into production, four lessons stand out.

Most of the work is around the rule, not in it. The regex is small. Most changes happen to the rules around it: when to use it, what never to do with it, how to apply it across tools. The filter is the mechanism, but the instructions are what make it reliable.

There is no final form. The filter will keep changing as the development environment changes. In a few months, it will look different, with new patterns added and others dropped. It’s a living document, not a finished artifact.

One filter, not two. If your docs carry two slightly different versions of the same rule, the agent latches onto whichever it hits first and uses it everywhere, forgetting the conditional logic in AGENTS.md about which belongs where. We had exactly that: two patterns drifting in two corners of the docs, one excluding test failures, one not. Both sort of worked … until they didn’t. The fix was a single pattern, defined once and referenced by name. Nothing to choose between, nothing to remember.

Forbid mutation. Given the slightest opening, agents will try to “improve” your filter for the one case they’re working on. So we added this line to the doc:

Append this filter exactly as-is. Do not add, remove, or change any patterns. Not even “just one more thing” for a specific situation. If the filter misses something, run the command without a filter as a separate follow-up call.

The phrase “not even just one more thing” is load-bearing. The regex does the filtering; the sentence keeps it from being quietly rewritten into something useless.

Smarter models will keep improving how agents pick tools and reason about problems. Recipes are what improve the parts in between, and recipes compound across every agent that touches your codebase. An afternoon spent writing one good filter outlives many model upgrades.

The Soufflé Problem: One Regex, 30% Fewer Tokens was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Tom Verilli | Chief Product Officer

In the last two years, 31,832 people applied to be a Product Manager at Whatnot. We hired one. You’re twice as likely to hit a hole in one as you are to get a job by simply applying.

That’s not a process failure. I’ve been building products — and product teams — for over a decade, and one of the biggest factors in deciding to come to Whatnot ~3 years ago was the very deliberate product culture. No one knows what it means to be a PM in the world of AI, but everything I see says the industry is moving towards us and how we build here — because no tool will make you useful if you aren’t doing the right job.

First: we have to acknowledge that the average PM is deeply average.

The product function emerged in response to scale — engineering teams got too big for CEOs or GMs to manage directly, so a business <> tech conduit was needed. Over time, we lazily generalized the role to “every time you hire an Engineering Manager, you hire a PM”. But where an Eng Director managed 30–40 people through their EMs, a PM Director just managed five. Incentives govern the world, so those directors’ jobs became “justify growing my eng partners headcount” so they could, in turn, grow theirs to become a VP. Slowly, the role of junior PMs shifted from “CEOs of the product” to “babysitters of buttons” and product-minded engineers to infantilized order takers.

Then COVID hit, and the industry hired a mind-boggling 500,000 new software engineers in just four years, and ~80,000 new PMs were minted to match. That’s 80,000 PMs buried within gargantuan teams at FAANG, far from any customer, 50 layers from the zoom where it happens, taught paint-by-number PM’ing at a product school, in an era of unearned engagement growth where seemingly anything worked.

The likelihood of someone emerging from that with great product instincts, experience, and grit actually feels less likely than hitting a hole in one.

Second: we made our best, worse.

When your job is supervising five people, all you can do with your day is get into other people’s work. They dislike that and label it micromanagement in an anonymous survey, so you back off. How then do you spend your time? You story-tell, shepherd things through review so your teams are ‘succeeding’, justify resources. But you don’t know what story to tell, so you stand up a user research team to tell you the jobs to be done, then a PMM function to tell that story to customers. The function that came to be strategically important because it gathered context and disseminated clarity abstracted itself out into increasingly ivory towers.

But the actual truth is in the data models of your systems, in sales calls, CX tickets, in the analytics — not in the pretty 2x2 made to simplify it all.

All the time you spend playing management means your innate understanding of the issues is getting stale, your instincts for your customer are duller, and the likelihood you’re right is dropping.

Our batting average as a function dropped both because the denominator expanded AND because its expansion meant everyone who was good at product seven years ago was promoted out of doing any actual work (or got rich enough that the incentive to stay and play politics was low).

The Whatnot Way

Since its earliest inception, the Whatnot product team has been built on a somewhat simple premise: we regret that product management exists. Sales and engineering got on just fine before we were hired, so where they can, they should just ship without procedural gatekeeping or nonsense paperwork. Product is a trade, not a qualification. Anyone who does it well learned by doing and by being around great people doing.

I was in an interview recently where someone told me Whatnot felt like Twitch and eBay had a baby — culturally it couldn’t be more wrong, but in terms of the product span, it’s a decent comp. A conservative estimate says those two organizations combined have >400 PMs. We have 20. 20 PMs for 1200+ total employees.

Our PMs are mapped to problems, not to EMs. Those two often overlap, but aren’t the same thing. If you’re building a new sales format for fashion sellers, you’re going to be pretty hand in glove with the EMs who own how listings and inventory work, but equally with the logistics and payments EMs.

Having to work across multiple stacks and weigh impacts to different customers isn’t easy — it requires broad context of the business, the ability to foresee downstream impacts of changes to any feature, prowess at context switching, the ability to build and spend trust across a whole org rather than with one partner. That’s why we hire almost exclusively senior PMs. PMs who are over endless alignment meetings and itching to build again. Or, we convert promising sales or ops folks and let them learn by doing. We’re always looking for the hole-in-one mid-career L5/L6 hire, but the stats don’t lie about how often we find them.

Finally, everybody ships, including me. I am always working directly with a team of engineers and designers to ship features as an IC, and so are both of our cofounders.

When it was time to test out if it was feasible for PMs to vibecode small features, I was the guinea pig. When it was time to onboard our first seller in Australia, it was our cofounder Logan doing it. When Zendesk started dropping customer tickets, it was our CEO Grant talking to their support engineer. As a company, we require every employee to sell, buy and do CX tickets, or we give them a “below expectations” rating. If we’re to lead a company with that commitment to customer centricity, we’ve gotta be deep in how things work and broad in why. We call it “being T-shaped” — having a breadth of context and the depth of your area, simultaneously. Depth and experience allow you to make decisions quickly, not having to wait for 5 layers of management to review it means those decisions become actions.

Members of Technical Staff

There is so much noise right now about “building”… No, product requirement docs aren’t dead. A PRD is just a vessel for thinking clearly about a problem and articulating that to others. Make yours interactive if you like, no one cares. No, the cost of shipping a bad product hasn’t gone to zero; it’s still paid by your customers. Throwing spaghetti at them 16x faster is not, in fact, a revolution. It’s simply annoying. And no, everyone’s not gonna be an S-tier Engineer and Designer and PM all in one. A few folks will be, but the same drivers of specialization — what people enjoy and are good at — will still drive how we work.

What is shifting is the realization that being an IC is a way better use of many people's skills, experience, and limited time on this earth than redrafting the same document for the 5th time to fit the current pedant's preferred formatting. A few PMs at Whatnot are managers, but each spends 90%+ of their time as an IC. There’s no distinction in our titles or comp for those who manage or don’t because we see no inherent virtue in it. AI gives us incredible leverage — I can move faster at just about every task in the development process, whether that’s understanding data that used to require a data scientist to untangle or turn a PRD into every permutation of CX SOPs that would normally be the subject of long nights in the office on launch week. I can build a bot to triage the 100 questions a week from sales or to find the localization gaps we left in a recent experiment.

The most disruptive thing about AI for PMs is its shown that the leverage of coaching people up and working through them just isn’t the singular source of leverage it once was. Especially if those people are — through no fault of their own — deeply average. But that leverage is only available to people who still know how to do the work.

What’s especially encouraging about this trend is that it’s going to entice the best PMs back to doing actual PM work. The thinking about the needs of the customer and the business, and the good taste to solve it in the best way. As a customer of other companies, I’m excited to see the luminaries of our industry getting back to building — it’s gonna make their products better. As someone obsessed with building the smallest, most highly leveraged PM team in history, I’m excited for it to release the incredible humans who’ve been phoning in roadmap reviews for half a decade.

Show, don’t tell

Below, I’m going to copy (in full) the one document we have about how we work on Product at Whatnot. If we’ve met even once, you won’t need me to tell you who the author is — this is how we talk and how we work.

You can also go look at who works on our team — there are at least six people on the team today who could be CPO at a series B-C startup who will spend their evening on the phone with sellers, 400 queries deep in a Hex Thread, or drafting v1 comms for tomorrow’s launch. It’s four former founders who have never in their lives agreed that something is outside their purview. It’s four ex-FAANG directors who no longer spend their days debating where people should live in a nine-box grid. It’s six early-stage PMs who have incredible taste, being told they need to try more things because we only learn by doing.

I doubt our erstwhile declaration of 20 PMs will last — the opportunity in front of us at Whatnot is so enormous that we won’t constrain ourselves arbitrarily — but the bar for who we hire will only go up as the industry and AI tools continue to reward great ICs with leverage. If you’re one of those folks, and what I’ve described above is what gets you going, you’ll work out how to get in touch with me.

Building at Whatnot

Building great products is hard. It’s not just that you have to have the right insight as to the problem, get the details right, take it to market right, measure it correctly so you understand its performance or iterate on it quickly. It’s that you have to do all those things or it doesn’t work. Worse, failing is expensive. We have few teams and a massive amount of opportunity in front of us — batting .300 is great if you play in the MLB but to realize our aspirations we need closer to .500. Without a high average we either constrain growth in the short term or we couple business growth to headcount growth and constrain ourselves in the long term.

This document has 2 parts:

1. Our philosophy — this will not change
2. Our processes — these will evolve and current SOT kept here

How we build gives us leverage

You can’t construct a building a room at a time, you have to design the whole building at once and build it all at once. Thankfully, we don’t work in construction, we work in software. Building iteratively is our superpower. We always launch the smallest unit that provides real user value and a solid user experience, but we design things further out to ensure we can scale it.

The happy path of a successful product here walks 7 steps consistently:

1) It’s something that matters for users and our business

Ruthlessly prioritize the most impactful things that solve for our users and business needs.

• You must be able to articulate that value explicitly: “enable scaled retailers to sell products stored in multiple locations in a single show — by updating ‘ships from’ to be a product field, not a show field”

Think about the system.

1. Is this product immediately valuable when launched?
2. Is it a ‘building block’ for other things?

If (1) isn’t true, do not proceed. If (1) is true, work out how it can become (2) over time.

2) It’s something people want. Understand their pain points, desires, and behaviors to create a product for them.

• You don’t know that unless you understand the user you’re building for in detail. Marry together qual and quant.
• Think about your product in the context of existing product workflow.
• Beware shiny objects. Especially shiny objects you’ve built elsewhere in the past.

3) Customer needs do not align with our org charts / are never met with a single feature

If you’re building locally you’re building naively.

• You must be working back from a complete customer experience, not code ownership. Go solve the problem, period.
• The inverse is true too — other PMs will need to push into “your area”. Help them.
• This principle is why we strive to have the smallest possible Product and Design team. The more folks there are whose role is narrowly defined the more myopic roadmaps get and the more time we waste on coordination and consultation.

4) It’s the simplest possible solution that solves the problem.

The key to building fast & reliable products that users love is avoiding unnecessary and non-impactful work

• Simple isn’t just fast to build, it’s typically also the most successful.
• Thinking about the system does not mean build the whole system upfront.
• The more you build before you know you’re right, the more expensive it is when you’re wrong.

5) It was validated with the smallest possible audience.

You’re just guessing until someone is using it.

• Get paper or clickable prototypes into the hands of sellers ASAP. Staff dogfooding catches bugs better than it validates a solution because we’re not our customers.
• Think about your GTM motion
• If you’re in validation mode, solving for awareness (internal or external) is a failure mode.

6) Once validated, we iterate like crazy.

Once live to customers, we’re shipping weekly if not daily improvements.

• If you hear “once we ship X we can move to Y” it is a giant red flag.
• Once we know it’s gonna be a thing you need to go back and solve for Catex and CX
• Launch, validate, measure, iterate, iterate, iterate > then move to the next priority.

7) We run through walls once in beta

Getting a spark is hard. Once you have one you’ve gotta pour on the fuel or it will die.

• Biggest risk to launching hyper-simple hyper-early products is they’re incomplete and thus not truly long term useful. Once you launch you’re on the clock to get from high potential to high impact.
• Zero in on maximizing the value you’re creating and not managing every little complaint, risk or repercussion.
• Working out what complaints, risks and repercussions to worry about is a judgment question for each launch. Worrying about non-risks is as dangerous as failing to worry about them

8) This ain’t the bachelor — decouple everything

There is a natural tendency when designing a system to want to ship multiple parts of it at once. In a sufficiently complex system like ours, chances are multiple teams are working on components of a system in parallel and it may seem sensible to ship them together so its one big change vs two. It’s also a trap.

• As long as each piece is independently viable and beneficial to customers, launch them ASAP

The role of reviews and feedback

We have documented a product process whose intent is to ensure we’re working on the right things and in the right way. This includes visibility, approval and accountability functions. More important than blindly adhering to that process however is internalizing the underpinning philosophy — which is well articulated in this twitter thread… (seriously, read it before proceeding)

1. In a complex system you need way more alignment than you think to actually get to the right answer. Because that word can be misinterpreted:
2. Autonomy at Whatnot is autonomy of implementation. No-one has or should expect autonomy of strategy. Without alignment autonomy is squandered.
3. To meet expectations at Whatnot a PM or Designer

Why Speed Matters

Our whole system is premised on maximizing the speed of shipping the right thing. 1–3 of the happy path are about working out what we think the right thing is, 4–7 about how we validate, iterate and scale that thing. We do that because:

1) Everything in our system compounds — the good and the bad

In 2025 we ran 750 experiments across about 250 business days, which comes out to roughly 3 ship/don’t ship decisions per day. If you simulate the long term impact of making each of those decisions just 3 calendar days faster the impact over a 2 year horizon is >$1.1B of incremental earnings for Whatnot Sellers. Not the impact of those products, just the impact of making those decisions fractionally faster. Every delay in shipping the right thing hurts our customers, and the bigger we get the more the opportunity/cost of speed.

2) Once speed is lost, it never comes back

Humans naturally conform and come to rely on process, so even ones invented for narrow use cases get applied more liberally than intended. Incentives shift to following the system vs having the impact the system was designed to ensure and the organizations muscle memory to “know, but go” is atrophied and lost. There is almost no single mistake we could prevent that would be a good trade off long term for slowing down the speed with which we build.

3) Speed isn’t the reason for mistakes/errors

Committees only prevent mistakes as a byproduct of preventing progress. Judgement is what actually prevents mistakes. Shipping more frequently builds our judgement — like an athlete we get stronger through reps. While building reps, teams can leverage the judgement of those with more reps and more context — continuous ad hoc guidance from product leadership, visibility for key risk mitigations like legal and comms at the earliest stages of planning (if there are hurricanes we need to avoid in the Atlantic, we need to know that when we’re plotting the course, not as we’re casting off), category or country leads who can proxy how specific customers may react. As a part of thinking about the system, PMs should seek to anticipate impacts of their launches, but are never gated either by having sought out, or by taking any feedback/input they get. Product review is the only gate in our development process.

Building, and Whatnot was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

A bad PR went out just as our hourly feature pipeline was about to run. The pipeline was responsible for recomputing fresh user behavior signals used in livestream ranking on the home feed. These features capture recent user intent: watches, bids, orders aggregated across users, sellers, and categories. A missed feature update means that the system stops responding to user signals immediately.

This PR release caused one of our feature groups to suddenly produce zero rows … but nothing broke!

Even though a critical feature group for the model stopped updating, the online feature store continued serving the last-known-good features, buffered by a 2-day TTL and nothing broke explicitly. Our anomaly monitors went off immediately, flagging the drop in row counts. The team identified the bad change and reverted it before the freshness gap grew large enough to matter.

No corrupted data reached production. The system absorbed the failure and degraded safely. This graceful degradation didn’t happen by accident; it was the result of hardened processes from years of lived experience, seeing the way these systems can degrade in obvious or subtle ways. In this post, we share some of those lessons learned.

The system: hourly feature pipeline

Whatnot is a highly dynamic marketplace where buyers’ preferences change at a moment’s notice. A buyer might research a laptop, join a football-card break a few hours later, and buy chocolate before logging off for the night. Daily feature updates capture long-term preferences but miss intra-day shifts.

To keep up with these rapidly changing buyer interests (in 2025) we moved from daily to hourly updates for features that power our ML feed ranking models. Fresher features are a critical component for capturing the live nature of the Whatnot marketplace, where the inventory available to purchase changes in real time. Moving from daily to hourly updates was a meaningful improvement. However, this update also came with a catch: the more frequently we update features, the less time we have to catch and correct issues before they impact users.

This is especially critical for new users. Their “aha moment” depends on how quickly recommendations adapt to their latest actions to help find the right livestream, with the right item at the right time.

Going hourly exposed failure modes that daily pipelines could mostly hide.

Our hourly pipeline for livestream ranking runs in three stages: raw engagement events (watches, bids, orders) are ingested and transformed via DBT into user-seller and user-category feature aggregations (~30 min total), then pushed into our feature store that uses Snowflake for training offline and Redis for real-time ranking (~5 min). Each stage has dedicated alerts for failures, runtime regressions, stale ingestion, and feature distribution anomalies, providing end-to-end observability across the ~37-minute pipeline. Hourly features have a TTL of 2 days in the online store to provide a buffer and preserve non-zero, stale values in case of >24 hours of ingestion failures.

On paper, this is simple. In practice, it spans multiple teams and a third-party system, with tight timing constraints. Every step can fail, and at an hourly cadence, we need to catch issues quickly.

What production actually threw at us

The bad PR: a dramatic setup, a deliberately boring ending

The failure mode described above, with a bad PR stalling feature updates, happened early in the rollout of hourly updates. We pushed a change that caused one of our feature groups to return zero rows. Our row-count anomaly alert fired within the hour. We found the bad change, reverted it, and by the time we’d done a full postmortem, production systems barely noticed. The online store kept serving last-known-good values the whole time, buffered by the 2-day TTL meant to handle cases like this.

A boring ending, which was the point. We’d designed for this class of failure, and when it finally showed up, the design held. A clear success story!

False alarms masked real issues.

As we expanded to new countries and categories, however, our feature distribution alerts started firing frequently, even when nothing was actually broken. Once we sliced aggregate alerts by country and category, normal marketplace volatility started looking like pipeline failure.

Some false positives are acceptable to avoid missing real issues. The problem was that we treated all alerts the same. Over time, the alerts started firing constantly, weren’t actionable enough, and on-calls started ignoring these alerts. The monitoring system we’d built faded into the background, and real issues weren’t being triaged.

The fix was to be more opinionated about what each alert meant. We split alerts into two tiers:

• High-confidence failure signals: e.g., zero-row counts. Binary, unambiguous, and almost always bad. These block ingestion and page on-call immediately.
• Low-confidence anomalies: distribution shifts with many benign causes. These go to a Slack channel for business-hours review, with runbooks for common patterns, and for quick diagnosis when needed.

Once we made this distinction, alerts became actionable again. When they fired, on-call teams trusted them—and responded accordingly.

The system that got slower and no one noticed

A few weeks in, something felt off. Hourly features weren’t updating as often as they should. The core table job had quietly shifted to running every two hours with no errors, no failures, no alerts. By every metric we were tracking, the pipeline looked “healthy.” It was just running half as often as intended.

At the same time, a slower trend was emerging. As Whatnot grew with more users, more categories, and more data, the pipeline got heavier. Feature models increased from ~2 to ~5 minutes. Feature store ingestion grew from ~5 to ~10. Again, no alerts, just gradual erosion of the buffer that kept us within the hourly window.

These were the same failure modes expressed differently: one abrupt, one gradual. Neither triggered alerts because nothing was technically “broken.”

For systems where cadence and execution time matter, the absence of failure is a weak signal. We need to explicitly monitor what matters: time between updates, end-to-end latency, and runtime trends. Otherwise, degradation goes unnoticed.

How we monitor and ensure reliability today

We’ve seen what production threw at us and realized there are many different ways that pipelines could break, from distribution skews to corrupt data models to ingestion failures, so we broke down issues into 3 tiers based on estimated GMV impact. We determined these thresholds based on evaluating the performance of a model trained with accurate hourly features but served using inaccurate hourly features, and came up with the following degradation tiers:

• Tier 0: Features missing or corrupted → ~ -2% GMV
• Tier 1: Delay >24h → ~ -1% GMV
• Tier 2: Delay <24h → degraded experience

Tier 0 or 1 issues constitute production incidents and trigger on-call immediately, while Tier 2 issues trigger alerts to be handled during business hours.

We also defined the following service level objectives (SLOs):

• 99.9% success for core tables + feature models + ingestion
• End-to-end runtime < 60 minutes (37 min at launch)

But what actually protected us in practice wasn’t any single monitor, it was defense in depth. We built layered coverage across the entire pipeline, with each layer designed to catch a different class of failure:

• Core data model failure detection and runtime monitoring
• Feature-level distribution checks and anomaly detection
• Ingestion health checks in Feature store
• Monitoring of features as actually served in production

Each layer assumes the previous one can fail. Online serving metrics act as the backstop for anything that slips through upstream, and TTLs provide a final safety net for anything that escapes monitoring entirely.

Ready and reliable

This layered approach is what makes operating fresh features viable in production. Freshness is exciting, but reliability is the real constraint, and now that we’ve made hourly features robust, we’re in a position to push even further. Our next step is moving to within-session and sub-minute updates, because when you’re the leading livestream shopping platform, every second counts.

If building systems at this scale sounds interesting to you in live commerce, real-time infrastructure, andthe intersection of community and technology , we’re hiring!

The ML Feature Pipeline That Got Slower and No One Noticed was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

A decade ago, one of the more useful ways to explain machine learning systems was with the diagram below: the ML model itself was a tiny box in the middle, surrounded by everything else you actually had to build to make it work in production.

The same thing is happening with LLMs now. Making the API call is the small box, maybe even smaller than the one in that original diagram. Calling the model is the easy part. The hard part is everything around it: there is less stable ground truth, inputs are harder to constrain, outputs are non-deterministic, and the system is much easier for users to push in unintended directions. The hard part is giving teams the ability to iterate fast, trust that the system is working, and know that it is getting better.

From our perspective, this is what an LLM platform actually has to solve for. It has to be reliable enough to support real product and operational workflows. It has to enable velocity, because many of the highest-leverage improvements are small changes that need to move quickly. And it has to create trust, so teams can understand output quality, catch regressions, and ship with confidence. These pillars reinforce each other. Reliability makes teams willing to depend on the platform in production. That production usage creates the data and feedback loops needed to build trust. And trust, in turn, makes it much easier for teams to move with velocity, because they can tell whether a change actually helped. The rest of this post is about those three strategic pillars and the concrete actions behind each of them that allowed us to build the LLM Platform at Whatnot.

We built on foundations that were already there

A big reason we were able to build the platform quickly, and with a small team, is that Whatnot had already spent years investing in the foundations underneath it. This was the result of earlier platform work across the company: building the modern data stack, scaling investments, and strengthening shared tooling and other internal platform primitives. Those investments meant integrations, logging patterns, analytics sinks, and internal tooling foundations were already there when we needed them. Because so much of Whatnot’s business logic is consolidated in a single codebase, exposing existing capabilities through the LLM Platform was often much easier than it would have been in a more fragmented architecture. Teams operating across many languages, many microservices, and many ownership boundaries would have a much harder time following the same path.

Building for velocity

One of the biggest differences between traditional ML and LLMs is who gets to participate.

Traditional ML has a relatively high barrier to entry as a contributor. Even getting a first useful version usually requires specialized workflows and expertise. LLMs are different. The interface is much closer to natural language. If you can explain a task clearly, critique an answer, or describe how something should behave, you already are much closer to making a contribution.

That matters more than it might seem. In many product-facing LLM systems, a meaningful share of the improvement does not come from rebuilding the model. It comes from changing the prompt, refining the examples, attaching the right tools, tightening the evaluation, or adding better context. Those are not purely engineering problems. In many cases, the people best positioned to make those changes are PMs, Ops leads, Policy Experts, and other subject matter experts who are closest to the actual user experience. That is also how we think about the platform internally: it should work for everyone, not just a single group of people.

For us, building for velocity meant building tight iteration loops. Two examples are prompt experimentation and the tool catalog.

Experimentation

We wanted prompt experimentation to feel like normal product work, not infrastructure work. If someone has a better idea for how the system should behave, they should be able to test it directly in the platform instead of writing code, updating an SDK, or waiting on a deployment. That matters because the people with the best ideas are often not engineers. But making experimentation self-serve does not mean making it loose. These experiments still run through the same experimentation framework and guardrails used in the our app product flow, so teams can move faster without giving up rigor.

In many LLM features, the two prompt variants return the same output most of the time. A standard A/B framework still counts all of those cases as exposures, even when the user experience did not actually change. That dilutes the signal and makes it harder to detect whether a prompt update really helped. So we changed the way exposure is logged: we only count an exposure when the two prompt variants produce different outputs. That lets us focus measurement on the cases where the experiment could actually matter and makes prompt iteration meaningfully faster (10x+ faster).

Tool Catalog

A lot of LLM product work stops being “just prompting” pretty quickly. The moment a workflow needs live data, or needs to take an action, the prompt alone is no longer enough. The system needs access to functions: look up an order, fetch shipment details, check refund eligibility, or update a ticket. At that point, the question stops being whether the model can call a tool. The harder question is whether teams can expose those functions in a way that is easy to reuse, easy to understand, and safe to test.

That is why we built around a tool registry instead of treating tools as one-off code attached to individual features. Internally, the goal was straightforward: engineers should be able to define a tool once in Python, register it, and have it automatically show up in a shared catalog for prompt authors. From there, PMs and engineers can browse available tools, inspect their schemas, understand what they do, and attach them to prompts without having to wire up custom agent loops or read through implementation details or write access control.

Building for Trust

When we talk about trust, we do not mean that people should simply feel good about the outputs. We mean that they should be able to look at the data and judge how the system is performing with clear eyes. LLMs are especially good at creating false confidence: a few strong examples can make a system look much better than it really is, and a fluent answer can hide a weak underlying behavior. That is why trust has to come from inspection, measurement, and repeated review of real outputs. We want teams to stay close to the data, because that is the only way to know whether a system is actually working and, more importantly, whether it is improving.

That belief shapes how we think about evals. Our goal is to measure whether the system is actually doing what a human wants. In practice, that means supporting both deterministic and non-deterministic evaluation. Deterministic checks (valid JSON, correct tool calls, required fields) are table stakes in many workflows, but they only tell you whether the system satisfied a mechanical requirement, not whether it did the right thing for the user. The higher-leverage work is messier. This is why we invested in LLM-as-a-judge and calibration workflows that let the domain expert teach the evaluator directly. LLM judges help because they can score open-ended responses against a rubric in a way that is much closer to human review. They replace a lot of the brittle rules, similarity metrics, and manual spot checking that do not scale well and often fail to capture what “good” actually means. Once that Judge is aligned, it becomes much more than an offline scoring tool: teams can attach it to a real production use case, monitor agreement over time, and catch degradation before it turns into product drift. The next step is just as important: using data mining to automatically surface weird, rare, or high-value examples for human review, so the evaluation set evolves with the product instead of freezing in time.

Building for Reliability

Reliability is what turns an LLM demo into infrastructure. If teams are going to build real workflows on top of the platform, they need more than a model that usually responds. They need high availability, predictable performance, and confidence that provider issues will not immediately take their product down. That is why we have treated reliability as a platform concern, not something each team should rebuild for itself. Concretely, that has meant adding multi-provider support, moving toward fallback behavior by default, investing in observability, and layering in platform primitives like caching, rate limiting, and guardrails. The goal is simple: when a team ships on top of the platform, they should inherit a more reliable system by default, rather than having to assemble one from scratch.

Conclusion

If there is one theme that cuts across all three pillars, it is that good LLM products do not come from the model alone. They come from tight feedback loops, strong platform foundations, and a clear understanding of what quality means to users. We built the platform this way because the problems we are solving with LLMs are real and specific: helping a trust reviewer process a harassment report in minutes instead of hours, giving a support agent the right context to resolve a buyer’s issue on the first try, making it possible for a seller to get a clear answer about a policy without filing a ticket.

The LLM Platform exists to make those kinds of experiences reliable, improvable, and trustworthy at scale. The model is the small box. Everything around it is what makes it work for people.

The model is the easy part: Building the LLM Platform at Whatnot was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Marcus Gee | Discovery

The hottest trends don’t happen over months, but start suddenly. At Whatnot, our best sellers update their strategies in real time to stay in step with the market. Connecting buyers and sellers in this fast-changing landscape is one of the most important problems Whatnot solves, but in 2025, it was a major challenge.

In this post, we’re going to walk through how we overhauled the taxonomy system to evolve more rapidly, so our team could help new communities grow.

There is No Single Taxonomy

Whatnot supports over 200 categories across seven countries, and that number increases weekly. Our taxonomy is a platform primitive that underpins nearly every discovery experience. It shapes onboarding (what categories new users see), browse (how they navigate), search (how results are ranked), and recommendations (what their feed looks like). A bug in taxonomy doesn’t break one feature; it degrades the entire discovery surface. That makes every change high-stakes.

Here’s what we learned the hard way: maintaining a single taxonomy doesn’t work. A single taxonomy can’t optimally serve onboarding, browse, and search at the same time, because these surfaces have different goals. Let’s look at two examples to understand why:

When new users join Whatnot, they choose categories they’re interested in. That choice shapes the first feed they see. If a user selects a niche category with limited supply, their initial experience may feel sparse or low quality. For this reason, onboarding taxonomies are country-specific and supply-aware. We fold categories into broader parents, reorder them, or exclude them based on local marketplace density.

When Labubu surged in 2025, it was subcategorized under Toys, and new users struggled to find relevant shows. When we elevated Labubu to the first page of onboarding (a similar treatment given to other big categories like Pokémon Cards), it reduced this friction and helped the category explode.

Browse has a different job: maximizing reach and discoverability. Categories are grouped logically to help buyers narrow from broad interests into more specific subgroups of shows and products. Subcategories may be cross-listed under multiple parents when buyer segments overlap. For example, the “Jewelry & Watches” subcategory “Watches” can also be found under “Men’s Fashion”. While the underlying category is the same, its placement reflects how different buyer journeys intersect.

Although these taxonomies represent the same marketplace, their structures differ based on product goals. This means we maintain multiple overlapping taxonomies, each optimized for a different surface. Managing this is the hard part.

The problem: customization, speed, and safety

The bottleneck to scaling category growth on Whatnot now became a tooling issue, not a supply issue. We needed to be able to evolve multiple purpose-built taxonomies–but quickly and safely. Drawn-out processes would miss emerging trends, and even minor bugs could have a widespread impact on the product experience.

Within a taxonomy, relationships are modeled as a collection of (subject, predicate, object) triples (e.g., (“football_cards”, “child_of”, “sports_cards”)).

The original system for updating taxonomies was built so that relationships in a taxonomy were modified at the record level. That meant manually adding or removing triples or modifying the subject/object of existing sets, one taxonomy at a time.

As Whatnot scaled to new communities and countries, this became increasingly difficult to manage. More communities meant more categories to update. More countries meant more taxonomies to maintain. The amount of work multiplied quickly. Taxonomy updates took days of engineering effort to create and roll out.

Engineers had to hold the entire graph in their heads. Small mistakes had real consequences. When we made large batches of manual, repetitive changes, it was easy to miss a step and forget to add or remove categories. Those mistakes caused taxonomies to drift out of sync and reduced category visibility. Because every step was manual, we didn’t catch those mistakes quickly and sometimes left them in place for months.

Shifting from record updates to taxonomy migrations

To solve this, we stopped treating taxonomy updates as record edits and began treating them as operations on subtrees. Across taxonomies, structures differed. Some allowed multiple parents, others enforced strict hierarchies. But every update could be decomposed into three primitives: add, move, or delete.

Take this recent taxonomy change, where we separated some of our Trading Card Games subcategories into an Entertainment cards category:

Under our old system, this would require the following seven changes for every taxonomy we want to update.

Six of those changes are all applying the same update. From this, it’s clear to see that we could group changes by the operation type and resulting object, so that they could be performed across multiple subjects at once. Taking it another step further, any taxonomies having this same subtree of TCG could all be batched together as well.

So in all, changes could be expressed as a transformation of categories from four inputs:

• the operation to perform (add, move, delete)
• the categories to apply the operation on (i.e source nodes)
• the new parent category of the source nodes (i.e., target node)
• the taxonomies to update

Now, under our new system we just need to define two changes!

Continuously evolving taxonomies

With this new approach, we had the opportunity to move to a continuous deployment model by introducing three process improvements: batching, scheduling, and validation.

Batching. Through abstraction, we introduced batching. Once updates were defined as composable primitives, repetitive record-level edits were replaced with coordinated operations across taxonomies.

Scheduling. We also introduced a cron-driven workflow to automate change execution. Now, updates can be sequenced and scheduled in advance.

Validation. We added dry runs. These output a JSON representation of the taxonomy, that we could visualize as a tree. In addition to manual verification of results, we added programmatic validations to detect invalid states such as loops, duplicate children, and orphaned subtrees. If validation fails, we roll back the change and alert on the offending portions of the taxonomy. These steps allow us to get ahead of any mistakes that may have been made.

These guardrails provide us with guarantees we never had in the old system. Now, every change is observable, reviewable, and predictable — allowing us to ship taxonomy updates quickly and confidently.

Ready for anything

The marketplace never stops moving. New trends will emerge quickly, buyer demand will shift, and the categories that matter most will not wait for a quarterly migration plan. Building for that reality meant changing more than our taxonomy data. It meant changing the system around it.

By moving from manual record edits to validated, schedulable taxonomy migrations, we made taxonomy updates faster, safer, and easier to scale across surfaces and markets. That gives us a much shorter path from marketplace signal to product change.

More importantly, it sets us up for a future where taxonomy evolution is no longer a bottleneck for growth. As new communities form and new demand appears, we can adapt the marketplace structure with the same speed and confidence that our sellers already operate with.

How Whatnot’s Taxonomy Keeps Pace with a Marketplace That Won’t Sit Still was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Paul Gut | CX & Trust Engineering

Every support org has them: thousands of pages of branching, conditional logic scattered across flowcharts, docs, and the heads of your most tenured agents. “If the label’s been generated, check the tracking status; if it shows delivered, verify the address on file…” The Standard Operating Procedures (SOPs) grow until they can no longer be managed. And when you’re the fastest-growing live commerce platform in history, it all catches up with you even faster.

At Whatnot, our agents handle everything from shipping questions on vintage sneaker orders to address changes on trading card shipments. Until recently, they were spending the majority of their time on work that required no human insight, like checking order prices, validating addresses, looking up tracking statuses — routine data checks interspersed with the actual decision-making. Even worse, our more experienced agents had memorized the SOPs and stopped referencing them altogether, which made propagating even the slightest changes a nightmare.

We needed SOPs to behave like code: automatable, deployable, version-controlled. And there’s no better way to make something behave like code than to simply turn it into code. So in mid-2025, we built Atlas: our new workflow execution and automation engine.

Formalize first, automate later

The idea behind Atlas is simple: for every ticket, the engine traverses a directed graph of nodes, eventually reaching a resolution the agent can execute. The graph is represented in code (more on that decision later).

We built a small Python DSL to represent these graphs. Each node could be a decision (“Has a label been generated for this order?”), an action (“Generate a shipping label”), or a resolution (“Send the label to the user”). We also made data points — reusable pieces of information like order IDs and tracking numbers — first-class citizens. Here’s a slightly simplified example of a decision node:

BinaryDecisionDefinition(
  title="Does the tracking show the item was delivered?",
  node_key=NodeKey("decide_delivery_confirmed"),
  data_points=[DataPointKey("tracking_status")],
  yes_node=NodeKey("check_address_match"),
  no_node=NodeKey("check_tracking_stuck"),
  automation=None,  # …for now.
)

Being engineers, our first instinct was to jump straight to automating things, but we held off. Instead, every node simply started out as “manual” — it would display the instructions and context to the agent and ask them to make the decision or take the action. Only once the workflow was fully formalized and used by agents did we add the automation function.

That took some self-control, but it turned out that simply breaking the workflow up into tiny pieces made many of the nodes almost trivial to automate. Every node was a simple question or action, complete with the data points needed to resolve it — more than enough for our LLM coding assistants to automate a given node in most cases.

Formalizing thousands of pages with an LLM

Note: I won’t call the LLMs “agents” here to avoid confusion with our customer support agents, but that’s what we’re talking about.

Then came the fun part: we had to formalize all these SOPs into Atlas workflows. We were a team of three, and between building the Atlas engine itself and writing automations, we didn’t exactly have the bandwidth to hand-translate thousands of pages of SOPs. So we reached for the most obvious tool for the job: LLMs.

It’s always good to try the simplest possible approach first. After all, these models are getting better and better — perhaps they could one-shot this task? So we tried giving the LLM a doc and asking it to formalize it. The results weren’t great. Some of these SOPs are close to a hundred pages long, so the model would quickly lose track of the branching logic and start making silly mistakes.

We iterated on the approach and landed on an agentic map-reduce style flow that looked something like this:

1. Sectioning: The LLM takes the doc, finds natural places to split it up, and creates several smaller “sub-SOP” docs.
2. Subgraph generation: Separate LLMs work on the sub-SOPs, converting each into a subgraph representing a part of the workflow.
3. Joining: A “big picture” LLM, aware of the full workflow but not necessarily all the details, joins these subgraphs into a complete workflow.
4. Validation and refinement: At each stage, we run a set of checks — some programmatic, some LLM-driven — to catch structural and stylistic mistakes. More on what these checks are in a moment.
5. Ops review: The ops lead responsible for that SOP goes through the workflow manually to confirm correctness.

This ended up working very well. The sectioning kept each LLM focused on a manageable chunk, but the real star was the validation suite, which we called “the harness.”

The harness

No matter how good an LLM is, it will still make mistakes — and we cannot afford mistakes in our workflows. That’s why we built the harness: three layers of checks that catch mistakes before they ever reach an agent.

The programmatic validator was the most straightforward piece. It let us define hard rules that, if broken, render a workflow “invalid”. Every workflow had to have exactly one root node; it could not contain any orphaned nodes; it could not reference nonexistent data points; etc. Since these rules were deterministic and fast to run, this was always our first line of defense. The LLM would run them after every single change, and they would also run in our CI.

Our principles validator was the piece we developed after realizing that some of our rules were more qualitative and, therefore, hard to encode. It takes the form of a simple MD file containing rules like:

• Break up multi-condition checks into separate binary decision nodes instead of cramming them into one node.
• Node keys should clearly indicate their purpose (e.g., “decide_delivery_confirmed”, not “check_status_1”)
• Only include data points that are actually needed for the current decision.

This piece worked extremely well. Every time we noticed the LLM falling into some bad pattern, we would add a rule around it to the principles file and watch the pattern disappear.

The principles file turned out to be so useful that we started including it in the initial context (the “required reading” if you will) of every LLM in the pipeline. The initial context also included examples of some of our best-designed workflows and common pitfalls. We found that this resulted in much more cleanly structured workflows right out of the gate.

The final piece was the human check. The creator or maintainer of the SOP, usually an ops lead, would go through the workflow and point out any discrepancies with the SOP. Sometimes these were mistakes made by the LLM; other times they were slight contradictions in the original SOP that did not confuse human agents, but needed to be ironed out in the formal version of the SOP.

Forget the database — just put it in the code.

One of the most common questions we get is why we chose to put these workflows in the codebase instead of the database. We recently announced our plans to eventually move them into the database, so now we also get asked whether putting them in the codebase was a mistake. On the contrary: it was probably the single best decision we made when designing Atlas.

When we started working on this project, we had just a few months to build this entire system from scratch in time for the holiday season. Our holiday season starts on Black Friday, and things don’t slow down until… Well, they never really slow down, because due to our growth, come spring, “Black Friday volume” usually becomes “regular Friday volume.”

Because of this, we needed to get clever about scoping this project out. But many requirements simply could not be cut: we needed version control for the workflows, we needed an approval process for all changes, we needed to ensure the workflows were always in a valid state, etc.

Building even a minimal version of this kind of change management system would likely have cost us weeks or even months. But we realized that we already used one such system every day — it was called GitHub.

By putting all the workflows in the code, we instantly hit all of our change management requirements. The version control was in Git. The approval process was handled by GitHub PRs. The CI made sure the workflows never became invalid. We used a GitHub Action to implement the changes requested by ops. This all ended up working quite well, and it’s still how Atlas works today.

The main downside of this approach is that any changes to the workflows require a deploy. That’s why we are planning to eventually move the workflows into the database. But we do not feel particularly rushed; here at Whatnot, we deploy many times a day, so this has not been a major bottleneck.

Where things stand today

Formalizing our very first workflow took three days of engineering work. Today, the same task takes under an hour of LLM time, and no engineering time at all. Our ops team edits the workflows in a visual admin tool, an LLM refines the changes, and an engineer merges them in. Nobody is operating outside their comfort zone.

If you’re sitting on a mountain of operations docs full of branching and conditional steps, just remember: it’s already code — you just can’t run it yet.

Whatnot is hiring great engineers!

If developing trail-blazing AI use cases to deliver tangible improvements to business outcomes excites you, we’re hiring. Join us in building the future of social commerce at whatnot.com/careers.

Your SOPs Are Already Code — You Just Can’t Run Them Yet was originally published in Whatnot Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.