The Interview Overhaul: The Interview Should Look Like the Job

I hold an unusual set of vantage points on technical hiring, and I want to put all three on the table before I make the argument, because the argument is blunt.

I run hiring loops as an engineering leader, which means I am partly responsible for the process I am about to criticize. I build recruiting software for a living — I lead engineering on AI intake and matching systems at Bullhorn, so the machinery of hiring is literally my day job, and you should weigh my claims accordingly. And not long ago I was a candidate again myself, sitting on the other side of the table, being asked with a straight face to perform algorithm calisthenics for a role whose actual work I could already describe better than the interviewer could.

From all three chairs, the conclusion is the same. Senior engineering interviews, as most of the industry still runs them, are a waste of everyone's time. Not suboptimal. Not "imperfect but the best we have." A waste — of the candidate's preparation, the interviewer's hours, and the company's money — because the thing they measure stopped predicting the thing they select for, and in the last two years AI quietly severed whatever connection remained.

That is the diagnosis. The prescription is one sentence long: the interview should look like the job. The rest of this piece is what that sentence means in practice, why the industry has resisted it for decades, and why the resistance just ran out of road.

The ritual, priced

Start with what the standard loop actually costs, because the numbers justify taking the problem seriously even before you get to whether the loop works.

A typical senior loop at a mid-size or large company runs four to six rounds: a recruiter screen, a technical phone screen, and an onsite block of algorithmic coding, system design, and behavioral rounds. Each round consumes an engineer-hour or two, plus preparation, plus debrief, plus scheduling overhead across time zones. When I wrote about the state of the interview pipeline last year in my breakdown of why engineering interviews are still broken, the working figure for fully loaded cost per engineering hire in the US was around $22,000, and nothing since has moved that number in a friendly direction.

Spending that money would be fine if the ritual selected well. The uncomfortable, decades-old finding is that it does not. Google's own People Operations organization — the most-copied interviewing culture in the industry — studied tens of thousands of its interviews and found the scores were close to worthless as predictors of on-the-job performance, a conclusion its former SVP Laszlo Bock stated in public, repeatedly, along with the finding that brainteaser questions predict nothing at all. The industrial-psychology literature has been even less kind for even longer: in meta-analysis after meta-analysis, the strongest widely available predictor of job performance is not a puzzle performed under observation but a work sample — a structured attempt at the actual work.

The industry read those findings, nodded, and kept running puzzle rounds. It is worth being honest about why, and I will be, a few sections down. But first I want to make the mismatch concrete, because "interviews don't predict performance" is an abstraction, and the thing I watch every week is not abstract.

What the job actually is

Here is a composite week from the senior engineers on my teams — composite, not hypothetical; every item on this list is something I have watched a senior engineer actually do in the last quarter.

A bug report comes in: intermittent, customer-facing, unreproducible in staging. The symptom is in the front end — a spinner that never resolves. The cause is not. Over two days the investigation moves from a React state race, to an API gateway timeout, to a backend service that retries non-idempotent writes, to a database lock contention pattern that only appears under a specific tenant's data shape, to a cloud networking configuration that silently drops long-lived connections. Five layers, four repos, three teams, two time zones. The fix is eleven lines. Finding the eleven lines is the job.

The same week, that engineer spends four hours in design review pushing back on a proposal — not because the design is wrong but because it solves a problem the roadmap is about to make irrelevant, and the kindest thing to do is kill six weeks of elegant work before it starts. Deciding what not to build is the job.

And threaded through all of it: reading. Reading other people's code, reading diffs, reading incident timelines, reading the AI-generated pull requests that now make up a substantial share of the review queue and that are wrong in newer, subtler ways than human code used to be wrong. Most estimates of how engineers spend their time have always put comprehension well ahead of composition, and the agentic-coding era has pushed the ratio further: when the writing is cheap, the reading is the work.

Treat the numbers as a sketch, not a survey — they are my read of my own teams, and your shop will differ. The shape, though, I will defend anywhere: writing fresh code from a blank page against a well-specified, self-contained problem is a minority activity for a senior engineer, and it is shrinking.

Now place the standard interview loop next to that week.

I have never once traversed a binary tree during a production incident. I have also never, in ten years of leading engineering teams, seen a senior engineer fail because they could not produce an optimal algorithm in forty-five minutes. I have watched several fail — expensively, slowly, painfully — because they could not read a system they did not write, could not get three teams to agree on an interface, or could not tell a beautiful design from a necessary one. The loop tests the first thing. The job is the second thing.

The steelman: why the ritual survived every critique

If the whiteboard ritual were merely stupid it would not have survived forty years of criticism, and pretending it is merely stupid guarantees the argument goes nowhere. The ritual survived because it delivers real administrative goods that its critics rarely price honestly. State them at full strength:

It scales. A puzzle round can be run by any trained engineer against any candidate for any team. A company hiring two thousand engineers a year can run tens of thousands of interchangeable rounds with a shared rubric and a calibration process. Bespoke, role-specific assessment is a boutique product; LeetCode-style screening is an industrial one.

It is comparable and defensible. Every candidate faces the same category of task under the same conditions. When a hiring decision is challenged — internally by a debrief committee, externally by a regulator or a lawsuit — "everyone answers structurally similar questions, scored on a rubric" is a position a company can defend. Structured process is not merely bureaucratic cover; unstructured interviews are where bias does its best work, and the literature is unambiguous that structure helps.

It protects the interviewer's time. An engineer can run a phone screen with thirty minutes of prep because the question bank is shared and the format is fixed. Work-sample assessment, done lazily, costs far more interviewer time per candidate — and interviewer time is the scarcest resource in any hiring system, a constraint I feel personally every quarter.

It filters for a real, if narrow, signal. Preparing for algorithmic interviews takes sustained effort. For years, willingness to grind several hundred practice problems correlated — roughly, unfairly, but genuinely — with a mix of motivation, working memory, and baseline fluency that was better than nothing, and much better than credential-reading.

That is the honest case, and for two decades I think it approximately held. The ritual was never a good measure of the job, but it was a cheap, scalable, legally defensible proxy with nonzero signal, and nobody had a replacement that preserved the administrative goods. Inertia did the rest.

Every load-bearing element of that case has now failed at once. What broke it was not a better argument. It was AI.

AI did not bend the signal. It severed it.

The puzzle round's residual value rested on one assumption: that performance in the round was produced by the candidate, from ability plus preparation, live. Every part of that assumption is now false or unverifiable.

Frontier models solve the overwhelming majority of the standard interview-question corpus — including most problems tagged hard on the major practice platforms — faster and more reliably than any human candidate who will ever sit in your loop. The corpus is in the training data; the puzzle format is precisely the well-specified, self-contained, blank-page task that models are best at. The same capability that made AI genuinely useful on my teams made the interview's core artifact reproducible on demand for the price of a token stream.

And the tooling to deploy that capability inside a live interview is not a hypothetical. By early 2025 a college student had been suspended for building an overlay tool whose explicit pitch was solving the coding interview in real time, undetectably, during the call — and rather than ending his career, the stunt raised venture capital at a striking valuation on the thesis that the interview format itself was indefensible. Whatever you think of the ethics, read the market signal: investors priced the whiteboard interview as an arms race the assessor loses.

The integrity problem generalizes past assistance into identity. I wrote up a senior frontend interview where the interrogation spent forty-five minutes on the wrong domain while the company's own codebase sat publicly exposed — a failure of assessor competence. But the mirror image is now routine too: candidates who are not who they claim to be at all, from proxy-interviewing services to the documented campaigns of North Korean operatives passing video interviews at US companies with stolen identities and AI-assisted personas. Remote hiring pipelines built on "watch a stranger solve a puzzle over video" were designed for a threat model that no longer exists.

Here is the part I find genuinely perverse: the standard loop now selects for AI fluency in exactly the wrong direction. It disqualifies candidates for using, in the interview, the tools I require them to use in the job — while rewarding whoever conceals the same usage best. I push AI tooling on my teams because the leverage is real; refusing it is malpractice. An assessment that bans the job's primary instrument, and cannot enforce the ban, is not measuring readiness for the job. It is measuring compliance with a fiction, or skill at violating it quietly.

So the equilibrium broke. The administrative goods — scale, comparability, defensibility — presumed the round measured something. Zero times scalable is zero. Zero times comparable is zero. Zero times legally defensible is a lawsuit with better paperwork.

Where the signal went

A theme I keep returning to on this site is that when a capability commoditizes, the value does not evaporate — it relocates to whatever is adjacent and scarce. It is true of models, it is true of code, and it is true here, because interviewing is just measurement, and measurement has to chase the scarcity.

Composition commoditized. A well-specified function, a known algorithm, a CRUD endpoint, a passing test suite for a described behavior — the machine does that now, cheaply and mostly competently. What did not commoditize is everything wrapped around composition: knowing what to ask the machine for, recognizing when its confident output is subtly wrong, holding the mental model of a system too large and too old for any context window, and making the judgment calls — architectural, organizational, economic — that determine whether the cheap code is the right code. Every one of those scarce skills is exercised on artifacts that already exist: real systems, real bugs, real constraints, real disagreements.

You cannot measure any of them with a blank editor and a puzzle. The blank editor was always the wrong instrument; now it is the wrong instrument measuring a commodity. An interview that wants signal has one place left to find it — contact with the real work — which is precisely the thing the industrial-psychology literature had crowned before most of us were writing software: the work sample.

The interview should look like the job. Here is what that means when you operationalize it.

The protocol: a loop that looks like the job

What follows is not a thought experiment. It is the shape of the loop I have been converging on in my own hiring, stated concretely enough to steal. It fits in roughly half a day of candidate time — less than the standard onsite — and every stage produces evidence about the actual role.

Some design notes that matter more than the stages themselves.

AI is inside the assessment, not banned from it. Every stage assumes the tools exist because the job assumes the tools exist. This dissolves the integrity arms race instead of escalating it: there is nothing to conceal when the assistant is on the table, and the signal moves to where it now lives — the quality of the candidate's judgment about the machine's output. You cannot cheat at judgment with the thing being judged.

The artifacts are real, which is why they resist rehearsal. A candidate can grind a thousand puzzles because puzzles are a closed corpus. They cannot pre-rehearse your codebase, your incident, or your design fight. The assessment inherits the unfakeable specificity of your actual system — the same property that makes the job hard.

Structure is preserved, so the administrative goods survive. Every candidate for a role gets the same excerpt, the same incident, the same design doc, scored on the same rubric by calibrated interviewers. This matters enormously: work-sample loops run lazily degenerate into vibes, and vibes are where bias lives. The defensibility case for a structured work sample is at least as strong as for the puzzle round, with the advantage that the thing being scored is job-relevant on its face.

The candidate learns as much as you do. A puzzle round tells the candidate nothing about the job. This loop is the job, in miniature — the codebase's actual texture, the incidents' actual messiness, the disagreements' actual stakes. Strong senior candidates consistently report that a loop like this sells them harder than any pitch, and the ones who recoil from your real codebase were going to recoil in week two anyway. Cheaper to learn it now, in both directions.

The one objection I take most seriously is interviewer time, because it is the objection I feel in my own calendar. The honest answer is that the old loop's cheapness was an accounting illusion: it moved the cost downstream, into mis-hires, into six-month discoveries that the puzzle champion cannot read a system, into the attrition of strong candidates who correctly read the whiteboard ritual as a signal about the company's engineering culture. The work-sample loop pays the cost where you can see it. Seeing costs is what engineering leaders are for.

Scoring it: rubrics that survive contact with humans

The fastest way to ruin a work-sample loop is to run it without deciding, in advance and in writing, what good looks like. The puzzle round's great hidden virtue was that it made scoring trivial — the solution compiles or it does not, runs in the right complexity class or does not — and any replacement that hand-waves the scoring problem will lose the argument with your recruiting organization, and deserve to. So the rubric work is not a detail. It is half the build.

The principle is anchored behavioral scoring: for each stage, before any candidate sits it, the interviewers who built it write down concrete, observable behaviors that define each level — not adjectives, behaviors. For the codebase walkthrough, a strong performance is not "understood the code well." It is: identified the load-bearing module unprompted; distinguished deliberate design decisions from accumulated accidents; proposed a refactor and then argued against doing it now on cost grounds. A weak performance is equally concrete: narrated the code line by line without forming a model of it; proposed rewrites reflexively; missed the one component the excerpt was chosen to surface. Write five or six anchors per level per stage. It is tedious. It is also the entire difference between an assessment and an impression.

For the incident replay, the anchors that have earned their place on my list all concern process rather than outcome: does the candidate form a hypothesis before touching anything, or flail; do they interrogate the AI assistant's confident wrong answer or accept it; do they narrate their uncertainty honestly; when the first hypothesis dies, do they update or dig in. Whether they actually land the root cause inside seventy-five minutes is the least informative bit in the stage — some of the best debugging performances I have scored ended unfinished, and said more than a lucky early guess ever could.

Calibration closes the loop. Every interviewer scores one recorded or shadowed session per quarter against the anchors, and the panel argues the deltas out loud. Where two calibrated interviewers diverge by more than a level, the anchor is ambiguous and gets rewritten — the disagreement is diagnostic data about the rubric, not about the interviewers. This is unglamorous machinery, and it is exactly the machinery the puzzle loop always claimed as its unique advantage. There is nothing about realism that forfeits rigor. You just have to build the rigor instead of inheriting it.

Two rules of hygiene, learned the annoying way. First, score independently before the debrief, in writing, every time — the moment scoring happens in the room, the loudest interviewer's impression becomes everyone's data. Second, rotate the artifacts on a schedule. A real incident stays unrehearsable only while it stays private; once a role family has cycled enough candidates, assume the excerpt is discussed somewhere and retire it. Artifact rotation is the work-sample loop's version of refreshing the question bank, at roughly the same maintenance cost.

Migrating without stalling the pipeline

None of this survives a memo that says "we are replacing our interview process," because hiring is a running system with an SLA, and no engineering leader gets to take it down for a rewrite. The migration that works is the one that looks like every other risky migration: shadow mode first, one surface at a time, with metrics that can kill the rollout.

Pilot on a single role family — one where you are feeling the old loop's failure most sharply; for me that was senior backend, where puzzle performance and job performance had visibly decoupled. Build the artifacts for that family only. Then, for one quarter, run the new loop as an additional signal alongside the old one rather than a replacement: candidates complete both, interviewers score both blind to each other's results, and the hiring decision still keys off the old process. This costs real candidate goodwill — a longer loop, temporarily — and you should be honest with candidates about why, which most take surprisingly well; being told "we are validating a work-sample process against the whiteboard one" is itself a signal about the company that strong candidates read correctly.

What shadow mode buys is the number the whole argument turns on: divergence. Track every candidate the two processes disagree on — the puzzle-strong, sample-weak cohort and the reverse — and follow the hires among them into their first two quarters. My experience, consistent with what the work-sample literature predicted decades before AI raised the stakes, is that the sample-weak cohort's struggles show up fast and in exactly the places the stages flagged: the engineer who narrated code without modeling it in the walkthrough is the same engineer whose first on-call rotation goes sideways. Once the divergence data points one direction for a couple of quarters, the argument inside the organization ends on evidence rather than on whoever argues loudest, and the old rounds can be retired one at a time with the paper trail that recruiting, legal, and the debrief committee all legitimately require.

Then instrument the steady state. Quality-of-hire is the metric everyone cites and nobody defines, so define it narrowly and in advance: manager-assessed performance against level expectations at two quarters, ramp time to first meaningful production contribution, and first-year regretted attrition, on both sides — regretted departures and regretted retentions. None of those is a clean number; all of them beat the metric the old loop optimized, which was, if we are honest, interviewer confidence at debrief — a measure of how persuasive the ritual felt from the inside, which is to say, a measure of nothing.

The other proxy is dying too

Interviews are one of hiring's two great proxies. The other is the resume — a self-authored, unverified claim sheet, parsed by machines, increasingly written by machines, screening candidates increasingly coached by machines. Everything AI did to the puzzle round it is doing to the resume, simultaneously: when generation is free, the self-authored artifact carries no signal, and the measurement has to relocate to something verifiable.

I have a dated, graded position on where that goes: my prediction that at least two top-ten talent platforms will ship verified-source candidate profiles that bypass the self-authored resume by January 2028 — profiles generated from linked repositories, verified employment records, and signed credentials rather than from a parsed PDF. I put it at 60 percent confidence, with the resolution criteria stated in advance, and my Bullhorn disclosure attached there in full. If it resolves yes, both of hiring's proxies — the claim sheet and the puzzle — will have collapsed into the same replacement: verified contact with real work. The direction of travel is one direction.

The macro context makes the timing less optional than it looks. Hiring volume in engineering is under structural pressure from the same AI wave — a dynamic I covered in the augmentation paradox playing out across radiology, paralegals, and agentic hiring — which means each individual hire carries more weight, which raises the price of a noisy selection instrument exactly when the instrument's noise floor gave out. Fewer, more consequential hires, chosen by a broken meter, is not a stable configuration.

The three chairs, one more time

From the hiring manager's chair: my loops exist to answer one question — can this person do the actual work with the actual tools under the actual constraints — and every round that does not produce evidence on that question is theater I am charging my team's hours for.

From the builder's chair: the tooling I work on all day makes matching and intake smarter, and none of it rescues an assessment that measures the wrong thing. Better plumbing for a broken meter yields precisely measured noise.

From the candidate's chair, most recently and most vividly: being asked to perform a rehearsable ritual, while both parties knew a machine in my pocket could outperform me at it, communicated exactly one thing about that company — that its process had not yet noticed the world changed. I declined to continue. So, increasingly, do the people you most want to hire.

The obituary for the whiteboard interview has been written many times, and I have learned to distrust obituaries — the profession's own has been premature for decades. So let me be precise about the claim. The interview is not dying. Selection is not optional; somebody has to decide who gets the job, and the decision should be structured, comparable, and fair. What died is the proxy — the puzzle standing in for the work — because the puzzle can now be performed by the tools it tried to exclude. When a proxy dies, you measure the real thing.

The interview should look like the job. It was always the right answer. Now it is the only one left.

Further reading:

• Beyond the Algorithm: why software engineering interviews are still broken, with the data
• The Live Coding Interview Fallacy: what live coding actually measures
• How AI will replace recruiters and HR professionals: the agentic hiring pipeline
• My graded prediction: verified-source candidate profiles replace the resume by 2028