AI 'Experts' Who Can't Debug: The Growing Skills Gap
Why many AI 'experts' can't debug their code: inside the growing skills gap between rapid code generation and true expertise.
The Problem Every Engineering Team Recognizes but Few Talk About
A developer joins the team with "AI/ML" plastered across their resume. They spin up impressive demos, generate code at remarkable speed, and pepper conversations with the right buzzwords. Then the first production bug hits — and they cannot explain what their own code does.
This pattern repeats across the software industry with increasing frequency. The tools that make AI accessible to everyone also make it trivially easy to fake expertise. And the cost of that gap between perceived and actual skill lands squarely on the teams left maintaining code nobody truly understands.
How AI Tools Turned Novices into Convincing "Experts"
The core issue is straightforward: large language models produce polished, confident-sounding output regardless of whether the person prompting them understands what they received. As one Hacker News commenter put it, "much of the illusion of capability lies in the expertise of the prompter." Flip that around, and the implication is uncomfortable — when the prompter lacks expertise, the output still looks capable.
This creates a specific failure mode. A developer can generate a working feature in hours, present it convincingly, and receive praise — all without understanding the architectural implications, edge cases, or maintenance burden of what they shipped.
Real numbers: According to IE Insights, researcher Damien Charlotin has documented close to 150 cases of AI hallucinations filed in legal proceedings around the world since June 2023. The most notorious involved Steven Schwartz, a lawyer with 30 years of experience, who submitted a ChatGPT-generated legal brief that appeared flawless on the surface. A judge discovered all six cited cases were fabricated. Schwartz could not even identify basic legal citation formats that he had supposedly used.
This is not a legal profession problem. It is a pattern that applies wherever AI-generated work replaces — rather than supplements — domain knowledge.
Why "Borrowed Comprehension" Costs More Than It Saves
One of the sharpest descriptions of this dynamic comes from a developer on Hacker News who described accepting AI-generated code without understanding as "borrowed comprehension, which you'll repay during maintenance with high interest." The same commenter estimated that what feels like free velocity is "probably more like tech debt at ~40% annual interest."
That metaphor deserves unpacking. A team member who generates 200 lines of working code per day using AI tools looks extraordinarily productive on paper. But if they cannot debug that code when it breaks, cannot refactor it when requirements change, and cannot explain it during code review — someone else on the team absorbs that cost. The 200 lines become a liability, not an asset.
Honest take: AI does not make weak engineers stronger. According to Karat's analysis of the AI engineering skills gap, AI acts as a multiplier, not an equalizer. Engineers lacking strong fundamentals may rely on AI without understanding the outputs, leading to an increase in coding errors. Meanwhile, top performers deliver outsized value. The gap between strong and weak engineers is widening, not shrinking.
The Multiplier Effect in Practice
Consider two developers tasked with building an API integration:
Developer A understands HTTP protocols, authentication patterns, error handling, and rate limiting. They use AI to accelerate boilerplate, generate test cases, and explore unfamiliar library syntax. When the AI suggests a retry mechanism without exponential backoff, they catch it. When it generates a SQL query vulnerable to injection, they fix it. AI saves them hours of typing while their judgment keeps the output production-ready.
Developer B prompts an AI tool to "build an API integration with Service X." They get working code, paste it in, and move to the next ticket. The code works in development. In production, it crashes under load because there is no connection pooling. It leaks API keys in error logs. It silently drops failed requests instead of retrying.
Both developers "shipped a feature." One created value. The other created an incident waiting to happen.
The Hiring Pipeline Problem
The challenge starts before someone even joins the team. Most interview processes were designed for a world where candidates wrote their own code. As Karat notes, organizations that delay updating interview rubrics and formats risk higher mis-hire rates, slower teams, and declining engineering productivity over time.
The signals that used to indicate competence — clean code in a take-home assignment, articulate descriptions of past projects, familiarity with trending frameworks — can now be generated in minutes. A candidate who spent an evening with ChatGPT preparing for an interview can sound indistinguishable from someone with years of hands-on experience.
The Pragmatic Engineer newsletter documented a case where a candidate claimed credentials from Warsaw University of Technology but could not speak Polish and had an accent inconsistent with their claimed background. The recommendation: record video interviews for later review, because "AI-disguised candidates might be hard to spot in a live scenario."
What Genuine AI Competence Actually Looks Like
According to Far Horizons' assessment framework, credentials mean less than learning velocity, theory matters less than practical application, and systematic thinking matters more than either. Their framework evaluates across four dimensions: technical foundations, practical application, strategic thinking, and learning trajectory.
The practical application dimension is where most fake expertise collapses. It is one thing to explain what a transformer architecture does. It is another to debug why a fine-tuned model produces inconsistent outputs in production, or to explain the tradeoffs between different embedding strategies for a specific use case.
Key takeaway for business: The distinction between someone who can use AI tools and someone who understands AI is the same as the distinction between someone who can drive a car and someone who can diagnose why the engine misfires. Both are useful. They are not the same skill.
Five Concrete Signals That Distinguish Real Expertise from AI-Assisted Performance
1. They Can Explain the "Why" Behind Every Decision
Ask a genuine expert why they chose a particular approach, and they will reference tradeoffs: performance vs. readability, consistency vs. flexibility, time-to-ship vs. maintainability. Ask someone leaning entirely on AI, and they will describe what the code does — which is just restating the prompt they gave the AI.
2. They Debug Methodically, Not by Regenerating
When code breaks, the real test begins. A competent engineer reads error traces, forms hypotheses, and isolates variables. An AI-dependent developer's debugging strategy is to paste the error back into the AI and hope for a different output. According to Karat's research, weaker candidates rely on AI blindly and struggle when asked follow-up questions, while strong engineers validate, refine, and debug AI output and can explain their reasoning.
3. They Push Back on AI Suggestions
Someone who understands the domain will reject bad AI output. They will say "this approach does not scale" or "this introduces a race condition." Someone who lacks that foundation accepts whatever the model returns, because they have no basis for evaluating it.
The IE Insights article highlights this exact danger: when AI fabricates plausible-sounding analysis "in domains you personally don't understand, the failure goes undetected." Novices lack the metacognitive skills needed to evaluate AI outputs — they do not know what they do not know.
4. They Understand Limitations and Failure Modes
Real AI practitioners can articulate where their tools break down. They know that LLMs hallucinate, that generated code often lacks edge case handling, and that impressive demos frequently hide fragile implementations. They treat AI as a tool with known failure modes, not as an oracle.
5. They Simplify Rather Than Accumulate Complexity
A Hacker News commenter challenged the claim that "only humans can decrease or resist complexity," calling it trivially false. But the deeper point stands: genuine expertise shows in the ability to produce simple, maintainable solutions. AI-dependent work tends toward complexity because the developer accepts whatever structure the model generates rather than reshaping it to fit the codebase.
How to Restructure Hiring and Team Evaluation
Update Interview Processes for the AI Era
Traditional coding interviews — timed algorithm challenges on a whiteboard — test a skill set that is increasingly irrelevant. But replacing them with take-home projects tests nothing at all, since candidates can generate entire applications with AI tools.
Here is what we recommend based on patterns observed across multiple engineering teams:
- Live debugging sessions: Give candidates a broken codebase and watch how they diagnose issues. This tests understanding, not generation.
- Architecture discussions with constraints: Present a system design problem with specific, conflicting requirements (low latency and strong consistency and minimal cost). AI tools struggle with genuine tradeoff analysis.
- Code review exercises: Show candidates someone else's code (AI-generated, with subtle issues) and ask them to review it. This tests judgment directly.
- Follow-up depth: For any technical answer, ask "why" three times. Surface-level knowledge collapses by the second follow-up.
According to Canditech's assessment guide, true AI competency for both technical and nontechnical roles centers on human judgment: asking better questions, validating outputs, and applying results with critical thinking. The tool itself is secondary to the operator's ability to evaluate what it produces.
Establish Clear Expectations Within Existing Teams
The problem is not that people use AI tools. The problem is that some people use AI tools as a substitute for understanding rather than as an accelerator for competent work. Making this distinction explicit within a team changes behavior.
Concrete practices that help:
- Require authors to explain their PRs verbally. Not a written description — a live walkthrough. If someone cannot explain the logic behind their changes, the PR needs revision regardless of whether it passes tests.
- Rotate debugging responsibilities. When production incidents occur, the person who wrote the relevant code should lead the investigation. If they cannot, that is a signal worth paying attention to.
- Track code churn per contributor. High velocity followed by high revision rates indicates the "generate and hope" pattern. Sustainable contributors ship less but rewrite less.
Invest in Fundamentals, Not Tools
The temptation is to send the team to "AI training" — prompt engineering workshops, tool-specific certifications, courses on the latest model capabilities. This inverts the actual priority.
Put simply: a strong engineer who learns AI tools in a week will outperform a weak engineer who has used AI tools for a year. The bottleneck is engineering fundamentals: data structures, system design, debugging methodology, and the ability to reason about code behavior. AI tools amplify whatever skill level exists. The investment should target the foundation, not the amplifier.
The Organizational Cost of Ignoring This Problem
Teams that tolerate the gap between perceived and actual expertise pay in several ways:
Maintenance burden shifts to the strongest engineers. When AI-generated code breaks and the author cannot fix it, senior engineers absorb the debugging load. This is the most expensive possible use of the most valuable people on the team.
Architecture degrades gradually. Each AI-generated module that "works but nobody understands" adds to a codebase that becomes increasingly difficult to modify. Over months, the system becomes brittle — not because any single piece is broken, but because nobody holds a mental model of how the pieces connect.
Hiring decisions compound. If the interview process cannot distinguish genuine expertise from AI-assisted performance, each new hire has a meaningful probability of being someone who will slow the team down rather than speed it up. Over several hiring cycles, this changes the team's capability profile fundamentally.
Real numbers: While exact figures vary by organization, the pattern is consistent — the cost of a mis-hire in engineering is typically estimated at 6 to 12 months of salary when accounting for onboarding time, lost productivity, and the impact on surrounding team members. When that mis-hire involves someone whose core skill is generating code they cannot maintain, the cost extends further into the codebase itself.
What This Means for Engineering Leadership
The disillusionment that engineering teams feel toward "AI experts" is not cynicism — it is pattern recognition. People who have watched colleagues generate impressive-looking code that collapses in production are identifying a real problem. The question is what to do about it.
Three principles worth anchoring on:
First, separate AI tool usage from AI expertise. Everyone on the team should use AI tools. That does not make everyone an AI expert, just as everyone using a search engine does not make everyone a researcher. Normalize tool usage while maintaining clear standards for code ownership and understanding.
Second, test for understanding, not output. Whether hiring or evaluating existing team members, the question is not "can this person produce working code?" The question is "can this person explain, debug, modify, and maintain this code?" Those are fundamentally different capabilities.
Third, protect the team's knowledge base. Every line of code that ships without someone on the team genuinely understanding it is a future incident. This does not mean banning AI — it means requiring that someone takes ownership of understanding what ships, regardless of how it was generated.
Key takeaway for business: AI tools are not going away, and teams that use them effectively will outperform those that do not. But "effectively" means using AI to accelerate competent work, not to simulate competence that does not exist. The teams that thrive will be the ones that can tell the difference — and that starts with being honest about the gap.
Frequently Asked Questions
How do we maintain code quality and architectural knowledge when AI generates most of the code?
Require every PR author to explain their changes in a live walkthrough. Rotate debugging responsibilities so the person who wrote the code leads incident investigation. Track code churn — high output followed by high revision rates signals a pattern of generating without understanding.
Is the pressure to ship more code with AI making our systems more fragile rather than better?
Often, yes. As described in Hacker News discussions, accepting AI-generated code without understanding creates "borrowed comprehension" that teams repay during maintenance. The fix is not slower shipping — it is ensuring that whoever ships the code can also own it in production.
How can we ensure our AI systems are explainable, reliable, and trustworthy rather than black boxes?
Start by ensuring the people building the systems can explain them. If a team member cannot articulate why their AI-assisted solution works, what its failure modes are, and how to debug it when it breaks, the system is a black box regardless of the underlying technology. Hire and evaluate for explanatory ability, not just output generation.
This article is based on publicly available sources and may contain inaccuracies.


