The Capability Illusion: is AI hiding your next talent problem?

Until recently, there was a clear relationship between the quality of someone’s work and the capability underneath it. If someone produced strong analysis, they likely had to do a fair amount of thinking to get there. But AI is making polished outputs a less reliable signal of actual capability.

We talk a lot about cognitive offloading, the idea of using external tools like AI to reduce the mental effort required for a task. With AI, a junior employee can now produce work that looks significantly more mature than their underlying ability to frame the problem, make a recommendation, or defend their reasoning. It’s creating a new challenge for leaders.

This isn’t an antagonistic discussion about AI; it’s a recognition that the efficiency of AI comes with a cost. As leaders, we don’t need to preserve the grunt work people used to learn through or force employees to sit there and make a PowerPoint deck so they have the experience of that struggle. But we do need to talk about the capability illusion and how leaders can be more deliberate about recreating the learning opportunities embedded in the work. The challenge isn’t just whether people can produce good work with AI. It’s whether organizations can still tell who is developing the judgment required for greater responsibility.

In the shadows

There’s a gap between how organizations adopt AI and how employees use AI. MIT’s Project NANDA research on the state of AI uptake found that while only 40% of companies say they purchased an official large language model subscription like ChatGPT or Claude, 90% of workers from those companies reported regular usage of personal AI tools for work tasks. 

“Shadow AI” use makes the problem harder to see, but it isn’t the problem itself. Whether the tool is sanctioned or not, leaders may know less than they think about how much of an employee’s output reflects their own judgment. That creates a capability illusion today and, over time, experience debt: people can produce stronger work while missing some of the repetitions, mistakes, feedback, and independent problem-solving that traditionally built judgment. The capability illusion is what leaders see now. Experience debt is what organizations discover later.

So how do you know who’s growing within their role versus who’s gotten better at the technology?

An open discussion

As leaders, we need to understand the how, when, where and why AI is being used by employees. You’re not doing your team members any favours by letting them send AI work out into the world without sitting down to interrogate the quality of the work and AI’s contribution. This should be a regular conversation in meetings. It normalizes it and makes the conversation more productive. 

At the individual level, start by making the thinking behind the work more visible. A few questions can help:

Walk me through how you got here. 

This works best as an invitation, a “help me understand” approach. Get someone to walk you through their process, where AI helped, where they disagreed with it, and what they changed. It gives you insight into where the tool’s contribution ends, and the team member steps in. It also reinforces where the hidden inefficiencies can be in workshopping an idea with an AI tool. 

What was the most important judgment call you made?

This is a chance to test out assumptions. It’s a glimpse of the team member’s ethics, technical foresight, and problem-solving abilities. Have them walk you through the options they considered and the trade-offs they made to solve the problem. Ask them what new information might cause them to change their recommendation. This allows them to go beyond the specific problem they solved and better recognize the judgment they’re building. 

What parts of your approach to this could translate to other challenges or projects within the organization?

Encourage your team members to share their process. Ask them what they learned from solving this problem that could transfer to another challenge the organization is experiencing. Have them identify where the tool slowed down their reasoning and what parts of the workflow from this particular project — in the absence of AI — could help lead to better outcomes. This prompt gives a team member the opportunity to zoom out and share how their approach fits within the wider organizational goals.

The systems lens

Those conversations are only one part of the response. Organizations also need to examine where capability is built. If AI removes an analytical task, a first draft, a difficult client interaction or some other piece of work, what developmental experience disappears with it? The answer isn’t to preserve inefficient work for its own sake, but to understand what that work was teaching and recreate the important parts deliberately. The same applies to assessment and promotion: polished output tells us less than it used to, so organizations need more direct evidence of how people frame problems, exercise judgment and make decisions under pressure.

Many talent systems were built for a world in which doing the work and developing through the work were largely the same thing. AI is beginning to separate those two things. If quality of output is no longer a reliable signal of readiness, organizations will need to become more deliberate about how they assess capability, create developmental experiences and decide who is ready for greater responsibility. The goal isn’t to preserve inefficient work for its own sake. It’s to make sure the efficiency we gain today doesn’t become experience debt we pay for later.