Skip to content
← All notes

Manual QA is coming back, and vibe coding is the reason

Generated code fails differently: locally perfect, globally wrong, and invisible to a suite derived from the same misunderstanding. That failure mode needs a human.

QA7 min read

For about a decade the industry told manual testers to learn automation or find another career. At the time that was reasonable advice, and a great deal of repetitive click-through work genuinely should have been scripted.

Something underneath it changed, and it changed faster than hiring plans did. The defect profile of the code being merged is different now, and the difference is exactly the kind that automated suites cannot see.

What vibe coding changed about the code being merged

A growing share of what lands in main was generated rather than written, and reviewed rather than understood. That is not a comment on careless teams. It happens on good ones, under normal deadline pressure, using tools that produce competent output most of the time.

Traditional bugs cluster where a person got confused: gnarly conditionals, off-by-one errors, the third state nobody considered, the error path that was never exercised. Generated code is frequently strongest exactly there, because those patterns are extremely well represented in what it learned from.

It fails somewhere else instead. It fails by being locally perfect and globally wrong.

The naming is consistent, the structure is clean, the logic is defensible, and it solves a subtly different problem from the one this system needed solved. Every automated check passes, because every automated check was derived from the same misunderstanding.

Code that is locally perfect and globally wrong: clean naming, clear structure, defensible logic, solving a subtly different problem from the one the system actually needed solved.
It does not fail where a confused person would have failed.

Why automated tests are blind to this failure mode

An automated test encodes an expectation somebody had. It catches regressions against anticipated behaviour, which is a genuinely valuable thing to catch.

By construction it cannot catch the case where the anticipation itself was wrong. If the developer misunderstood the requirement and the tests were written from the same understanding, or generated from the resulting code, then the suite agrees with the bug.

A person using the product notices in under a minute. Not because they are cleverer than the suite, but because they are doing something it never does: forming an expectation from outside the code, from what a reasonable user would want, and feeling the friction where reality diverges from it.

That is exploratory testing, and it is the specific skill the industry spent a decade describing as obsolete.

A person using the product the way a person would, noticing that a confidently produced result is wrong, where a suite built from the same misunderstanding as the code agrees with it and passes.
The suite and the code share a blind spot. A human does not.

What good exploratory testing actually involves

It is worth being precise here, because manual testing has a reputation problem that was honestly earned by the version of it that really was replaceable: reading a script of fifty steps and ticking boxes. That work should be automated, and largely has been.

The version that is now scarce looks nothing like it:

  • Building a mental model of what the system is for, then deliberately hunting where reality and the model come apart.
  • Noticing that a flow is technically correct and practically unusable. The confirmation dialog nobody would ever find. The error message that says what went wrong but not what to do about it.
  • Following a suspicion across feature boundaries. Real defects live between components, and suites are almost always organised inside them.
  • Writing a defect report a developer can act on immediately, which requires understanding the system well enough to describe a probable cause rather than a symptom.
  • Knowing when to stop. Exploratory testing without a time box expands to fill any schedule you give it.

The hiring problem this creates

Teams that now want this skill are finding it hard to source, and the cause is self-inflicted. Ten years of framing manual testing as a stepping stone pushed the people who were good at it into automation roles, into product, or out of the industry entirely.

It is also genuinely awkward to assess. Automation skill is demonstrable: show me the framework, show me the repository, walk me through the flaky test you fixed. Exploratory skill shows up as a track record of having found things, which is easy to claim and hard to evidence.

The approach that works is to test it directly. Give a candidate a real build with real defects seeded in it and half an hour with no script. What they find matters less than how they decide where to look, and that conversation tells you more than any CV.

This is the same shape of problem we hit building skill assessments in general: for anything where the output can be faked, you have to observe the process rather than grade the artefact.

The honest caveat

This is not a reversal, and anybody selling it as one is overclaiming. Automated suites are not going away. Regression testing should stay automated. A team that reads this, hires six manual testers and lets CI rot will get exactly what it deserves.

The claim is narrower than that. The ratio moved. When code volume rises and the number of people who have genuinely read that code falls, the value of somebody who can look at working software and tell you it is wrong goes up.

That is a real change in what capability is worth, and most organisations' job architectures have not caught up with it.

Common questions

Is manual testing making a comeback?
Exploratory testing is, and scripted manual testing is not. The repetitive script-following work was correctly automated. What has become more valuable is the skill of forming expectations from outside the code and finding where working software is subtly wrong, because that is the failure mode generated code produces most often.
Why can't automated tests catch bugs in AI-generated code?
They can catch regressions, but not misunderstandings. If the code implements the wrong requirement and the tests were written from the same understanding, or generated from the code itself, the suite will confirm the bug. The failure is not in the assertions, it is in the expectation they encode.
What is the difference between exploratory testing and manual testing?
Manual testing often means executing a predefined script by hand, which is a good candidate for automation. Exploratory testing means simultaneously designing and running tests based on a mental model of the system, deliberately looking for where reality and that model diverge. Only the second is hard to automate.
How do you assess exploratory testing skill in an interview?
Give the candidate a real build with defects seeded in it and a fixed time box, with no script. Pay more attention to how they choose where to look and how they describe what they find than to the raw count of defects. A written report you could hand to a developer is a strong signal.

PDP Quest exists because of the problem underneath all of these: when output stops indicating capability, you need another way to know who can actually do the work.

Keep reading

Three more notes on the same argument.