A product spec that used to take a day now takes twenty minutes, and it comes out better organised than the one that took a day. Most teams have booked that as a productivity win.
It is not, quite. The day was never the constraint. Writing the spec was what forced you to find out what you did not know, and removing the writing removed the finding out.
I want to be specific about what changed, because the useful response is not to stop using the tools. It is to notice which signals your organisation was reading, and which of them have quietly stopped carrying information.
What AI in product management actually replaces
Start with a definition, because the term covers two very different things and the difference matters.
Generative work is producing an artefact from context you supply: the spec, the update, the ticket descriptions, the research summary, the competitive teardown. Models are genuinely excellent at this now, and a PM who refuses to use them is choosing to be slower for no benefit.
Judgement work is deciding what is true and deciding what matters. Which of the three user problems you are conflating is the real one. Whether the number in the business case is measuring what its label claims. Whether this quarter goes to the thing that is easy to justify or the thing that is right.
Every credible use of AI in product management sits in the first category. The confusion comes from the fact that judgement work has historically been delivered through artefacts, so improving the artefact looks like improving the judgement.
Why the friction in writing a spec was load-bearing
Think about what actually happened when you wrote a spec by hand. You would get four paragraphs in and hit a sentence you could not finish, because finishing it required a fact you did not have.
That stall was the mechanism. It is where you discovered the dependency nobody had raised, or that two stakeholders meant different things by the same word, or that the success metric could not be measured with the events currently being tracked.
The document was the by-product. The stalling was the process.
Generate the document and the stalls do not happen, because the model does not stall. It has seen ten thousand specs and it will smooth straight over the gap with something plausible. You get a complete artefact that reads as though every question was answered, and no list of the ones that were not.
Fluency and judgement are now indistinguishable on the page
Before, a well-argued, well-structured, comprehensive document was weak evidence that somebody had done the work. Not proof, but it correlated, because writing that well about a problem was hard unless you understood it.
That correlation is gone. Fluency is free now and judgement is not, and a reader cannot tell them apart from the artefact alone.
A stakeholder reading a strong strategy document has no way to know whether it came from six weeks of customer conversations or from a prompt containing the phrase "B2B SaaS onboarding". Both produce confident prose with a plausible structure and a tidy set of next steps.
This has a direct organisational consequence that I have not seen many teams address. If your performance conversations are informed by what a manager remembers, and what a manager remembers is the documents, then you are now evaluating people on a signal that stopped working.
The three product management skills that got more valuable
Being concrete about what survives is more useful than the general worry, because these are the things worth hiring and developing for:
- Establishing what is true. Talking to customers yourself, reading the raw data, noticing when a metric is named after one thing and measuring another. A model can summarise research you give it. It cannot tell you the research was of the wrong twelve people.
- Deciding what matters. Prioritisation is a claim about the world, not an argument. It says this problem is more painful, for people we can actually reach, than that one. It is accountable to outcomes rather than to how well it was written up.
- Holding a position in a room. Most bad roadmaps are not analysis failures. They are what happens when nobody is willing to tell a senior person their idea is not worth a quarter, and generated evidence does not make that conversation easier.
How to tell whether a product manager can actually do the job
This is the part we have direct experience with, because it is the problem our own product exists to solve, and building the assessment engine taught us something we did not expect.
The instinct is to assess knowledge: ask what a good PRD contains, what frameworks they know, how they would run discovery. Those questions are now close to worthless, because they have well-known answers that anybody can produce fluently whether or not they have ever done the work.
What holds up is putting an ambiguous, realistic situation in front of somebody and watching what they do with it. Not "what is the RICE framework" but "here is a request from a large customer, here are three conflicting data points, what would you do first and what would you need to know". The answers separate people immediately, and they cannot be prepared for in the way a definition can.
One design decision we made and would defend: our assessments are five questions with four correct required to pass, written for one specific level rather than for a topic. A shorter test with a lower bar produces a number that everybody clears, and a number everybody clears tells a manager nothing they did not already assume.
What to do differently on Monday
Keep generating the artefacts. They are better and faster, and the time saved is real.
But split the two jobs deliberately. Before you generate anything, write down by hand the three things that would have to be true for this to be worth doing, and how you would find out whether they are. That takes about fifteen minutes and no tool can do it for you, because it is a claim about your specific customers rather than about product management in general.
Then judge the work, yours and your team's, on whether those three things turned out to be true. That is a capability signal that survives the tooling. The document is not, and it has not been for a while.
