Home • About • Projects • Blog • Resources • Contact
AI Native Products: What Every PM Needs to Know (and Do)
A brilliant overview of what it means to build AI-native products — not just adding AI to existing features, but rethinking product strategy, evaluation, and architecture around the probabilistic nature of AI systems.
It all starts with knowing the problem and whether AI should be used to solve this.
Key Takeaways
🎯 Shift from Deterministic to Probabilistic Outputs
In traditional product you are building deterministic outputs; in AI Native Product, the outputs/outcomes are always probabilistic. Do we want to do 10 things really well, or 10K things 90% well? There are cases where you really want determinism, and cases where it’s ok to have probabilism. AI PM is a mindset difference - need to understand how to manage trade-offs between determinism and non-determinism.
📏 New success metrics
The success metrics are a lot different now. Need to think about hallucination rates, buy-in rates, how to measure whether it’s ethically and responsibly safe - AI products can cause unimaginable danger, need to think about what guardrails need to be put in place.
📐 Build an evaluation-first mindset
Evaluation focussed mindset - so many agents being launched, difficult to know how to measure - how do we know it’s driving value and not going rogue? In order to write good metrics for these, you need to understand the underlying mechanisms otherwise you won’t be able to write good rubrics.
🎂 Sprinkle cake vs Layer cake
- Sprinkle cake = bolt-on AI
- Layer cake = AI from the ground up
Be deliberate: sometimes sprinkles are fine, sometimes you need foundational change
🤔 Is the problem suitable for AI?
Ask yourself:
- Do you have enough data (thousands+ rows)?
- Do patterns exist in that data?
- Does the solution require personalisation, automation, or augmentation (human-in-the-loop)?
🛠 Prototype with care
What’s a small wrapper that could be built that would maximise latent value. Don’t launch anything that could hurt users or business.
If you have an opportunity to invest more, you need a strategy that looks at how to build the layer cake from the ground up.
Do you have consent from the users? What are your users expectations?
Make sure you’ve built up the right architecture to support your AI products.
💡 Fluency, not just hype
- Know when not to use AI — heuristics may be better
- Understand the toolbox: LLMs, RAG, fine-tuning, agents, heuristics, symbolic approaches
- Evaluate models during and after deployment: monitor for drift, ethical impact, UX degradation
Importance of having good product sense/taste; a customer first approach; thinking carefully about whether AI is the right tool or something else would be better. Have enough AI fluency that you can understand what the right problems to go after is; understand user experience and put a high bar on this; think about fail cases and edge cases. Knowing how to translate between business metrics and model metrics - what does good look like and when do we call it good? This is what a good UX experience should look like, and when it fails this is what I want to happen so that it fails gracefully.