How AI products are actually built at US Big Tech companies: approaches, mistakes, and the new role of the PM
Abstract
AI products break the assumptions most product management practice is built on: deterministic outputs, stable specs, and a clean line between engineering and design. Drawing on direct experience building AI features inside US Big Tech companies, this talk examines how product development actually happens once a model enters the stack, and where teams go wrong. We look at recurring mistakes: shipping on demo quality instead of eval quality, treating prompts as throwaway code, and letting model releases dictate the roadmap instead of user needs. We also trace how the PM role itself is shifting: from writing specs to defining evaluation criteria, from sole roadmap owner to translator between research, safety, and engineering, and from planning in quarters to iterating in days. The talk closes with a practical framework, three habits any PM can adopt to work effectively with AI, regardless of company size. Aimed at PMs, engineers, and anyone shipping AI features, it trades slogans for the operational detail of how this work actually gets done.
- Published in
- Proceedings of the Raptors Conference, Vol. 1 (2026)
- Conference
- Raptors Conference 2026, 21 October 2026, online
- Accepted
- 2026-09-16
- DOI
- https://doi.org/10.68495/rc.2026.C005
- Presentation
- Slides (PDF)
- Licence
- © 2026Andrei Miloserdov. Published under Creative Commons Attribution 4.0 International (CC BY 4.0).
- Publisher
- Hackathon Raptors, Community Interest Company 15557917, London, United Kingdom
- Permanent address
- https://proceedings.raptors.dev/miloserdov-ai-products-are-actually
Open access. Every entry in the Proceedings is free to read, with no charge to authors and no charge to readers. Slides are served from this site, so they stay available at the paper's permanent address.
Cite this paper
Andrei Miloserdov (2026). How AI products are actually built at US Big Tech companies: approaches, mistakes, and the new role of the PM. Proceedings of the Raptors Conference, Vol. 1 (2026). Hackathon Raptors, London. https://doi.org/10.68495/rc.2026.C005
@inproceedings{raptors2026-c005,
title = {{How AI products are actually built at US Big Tech companies: approaches, mistakes, and the new role of the PM}},
author = {Miloserdov, Andrei},
booktitle = {Proceedings of the Raptors Conference},
volume = {1},
year = {2026},
publisher = {Hackathon Raptors},
address = {London, United Kingdom},
doi = {10.68495/rc.2026.C005},
url = {https://doi.org/10.68495/rc.2026.C005},
abstract = {AI products break the assumptions most product management practice is built on: deterministic outputs, stable specs, and a clean line between engineering and design. Drawing on direct experience building AI features inside US Big Tech companies, this talk examines how product development actually happens once a model enters the stack, and where teams go wrong. We look at recurring mistakes: shipping on demo quality instead of eval quality, treating prompts as throwaway code, and letting model releases dictate the roadmap instead of user needs. We also trace how the PM role itself is shifting: from writing specs to defining evaluation criteria, from sole roadmap owner to translator between research, safety, and engineering, and from planning in quarters to iterating in days. The talk closes with a practical framework, three habits any PM can adopt to work effectively with AI, regardless of company size. Aimed at PMs, engineers, and anyone shipping AI features, it trades slogans for the operational detail of how this work actually gets done.}
}
TY - CONF TI - How AI products are actually built at US Big Tech companies: approaches, mistakes, and the new role of the PM AU - Miloserdov, Andrei T2 - Proceedings of the Raptors Conference VL - 1 PY - 2026 DA - 2026/09/16 PB - Hackathon Raptors CY - London, United Kingdom DO - 10.68495/rc.2026.C005 UR - https://doi.org/10.68495/rc.2026.C005 AB - AI products break the assumptions most product management practice is built on: deterministic outputs, stable specs, and a clean line between engineering and design. Drawing on direct experience building AI features inside US Big Tech companies, this talk examines how product development actually happens once a model enters the stack, and where teams go wrong. We look at recurring mistakes: shipping on demo quality instead of eval quality, treating prompts as throwaway code, and letting model releases dictate the roadmap instead of user needs. We also trace how the PM role itself is shifting: from writing specs to defining evaluation criteria, from sole roadmap owner to translator between research, safety, and engineering, and from planning in quarters to iterating in days. The talk closes with a practical framework, three habits any PM can adopt to work effectively with AI, regardless of company size. Aimed at PMs, engineers, and anyone shipping AI features, it trades slogans for the operational detail of how this work actually gets done. LA - en ER -