In our latest episode of Deployed, we sat down with Tyler Phillips, AI Product Lead at Apollo.io, to discuss the details of what it looks like to building AI products at a PM at one of the fastest-growing sales platforms.

If you don't know Apollo yet, you should! They've been at the forefront of using AI to transform how sales teams prospect, engage, and convert leads. They've quickly become a powerhouse in the sales space, serving millions of sellers across 500,000 companies and crossing the $100M ARR milestone. They raised their Series D at more than a $1B valuation, and are backed by top investors like Bain Capital, Sequoia, and fellow Freeplay investor Pathlight.

In this conversation, we get into the role of product managers in building AI products, and explore what it really takes to build effective AI products in this space. As Tyler shared with us, the day-to-day reality of AI product management is quite different from what many expect — it can be both more in the weeds, and more strategic at the same time.

In this conversation, we talk about:

• The "unsexy" but critical PM work that makes great AI products possible

• Why domain expertise trumps technical knowledge for certain types of AI prompting

• Apollo's framework for evaluating AI quality across their products

• What aspiring AI PMs should focus on to succeed in this rapidly evolving field

Check out the full episode on Spotify, Apple Podcasts, and YouTube. Some of our favorite highlights below.

The #1 Job of an AI Product Manager: Delivering Quality

If there's one takeaway from our conversation with Tyler, it's that delivering a quality product should be the north star for AI product managers.

"Quality is like the number one thing you can deliver as an AI PM… UX is also important, but quality is definitely number one. So I would argue like if you're not getting involved in that process as a PM, it's very hard for you to have a highly retentive product."

Tyler described how Apollo has created a systematic evaluation process for their AI outputs, scoring each on four key criteria: accuracy, relevance, clarity, and tone. Each dimension gets a score from 1-3, with a target average above 2.5 before shipping.

What's striking is how hands-on he's been with this process. Sometimes people have a perception that PMs should stay at a high level and not get into the details with AI products. But for one of their most successful AI agent products, Tyler spent 2-5 hours every week personally reviewing spreadsheets of outputs across dozens of different company examples to ensure quality.

Apollo's Primary Evaluation Framework

• Accuracy: Is the information factually correct?

• Relevance: Does it address what the user actually needs?

• Clarity: Is it easy to understand and actionable?

• Tone: Does it match the appropriate voice for sales communications?

Domain Experts Should Write Prompts, Not Engineers

Another insight from our conversation was about who should be responsible for prompt engineering. At Apollo, they made a strategic decision to hire dedicated prompt writers with sales expertise rather than having their ML engineers write prompts.

This perspective aligns with what we've observed as being increasingly common across the industry. The most effective AI products depend on really nailing the details for a given domain, and often the best people to help do that aren't engineers. Instead, domain experts teamed up with strong engineers can learn best practices with prompt engineering and other generative AI techniques, and help crank out great products. (See also: our recent post about the "AI Quality Lead" role)

Apollo's AI Team Structure

• AI/ML Engineers: Focus on model implementation, APIs, and instrumenting agentic workflows

• Prompt Writers: Domain experts who craft effective prompts for specific use cases

• AI Product Managers: Define quality standards and coordinate evaluation efforts

• Sales Reps: Provide real-world testing and feedback on AI outputs (internal dogfooding)

This combo of roles enables faster iteration and higher-quality outputs by bringing subject matter experts directly into the development process.

Practical Advice for AI Product Managers

For those looking to build or improve AI products, Tyler offered several actionable recommendations:

1. Start with the problem, not the technology

Before jumping to AI as a solution, establish a clear framework for when AI is appropriate for your product. "Everyone defaults to AI as a solution, but that's not really the right way to think about it," Tyler noted.

2. Focus on UX transparency

Users need to understand what the AI is doing and be able to verify its outputs. This builds trust and allows for correction when needed.

3. Immerse yourself in AI

"Simply immerse yourself in AI every day. Every single thing that you do, try to use AI for it," Tyler advised. Being your own power user helps build intuition about what works and what doesn't.

4. Learn by doing

Start small, pick a specific manual workflow you could enhance with AI, and build an MVP. "That feedback loop will at least give you the experience to say, you know what, I've built an AI product that's been successful."

5. Don't assume everyone wants to be an AI expert

One of Apollo's key learnings was that most users don't want to write prompts or understand AI internals—they just want to get their job done faster. This led to a pivot from open prompting interfaces for users, to more guided experiences where "we'll do all the work in the background."

The key insight driving all these directions is that AI should simplify workflows, not create new complexity. "The only way for mass adoption is to just simplify their (customers') workflow with AI working magically in the background," Tyler explained.

Conclusion

For everyone thinking about the role of AI PMs, Tyler offers some important insights. AI product management is less about chasing the latest technology trends and more about the disciplined, sometimes unglamorous work of ensuring quality and product relevance.

For product and engineering leaders looking to build effective AI features, the message is clear: invest in quality evaluation processes, bring domain experts into your development team, and focus relentlessly on solving real user problems rather than showcasing AI capabilities.

Want to hear more insights from Tyler? Listen to the full episode on Spotify, Apple Podcasts, and YouTube.