How Stacker Built an AI Recommendation Engine Clients Actually Trust

Watch this episode of Building For Others

When Ken Romano and his team at Stacker looked at their mountain of first-party data—years of stories distributed, publishers who picked them up, page views generated—they saw an opportunity. They solved their clients' most persistent question: "What should I write next?"

Writing suggestions come in a million shapes and sizes, and few of them are data-backed. Everyone’s got ideas, but how do you know which will hit?

The result was Sparks, an LLM-powered recommendation engine that now delivers thousands of personalized content suggestions per week to brands across different industries. The real game-changer here is how they combined data and notoriously unreliable large language models into statistically reliable strategic guidance, passed rigorous quality bars, and got people to actually use it.

The Problem: Quality Strategic Guidance Doesn't Scale

Stacker operates as the first earned distribution platform (think AP or Reuters, but sourcing quality journalism from brands rather than traditional newsrooms). Their content strategists advise clients on what types of stories would perform well with publishers. It works spectacularly, but in depth strategic guidance is difficult to scale.

"The biggest question we get from brands when they want to syndicate stories is tell me what type of story to write," Ken explained. "In its simplest terms, the Sparks tool tells them what types of stories that they should write."

The challenge involved maintaining the quality and personalization of human strategists while serving hundreds of clients simultaneously.

How They Built It: Data + Documentation + Iteration

Here's what made Sparks work, broken down into actionable components:

1. They Had the Right Ingredients Before They Started Cooking

Stacker didn't just wake up one day and decide to build an AI tool. The 90-day build was possible because of a year's worth of groundwork:

• First-party data goldmine: Every story ever distributed, who picked it up, and how it performed
• Documented institutional knowledge: Their content strategy team had already created extensive documentation on what makes stories successful
• Clear success criteria: They knew exactly what "good" looked like because humans had been providing data-backed guidance manually

Takeaway: Before building AI solutions, audit what you already have. Your data and your team's documented expertise represent your most valuable assets.

2. They Built Quality Control Into the Algorithm, Not Just the Output

Ken's team didn't just generate recommendations and hope for the best. They built recursive self-evaluation directly into the system:

"We have the algorithm actually score itself and essentially have a conversation with itself to say, am I hitting these five points that need to be met for this to be a quality recommendation? And then it self-corrects before it actually spits out the final output."

The system checks against five specific criteria before any recommendation reaches a client. Crucially, they had their content strategy team test it for weeks before launch, treating the AI's suggestions exactly like they'd treat a new team member's work. The team still tests it on an ongoing basis, to ensure continuing quality.

Takeaway: Build quality assurance into the process, not just at the end. If you can't articulate what "good" looks like in specific, measurable terms, you're not ready to automate.

3. They Started in a Controlled Box

One of Ken's smartest decisions was limiting the scope initially. There was no open-ended chat. There was no ability for clients to submit their own ideas and get AI feedback. Just four curated recommendations every Monday.

"I did not want us to have to worry about every single thing that someone might ask off the top of their head," Ken said. "So in our first iteration of this, it's very much in Stacker's control... it's a pretty safe box to be in to start."

Takeaway: Don't try to boil the ocean. Start with a defined, controllable use case where you can monitor quality and iterate based on real feedback.

The Rollout: Why Great Tech Isn't Enough

Here's where things get really interesting. Stacker's first attempt at putting their data to work with AI actually flopped.

They integrated their entire dataset into an AI assistant so their team could ask questions and get instant answers. Sounds perfect, right? Nobody used it.

"Just staring at that blank screen of entering what you're looking for was way too overwhelming," Ken admitted.

The fix involved two simple changes:

1. Made it available in Slack (where people already were)
2. Gave people specific examples of what to ask

Usage skyrocketed.

The Results: When AI Gets It Right

The best validation came from a healthcare client who produces content about supplements and longevity. Their feedback on Sparks' recommendations:

"These are almost exactly the types of stories that we typically write."

Sparks gave them high-quality inspiration for stories they already wanted to write, validated by data on what actually performs.

How to Approach This at Your Organization

If you're sitting on data and wondering how to turn it into something useful, here's your starting playbook:

Step 1: Audit Your Assets

• What data do you have that tracks success/failure?
• What institutional knowledge exists (even if it's just in people's heads)?
• Who on your team is already solving the problem you want to automate, and how?

Step 2: Document What "Good" Looks Like

• Before you automate anything, can you articulate specific criteria for quality?
• Have your experts write down their decision-making process
• Create examples of great outputs vs. mediocre ones

Step 3: Start Small and Contained

• Pick one specific, high-value use case
• Limit the scope so you can monitor quality
• Build in quality checks before outputs reach end users

Step 4: Design for Adoption, Not Just Functionality

• Meet people where they already work (Slack, email, etc.)
• Provide specific examples and use cases, not just capabilities
• Find your champions (don't rely on top-down mandates)

Step 5: Test Like You're Hiring a New Employee

• Have experts evaluate AI outputs the same way they'd evaluate a junior team member's work
• Iterate based on real feedback from real users
• Be willing to go back to the drawing board (like Stacker did with their first attempt)

The Bigger Picture

Stacker's story illustrates three powerful principles for production-grade AI:

1. Strategic AI scales expertise rather than replacing it. Sparks works because it captured and amplified what content strategists already knew. It didn't invent something entirely new.
   
   
2. Quality at scale requires quality by design. Building recursive self-evaluation and clear success criteria into the system made 400+ weekly recommendations possible without sacrificing reliability.
   
   
3. The hardest part involves adoption, not technology. Even brilliant solutions fail if you don't design for how humans actually work and learn.
   
   

Want to explore how to unlock insights from your own data? Stacker used elvex to make their distribution data queryable in natural language. While their first rollout taught them important lessons about implementation, the underlying capability remains powerful. Teams continue turning complex datasets into accessible strategic guidance. Learn more about elvex and how teams are using AI to get new leverage from their core business assets.