Measuring DevRel Impact in the Changing Landscape — A Conversation with Mary Thengvall
In this interview, I sat down with Mary Thengvall — author of The Business Value of Developer Relations, founder of Persea Consulting, and former Head of Developer Relations at Camunda, and personally a mentor to me. We talked about what it means to measure DevRel impact when AI is quietly eating the top of the funnel.
Mary's book was one of the first resources I turned to when I was finding my footing as a DevRel practitioner, and she's been just as generous in person, always willing to think through ideas together.
We discussed how measuring DevRel impact is changing (or remaining the same) as AI content generation tools evolve.
We explored it via five questions and some brutally honest answers.
The Metrics That Have Stayed Honest
Question 1. What metrics did you rely on two years ago to justify DevRel investment, and which of those do you still trust today?
Two years ago, Mary’s team at Camunda was running Common Room across their entire community surface — forums, meetups, Reddit — piping identity resolution back into Salesforce to segment engagement by customers, prospects, and partners.
The goal was nuanced: were people engaging more after becoming customers, or going quiet once they had a CSM? That signal shaped how the team positioned community resources internally — as a deflection layer for support, or as a retention driver for existing customers.
That framework hasn’t fundamentally changed. “I think AI presents another way for us to look at metrics, but it’s a question that’s always up in the air and something you need to keep track of,” she said.
What has changed is where engagement happens. Forum traffic is down — but crawler traffic is up, which tells a different story. “We’re not getting as many viewers on the website, but we are getting more information from crawlers, which says to me that people are still asking the questions — they just aren’t having to go direct to the forum.”
Attribution Was Always Broken
Question 2: When a developer discovers and adopts your product through an AI-generated recommendation or code snippet, how do you attribute that? and does it even matter?
When a developer lands on your product after an AI-generated recommendation, does it matter how they got there?
Mary’s take is that attribution through AI or other intermediaries isn’t a new problem — it’s a louder version of the old one. “Did they just use a different email address and we don’t know it, or did they see us at a conference and have an in-person conversation but weren’t ready for us to scan their badge?”
Image generated using Google Nano Banana2 AI
The more useful reframe isn’t how did they arrive — it’s what happens after they do. That’s always been a DevRel responsibility, and AI actually raises the stakes: “How do we improve the AI answers that people are seeing and receiving, and then also improve the experience that they have once they do interact with the website.”
Accurate, up-to-date documentation matters more now, not less. When AI is stitching together an onboarding path from your public content, stale docs become a product defect.
ROI Pressure Has Gotten Real
Question 3: Has the pressure to prove DevRel ROI increased as AI has taken over more of the content work? How are you responding to that internally?
We’re seeing across the industry that DevRel teams are getting cut. Mary didn’t soften this - “We’ve seen so many DevRel teams dissolved — really good, solid programs that have just gone away over the past couple of months. And I think a lot of it has to do with the perception that, well, it’s marketing and content and resources, and AI can write all of that itself.”
Her counterargument, though, is that the things that make DevRel effective aren’t the things AI replicates well. “Developer advocates are always going to know what topics their community members are most interested in, what type of humor is going to resonate, what the in-jokes are.”
She’s not anti-AI — she uses it to draft outlines, sanity-check gaps, brainstorm. But she’s clear-eyed about where it breaks down: “If it’s already that well-known of a topic, do you actually need to publish a blog post about it? Because everyone should already have that information.” Original perspective, earned trust, community instinct — those still need a human.
Leading Indicators That Still Hold
Question 4: Are there leading indicators you’ve found that reliably predict developer adoption or retention, things that hold up even as the top-of-funnel gets noisier?
Predicting adoption is still hard. But our discussion kept coming back to network effects — specifically, the word-of-mouth layer that persists even as AI intermediates more discovery.
Mary says, “There are still plenty of Slack communities that are completely volunteer-run where people go and say, ‘Hey, what tools do people use for X, Y, and Z?’ And those people are also asking AI — but they’re still relying on the people they know and trust and those firsthand testimonies.”
That’s where DevRel’s storytelling function has outsized leverage. You can’t manufacture a peer recommendation, but you can surface the ones that exist and make sure they’re heard at the right moment.
If You Had to Boil It Down
Question 5: If you had to reduce DevRel to one or two outcomes that the rest of the business unambiguously cares about, what would they be?
Mary pushed back on the framing of universal DevRel outcomes — and she’s right to. “The goals of the company often dictate the goals of the DevRel team, and I don’t think that’s a bad thing.” A seed-stage company needs awareness and word-of-mouth reach. A 20-year-old company needs community depth and change management.
At Camunda, the anchor metrics were developer experience improvement (directly tied to retention) and engagement continuity through product transitions. The discipline was always the same: draw a straight line from team goal to company goal, then show contribution at review time.
“You can directly go back to your goals that you had for your team and say, ‘Here’s how we directly contributed to those goals’ — which hopefully helps with those conversations of, okay, but why is your team still important?”
As Kelsey Hightower once put it: DevRel is what your employer pays you to do. Whoever funds the team defines the function. The job is to make sure that definition maps to something the business genuinely can’t live without.
Disclaimer
AI was used to generate illustrations/images (as noted), to fix typos, and/or improve the grammar of the written piece.