Most AI visibility tools run a prompt once a day and report the result as fact. In a new two-host format, Paris Childress and Simon Yong of Question.Marketing debate why that measurement is probably wrong, why recommendations matter more than citations, how to handle negative PR in AI answers, how to attribute revenue to AI search and whether AI agents will make traditional GEO obsolete.
Most AI visibility tools check a prompt once a day and call it a number. In a new two-host format of The GEO Show, Paris Childress, founder of Hop AI and co-founder of GEOforge, and Simon Yong of Question.Marketing debate why that measurement is probably wrong and what to track instead.
Walking the exhibitor floor at brightonSEO, Paris found almost every vendor selling visibility tracking, and most ran just one check per prompt per day. Because LLMs are probabilistic rather than deterministic, a rank-tracker mindset doesn't work: you need a large sample and a confidence interval alongside any share of voice figure. Simon adds that nobody yet knows the real volume behind a prompt or the thousands of ways it can be phrased.
Being cited in a source panel isn't the same as being recommended in the answer, and neither is the same as being chosen. Paris suggests looking downstream at the pages that win citations and how that traffic behaves, while noting that ChatGPT links are reported to be losing their referral data, which makes attribution harder.
Simon describes a client whose negative coverage is being picked up by ChatGPT. Paris argues it's even harder to untrain AI than to bury a bad Google result, which is why now is the window to shape how AI understands your brand. Smaller brands can leapfrog bigger competitors if they out-execute them.
Paris's recommendation: ask every new customer or lead how they heard about you, and list ChatGPT, Gemini, Claude and Google AI Overviews as separate options, then track the trend in your CRM. Simon, with 20+ years in Google Ads, argues attribution has never been solved and that agentic sales may finally deliver full attribution.
The pair imagine personal agents introducing brand sales agents into conversations, ChatGPT Ads that work like custom sales agents armed with a brand's knowledge base, and a future where the "clicks" are just answers to your agent's questions.
Paris: Hi, everybody. Welcome to episode 62 of The GEO Show, brought to you by GEOforge, full self-driving for AI visibility. So today, we are testing a new format. I'm here with Simon Yong from Question.Marketing. You may have caught some of our earlier episodes on LinkedIn. What we're going to do today is we're going to cover five big themes that have been recurring over the past few weeks in past episodes. So just to do a recap here, I'm getting this daily intelligence brief that I'm using as the basis for most of my episodes, I've been noticing a lot of recurring themes, and so I asked Simon to join me today to cover some of these themes, and we can get his point of view and maybe debate any of them that, where we may fall on different sides of it. So first, let me welcome Simon. Hey, how you doing today, Simon?
Simon: Yeah, good. Busy as always. Learning as always, which is I think what excites me most about the space we're in, so I'm sure you're the same.
Paris: Yeah, absolutely. It's coming like an avalanche right now. I'm trying to keep up. But it is fun, and it's never been more interesting times, I think, than right now. So let's get right into the first theme of, the big themes from the past few weeks, and that is this question: Is AI visibility measurement fundamentally flawed? So are GEO platforms selling marketers precise-looking numbers, but that aren't statistically reliable? So this gets to the measurement debate, and I can tell you, from being recently at brightonSEO and basically walking around and talking to of 50-plus vendors that are exhibiting, almost all of them were selling a platform or a tool did visibility tracking. What I noticed was that, practically none of them were doing it at the level of rigor as GEOforge. Most of them were basically doing one prompt run or one check per day for a prompt, and they would do that across several engines, then that was it. That was the number for the day, and then they did it again the next day. What's your take on this, Simon? I definitely have a firm position, but measurement right here is very loose. The market is, really easily duped I think this kind of unethical measurement, approach. But what's your opinion here?
Simon: But you wouldn't be talking about the SEO industry in terms of, you know, we're going to sell you something and we're going to tell you that's true. That's the problem. I think one of the things that really annoys me at the moment is you keep seeing, "Oh, we've got this great tool." We, you know, and they'll show a graphic of, here's...
Simon: Oh, we had 20 million impressions, and we got this number of clicks and... But, and I'm like, I keep asking these people, "Well, whose data's that?" Because so many of these adverts, it's made up, you know, it's absolutely manufactured. But to talk to your point about which tools are the best, I don't think any of them even come close to what you're doing, just because they're not...
Simon: Like you say, there's tools out there. I could... Well, tell me if I'm wrong, but I could probably make a tool myself that would track one prompt and go out and test it on ChatGPT once a day and, tell me whether or not I'm there, right? That doesn't prove anything.
Paris: Yeah, exactly. Oh, I think here it's about confidence intervals, and one thing that I saw missing from most of the industry is that there was-- Because these are probabilistic systems, they're not deterministic, but most people are still thinking about this in the lens of a rank tracker. They have used SEO tools, and SEO tools typically have a rank tracker that are they're pretty reliable.
Paris: They'll tell you that you rank in this position, and when that moves, and it doesn't move too often usually. And that same mindset has carried over to GEO where, okay, some people even think about ranking, but generally they're thinking, "Okay, it's a tool that's going to look up whether I'm there or whether I'm not there, and it's going to tell me yes, no, it's going to update that every day.
Paris: Okay, great." But what they're missing is the probabilistic nature of LLMs versus the deterministic nature of rank tracking, means that you need to build a large pool of a sample. You need to do a lot of sampling, a deep degree of sampling. And then you need to demand that the tool gives you some kind of a confidence interval. Your share of voice is this percent, but here's how confident I am because it's all probabilistic. That's not happening. It's a shame. I think it's an opportunity for us, but we're fighting an uphill battle the market just isn't really educated on this.
Simon: No, and you've got.
Paris: Reality.
Simon: One of the things that I'm seeing in some of the projects that we're running, and it'd be interesting to know if this is true for what you're seeing as well, when you see, um... Well, you've got the difference between citation and recommendation for a start, which a lot of them don't necessarily differentiate between.
Simon: You've got some level of visibility, you've got some level of confidence. But what I'm also seeing is that let's say you do manage to get a piece of content cited or recommended, the nobody knows, I don't think anybody on Earth knows yet how well that is spread across, you know... What is the volume of that search?
Simon: Don't know. What are the volume of similar searches that could be interpreted as the same prompt? Don't know.
Simon: Could use a fan-out query to say, "Okay, somebody's asked X," there are... There could be 1,000 questions that are very similar.
Paris: Yeah.
Simon: Of the answer are you then? I don't know. So.
Paris: Yeah.
Simon: There's a lot of people selling a very basic tool that...
Simon: And you're right, it is... Well, at the end of the day, it's marketing, right? They're marketing a tool that people are getting sucked into. I've been doing a load of outreach myself, as you know, and a lot of people will just come back and say, "Yeah, we've got it covered. Our SEO agency's doing this." And I was speaking to a client yesterday who has been working with an SEO agency on their AI visibility, and two months in, the SEO agency are still talking about on-page adjustments.
Simon: They haven't done any content yet, and they haven't looked at any gaps, and they've done... And that just for me is selling, well, BS. It's just not, that's not what we're doing. But to say to track it, surely we have to end up at a scenario where it's not whether you're cited, it's not whether you're recommended, it's whether you're actually chosen, whether the business comes to you and you become the answer that sells the product.
Paris: Exactly. Yeah. You know, Simon, this takes us, well, to what was going to be the third theme, but I'll go right to it, which is are citations the wrong metric? Should GEO focus on recommendations? That's exactly your point, which is that, all right, there's an infinite long tail of prompts, and, we're never, we're never going to be able to capture that the way we used to capture keyword research. It doesn't exist. I don't think there'll ever be a tool for it. We do have some query fan-out data now that shows up in Bing Webmaster Tools, and that's helpful, but that's a little bit different because that's that's going from the user's prompt to, a set of queries that, is the purpose of retrieving the answer, but it's not actually giving us a broader view into all possible prompt variations from the user. So people are focused a lot on these prompts right now, but we need to, we need to look downstream, I think. So regardless of what the prompt is, look at all the, look at all the content pages that are getting citations and then see if any of that turns into... There's a few pathways. People can click on those citations, that's the most straightforward, and you can see that referral traffic. How does it behave? How does it convert? But then there's a lot of indirect paths. ChatGPT has recently started to strip out the UTM parameter from those links, the links from the referral. Basically, when you get a mention with a link, those UTM parameters are gone, so the ChatGPT referral traffic reported in GA4 has tanked in recent days. I don't know if that's, on accident or if that's something they're going to fix or bring back. Don't know what their motivation for that would be, but that only makes it harder to attribute.
Simon: Maybe they don't want to give that data to Google.
Paris: I guess not. But what, why, but then you prevent the customer from seeing the ROI, I guess. Not, well, yeah, I mean the marketer, the brand. It could be that they also want to, they want those people to spend more on ChatGPT ads, because certainly you can attribute all of the clicks and impressions from ChatGPT ads. So maybe they, by making organic, worse as a attribution picture, it may encourage people to go faster into the ad space,.
Simon: But something I wasn't aware of until a couple of weeks ago, and I found a piece of research that apparently, you tell me whether you believe this to be true, apparently 15% of everything that is typed in on a daily basis into a search engine or an LLM is unique. So 15% on a daily basis, because we're all sat here presuming we would type something in a particular way, and of course I would.
Simon: My tone of voice and the way I speak is me, right? But when you start going to the point of, well, here's a prompt, where if you took 1,000 people, surely, you know, not everybody would type it in all the same. We would all word that slightly different, and that's what that statistic says is, you know, do we ever get to the point of we cover off all of those prompts, if you want to call them, or those search queries?
Simon: We don't, because it's... There's that much that's going in that's unique on a daily basis. Obviously, the AIs want to harvest the way we talk and everything that we put in to produce the answers, but then therefore they have to also interpret what is put in to then... 'Cause I've seen it with a client, where we put in things like, "Is bulk WhatsApp marketing, legal under PECR regulations in the UK?"
Simon: Huge prompt string, right? Very what you would call traditionally long tail, right?
Paris: Yeah.
Simon: But then they started coming up for, WhatsApp marketing regulations, just that. So that's far less broad and searched far more presumably, or if you took 1,000 different ways of wording that, you've got to be able to serve the answer.
Simon: So presumably they're cited and then picked up because of the content that was built in the first place. So you... I don't think we're going to go... People might say, "Oh, just go and build 1,000 pieces of content that match every single possible, you know, way somebody could type in a particular query." Well, that would be like going back to the olden days in SEO and saying, "I'm going to build a page for every single location and then try and put on there..."
Simon: Do you see what I mean?
Paris: Yeah, or every long tail keyword it's-- Yeah, at some point you have to draw the line. I totally agree here, and I think what this question is implying is both, trying to, encapsulate all the prompts and also putting a lot of weight on citations is the wrong metric because citations is re-really a source panel that gives the user...
Paris: If the user decides that they care about That, it shows the user these are the sources that we primarily drew from when we synthesized this answer. It-- There's no real call to action or urge to tell people to click on those links. And most people, I mean, judging by my behavior, I rarely even look through that.
Paris: I mean, oftentimes it's a long scroll, and I'm focused on the answer, so what I really care about is getting into the answer. But even there, is that enough? Because is the answer recommending you, or is it telling the users maybe the prompt was, "Hey, which marketing agencies in the UK should I absolutely avoid?" You know? And maybe you're that.
Simon: That is a.
Paris: Consider.
Simon: Really, good point because I was talking to a client yesterday who has, some negative PR out there, and it is being picked up by ChatGPT, etc. And their question was, "Well, how do we overcome that?" Because it's already cited.
Simon: Now that will be a whole nother debate that we'll have in the future.
Paris: Yeah, I think there's going to be a new version of ORM, online reputation management AI visibility. Yeah, I actually debated that recently with,.
Simon: Did you just invent ARRM?
Paris: Something like that, yeah. AI ORM, but whatever that turns into. But yeah, that's the same kind of thing where if you have, if you have negative PR coming from an authoritative source, could be a real problem.
Paris: You've got to, you've got to over, overstack, you know, you got to stack the deck on the other side and outweigh that somehow. In the old world of ORM, the way that I see it, the way I argued in that last debate too, that the SEO ORM game was just about getting something that was ranking, a negative story that was ranking high and sometimes even ranking number one for your brand name. You would have to push that down the rankings. That means you would have to boost pages, well, like your homepage or even your social media profiles. You would have to get other stuff to outrank it so you could push that negative story down, and it's probably a similar case here with, AI ORM.
Simon: But one of the, one of the problems that you might have, and as you know, if information doesn't agree across different surfaces, then typically the AI will ignore it. So you might get to a scenario where you're fighting, you know, I've got negative reviews from two, three years ago, and now I can't get rid of it, and I can't get it to agree, and I can't get above, you know, I can't get above my competitor.
Simon: I can't become recommended.
Paris: Mm-hmm.
Simon: Or the, you know, the agent might say, "Well, I can't recommend them because that might be true."
Paris: Yeah. I think that's a, that's an even bigger problem than the ORM in the SEO world where you just had a negative story ranking too high and you had to bury it. And you do that really by getting a lot of links to your positive stories and your other properties so that eventually with the, you know, the link juice could start to down the old story, and then the old story would naturally age, and sometimes that would help. But if AI has cemented its understanding of your brand in some negative way, and there's just enough... It's just convinced that this is a, this is a shady business, and if people are really going to ask me about this, I'm going to let them know that people have said these things and watch out, be careful. Very hard-- I think it's even harder to untrain AI once it has cemented its opinion. And that's what we're trying to do now is I think we're in this window where there is, for many brands, there is an opportunity to really cement AI's understanding of your brand a way that's going to put you in a really competitive position, and it's going to cement the positive story, and it's going to cement the-- it's going to lead to AI recommending your brand in a positive way. And I think that's why the time to move into GEO is so perfect now actually because there's... AI's, AI is still kind of like an early infant's brain, I guess. I don't know if that's a good analogy. There is a great opportunity to shape and to mold, opinion right now and eventually later when that cements, it's going to be hard. If you do it the right way, then it's going to be hard to reverse.
Simon: Well, it's,.
Paris: So.
Simon: It's almost like if you had the opportunity now to go back to the very start of the internet, imagine how well you could do. Like, you could quite easily make yourself millions and millions just by.
Paris: Yeah.
Simon: Putting up pages in the right way. And if people are listening to this or watching, think of it that way because this is...
Simon: It feels to me like a bigger opportunity than where we were when the internet started, because it just went slowly to begin with, and no one really realized the massive opportunity you had. And it all sort of came, you know, a couple of three years in where everyone was like, "Wow, well, I can make millions with an e-commerce store, can I?"
Simon: But maybe we are in that position now where if you can start and put a footprint down before anybody else, you could win all of that recommendation and being chosen.
Paris: Yes. And even if you're not doing very well in SEO, that shouldn't be a reason to discourage your, you or your brand from diving in headfirst into the GEO game, because I think that, if you're a smaller player and you just don't have the resources and have never been able to outrank larger competitors in Google, you almost have a clean slate now, and you could-- If you outwork those guys now, you can actually start winning recommendations. There was a great example, Profound, one of the co-founding guys at Profound presented at brightonSEO, and he presented this case of a running shoe company, and I can't remember their name, but I had never even heard of them. It's, kind of obscure. And he said they have a large data set, of course, across lots of, niches and running shoes is one of them. And this relatively obscure running shoe website brand, they had been producing such great content that they were getting-- they were crushing Nike, Adidas, and, other major brands in prompts and AI visibility around running shoes. And it's just a great example you can... Right now, the opportunity is big as it's ever going to be to leapfrog over your your competitors that might have dominated you on SEO for the last 10, 15 years, and now's the opportunity where you can leapfrog them in GEO if you're able to execute the right way. All right. Well, the last point I want to make on this, and then I want to get to the last story and give us enough time for that, is that in terms of measurement and how to measure, I think I-- This was my final slide at brightonSEO, which is that AI attribute-- AI-driven attribution is going to be fundamentally flawed for a long time.
Paris: I don't think this is going to get solved anytime soon. And until we get there, the best thing you can do is to ask every new customer when they come through the door or every lead that fills out the first form, ask them, "How did you hear about us?" And my old recommendation used to be of those choices should be something like an AI chatbot. But someone, can't remember, gave me this tip to take it one step further and to say, of those seven or eight or nine options, it should be actually ChatGPT, Gemini, Claude, Grok maybe, I don't know, Google AI Overviews. And people are going to likely remember if they had a long conversation that led them to then go out and do a search and they found your brand, they converted, or they filled out the lead gen form, most likely they're going to have a good memory of a long conversation that started their journey with ChatGPT, with Claude, and they're going to answer that with a high degree of reliability, I think. And then when you have that data, then basically you flow that all the way through to your CRM, you can map that to pipeline, you can map that to opportunities and start by baseline it, and then you can watch the trend over time. As you invest more and more into GEO strategies, if more people are basically saying they found you through AI search and through any of these chatbots, to me, that's the strongest signal that what you're doing is moving the needle, that things are going in the right direction.
Paris: Keep doing what you're doing. Do more of it. Of what your share of voice report tells you, regardless of what your citation impressions reports are showing anything else really, ask the customer, "How did you hear about us?" And trust that they're going to remember the chat conversation they had and even the engine that they were in.
Simon: So I get that. I have a slightly different prediction or where I think it should go, because.
Paris: Mm-hmm.
Simon: Somebody who's done 20-plus years of Google Ads, attribution has always been really difficult, right? There has never been, what you would call a single source of the truth, even in Google itself. So you would think, right...
Simon: And done properly, and I'm not going to argue with, you know, anyone who says anything else, but if it's done properly and you are using all of the tags and you're pixeling people, etc, even then, it still breaks because of privacy or cookies or,.
Simon: You know, or somebody, you know... The argument would be, well, okay, and I've sat in agency meetings where I've been running the Google Ads for a client, and you've got the agency who's running the Meta Ads, and they're claiming a percentage of conversion and-.
Paris: Claiming more, a larger share of the conversions.
Simon: Of course, right?
Simon: We're all claiming more, right? But, and you still can't prove it because the trouble is somebody could see a Google ad, they could then spend a few days thinking about it. They then see the products again on Facebook, and then they go and buy it. Well, who got it? What was it first click, last click, what, right?
Paris: Yeah. Mm-hmm.
Simon: My take would be to answer the... And yes, we can only do what you're talking about at the moment. So I'm not saying this is here now, but it will be. At the moment, ask them. You know, you can't get a better source than somebody goes on and they go, "Yeah, I found you on ChatGPT, attribute it." But why wouldn't we move into a world where me as a eCommerce store owner, I'm using ChatGPT to track my attribution in ChatGPT, right?
Simon: It should be able to tell me, because eventually we're bound to get to agentic sales, right? We must get to the point where I go, "I want that product, send it to me." And then it's fully attributable because maybe, like you've talked about in the past, it is the, an agent selling you a product, going into sales mode and saying, "Hi, Simon, do you want these golf clubs?"
Simon: Right? And eventually I go, "Oh, you remember you spoke to me a couple of weeks ago again about those TaylorMade golf clubs. Can you just get them ordered for me?" And then absolute 100% attribution, right? So we...
Paris: That case, yeah, then there's no doubt. That would solve it, yeah.
Simon: But it's surely each of the, each of the models must build that, right? They must.
Simon: Not to say that there's going to be an overarching separate tool that is, you know, does it track my conversions in all of the different agents. In each of the agents, it's got to track what it did, where it went. Maybe this is, you know, behind the scenes. So say you're talking about ChatGPT removing UTM tracking links, probably there is some other pixel or, ID that goes along with these pieces of data that go from user to website, etc.,.
Simon: That are then tracked in some other way.
Paris: Yeah. I mean, if you get to a point where the agent, Well, your personal agent, actually, this is a perfect segue to the last question, which is: Will AI agents make traditional GEO obsolete? But you've already opened the topic with the agent comment, because agents are now, personal agents are going to-- That's the next big harness.
Paris: That's the next big wave of AI, personal agents, and all the rest. So these agents are going to, I've heard different versions of it, but either the agent is going to put its sales cap on and, then clearly identify that, "Hey, I'm now selling you something," or it's going to say, "Hey, let me introduce you to my agent friend over here, who is a salesperson who I trust, and, I'm going to I'm going to introduce them to this conversation.
Paris: Are you okay if I bring them into the conversation because they have a great set of golf clubs that they can offer you?" And the user says yes, and they opt into that, and then that agent partner steps in, makes the sale, closes the the sale probably inside of ChatGPT. Why even go to the website, you know? I can see pictures, I can see videos and everything else right inside of ChatGPT. I'm convinced. Make the sale, attribute the sale 100% to ChatGPT. That would be a whole different world, actually, because that's stronger attribution than has ever existed in digital marketing.
Simon: And we've got to obviously improve, right? Because marketing needs to do that, but I think surely it has to be the agents in themselves that do that. But then do I have my own personal negotiation bot that, you know, is then talking back to my agent and saying, "Right, okay, I've found this and I've found that, and.
Simon: Well, these guys have got better-.
Paris: You know, I think the future of ChatGPT Ads interface for the advertiser may be that you are building a custom ad bot, or you're building a, you're building a custom sales agent that's prepared to be introduced and to step into conversations when the user-- when the customer's agent invites you in. So you're standing ready to be invited into conversations, and once you get invited into those conversations, you are trained with all the knowledge about that brand, and of course, you have also sales techniques and... But you're armed and ready to go. And the advertiser's job is to all the assets that that sales agent needs so it can win deals when it gets introduced. And that might be all we ever do in ChatGPT Ads.
Simon: But do we.
Paris: We're not picking prompts. We're not saying themes, this or that. It's just saying, "Whenever you think it's right to insert me in, everything that this, sales agent needs to be trained on," and, you're just supplying, kind of like Google Ads now, you're really supplying all the assets.
Paris: And Google says, "We're going to find you the audience. We're going to find the intent. Don't worry about trying to do keyword research. Just give us all your creatives, give us all your messaging, all the assets, give us your data, and we'll bring you, we'll bring you the leads, we'll bring you the conversions."
Simon: But if you've, like, I I, and I think sometimes we're a little bit too close to it because I spoke to another client yesterday and I said, "Right, have you got a knowledge base?" And they said, "Well, we've got a blog." I think that was the disconnect between.
Paris: Not being entirely wrong either. I'm sure their blog contains a lot of knowledge.
Simon: No, but they haven't got... So when we get... If what you're talking about is if I had everything that we possibly know about a business that down to what a product is made from, you know, where we get it from, what, like, how quickly we can deliver it, what experience others have had with it, all of that, right?
Simon: That's your knowledge base, and at that point, if you attach that to a sales agent,.
Paris: Yeah.
Simon: You don't ever have a website. You just have a digital footprint that is your knowledge base that is prepared to interact with agents, and it'll say, maybe they'll come and say, "Well, look, we are selling 1,000 sets of golf clubs, this week in ChatGPT in the UK.
Simon: Do you want to be part of that?" Right? "What have you got to offer?" And you go, "Well, my knowledge base and my stock system says I've got 40 sets of those." And it might say, "Oh, are you prepared to let, you know, some of them go at a little bit of a discount to move them more quickly?" It might, like, who knows where we end up going.
Paris: Yeah. Wow. Yeah, that's amazing. I think there's a vision forming here for GEOforge, for BaseForge in a world of advertising, because so far right now, GEOforge is all about organic AI visibility. It's not about ChatGPT Ads. But, you could, in the future, if we could plug BaseForge into ChatGPT Ads platform and allow ChatGPT Ads...
Paris: Imagine this is the world where now the agents are selling, and I can just say to my sales agent, "Here's BaseForge. I'm going to plug this in. It contains all of our proprietary data, knowledge, sales call transcripts, and, basically you use everything in here to sell. Whatever you find, and when you get invited into a conversation by a user's personal agent, there's your assets, there's your resources."
Paris: And basically BaseForge would kind of plug in to ChatGPT Ads.
Simon: Well,.
Paris: Support.
Simon: Do are we going to get into a world where, and this'll be another episode for us, but I think we obviously get to a world where I don't want to click an ad, right? I want to have the conversation. I want to interact with the agent. I don't want to go off and ... I why would I do that, right?
Simon: That's-.
Paris: No, there there won't be any, there won't be any clicks other than just answering multiple choice questions from your agent. You know, would Just do you want to talk to Simon's, do you want to talk to, with TaylorMade's sales agent? Can I introduce you right now or would you prefer not?
Paris: Yes, no. I mean, those are going to be the clicks. Simon, I've got to wrap up unfortunately, but, this was fun. I think we should redo this format again. And, thanks to everybody listening, and we will see you all on the next episode.
Simon: Yeah. See you soon.
