If an agency pitches you on "AI SEO" and starts talking about "leverage" or "synergy," hang up. You aren’t hiring them to use buzzwords; you are hiring them to solve for a chaotic, non-linear search environment. Since 2013, I’ve seen enough "SEO revolutions" to know that when the technology changes—from the blue link era to the Large Language Model (LLM) era—the grifters double down on vague promises.
You need hard data. You need dev-ready specs. You need to know exactly what I would screenshot to prove this changed, or it didn’t happen.
Here is how to vet an agency before you waste a single dollar on a consultant who doesn't understand Retrieval-Augmented Generation (RAG) or entity graph mapping.
Is this agency actually monitoring AI-specific referral traffic?
Most agencies are still obsessed with organic click-through rates from Google Search. But if your brand isn’t showing up in ChatGPT, Perplexity, or Gemini, your traditional SEO work is only half the battle. You need to ask them: "How are you tracking my brand’s presence within LLM responses?"
If they tell you that Google Analytics 4 (GA4) is all they use, ask them how they are filtering for AI referral traffic versus standard web crawl traffic. If they cannot show you a custom dashboard or a filtered view in GA4 that isolates referral sources like chatgpt.com or perplexity.ai, they are treating AI like a static document index, not an evolving knowledge base.
Ask them: "Can you show me a screenshot of a custom GA4 report identifying traffic from an AI chatbot? If not, what tracking parameters are you suggesting we implement to monitor this?"
Do they understand the difference between indexing and live web retrieval?
Traditional SEO is about ranking. AI visibility is about retrieval. When a user https://stateofseo.com/what-does-recommendation-position-mean-in-ai-answers/ asks an LLM a question, the model performs a RAG process—it fetches data from the web to supplement its training data. If your site structure is bloated or your technical SEO is poor, the model ignores you.
Agencies like Four Dots have been ahead of the curve on the technical aspects of how crawlers interact with content. You need to ask your agency how they optimize your content for RAG. Do they understand how to structure technical documentation so a model can effectively "read" it during a live search?
Ask them: "How do you optimize my site for live web retrieval? Are you prioritizing semantic clarity, or are you still stuffing keywords into meta tags like it's 2015?"
How do they handle entity optimization and knowledge graphs?
This is where most agencies fail. They think "topics" are keywords. They aren't. Entities are distinct, defined objects—people, companies, products, concepts—that search engines link together in a knowledge graph. If Google and other LLMs don't recognize your brand as a clear entity, you will be drowned out by competitors who have successfully built their own knowledge graph footprint.
You want to see a map. You want to see how your site connects to industry-standard databases (like Wikidata or Google’s Knowledge Graph). If they can't explain how your brand's @id links to external authoritative sources, they are missing the core foundation of modern visibility.
What is their process for Schema.org and @id linking?
Broken schema is the bane of my existence. It looks "fine" in the CMS, but it fails validation because the nesting is wrong or the ID linking is inconsistent. If you don't have clear @id linking, the AI cannot disambiguate your company from every other company with a similar name.
Ask them: "Can you provide a Google Rich Results Test report for a sample page, and show me how you are using @id linking to connect my brand entity across my site's entire structure?"
The Comparison: Traditional SEO vs. AI-First Visibility
Feature Traditional SEO AI-First Visibility Primary Goal Blue link ranking Knowledge graph attribution Content Structure Keyword-focused Semantic entity-focused Testing Rank tracking RAG response verification Technical Focus Page load speed Structured data (@id linking)What is the state of their dev-ready specs and QA cycles?
If an agency sends you a document full of "strategic recommendations" without a single line of actionable code, you are losing money. An agency worth their salt provides dev-ready specs. This means they should hand your engineering team a json-ld id for knowledge graphs JSON-LD snippet that is ready to deploy, or a specific robots.txt rule modification to handle the myriad of AI scrapers and crawlers currently hitting your server.
I keep a running list of bots that I block in robots.txt—some are legitimate, some are just scraping data to feed smaller, low-quality models. Your agency should be managing this list for you. They should also be running QA cycles on every technical change. If they push a site change and don't re-test the schema in the Rich Results Test tool immediately, they aren't doing the work.
Ask them: "Can you provide an example of a recent 'dev-ready' document you sent to a client's engineering team? What does your QA cycle look like when a schema update fails validation?"
What do they know about tools like FAII.ai?
There are tools emerging that help bridge the gap between human-led content and machine-readable data, such as FAII.ai. A forward-thinking agency should be exploring these platforms to ensure your content isn't just optimized for human readers, but for the machines that curate the information for those humans.
If they haven't heard of these platforms or don't have a perspective on how LLM evaluation tools impact their strategy, they are likely using 10-year-old playbooks.
The Final Interview Questions
When you sit down with them, make them answer these four questions. If they waffle, look for the door.
"What would I screenshot to prove your strategy changed our visibility in an LLM?" (Force them to define a success metric that isn't just "organic traffic.") "Which AI-crawlers are currently hitting our site, and which ones are you recommending we block or allow in robots.txt?" (If they don't have an answer, they aren't monitoring your logs.) "How do you manage entity disambiguation for my brand?" (Listen for mentions of Knowledge Graphs and @id linking.) "Can you provide a technical specification document for a recent Schema implementation?" (Ensure it includes validation steps and QA logs.)Ultimately, AI visibility is about proof. It’s about building a digital architecture that makes it impossible for a machine to misunderstand who you are, what you offer, and why you are the authority. Don't settle for agencies that hide behind "industry-leading" claims. Demand the technical specs, demand the QA evidence, and demand results you can actually measure.