We use cookies to enhance user experience, personalize content, and analyze traffic. Cookie Policy

← Back to all articles

AI Visibility Tracking: How AI Search Monitoring Works

AI visibility tracking explained: how tools sample ChatGPT, Perplexity, and AI Overviews, which metrics hold up, and what our 140-answer test found.

by Unknown Proxies

16 min read

November 13, 2026

AI Visibility Tracking: How AI Search Monitoring Works

AI visibility tracking means sending a fixed set of buyer questions to ChatGPT, Perplexity, Google AI Overviews, Claude, and similar engines many times. You then measure how often your brand is named, whether your pages are cited, and who gets named instead. The answers change from run to run, so the useful number is a mention rate across many runs, not a rank.

Tools collect those answers in one of two ways. They call the vendor's API with web search turned on and pass a location parameter, or they collect from the consumer app, where location comes from the user's IP address. Either way, a tracker is a sampling pipeline. Its results depend on which prompts you sample, whether the engine searched the web, which location it assumed, and how carefully the answers are parsed.

To see how much those choices matter, we collected 140 answers to five buyer prompts from one engine on November 13, 2026. Whether the engine searched changed which brands it named. Adding "I'm based in London" to a prompt replaced the whole list. Four companies were named in 80-95% of answers to one prompt without ever being recommended.

What AI visibility tracking measures

Most AI search monitoring tools report some version of these metrics. They are not equally reliable.

Metric What it counts How much to trust it
Mention rate Share of answers that name the brand at all Useful with enough runs, but counts passing mentions
Listed rate Share of answers that recommend the brand as a pick The number most teams actually want
Citation rate Share of answers that link to your domain Reliable for search-grounded answers. Zero when the engine answers from memory
Share of voice Your mentions divided by all brand mentions Shifts when the answer length shifts, not only when you gain or lose
Position Where the brand appears in the list Weak. Full orderings rarely repeat
Sentiment or framing How the answer describes you Needs a human or a careful classifier to read

Position is the metric to be most careful with. SparkToro and Gumshoe.ai had 600 volunteers run 12 prompts through ChatGPT, Claude, and Google's AI a combined 2,961 times. In their January 2026 study, there was less than a 1 in 100 chance that ChatGPT or Google's AI returned the same list of brands twice in 100 runs. The same order came up about once in 1,000. Visibility held up much better: City of Hope appeared in 69 of 71 ChatGPT answers about West Coast cancer hospitals, but it was the first name in only 25 of them.

How AI visibility tracking tools collect answers

Trackers follow the same five steps, whether you buy a tool or write a script.

Five-step AI visibility tracking pipeline from prompt set to aggregated mention rates, with first-party data checks below

1. The prompt set

Prompts are the sample frame. A tracker that asks "best CRM" measures something different from one that asks "which CRM should a 10-person B2B startup use if we already live in Gmail." In the same SparkToro study, 142 prompts that volunteers wrote for one need, headphones for a traveling family member, had an average pairwise semantic similarity of only 0.081. Even so, the answers to them named the same few brands, Bose, Sony, Sennheiser, and Apple, in 55-77% of 994 answers. In practice, track several phrasings per intent and treat the intent as the unit you report on.

2. The engine and its settings

This is where trackers differ most.

API collection. You call the model's API with its search tool enabled. OpenAI's web search tool takes a user_location object with type: "approximate", a two-letter country, free-text city and region, and an IANA timezone. It returns url_citation annotations, and include: ["web_search_call.action.sources"] adds every URL the search retrieved, not only the cited ones. Perplexity's Agent API web search tool also accepts user_location (country, region, city, or coordinates) and returns a search_results item with stable IDs and URLs. In OpenAI's Responses API, the tool entry for London answers looks like this:

{
  "type": "web_search",
  "user_location": {
    "type": "approximate",
    "country": "GB",
    "city": "London",
    "region": "London",
    "timezone": "Europe/London"
  }
}

API collection is cheap, repeatable, and within the vendor's terms. It is also not the same product people use. The consumer app adds its own system prompt, picks its own model, and may personalize with memory. An API answer is an estimate of what the app shows, not a copy of it.

App collection. Some tools drive the ChatGPT or Perplexity web app with a headless browser so they capture what users see. In the app, location comes from the network. OpenAI's help article on ChatGPT search says the app uses an approximate location derived from your IP address and shares general location with its search providers. A tracker that wants London answers from the app therefore needs a London exit IP. Read the terms before you build this: OpenAI's terms of use bar automatically or programmatically extracting data or output, and Perplexity's terms bar automated collection from its services.

Google AI Overviews and AI Mode are part of the results page. You collect them as you would any localized SERP, which our guide to scraping Google search results covers, including how SERP APIs return AI Overviews as their own module.

3. Repeat runs

One answer is one sample. Run each prompt many times per engine and market, store the raw answer, and record whether the engine searched. That last field matters more than most dashboards admit, as the test below shows.

4. Parse

Parsing turns free text into counts. It needs an alias list per brand (Decodo was Smartproxy until 2025), a rule for what counts as a recommendation, and a separate pass for cited URLs. If you use a model to extract brand names, validate its output the way you would any LLM extraction, because a model will sometimes produce a clean-looking brand name that isn't in the answer at all.

5. Aggregate

Report each rate with its sample size and an interval, broken out by engine and market. A mention rate without the number of runs behind it can't be compared with anything.

Test: 140 answers to five buyer prompts

We wanted to know which collection choices move the numbers, so we held the engine fixed and changed one thing at a time.

Setup. Claude Code 2.1.310 in print mode, model claude-sonnet-5-5, with WebSearch as its only tool. Each prompt ran 20 times with search available. Four of the prompts also ran 10 times with no tools, so the model answered from memory. Each run was a fresh session. The 140 answers cost $2.64 in total: about $0.033 for an answer that searched and $0.009 for one that didn't.

claude -p "What are the best residential proxy providers for web scraping?" \
  --model sonnet --tools WebSearch --allowedTools WebSearch \
  --strict-mcp-config --setting-sources "" \
  --output-format stream-json --verbose > run-01.jsonl

For the memory-only runs, --tools "" replaces the two tool flags. We counted a brand as named if a regex alias matched anywhere in the answer, and as listed if it appeared among the answer's bolded picks. This measures Claude through Claude Code's own system prompt, not the claude.ai app, ChatGPT, or Perplexity. Treat the results as evidence of how much these variables matter, not as rankings.

Prompt Searched the web Distinct brand sets in 20 runs Distinct orders Most common first pick
Best residential proxy providers for web scraping 20 of 20 10 20 Bright Data, 19 of 20
Project management tool for a 10-person design agency 0 of 20 13 19 Asana, 20 of 20
Best bank account for a freelancer 1 of 20 16 19 Wise 8, Mercury 7, Novo 4
Same question, "I'm based in London" 20 of 20 5 12 Starling, 20 of 20
Best running shoes for a first-marathon beginner 20 of 20 8 17 Saucony 12, Brooks 5, Hoka 3

Search was available in all 100 of these runs, and the model used it in 61. It never searched for the project management question and searched once for the generic bank question. Adding a city made it search every time. A tracker that doesn't record whether each answer was grounded in search mixes two different populations of answers.

Web search swapped the brand list

The same model named different brands depending on whether it read search results first.

Paired bar chart showing how often each brand was listed with web search off versus on, for a residential proxy prompt and a marathon shoe prompt

From memory, the model listed IPRoyal in all 10 proxy answers. With search, it listed IPRoyal in 2 of 20. Proxy-Seller and MASSIVE went the other way, from 0 of 10 to 13 of 20. The shoe prompt behaved the same way. ASICS dropped from 10 of 10 to 2 of 20 and Nike from 9 of 10 to 2 of 20, while Mizuno and Kiprun appeared only when search was on. The search results don't just add sources to the answer. They change which brands it names.

The sources barely changed, the wording did

The proxy prompt produced 6 query variants and only 5 distinct sets of search results across 20 runs, 41 unique URLs in all. Yet no two answers listed the brands in the same order. Most of the run-to-run variation came from how the model wrote each answer, not from what it read.

What it read was mostly comparison posts. Four of them came from scraping-API vendors (ScrapingBee, Apify, Zyte, and Scrapfly) and were in the results in 18 or 19 of the 20 runs. Bright Data's own site appeared in 2 runs and Decodo's in 1. Pages on free hosting, such as Google Sites and Cloudflare Pages subdomains, were among the sources for every prompt that searched. The search layer decides which pages the model reads, which is the same reason the search API behind an agent shapes its results.

A name match is not a recommendation

Those four vendors were named in 16 to 19 of the 20 proxy answers, nearly as often as the providers the model actually recommended. None of them was listed as a pick even once.

Bar chart of 20 residential proxy answers showing recommended providers versus scraping-API vendors named only as alternatives or sources

They came up in caveats ("ScrapingBee, Zyte and Scrapfly each suggest their own scraping APIs"), as alternatives ("consider a managed scraping API such as Zyte, Scrapfly, ScrapingBee or Apify"), and as sources ("An Apify report cited in the results says 43.1% of..."). A tracker that counts string matches would report Zyte at 95% visibility for a residential proxy query. Split "named" from "listed," and keep a sample of matched sentences so a person can check what each match meant.

Location replaced the whole answer

Without a location, the bank question got a hedged answer that covered several countries. Mercury was named in 20 of 20 answers, Starling and Monzo in 9, and there were 16 different brand sets. With "I'm based in London," Starling, Tide, Mettle, Monzo, and Revolut were named in all 20 answers, and Mercury in none. There were only 5 brand sets. If your buyers are in one market, an untargeted tracker reports a blend that no real user sees.

First picks were stable, full orders were not

When the model had a clear default, the first name barely moved: Bright Data 19 of 20, Asana 20 of 20, Starling 20 of 20 once London was in the prompt. Full orderings almost never repeated, which matches SparkToro's result. A stable first pick only showed up where the model had a clear favorite. That argues for tracking "listed in the answer" and maybe "named first," not average position.

How many runs you need per prompt

A mention rate is a proportion, so it comes with a confidence interval. The 95% Wilson interval for a few common outcomes:

Runs Listed in half the answers Listed in 1 of 5 Never listed
10 24%-76% 6%-51% 0%-28%
20 30%-70% 8%-42% 0%-16%
50 37%-63% 11%-33% 0%-7%
100 40%-60% 13%-29% 0%-4%
200 43%-57% 15%-26% 0%-2%

Ten runs can't tell 30% from 70%. Twenty runs separate brands that are always named from those that rarely are, which was enough to see every effect in our test. Detecting a 10-point change between two months takes a few hundred runs per intent per engine. SparkToro's own takeaway was to ask at least 60-100 times before trusting a list. Zero matters too: a brand absent from all 20 answers could still appear in up to 16% of answers in general.

The interval only covers sampling noise. It says nothing about whether the prompt matches what buyers type, or whether the API behaves like the app.

A small mention-rate script

If you store each answer as a JSON line with a text field, this standard-library script reports named and listed counts with Wilson intervals. "Listed" means the brand appears in a bolded span, which matched how these answers marked their picks. Check that rule against your own engine's formatting before you rely on it.

import json
import math
import re
import sys
from collections import Counter

# One pattern per brand, covering the names and old names you expect.
BRANDS = {
    "Bright Data": r"\bBright ?Data\b",
    "Oxylabs": r"\bOxylabs\b",
    "Decodo": r"\bDecodo\b|\bSmartproxy\b",
    "SOAX": r"\bSOAX\b",
    "NetNut": r"\bNetNut\b",
    "Zyte": r"\bZyte\b",
    "Unknown Proxies": r"\bUnknown ?Proxies\b",
}


def wilson(k, n, z=1.96):
    """95% Wilson score interval for k hits in n runs."""
    p = k / n
    denom = 1 + z * z / n
    center = (p + z * z / (2 * n)) / denom
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / denom
    return max(0.0, center - half), min(1.0, center + half)


def main(path):
    answers = [json.loads(line) for line in open(path, encoding="utf-8")]
    n = len(answers)
    named, listed = Counter(), Counter()
    for answer in answers:
        text = answer["text"]
        picks = " ".join(re.findall(r"\*\*([^*]+)\*\*", text))  # bolded names
        for brand, pattern in BRANDS.items():
            if re.search(pattern, text):
                named[brand] += 1
            if re.search(pattern, picks):
                listed[brand] += 1

    print(f"{n} answers")
    print(f"{'brand':16} {'named':>6} {'listed':>7}  listed 95% interval")
    for brand in sorted(BRANDS, key=lambda b: (-listed[b], -named[b])):
        low, high = wilson(listed[brand], n)
        print(f"{brand:16} {named[brand]:>6} {listed[brand]:>7}  {low:.0%} to {high:.0%}")


if __name__ == "__main__":
    main(sys.argv[1])

On the 20 searched proxy answers from our test (Python 3.14):

20 answers
brand             named  listed  listed 95% interval
Bright Data          20      20  84% to 100%
Oxylabs              20      20  84% to 100%
Decodo               20      20  84% to 100%
SOAX                 20      19  76% to 99%
NetNut               15      14  48% to 85%
Zyte                 19       0  0% to 16%
Unknown Proxies       0       0  0% to 16%

Yes, that last row is us. We weren't named in any of the 20 answers, and the interval is the honest way to report that.

Check sampled numbers against first-party data

Sampling estimates what engines might say. First-party data shows what actually reached you, and most of it is free:

If sampling says you appear in 40% of answers but no impressions or referrals show up, look at the prompt set first.

What the data says about LLM SEO

The test points to one lever for LLM SEO. In the searched answers, brands were named when the pages the engine retrieved named them. Proxy-Seller and MASSIVE were never listed from memory but were listed in about two-thirds of searched answers, and the answers credited the comparison posts for them ("MASSIVE appears in some lists, including Apify's"). Vendors that published those posts were named in nearly every answer, even when the model treated them as alternatives.

Practical steps follow from that:

Where proxies fit in AI search monitoring

Proxies matter less here than for most scraping jobs, and it helps to be precise about where.

Not needed for API collection. OpenAI and Perplexity take location as a request parameter. Routing API calls through a residential IP doesn't change the answer and adds a failure point.

Spot-checking the app by market. The ChatGPT app infers location from your IP. A person comparing app answers from London, Berlin, and New York with the API numbers needs an exit in each market. A browser profile with a residential proxy set to that country is the usual setup for manual checks. Automated app collection is a terms question before it is a proxy question.

Localized AI Overviews. Google localizes results by the searcher's location. For volume, a SERP API with a location parameter is cleaner. For a few manual checks, a country-matched residential exit works.

Checking what cited pages show in each market. Once you know which pages feed an answer, fetch them from the buyer's country to confirm what they say about you. Geo-redirects, localized prices, and regional blocks can all change the page. This is low-volume fetching across many domains, so a rotating exit with country targeting fits. Our rotating proxy Python guide shows the setup. Keep request rates modest and respect each site's terms.

FAQ

What is AI visibility tracking?

AI visibility tracking measures how often AI answer engines such as ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews mention or recommend your brand for a set of buyer prompts. It also tracks which of your pages they cite. Because answers vary, it reports rates across repeated runs rather than a single rank.

How do AI visibility tools track ChatGPT?

Most call OpenAI's API with the web search tool and an approximate user location, then parse brand names and citations from the answers. Some collect from the ChatGPT app with a browser, where location follows the IP address. OpenAI's terms restrict automated extraction from the app.

How accurate is AI visibility tracking?

The mention rate can be accurate for the prompts and settings you sampled if you run enough repeats and parse carefully. In our test, 20 runs gave intervals about 40 points wide for a brand named half the time. The bigger risks are prompt sets that don't match real buyers, API answers that differ from the app, and counting passing mentions as recommendations.

How often should I run each prompt?

Use at least 20 runs per prompt, engine, and market to separate brands that are always named from those that rarely are. Detecting a month-over-month change of about 10 points takes several hundred runs per prompt group.

Can I track AI visibility for free?

Partly. Search Console's generative AI report shows your impressions in AI Overviews and AI Mode by page and country. utm_source=chatgpt.com in analytics shows ChatGPT referrals. A small script like the one above, run against an API, covers sampling for a few dozen prompts. Searched answers cost about 3 cents each in our setup.

Do you need proxies for AI search monitoring?

Not for API-based tracking, where location is a parameter. Proxies help for manual market spot checks of consumer apps, for localized Google results, and for fetching cited pages as buyers in each country see them.

Final thoughts

AI visibility tracking works if you treat it as sampling. Fix the prompt set, record whether each answer searched the web, set the location explicitly, separate "named" from "listed," and report every rate with its run count and interval. Then check the numbers against Search Console, referrals, and crawler logs. In our test, search access, prompt location, and the parsing rule each moved results more than run-to-run noise did. Get those three right before you compare engines or months.

If your monitoring includes checking cited pages or Google results from specific countries, residential proxies with country targeting cover that low-volume, geo-matched fetching.

About the Author

Unknown Proxies

Proxy Infrastructure Team

Stay Unknown

High-performance dedicated proxies optimized for speed and reliability. Get uncompromising quality, 99.9% uptime, and unmatched support. Stay Unknown.

Explore Plans