Skip to main content

Legal Technology · AI Visibility

My own website had been invisible to Bing since 2020 and nothing inside it ever told me. This is what I found when I audited my own AI visibility properly, and what it means for any law firm.

◆ The argument in one minute

A growing share of prospective clients now ask an assistant a question rather than searching. If your firm is not in the paragraph that comes back, you have not lost a ranking position — you are not in the conversation. Getting into it is mostly not a technology problem. It is an identity and evidence problem, and most of it happens on pages you do not control. Here is the documented evidence on what works, what does not, and what I found when I tested it on myself.

Last week I discovered that my own website had been invisible to Bing since July 2020.

Not ranked badly. Not penalised. Invisible. Bing’s last recorded crawl of philarist.com was six years ago, and nothing inside my website ever showed it. No warning, no error, no dip in a dashboard. The site looked perfectly healthy from the inside for six years while one of the two indexes that matter had simply stopped looking at it.

The cause turned out to be a single firewall rule on my server. Someone had, entirely reasonably, told it to block visitors whose identification string contained “bingbot” — a sensible-sounding defence against badly behaved robots. The rule also caught OpenAI’s search crawler.

Here is why that matters more than it used to. Bing’s index is what Microsoft Copilot answers from, and what ChatGPT’s search has historically drawn on. So for six years, two of the four major AI assistants could not read my site at all. The only way to find out was to pretend to be Bingbot, request my own homepage, and watch my own server refuse me.

Practical implication

Your website’s dashboard will never tell you this. Crawler access can only be tested from outside, by requesting your own pages while identifying as each crawler and checking the response code.

Watch the briefing

The same briefing as a nine-minute video overview, with chapters.

1How the answer engines choose their sources

Start with what the platforms themselves say, because it cuts through a great deal of noise.

Google is the only one with explicit publisher documentation for its AI features. Its position, quoted verbatim from its “AI features and your website” page: “You don’t need to create new machine readable files, AI text files, or markup to appear in these features. There’s also no special schema.org structured data that you need to add.” A page needs to be indexed, eligible to appear in Search with a snippet, and to meet the normal technical requirements. That is all.

Microsoft gives the most useful mechanical description. In its October 2025 guidance it explains that AI assistants “don’t read a page top to bottom like a person would. They break content into smaller, usable pieces.” Headings “act like chapter titles that define clear content slices.” Question-and-answer pairs can be “lifted word for word.” Its don’ts are worth memorising: avoid walls of text, do not hide answers in tabs or accordions, do not put core information only in PDFs or images.

OpenAI documents very little about ranking, but one sentence matters enormously: “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.” Block that one crawler and you are out of ChatGPT’s search results entirely.

Perplexity and Anthropic publish essentially nothing about how they rank. Any article claiming to know “Perplexity’s ranking factor weights” is inventing them.

Now the finding that reorders everything else. Traditional search optimisation assumed that ranking in the top ten produced citations. That link has weakened sharply. Ahrefs analysed 863,000 keywords and four million AI Overview citations in March 2026 and found only 38% of cited pages ranked in the organic top ten — down from 76% in their study eight months earlier. The remainder came from positions 11 to 100, or from beyond position 100 entirely. Google calls the mechanism “query fan-out”: the engine breaks a question into several sub-questions and pulls a passage for each.

AI engines retrieve passages, not pages. A single clear paragraph that answers one question well is the unit of visibility.

2Three kinds of crawler, and why the distinction decides things

Most firms treat “AI bots” as one category to allow or block. They are three, doing three different jobs, and confusing them is how good decisions go wrong.

KindExamplesWhat blocking it costs you
Training crawlersGPTBot, ClaudeBot, Applebot-Extended, meta-externalagentYour site can never enter a model’s trained-in knowledge. Anything an assistant says about you “from memory” will come from other people’s pages.
Search and retrieval crawlersOAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, BingbotYou are out of the live answers entirely. These build the indexes the assistants cite from today.
User-triggered fetchersChatGPT-User, Claude-User, Perplexity-UserAn assistant cannot read your page when a real person asks about you in that moment. Several of these ignore robots.txt, because a human requested the fetch.

Blocking the training crawlers is a perfectly legitimate choice, and many publishers make it. Just make it deliberately, knowing what it costs. Blocking the retrieval crawlers, by contrast, is almost never what a firm intends — and in my case it happened by accident.

Two traps are worth knowing. Firewall rules match on fragments of text, so a rule aimed at one bot catches others by accident — which is exactly what happened to me. And OpenAI changes its version numbers; GPTBot went from 1.1 to 1.4 inside a year, and OpenAI states the number may change. Test for the crawler’s name, never its full version string, or you will get a clean-looking result that is wrong.

3Why the assistant describes you as who you used to be

Fixing the crawlers was the easy half. The harder discovery was what the assistants said about me once they could read anything at all.

I asked Perplexity, cold: “Who is Philippos Aristotelous?” It answered confidently, cited my own website first, and described me as a lawyer-turned-business-strategist and coach advising on leadership, culture and engagement. Every word accurate — about 2019. Not one mention of the legaltech and AI implementation work that is now most of my practice.

The explanation is uncomfortable, and it applies to almost every professional and every mid-sized firm. A language model has no stored knowledge of someone like me. Research presented at ACL 2023, covering fourteen thousand questions about named entities, found that models “struggle with less popular factual knowledge, and scaling fails to appreciably improve memorization of factual knowledge in the long tail.” A consultant, a partner, a twenty-lawyer firm — all long tail by construction. The assistant does not remember you. It assembles you, live, from whatever pages it can find, and it looks for agreement between them.

So I went and looked at the pages about me. A speaker bureau profile calling me an “award-winning lawyer, business strategist, CEO/executive coach” with six talk titles about leadership and happiness and not one word about AI. An author bio in the same vein. An author page publishing a dead LinkedIn address that contradicted my real one — and two different identifiers for the same person is precisely what breaks the matching these systems depend on.

Eight well-indexed pages saying “coach”. One recent newspaper column saying “AI adviser”. The assistants were not wrong. They were reporting a consensus, and the consensus was six years out of date.

Most of the pages that define your firm are not your firm’s. If they disagree with your website, the older and more numerous version usually wins.

Directories, bar listings, conference profiles, the partner bio an old employer never updated, the podcast episode description, the book page. Correcting mine took about a day and involved no web development at all — and within hours of one profile being corrected, an assistant was already citing the new version.

4What the evidence says actually works

There is a great deal of confident nonsense written about this, so I went looking for studies with methods attached.

The most useful is a peer-reviewed paper from KDD 2024, which built a ten-thousand-query benchmark, tested nine different ways of changing content to see which got it cited more often, and validated the results on the live Perplexity engine.

Change to the contentEffect on citation
Adding direct quotations from named sources+41%
Adding concrete statistics+33%
Improving fluency (plain, readable prose)+29%
Citing sources+28%
Keyword stuffingNegative

The four that worked are the same underlying move: put verifiable, attributable evidence next to your claims. Name the regulation. Give the number. Quote the person who said it. The one traditional search tactic tested actively hurt.

For a law firm this is good news, because it is also simply better professional writing. “We help firms adopt AI responsibly” is invisible. “The CCBE’s October 2025 guide on generative AI states that lawyers could face sanctions for professional misconduct if they use AI output without verification — here is how we build that check into a firm’s workflow” is citable, and more useful to the reader besides.

Freshness matters, but not the way people think

Ahrefs found AI-cited pages were about 25% newer on average than organic top-ten results. But a separate study by Seer Interactive, covering 47,000 citations across 7,683 pages, found something more useful: pages cited consistently, month after month, averaged six months old, while one-off citation spikes averaged two months. Over a quarter of “fresh” cited pages had originally been published two or more years earlier.

Durable visibility comes from maintaining a small number of strong pages, not from constant publishing. Update the twenty pages that matter; do not churn out two hundred.

And the finding that surprised me most

Profound analysed 1.4 million citations across six AI models and found LinkedIn is the single most-cited domain for professional queries on every one of them. Semrush, across 325,000 prompts, found LinkedIn appears in 14.3% of ChatGPT search answers. Two details matter for a firm: long-form articles carry 50 to 66% of those LinkedIn citations while ordinary feed posts carry 15 to 28%, and accounts with fewer than 500 followers are cited about as often as large ones. Publishing frequency tracks citation; audience size does not.

In my own case this was not theoretical. When an assistant was asked whether I work on AI implementation for law firms, it answered correctly and in detail — and every source it cited was LinkedIn. My website, with three pages on exactly that subject, contributed nothing.

5What does not work

llms.txt. A proposed file listing your site’s content for AI. Google’s John Mueller compared it to the keywords meta tag, and Google’s own documentation says you do not need “AI text files.” A fourteen-day study of server logs covering 147,000 requests from eighteen AI crawler families found 723 requests for robots.txt and zero for llms.txt. It is harmless, and it does nothing.

Structured data as an AI lever. Schema markup is the most oversold item in the field. Cited pages are about three times more likely to carry it — which is why it gets sold hard. But when Ahrefs ran the only controlled test, following 1,885 pages that added it against matched controls, the citation change was statistically indistinguishable from zero on ChatGPT and Google AI Mode, and slightly negative on AI Overviews. The correlation exists because well-run sites do everything else right too. A separate experiment placed a fact only inside the markup and asked five AI systems to retrieve it; none could, because the markup is stripped before the model sees the page. Keep your structured data correct as basic hygiene, and expect nothing from it.

FAQ markup specifically. Google removed the FAQ rich result entirely in May 2026. The visible question-and-answer format on the page is what Microsoft describes as working. The hidden code around it does nothing.

Any percentage attached to “author credentials” or E-E-A-T. Figures such as “pages with visible author credentials are 41% more likely to be cited” circulate widely. I could not trace a single one to a published study with a stated method. Write a proper author bio because readers and consistency deserve it, not because of a number somebody invented.

Buying an “AI visibility score”. No platform publishes one. Every tool selling one is inferring.

6A practical sequence for a law firm

In the order the evidence supports, cheapest and most certain first.

  1. Test your crawler access from outside, this week. Request your own homepage, robots.txt and sitemap while identifying as Bingbot, OAI-SearchBot, PerplexityBot, Claude-SearchBot and Googlebot, and confirm each returns 200 rather than 403. Ask whoever runs your server for the list of blocked user agents. Check also for noarchive and nosnippet directives — Bing has stated that noarchive excludes a page from Copilot answers.
  2. Write one sentence describing the firm, and use it everywhere, word for word. Then work through every third-party page about the firm and its partners and make them agree. Fix conflicting links, especially two different website or LinkedIn addresses for the same person. This is the highest-leverage work on the list and it needs no developer.
  3. Make every service page answer one question in its opening paragraph, and name the firm in it. My own AI services page, well written and technically clean, never actually stated my name in a sentence — it was all first person. A machine found the service and the country but no subject.
  4. Put the evidence into your best pages. Quotations from named authorities, real statistics, citations to the regulation or guidance. This is the +41%, +33%, +28% lever.
  5. Use visible, question-shaped headings, and do not hide the answers in accordions, tabs, images or PDFs.
  6. Publish long-form under a named person, on a cadence. Every two to three weeks beats a burst followed by silence.
  7. Maintain twenty pages properly rather than publishing two hundred thin ones.
  8. Get quoted elsewhere. Muck Rack’s analysis of 25 million AI citations found 84% trace to earned media. A controlled test by Stacker found the same article’s citation rate rose from 8% to 34% when it also appeared on third-party outlets.

7Measuring it honestly

Most of what is sold as measurement is noise. Three things actually work.

A fixed panel of buyer questions. Write down five to eight questions a real client would ask — “who helps a fifteen-lawyer firm in Cyprus implement Claude?”, “law firm management consultant Limassol” — and put them to the same assistant at fixed intervals. Record two things for each: whether your website appears in the source list, and whether the firm is named in the answer text and from which source. Keep the wording identical every time, or the runs are not comparable.

Three to four weeks between readings, not three hours. I re-ran my own panel on the same afternoon as a batch of changes and, predictably, nothing had moved. Indexes have to re-crawl before anything can change. More frequent runs produce noise that looks like signal.

Bing Webmaster Tools’ AI Performance report is the only first-party instrument that exists. It reports citations in Copilot and, uniquely, the “grounding queries” — the internal sub-questions Copilot actually asked before citing you. That is the only place any platform tells you what an AI wanted to know.

A warning worth passing on

One of my own measurement runs reported “zero citations, unchanged”. It was wrong. The script behind it had failed silently, every query had returned an error, and the counter dutifully found zero mentions in the error messages. Smoke-test the instrument before trusting a null result. A confident zero is the easiest number to produce by accident.

8Where I have got to

For transparency, since this article is built on my own case. Every AI crawler now reaches my site. Every external profile has been corrected. My pages state in plain sentences what I actually do. And asked those buyer questions today, the assistants name me in three of five — first, for one of them.

Every one of those mentions still comes from LinkedIn. My website is cited zero times.

That is honest, and it is expected: the pages are days old and the crawlers only regained access this week. The next reading that means anything is three weeks away, and it is already in the diary.

Which is the right note to end on. This is not a project a firm finishes. It is the habit of keeping the pages about you — yours and everyone else’s — consistent, evidenced and current, and then being patient enough to measure properly.

Wondering what an assistant says about your firm?

A short conversation about where your firm stands, and what would change it.

Legaltech & AI implementationArrange a conversation →

Sources & notes. Google Search Central, “AI features and your website”. Microsoft, “Optimizing your content for inclusion in AI search answers” (October 2025) and the Bing Webmaster Tools AI Performance report (February 2026). OpenAI crawler documentation. Anthropic’s crawler support article. Aggarwal, Murahari et al., “GEO: Generative Engine Optimization”, KDD 2024 (arXiv:2311.09735). Ahrefs: AI Overview citation-versus-ranking analysis (March 2026), the schema and AI citations controlled study (May 2026), and the freshness analysis of approximately 17 million citations. Seer Interactive, content recency and AI visibility (7,683 pages, 47,097 citations). Mallen et al., “When Not to Trust Language Models”, ACL 2023 (arXiv:2212.10511). Profound, LinkedIn as the most-cited domain for professional queries. Semrush LinkedIn citation study (325,000 prompts). Muck Rack, “What Is AI Reading?” (May 2026). Stacker earned-media citation-lift test. A fourteen-day AI crawler server-log study (July–August 2026). searchVIU schema markup and AI systems test (December 2025). Figures are quoted as published; where a widely circulated claim could not be traced to a stated method, it is named as untraceable rather than repeated. Related reading on this site: what a law firm’s first AI pilot really tests and how Cyprus law firms can actually use Claude.

Disclaimer

This article is published for general information and education. It is necessarily general, is not legal, tax, accounting, financial or other professional advice, and should not be relied on in place of advice tailored to your circumstances. Reading it creates no client or advisory relationship. Reasonable care is taken in preparing what is published here, but platforms, guidance and research change quickly in this area, and no warranty is given as to continuing accuracy or completeness. To the maximum extent permitted by law, Philippos Aristotelous and Philarist accept no liability for loss arising from reliance on this article.

Talk about your firm

Book a free 30-minute call. When you book, use the Notes box for a short brief: who you are, your firm and what you would like to discuss. I read every brief before the call. If I am not the right person, I will tell you within one working day and suggest a better route.

Book a call or write to me
Close Menu