TL;DR
- Do GEO tricks work? Some move an AI ranking a little, on the lists they were tested on. Among the rewrites most GEO pitches sell (adding quotes, statistics or citations), the ones that move it most are the ones a filter flags most. And the uplift number you get quoted depends on who wrote the test.
- The evidence is GEO-Bench, an academic benchmark from the University of Southern California and Arizona State University (arXiv 2605.29107, version 3, 12 October 2026, not peer reviewed). It scores 15 ways of editing a piece of content to rise in a list that an AI model ranks.
- On Llama-3.1-8B, rewrites that add quotes, statistics or citations lifted a target more than strings optimized on the ranking model itself. An attack that searched with feedback from the ranker lifted it most.
- The same attack text scored about five times higher under its authors' released evaluation script than under the benchmark's shared one (Llama-3.1-8B, mean over the five datasets). Without its method, a GEO uplift figure cannot be checked.
- All of it happens after retrieval, on a list that is already fixed. Nothing in the benchmark says how to get into that list, and that step comes first.
GEO, generative engine optimization, is now sold the way SEO once was: a list of tactics, a before-and-after percentage, and a promise that AI answers will name you. If GEO is new to you, our GEO overview covers the basics. This post is for the person evaluating an AI SEO or GEO agency's pitch. One benchmark has now put most of the tactics in that pitch through the same test, and its results are more useful to a buyer than to a seller.
What is GEO-Bench?
Before GEO-Bench, each manipulation method had been tested with its own data, its own prompt, its own metric and its own AI ranker, so nobody could say which one was stronger. GEO-Bench runs 15 methods through one protocol. They come from three families:
- Black-box prompt attacks: an AI attacker writes text meant to push a target up the list, in some methods querying the ranker and learning from its answers.
- White-box gradient attacks: strings of tokens optimized directly on the ranking model's own weights. They usually read as gibberish.
- White-hat rewrites: the tactics in most GEO pitches. Add quotes, add statistics, add citations, take an authoritative tone, or rewrite for general content quality.
GEO-Bench takes these rewrites from the original 2023 GEO paper (arXiv 2311.09735) and a 2025 follow-up benchmark, C-SEO Bench. The 2023 paper is the source of the "up to 40 percent" figure our GEO overview cites. That figure measured visibility inside generated answers on the authors' own benchmark; GEO-Bench measures rank in a short list under one shared script. The two numbers measure different things and cannot be compared directly, which is the point of the section on scripts below.
The test set is 729 targets in 83 lists of 7 to 10 items, drawn from five datasets in the GEO research literature, almost all of them English product lists. Each edit is scored on two things: how far it moves its target, and how detectable it is. Movement is reported as normalized rank gain (NRG): 1 means the target went from last place to first, 0 means it did not move. The main ranker is Llama-3.1-8B, an open model, and the results are re-scored on three more (Qwen2.5-14B, Mistral-7B and GPT-4o-mini) plus a cross-encoder reranker.
Read everything below with its limits attached. This is an academic paper, not yet peer reviewed. It uses one AI judge with one judge prompt for detectability, has no human study, and ranks short English lists with open 7B to 14B models and GPT-4o-mini, not with ChatGPT search, Perplexity or Google's AI features as deployed. There is no first-stage retrieval: the list is handed to the ranker already built.
Which GEO tricks actually move an AI ranking?
In GEO-Bench, on Llama-3.1-8B averaged over five datasets, rewrites that never saw the ranker moved a target further up an AI-ranked list than strings optimized on the ranker's own weights, and a search attack that learned from the ranker's answers moved it most (0.179 NRG).
| Method | Family | Rank gain (NRG) | AI judge flags, at a 6% false-positive rate | Gain left after either filter |
|---|---|---|---|---|
| TAP, a tree-search attack that queries the ranker (Llama-3.1-8B attacker) | Black-box attack | 0.179 (24% of targets moved to first place) | 60% | 0.054 |
| Content improvement rewrite | White-hat rewrite | 0.125 | 31% | 0.015 |
| Add quotes | White-hat rewrite | 0.105 | 76% | 0.007 |
| Add citations | White-hat rewrite | 0.098 | 63% | 0.023 |
| Add statistics | White-hat rewrite | 0.090 | 74% | 0.022 |
| Authoritative tone | White-hat rewrite | 0.083 | 10% | 0.043 |
| Zero-shot, an AI attacker with no ranker feedback | Black-box attack | 0.053 | 15% | 0.030 |
| STS, RAF and StealthRank suffixes | White-box gradient attack | 0.013 to 0.031 | 55% to 100% | 0.000 or below |
Three readings hold up. First, the gains are modest. On Llama-3.1-8B, averaged over the five datasets, a content improvement rewrite gained 0.125 NRG: one eighth of a full last-to-first climb. Second, what separated the top attack from the zero-shot one was feedback from the ranker, not a smarter attacker: a Qwen2.5-14B attacker matched the Llama-3.1-8B one (0.177 against 0.179 NRG, Llama-3.1-8B ranker, five-dataset mean). On Llama-3.1-8B, TAP's lead over the content improvement rewrite is not statistically significant once multiple comparisons are corrected for. Third, access to the model's weights did not translate into advantage. The gradient strings were optimized on Llama-3.1-8B's own weights and still trailed every attack and every effective rewrite in the table (0.013 to 0.031 NRG, five-dataset mean). They were scored under a ranking prompt they had not been tuned on, which is the position of anyone who cannot see a platform's ranking prompt; the authors say this suggests persuasive content transfers across prompts better than optimized tokens. They also note that a gradient attacker that knows the deployed prompt could be stronger.
Do GEO tricks get caught?
In GEO-Bench, the GEO tactics that work can be detected, but a filter run at one fixed threshold, as a platform would run it, catches only some of them. GEO-Bench asked an AI judge (Qwen2.5-14B) whether each text was "manipulative toward a ranking system." Measured across all thresholds, the judge separates every method that gains at least 0.05 NRG from clean text, with an AUROC between 0.72 and 0.98 (Llama-3.1-8B, five-dataset mean; 0.5 is a coin flip, 1 is perfect separation). A real filter does not run across all thresholds. It runs at one, chosen to avoid flagging honest content.
At a 6% false-positive rate, averaged over the five datasets, the judge flags only 10% of authoritative-tone edits and 31% of content improvement edits, and, against the judge alone, those keep most of their gain on Llama-3.1-8B. A simple keyword filter flags a different set: 74% of content improvement edits, but only 19% of statistics edits (keyword violation rate, five-dataset mean). Combining the two, so that an edit is flagged when either filter fires, with the pair tuned to flag 10% of honest originals, leaves every rewrite at 0.046 NRG or less (Llama-3.1-8B, five-dataset mean). The search attack held up best: run with a Qwen attacker, it kept 0.081 NRG on Llama-3.1-8B after both filters (five-dataset mean), the most of any method. Measuring how predictable the text reads (perplexity) only exposed the gradient strings. Every attack and rewrite outside the gradient family read as more predictable than the original (five-dataset mean).
For a buyer, the point is the link between how much a tactic lifts and how easily it gets flagged. In this benchmark, the rewrites that lift the most (content improvement, quotes, citations, statistics) are also the ones a keyword filter or an AI judge flags the most. A plain authoritative rewrite gains less, and the AI judge barely flags it (10% of edits at a 6% false-positive rate, five-dataset mean), though 31% of its edits tripped the keyword filter (keyword violation rate, five-dataset mean). Whatever lift a tactic buys depends on where a platform sets its filter, and that is a setting the platform can change on any day.
Why do GEO results differ between agencies?
GEO-Bench did not test agency reports, but it shows why their numbers diverge: the test script can move the measured gain as much as the tactic does. This is the most useful result in the paper for anyone reading a GEO case study.
The authors took the attack strings produced by one published method, StealthRank, and scored the same strings two ways on the same ranker, Llama-3.1-8B. Under the method's own released script, on Llama-3.1-8B, the mean NRG over the five datasets was 0.25. Under GEO-Bench's shared protocol, on Llama-3.1-8B and the same five datasets, it was 0.05. Nothing about the attack changed. These are the strings StealthRank's own released code produced; GEO-Bench's own re-run of the method, in the table above, scored 0.031. The scripts differ in the chat template, the way the list is formatted, the wording of the query, and the shared protocol's use of the median rank over ten random product orders. The authors did not test those factors one at a time, so they cannot say which one matters most.
The dataset matters too. Under the same protocol on Llama-3.1-8B, the search attack reached 0.32 NRG on one dataset (Ragroll), while on another (STSData) no method passed 0.07. A gain measured on one dataset says how that dataset behaved. It does not say what will happen to your pages.
So an uplift percentage on its own tells you about the test that produced it. Two percentages from two different tests cannot be compared directly.
What does the benchmark not tell you?
- Whether you get found. Every list in GEO-Bench is fixed before the ranker sees it. Real AI search first decides which pages it can fetch and which it retrieves, and the benchmark measures none of that. We covered the documented part of that layer, which crawler each engine uses, in how AI chooses what to cite, and what models prefer once the candidates are fixed in why AI keeps citing the same sources.
- How big the effect is on the engine you care about. The order of methods carried across the four AI rankers (rank correlation 0.84 to 0.94), but the size of the gains did not. The content improvement rewrite gained 0.082 NRG on GPT-4o-mini and 0.279 on Mistral-7B (equal-weight mean over the datasets), and GPT-4o-mini was the hardest ranker to move. The cross-encoder reranker ordered the methods differently again, and quotes, statistics and citations left it flat or lower.
- How a human reads the edited text. There is no human study, and detectability rests on one judge with one prompt.
- Anything about ChatGPT, Perplexity or Google's AI features as they run today. None of them was tested, and none publishes its ranking method.
How do you evaluate a GEO agency's pitch?
- Which engine, and which model? A gain on one model is not a gain on another. Even inside this benchmark, the size of a gain changed several-fold between rankers.
- Which questions, and when? AI visibility is a measurement over a fixed, dated set of questions, re-run over time. Our AI visibility funnel lays out what to measure at each stage.
- Whose script? Ask for the method behind the number: the prompt, the list, the model, how many runs. Without it, the figure cannot be checked or compared.
- Was retrieval measured first? If your pages are not retrieved for the questions your buyers ask, ranking tactics are working on a list you are not in.
- Is the tactic one a filter would flag? Rewrites that add quotes or statistics were the rewrites this benchmark's AI judge flagged most often: 76% of quote edits and 74% of statistics edits at a 6% false-positive rate (Llama-3.1-8B setting, five-dataset mean).
None of this says the white-hat tactics are worthless. Adding a real statistic with its source, or quoting someone who knows the subject, can make a page more useful. The lift is real on a list, but it is small, it depends on the ranker, and the biggest lifts are the easiest to flag. On its own, it is not a strategy.
What we do instead
Our GEO work starts by measuring: we run the questions buyers actually ask through the AI engines that matter, record whether a client's pages are retrieved, cited or passed over, then track what happens, keeping what works and dropping what does not. Page changes come after that, aimed at answering those questions well rather than at tactics a filter learns to flag (see our SEO audits).