Design a GEO study without overstating the results

“Did visibility improve?” is too broad without a defined outcome. Ask whether a fixed set of unbranded discovery answers mentions the brand, whether eligible citations point to owned domains or whether specific product claims are supported. Keep these outcomes separate.

The GEO paper motivates purpose-specific visibility measures, while ALCE studies answer and citation quality. Their metrics answer different questions. Neither authorizes mixing mentions, referrals and factual correctness into an unexplained universal score.

Version the prompt set and specify inclusion rules before collection. Separate category discovery from branded verification. Record assistant, exposed model label, mode, language, location, conversation history, memory setting where available and time window. Keep comparable settings across observation periods.

OpenAI documents query rewriting and possible memory or location influence. A fresh conversation does not necessarily remove every personalization effect, so record what was controlled and what was unknown. Repeat prompts under a declared schedule rather than keeping only the most favorable answers.

For mention rate, declare which successful unbranded answers count and how names are disambiguated. For citation share, define eligible link types, owned domains and duplicate handling. Also report how many answers had no eligible citations. Failed requests belong in an attempts-and-failures log, not silently inside or outside every denominator.

If there are no eligible citation occurrences, share is not applicable. That does not prove a zero share of user attention. If the prompt set changes, publish the new version and avoid presenting it as a like-for-like continuation.

Fictional observations: one period contains six owned citation occurrences out of 24 eligible occurrences, or 25%. A second period contains six owned occurrences out of 18, or about 33.3%. The owned count is unchanged; the share increased because the denominator decreased.

Now suppose brand mentions move from 12 of 30 successful answers to 15 of 30. That is an observed increase from 40% to 50% in this cohort. It is not proof of a content-edit effect, a population estimate or evidence of more visits. Preserve the answers and review whether recommendations and claim support changed as well.

Between periods, assistants, search indexes, competing pages and user context can change. An unchanged comparison cohort can provide useful context, but does not automatically remove those confounders. Where a controlled experiment is feasible, declare assignment and evaluation rules in advance. Otherwise describe association rather than causation.

Repeated runs of the same prompt are not automatically independent observations. If you calculate uncertainty, use a method appropriate to that repeated structure and explain the sampling limits. Do not claim a representative industry benchmark from a convenience sample.

• Research question, metric definitions and prompt-set version.
• Dates, assistants, settings and known uncontrolled conditions.
• Attempts, failures, successful answers and eligible citation counts.
• Raw numerators and denominators, not percentages alone.
• Claim-review rubric, unresolved cases and reviewer disagreements.
• Page changes, competing explanations and study limitations.

The 2024 GEO study reports interventions in its own experimental setting. Treat it as historical evidence to examine, not a guarantee for a current product or your website. Keep business outcomes such as referrals and conversions in separate measurements.