For many years, SEO professionals have worked under a fundamental belief: the broader the scope of your content, the higher the chances it has of appearing in AI-generated responses. Indeed, each “best practice” in traditional SEO content encourages you to create more: additional subtopics, extra sections, and increased word count. Craft the “definitive guide.”
An examination of 815,000 query-page combinations from 16,851 queries and 353,799 pages presents a different conclusion:
- The coverage of fan-out has little significance concerning citation rates.
- There are two indicators that can effectively forecast if ChatGPT will reference your page.
- Implementing six specific modifications to your current content library can be beneficial.
Sure! Please provide the text you would like me to paraphrase, and I’ll be happy to help.
Please provide the text you would like me to paraphrase, and I’ll be glad to assist you!
1. The Research
AirOps executed 16,851 queries via ChatGPT three times utilizing the user interface, documenting every fan-out sub-query, each URL investigated, all citations referenced, and every page that was scraped. The pipeline was developed by Oshen Davidson, while I focused on analyzing the data.
Every query results in an average of two fan-out queries. ChatGPT fetches approximately 10 URLs for each sub-search, reviews them, and then decides which ones to reference. We evaluated the alignment of each page’s H2-H4 subheadings with those fan-out queries by utilizing cosine similarity on bge-base-en-v1.5 embeddings. This evaluation is referred to as fan-out coverage: the percentage of subtopics that a page covers at a 0.80 similarity threshold. (The 0.80 threshold was established to determine if a subheading qualifies as a match to a fan-out query, serving as a bar for relevance.)
The inquiry: Are pages that have greater fan-out coverage cited more frequently?
Additional details can be found in the jointly authored AirOps report.
2. Density Makes Minimal Impact
In a dataset comprising 815,484 rows, there exists a weak correlation between fan-out coverage and citations.
Addressing all subtopics results in an increase of 4.6 percentage points compared to not covering any. This difference diminishes even more when adjusting for query match (the degree to which the main heading of the page aligns with the original query). For pages that demonstrate a strong query match (>= 0.80 cosine similarity):
Image Credit: Kevin Indig
Moderate coverage, ranging from 26% to 50%, is more effective than comprehensive coverage. In fact, pages that attempt to encompass all aspects tend to achieve lower scores compared to those that delve into about a quarter of the subtopics. The approach of creating an “ultimate guide” yields poorer outcomes than a targeted article that thoroughly addresses two to three related perspectives.
3. What Truly Determines Citation
The two primary signals are: query match and retrieval rank.
1. Retrieval rank serves as the most significant predictor by a considerable margin. A webpage listed at position 0 in ChatGPT’s search results (which is the first URL provided by its search functionality) has a citation rate of 58% citation rate. However, by the time one reaches position 10, this percentage declines to 14%. For this analysis, we executed each prompt three times in succession, and the pages that were cited in all three instances had a median retrieval rank of 2.5. In contrast, pages that were never cited had a median rank of 13.
Image Credit: Kevin Indig
2. Query match (the cosine similarity between the query and the most relevant heading on the page) serves as the most significant content indicator. Pages that achieve a heading match of 0.90 or higher experience a citation rate of 41%, while those with a match below 0.50 see only a 30% citation rate. Furthermore, within the top-ranked pages (positions 0-2), an increased query match contributes an additional 19 percentage points.
Fan-out coverage, number of words, count of headings, and domain authority: these are all of lesser importance. Some are uniform, while others show an inverse correlation.
4. The Wikipedia Exemption
One category of websites deviates from the norm. Wikipedia ranks the lowest in retrieval within the dataset (median 24) and has the least favorable query match score (0.576). Despite this, it boasts the highest citation rate at 59%.
Wikipedia articles typically consist of around 4,383 words, include 31 lists, and feature 6.6 tables. They embody the essence of an encyclopedia. ChatGPT references Wikipedia from the depths of search results, where other types of websites tend to be overlooked.
This density functions as a signal, yet it operates on a level that no publisher can duplicate. The content on Wikipedia is comprehensive, intricately organized, and interlinked across countless subjects. In contrast, a 3,000-word corporate blog article featuring 15 subheadings does not compare.
5. The Bimodal Reality
In this dataset, 58% of the pages obtained by ChatGPT are not cited at all. Meanwhile, 25% are consistently referenced whenever they are used. Only 17% are cited sporadically.
The groups that are frequently referenced and those that are rarely mentioned appear almost the same across various content metrics. They have comparable word counts (approximately 2,200), a similar number of headings (around 20), alike readability scores (approximately 12 FK grade), and nearly the same domain authority (about 54). The measurable on-page indicators do not distinguish between successful and unsuccessful content.
The distinguishing factor is the retrieval rank. Pages that are frequently cited tend to appear at the top when they are found, while those that have never been cited fall into the lower half. The retrieval system, regardless of the internal signals it utilizes, acts as the gatekeeper, with all other elements serving as tiebreakers.
6. Implications for Your Content
Traditional wisdom in SEO content writing suggests that one should address a larger number of subtopics, include additional sections, and enhance content density. However, data indicates that this conventional method results in “mixed” pages, with 17% falling in a gray area where they are occasionally referenced but often overlooked.
Pages that are mixed in nature exhibit the greatest word counts, contain the most headings, and possess the highest domain authority within the dataset. They are regarded as the “ultimate guides.” However, they are also the least dependable performers when it comes to ChatGPT.
Consistent winning pages maintain a clear focus. They:
- Directly align the query with their titles,
- Usually maintain a concise length (the ideal citation range is between 500 and 2,000 words), and
- Include sufficient organization (7-20 subheadings) to arrange the information without compromising its quality.
Create a page that provides the most comprehensive answer to a single question, rather than one that merely addresses 20 questions adequately.
If you’re ready to put these digital marketing strategies to work for your business, we’d love to help. Digital Assets Marketing specializes in SEO, Google Business Profile management, and data-driven advertising tailored to small businesses in Central California. Schedule an appointment with us today and let’s discuss how we can grow your online visibility and drive real results for your company.
