Google Deduplication Of Top Stories From Web Search Results
We know that Google commits to de-duplicate when a featured snippet appears so they can remove the snippet from Top Web Results. display it in the main web results.
Google may, in some cases, remove the web search result snippet when it displays the same result in the Top Stories section of Google search results.
The rest of the story continued but the update is in the tweet above.
Google’s Danny Sullivan had this to say after a complaint from The Verge on Twitter. Danny said that “it shows up in the top stories, it’s deduplicated from the rest of the site. Deduplication can often be helpful.Performing this search as the user would, using solution search terms rather than uncommon terms inthe title tops the Top Stories, and the deduplication means there is more variety compared to other posts. During such searches, our systems also generally attempt to display the most useful and reliable information they can. Because of this, you won’t see many duplicates of Duplicates certainly exist, but it doesn’t make as much sense to show them. This leads to headline-oriented searches. As I said before, this is very common among writers. I used to do that myself.However, headline searches often contain a large number of terms, so our systems switch to returning pages containing those terms. That means authors are more likely to find duplicates, even though they probably won’t turn up in typical searches that readers would do. our deduplication feature can still be enabled for these as well, as was the case in this case. As said, deduplication can be useful.But we also understand the concern it might cause.We’ve been doing this with Top Stories since last May, but we’ll revisit it to see if it’s appropriate to continue or perhaps make further changes. Also, I’m still checking, but I think this deduplication is particularly unique as it only occurs on highlights when there is only one story. shown or maybe just for the first story shown.
As you can see, Danny gives this as an excuse for why other posts appear in the web results and not The Verge for this query. But as you can also see it seems to explain that sometimes it doesn’t work that way.
The interesting part is that it looks like Google is really deduplicating here when it comes to the main stories.
Amusingly, a few hours earlier, News SEO Barry Adams had a whole Twitter thread about it:
So I pointed out to Barry Adams the tweet from Danny and his response:
So I did some sample searches and Google seems to be removing a recent story from this site from the top news and showing it in the web results and at the same time removing those stories from the top news from the web results. top stories here but are in web results and at the same time SEL and SEJ are in top stories but not in web results (click to enlarge):
It is therefore unclear when and why Google might de-duplicate a URL from display in web search when it is also displayed in headlines. It could be a timing issue or something else. Danny Sullivan said, “Also, I always check, but I think this deduplication is particularly unique because it only happens with Top Stories if only one story is displayed or maybe just for the first story displayed.