What you need to know
- Google has launched a new opt-out experience dubbed Google-Extended, which will help publishers restrict Bard from using their content to generate responses.
- The company indicated that this new feature is designed to help publishers establish more control over their content.
- The emergence of AI and chatbots has negatively impacted website traffic, revolutionizing how people run searches while using the web.
The emergence of AI and chatbots has revolutionized how users interact with the web, especially when running searches. It's now possible to get tailored answers within chatbots.
While this is an impressive feat, it has raised much concern among publishers and the media. This is because chatbots scour the internet for information and package it as per the user's prompt. The main issue is how these companies will compensate websites and publishers for the information chatbots pick from them. There's also the issue of attribution.
Google seems to be listening to the publishers' plight and is now rolling out a new control dubbed Google-Extended. It will let publishers restrict Bard from accessing and using their content to generate responses.
As highlighted by Google, the feature is part of its robots.txt indexing file that publishers can leverage and include in their site's documentation to inform Google not to use their content to "help improve Bard and Vertex AI generative APIs, including future generations of models that power those products."
According to Danielle Romain, Google's VP of Trust, the company decided to incorporate this new addition to its AI ventures to provide publishers with more control over how their content is used "for emerging generative AI use cases."
Analysis: Google on Microsoft's neck
Chatbots like Bard and Bing Chat heavily rely on information from websites when giving responses to queries. This has raised much concern among publishers and the media, and for good reason.
Chatbots pick information from multiple websites and present it in a compact and summarized format, including all the necessary information. This has greatly impacted how end-users interact with search while using the web, negatively impacting website traffic.
Google has made a step in the right direction by giving publishers control over how the AI chatbots interact with their content. Microsoft made a similar move last week, adding a new set of tools designed to help publishers "control indexing and snippet length on Bing."
The company revealed that this move aims to help publishers establish more control over how their content is being used on the web. Publishers will have access to a wide variety of controls that will help them determine how Bing Chat uses their content, as listed below by Microsoft:
- No action is needed to remain in Bing Chat. Content without NOCACHE tag and without NOARCHIVE tag may be included in Bing Chat answers and will benefit from AI's ability to generate more helpful answers and to increase your ranking opportunities in Bing Chat; site content may be used in training our generative AI foundation models.
- Content with the NOCACHE tag may be included in Bing Chat answers. We will only display URL/Snippet/Title in the answer; Going forward, for content in our Bing Index that is labeled NOCACHE, only URLs, Titles and Snippets may be used in training Microsoft’s generative AI foundation models.
- Content tagged NOARCHIVE will not be included in Bing Chat answers, not be linked to in the answers. Going forward, for content in our Bing Index that is labeled NOARCHIVE, we will not use the content for training Microsoft’s generative AI foundation models.
- If content has both NOCACHE and NOARCHIVE tags, we will treat it as NOCACHE.
At the beginning of the year, Microsoft responded to the woes of publishers by introducing a new compensation scheme. The scheme is in place to ensure that publishers are compensated every time Bing Chat picks information from their websites. Microsoft further added that its top priority is to "drive more traffic to publishers in this new world of search."
It's worth noting that if you opted for the option of not letting Google's AI-powered chatbot, Bard, use your content when generating responses, and it will also apply to future models that will come up. It'll also be interesting to see how this change will impact the chatbot's accuracy.
Do you think publishers having more control over their content will harm the accuracy of chatbots? Let us know in the comments.
UPDATE ON SEPTEMBER 29, 2023: The article has been updated with information about Microsoft's options for webmasters to control the usage of their content in Bing Chat.
