ChatGPT Search Shows 76.5% Error Rate in Attribution Study

OpenAI’s ChatGPT Search struggles to accurately cite news publishers, says a study by the Tow Center for Digital Journalism at Columbia University.

The report found frequent misquotes and incorrect attributions, raising concerns among publishers about brand visibility and control of their content.

Additionally, the findings call into question OpenAI’s commitment to the responsible development of AI in journalism.

ChatGPT Research Background

Launch of OpenAI Search ChatGPT last month, saying it collaborated extensively with the news industry and incorporated editors’ comments.

This contrasts with ChatGPT’s initial rollout in 2022, where publishers discovered their content had been used to train AI models without notice or consent.

Now, OpenAI allows publishers to specify via the robots.txt file whether they want to be included in ChatGPT search results.

However, the Tow Center’s findings suggest that publishers face the risk of misattribution and misrepresentation regardless of their opt-in choice.

Accuracy issues

The Tow Center evaluated ChatGPT Search’s ability to identify citation sources for 20 publications.

Key findings include:

Out of 200 queries, 153 answers were incorrect.
The AI rarely admitted its mistakes.
Phrases such as “maybe” were only used in seven responses.

ChatGPT often prioritizes user satisfaction over accuracy, which could mislead readers and damage publishers’ reputations.

Additionally, researchers found that ChatGPT Search is inconsistent when asked the same question multiple times, likely due to the randomness built into its language model.

Citing copied and syndicated content

Researchers discover that ChatGPT Search sometimes cites copied or syndicated articles instead of original sources.

This is likely due to publisher restrictions or system limitations.

For example, when asked for a quote from a New York Times article (currently embroiled in a lawsuit against OpenAI and blocking its crawlers), ChatGPT linked to an unauthorized version on another site .

Even with the MIT Technology Review licensing OpenAI’s crawlers, the chatbot cited a syndicated copy rather than the original.

The Tow Center found that all publishers are at risk of misrepresentations by ChatGPT Search:

Enabling crawlers does not guarantee visibility.
Blocking crawlers does not prevent content from appearing.

These issues raise concerns that OpenAI’s content filtering and approach to journalism could drive people away from the original publishers.

OpenAI response

OpenAI responded to the Tow Center’s findings by stating that it supports publishers with clear attribution and helps users discover content with summaries, quotes and links.

An OpenAI spokesperson said:

“We support publishers and creators by helping ChatGPT’s 250 million weekly users discover quality content through summaries, quotes, clear links and attribution. We’ve worked with partners to improve the accuracy of inline citations and respect publisher preferences, including enabling them to appear in search by managing OAI-SearchBot in their robots.txt. We will continue to improve search results.

Although the company has worked to improve citation accuracy, OpenAI says it is difficult to resolve specific issues of misattribution.

OpenAI remains committed to improving its search product.

Looking to the future

If OpenAI wants to collaborate with the news industry, it must ensure that publishers’ content is accurately represented in ChatGPT Search.

Publishers currently have limited power and are closely monitoring legal action against OpenAI. The results could impact content usage rights and give publishers more control.

As generative search products like ChatGPT change the way people interact with news, OpenAI must demonstrate its commitment to responsible journalism to earn users’ trust.

Featured image: Robert Way/Shutterstock