Study of ChatGPT citations makes dismal reading for publishers

The study reveals ChatGPT's citation issues, impacting publishers' content accuracy and attribution.

: Tow Center's study highlights ChatGPT's citation inconsistencies affecting publisher content accuracy. Researchers found inaccurate sourcing in 153 instances, with only seven instances acknowledging uncertainty. Publishers face credibility and commercial risks due to these errors, while licensing deals with OpenAI don't guarantee better citation accuracy. The study calls for better scrutiny of OpenAI's technology handling journalism.

The Tow Center for Digital Journalism's study reveals significant issues with ChatGPT's handling of citations, showing that publishers are vulnerable to the tool's inaccuracies. Researchers tested ChatGPT with 200 quotes from various publishers, including The New York Times and The Financial Times, and found 153 incorrect citations, with ChatGPT rarely admitting uncertainty.

The study found that even publishers with content licensing deals with OpenAI faced erratic citation accuracy, putting their reputations at risk. Instances of erroneous attribution and content misrepresentation were noted, highlighting concerns about traffic diversion and dependence on ChatGPT for visibility.

Furthermore, the researchers pointed out that OpenAI treats journalism as decontextualized content, disregarding its original context. These findings indicate that publishers face challenges in managing how ChatGPT references their content, prompting the need for more robust measures to ensure accurate representations.

ChatGPT citationsdigital journalismcontent licensing