Hallucinated Sources

mechanism

When a model lacks evidence but is still pushed to answer, it may preserve the appearance of competence by inventing both the claim and its provenance. Verification must therefore test whether the cited source exists as well as whether it supports the claim.

The answer arrives with a reassuring detail: it supposedly came from a website. There is just one problem—the website does not exist. Unable to find the requested data, the model has produced “a nice answer” and fabricated the evidence beneath it.

E1

Plausibility fills the evidence gap

The failure has three moving parts: missing information, pressure to complete the response, and a system optimized to generate convincing language. Instead of stopping at “I cannot find that,” the model predicts a satisfying answer—and may predict a credible-looking citation along with it. The invented source makes the claim feel independently grounded even though both were generated together. This is why checking only whether an answer sounds right is insufficient: fluent prose and plausible provenance can be products of the same fabrication.

E1

Where it shows up

The website that never existed

A model cannot locate the data, yet answers anyway and attributes the result to a nonexistent website. The citation does not merely fail to prove the claim; it hides the absence of evidence.

E1

A missing source settles only part of the question

This example shows that models can fabricate provenance when evidence is unavailable; it does not establish that every unsupported answer is false or that every model response contains invented citations. A nonexistent source disproves the attribution, not automatically the underlying claim—which may still be confirmed elsewhere.

E1

Run a two-link citation check

Before using a consequential model-generated claim, open one cited source and perform two separate checks: confirm that the source actually exists, then locate the passage or data that supports the claim. If either link fails, treat the answer as unverified rather than repairing the citation from guesswork.

E1

Episodes that teach this