Why AI Invents Sources, and the Five-Minute Check That Catches One

A fabricated citation is not a broken citation. It is a well-formed one: a plausible author, a plausible title, a real journal, a volume and page range that sit exactly where they should, and often a DOI. Nothing about it looks wrong, which is the entire problem. The only reliable way to tell is to take the reference somewhere the model cannot follow it — a registry that either holds a record for it or does not.

That check takes about five minutes per reference at most, and often far less. Below is the procedure, plus the part almost nobody prints: what each step proves and what it leaves wide open. A citation has to answer three questions, and only two of them can be checked against a registry.

What a fabricated citation actually looks like

Here is one that is documented. In Kohls v. Ellison, a Minnesota federal case about deepfake law, an expert declaration cited a reference that the declarant himself later told the court does not exist:

Hwang, J., Zhang, X., & Wang, Y. (2023). The Influence of Deepfake Videos on Political Attitudes and Behavior. Journal of Information Technology & Politics, 20(2), 165-182.

— documented in Kohls v. Ellison, Doc. 39, as a nonexistent article. Do not cite it.

That article does not exist. The declarant said so himself in a later filing, describing it in his own words as "a citation to a nonexistent 2023 article by Hwang et al." It is safe to reproduce here only because it sits on a public federal docket, labeled as fabricated by the person who filed it. Nothing in it should be relied on except the two elements noted below.

How it got there is the part students should sit with. He describes a survey phase in which he used Google Scholar and GPT-4o "to identify articles that were likely to be relevant to the declaration," and an analysis phase in which the model summarized some of them. But the citation errors, he tells the court, "occurred in the drafting phase." He sometimes pasted his own bullet points into GPT-4o to have it draft a paragraph — by his best recollection, under prompts like "draft a short paragraph based on the following points" — leaving the word "[cite]" among them as a note to himself to add the real reference later. The model, he writes, "must have interpreted my note to myself as a command," and its response "was to generate a citation, which is where I believe the hallucinated citations came from." He says he never meant it to supply one. The request was for prose, and a citation arrived attached to it.

Then he had GPT-4o build the reference list from the draft, and did not run it through the reference software he normally uses — software that, he says, "might have caught the hallucinated citations." What he skipped was a check outside the model, of the same kind this article teaches.

Now look at what is real inside it. The DOI attached to that reference begins 10.1080, a prefix registered to Informa UK Limited. The Journal of Information Technology & Politics is a real journal with a real ISSN, published by Taylor & Francis. The parts a reader would use for a quick sanity check — the publisher behind the prefix, the journal, its ISSN — are all genuine. The article is not.

Coverage calls this shape a chimera citation, and uses "Real Authors, Wrong Paper" for the version where the named authors are real people (whether the three named here exist was not checked for this article). A close cousin, a real source attached to a claim it never made, is sometimes called hybrid fiction. Both are well described. What is harder to find is a plain statement of why it happens, and that turns out to explain the whole procedure.

Why a language model produces one

A citation is two things at once, and a model is good at one of them and structurally bad at the other.

The first is form. A reference is one of the most patterned strings in academic writing: surname, initial, year in parentheses, title, journal in italics, volume, issue in parentheses, page range. That pattern is everywhere in the training data, so it is learned thoroughly, and closely enough that the differences between an APA 7 and an MLA 9 entry survive intact inside a fabricated one. It is why a fabricated reference is rarely mangled: the shape is the easy part.

The second is content: which specific people wrote which specific title, in which volume, on which pages. There is no pattern in that. A 2025 paper by researchers at OpenAI and Georgia Tech — a preprint, not a version of record — names that second case directly, separating errors that "arise due to poor models" from errors caused by "arbitrary facts when there is no pattern in the data." Their example is a birthday: nothing about a person predicts it. A citation is the same kind of fact, wearing a far more patterned costume, which is what makes a fabricated one so convincing.

The same paper adds the part that explains why better models have not made this go away. The authors argue that models "hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty" — a system scored on whether it answered correctly gets nothing for saying "I don't know" and something for a plausible guess.

The 2023 study that measured citation fabrication describes the tool in the same terms as the split above. ChatGPT, its authors wrote, "is fundamentally not an information-processing tool, but a language-processing tool. It mimics the texts—not necessarily the substantive content—found in its information base." On why fabricated citations persist, they are openly tentative: in their experience the software "may attempt to recognize bibliographic citations and to treat them differently than regular text," so fabricated ones "may therefore represent an inability to fully recognize which specific parts of the text should be treated as bibliographic data."

That is why the check below works the way it does. A model answering from its own training, with no search step, has nothing to look a reference up in, so you have to do the lookup yourself, somewhere else.

The five-minute check, in the order that settles it fastest

Run these in order, cheapest first; each one tells you whether you need the next. A DOI rarely settles the question on its own: a DOI that fails sends you to step two's title search, and one that resolves still needs step two's record and step three's comparison.

Google Scholar sits alongside this rather than inside it. It is excellent for finding a work you half-remember, but Scholar's own help is careful about coverage: while it tries to be comprehensive, "it isn't possible to guarantee uninterrupted coverage of any particular source," and it indexes "papers, not journals." So a Scholar hit is useful corroboration and a Scholar miss proves very little. Use it to find things, not to rule them out.

What each check proves, and what it does not

The check, run end to end on two references

Both runs below were first executed on 22 August 2026 and run again on 8 October 2026, with the same results. The HTTP codes are what came back.

A reference documented as nonexistent

Taking the DOI from that nonexistent reference in the Kohls declaration, 10.1080/19331681.2022.2151234:

• doi.org returned HTTP 404 with the DOI system's standard message that the identifier cannot be found.
• Crossref returned HTTP 404 and the body Resource not found.
• A Crossref title search for the exact title returned no match — the closest results are unrelated records on adjacent topics.
• The prefix, queried on its own, returned Informa UK Limited. Real.
• The ISSN returned Journal of Information Technology & Politics, publisher Informa UK (Taylor & Francis). Real.

Read those five results carefully, because two of them are not independent evidence. Crossref is the registration agency behind the 10.1080 prefix, so the doi.org miss and the Crossref miss are largely the same fact arriving twice. What makes the finding solid is that the registry which would hold this record confirms the publisher and the journal, and has nothing at all for the article — alongside the declarant's own statement.

One honest limitation: the journal's own issue page refused every request from this machine with HTTP 403, so we never read that issue's contents and claim nothing about them. Say what you checked, not what you assume.

A reference that is real

The same chain, run on 10.1038/s41598-023-41032-5:

• doi.org answered with a redirect (HTTP 302) to the article on the publisher's site, which loaded normally (HTTP 200).
• Crossref returned HTTP 200 with a full record: title Fabrication and errors in the bibliographic citations generated by ChatGPT, journal Scientific Reports, volume 13, year 2023, authors Walters, William H. and Wilder, Esther Isabelle, publisher Springer Science and Business Media LLC, type journal-article.

Every field matched. That is the whole check, and it took under a minute. It is also the paper most of the numbers in this article come from, which means you can run the same two commands and confirm our sourcing while you learn the method.

The two failures an existence check will never catch

Walters and Wilder generated 84 short literature reviews and examined all 636 references in them. Their headline numbers are the ones that get quoted: 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated. Those figures come from texts generated in April 2023 and describe those two models, not whatever you used last week.

The numbers underneath are the ones that matter for the procedure, because they describe references that survive step one.

Two findings from the same paper should change how you read a reference at a glance. First, "most of the fabricated article, book, and website citations include the names of real journals, publishers, and organizations." Recognizing the journal is not reassurance; it is the expected condition of a fabricated reference.

Second, and more counterintuitive: hyperlinks were rare in these citations overall, but "links are actually more likely to be found within the fabricated citations" than within the real ones — and of the links attached to real works, about a third pointed somewhere wrong. The instinct that a reference carrying a link is the safer one has the evidence backwards. Follow it; do not be reassured that it is there.

A newer test, on a newer model, extends that lesson to DOIs. In June 2025, researchers writing in JMIR Mental Health had GPT-4o write six literature reviews on mental-health topics and checked all 176 citations: 35 (19.9%) were fabricated, from 6% on major depressive disorder to 28% and 29% on two less-studied disorders. Of the 33 fabricated citations that came with a DOI, the authors report that 21 carried one that was "valid but linked to irrelevant and unrelated journal articles." Those pass step one and return a record at step two; comparing that record field by field, in step three, is what catches them. Among the real citations, the most frequent error was "incorrect or invalid digital object identifiers" — so a DOI that fails does not, by itself, prove the work was never published.

Asking the model to confirm its own citation is not a check

This is not a hunch. Walters and Wilder note that ChatGPT "often provides inaccurate responses when asked to verify the legitimacy of the works it cites," citing five sources for it, and that it "often stands by its incorrect statements when asked to verify them." One guide in this space does list "ask the AI to verify" as a hedged option; for a tool that is not searching, the evidence does not support even the hedge. Spend the same thirty seconds on doi.org instead.

A tool with genuine retrieval — one that searches and returns links it actually fetched — is a different situation, but only up to a point. It can still attach a real, resolvable source to a sentence the source does not support, and that is step five's problem, not step one's.

What your school actually calls this

Where this topic's coverage raises policy at all — and most of it does not — it says something like "Many U.S. universities classify fabricated citations as academic fraud, regardless of intent," and stops there, without quoting any school's code. That is vague in a way that matters, because both codes quoted below give fabrication its own category, separate from plagiarism.

Indiana University's Code of Student Rights, Responsibilities, and Conduct lists seven categories of academic misconduct, of which Cheating, Fabrication and Plagiarism are three distinct entries. Its fabrication clause reads: "A student must not falsify, invent, or misrepresent any information or data, or manipulate materials, equipment, or processes, without authorization, in any academic exercise, including, but not limited to: records or reports, laboratory equipment and results, and citation to the sources of information." Citations are named in the definition. The University of Arizona's Code of Academic Integrity likewise prohibits "Cheating, fabrication, facilitating academic dishonesty, and plagiarism" as four separate forms, and takes its definitions from the Arizona Board of Regents' Student Code of Conduct, where fabrication is "falsification or creation of false data or information" — broader wording that does not mention citations at all.

Why the distinction is worth knowing: plagiarism is about whose words these are, and it is the category students expect. Fabrication is about whether the record you presented is true. An invented reference is a problem of the second kind, and a defense built on "I didn't copy anyone" answers a charge nobody made — which is also why the habits that keep a paper clear of plagiarism do not, on their own, keep it clear of this. Two named codes are two named codes, though — policy here is institution-specific, and the only version that governs you is your own school's. Find it before you need it, the same way you would read the consequences section before an exam rather than after.

One thing this is not is a question for an AI-writing score. That score estimates how a passage was produced; it is not a check on whether a journal article exists. Automated citation checkers are a separate product — at least one AI-detector brand now sells one, marketed to educators — and whatever one reports, the registry steps above are how you confirm it. A citation is checked bibliographically, against registries, and what comes back is a record or the absence of one, not a probability — unlike a detection score, and unlike the separate question of where proofreading help stops and authorship starts.

What to do with a reference you cannot verify

The rule is short: do not submit a reference you have not opened. If it will not verify, you have three options and only three.

1. Replace it. Find a real source that actually supports the point, using the AI output only as a search hint — a topic, a research area, a name to look up in your library's databases.
2. Cut the claim. If no real source supports it, the sentence was never supported, and the citation was doing the work of hiding that.
3. Disclose it before it is found. If a reference is already in something you turned in, email your instructor and say what happened. A student who reports it is in a different conversation from one who is asked about it.

Do the check as you build the reference list, not the night before. A few minutes per reference is easy to absorb across a week, and a bad afternoon if you leave the whole list to the end — which is the argument for keeping a running record of what you have actually read as you go, and for building the paper around real sources from the start.

None of this makes AI unusable for research. It makes it usable in one direction: good for finding out what a field talks about and which names recur, unreliable for the bibliographic details, and no substitute for reading the paper. Where that line sits for your own work is set by your assignment brief and your instructor, not by a vendor and not by us.

If the check has left you with a paper standing on sources you cannot use, rebuilding it is real work. Apex Essays writes from sources our writers have actually read, and custom essay writing starts from $7.89 a page at the simplest configuration — a floor for standard levels on longer deadlines, with rush work and higher levels costing more. If it is the reference list specifically, APA paper help is the narrower version of the same service.