Why I Stopped Asking ChatGPT for Legal Citations

I will keep this short. A general-purpose chat model is not a legal research database. Asking it for case citations and trusting the answer is how lawyers end up sanctioned. The published opinions on this from courts across the country are clear. The Bar opinions are clear. The risk profile is clear.

What is less obvious is what to do instead, because the underlying impulse is correct: AI can accelerate legal research. The question is which tool, doing which job, with which verification step.

I am a Florida trial lawyer pivoting into AI applied to legal work. I run multiple AI agents across my practice. I have hundreds of hours of hands-on experience with what these systems do well and where they fail. The citation problem is one of the cleanest places to draw the line.

What “hallucinated citations” actually means

A general-purpose model trained on a broad mix of internet text learned what legal citations look like. The format. The reporters. The case-name conventions. What it did not learn, in any reliable way, is which citations point to real cases.

When you ask the model for “a Florida case holding X,” the model will give you something formatted like a Florida case holding X. The volume number, page, year, and court will all look correct. The case may or may not exist. The holding may or may not be accurate. The model is not lying. It is completing a pattern. The pattern includes “looks like a real cite.”

The lawyer who copies that into a brief without pulling the case in the reporter has filed a fake citation. Judges have been finding these for two years now. Sanctions are being issued. State bars are opening discipline files. The professional consequences are not theoretical.

What changed in my workflow

I used to test general models on legal questions to see how close they could get. The answer was sometimes very close, sometimes very far, and always close enough on surface to fool a reader who did not pull the case.

I no longer ask general chat models for case citations at all. The cost of a wrong citation in a court filing is high enough that the time saved by even a correct citation does not justify the verification burden across a hundred queries.

What I do instead is straightforward. Legal research that has to land in a brief comes from a real legal research database. Westlaw. Lexis. Fastcase. The case is pulled in full. The holding is read in full. The pin cite is checked against the actual page.

AI accelerates a different layer of the work. It helps me think about what arguments to make, summarize a long opinion I have already pulled, organize the cases I have already verified into a draft outline. The citations themselves come from databases that are responsible for the citations being real.

The duty of competence

The professional rules have not changed. The lawyer is responsible for the work product. The lawyer signs the brief. The lawyer files the motion. Whatever tool the lawyer used, the duty of supervision and verification stays with the lawyer.

State bars across the country have issued guidance over the last two years saying the same thing in slightly different words. Use the tools, but verify the output. Disclose where required. Maintain confidentiality. The duty is unchanged. The instruments are new.

A lawyer who relies on a general chat model for citations and ends up filing fake cases has not just made a research mistake. The lawyer has fallen below the duty of competence in a way the rules clearly contemplate. That is a hard fact, not a future risk.

What works in practice

Three things have worked in my own workflow.

One. Use legal research databases for citations, full stop. The databases are not as fast as a chat model. They are not as conversational. They are reliable on the one thing the chat model is not reliable on: the case actually exists, the holding is what they say it is, the pin cite points to the right page.

Two. Use AI for the layers around research. Summarize an opinion you have already pulled. Outline the structure of a brief from cases you have already verified. Draft factual sections you can check against the record. Translate a section of law from a foreign-law treatise. Useful work that does not depend on the AI inventing cites.

Three. Read every line that goes in a filing. Cite-check every cite. Spot-check every fact. The lawyer's name is on the document. Treat the AI output the same way you would treat draft text from a junior associate you do not entirely trust yet.

The honest summary

AI is genuinely useful for legal work. The mistake is asking a general-purpose chat model to do a job a legal research database is built for. Once you draw the line in the right place, the rest of the workflow gets cleaner.

I am still bullish on AI in law. I am also bullish on lawyers reading their own briefs and pulling their own cases. Those are not in tension.

Frequently Asked Questions

What is a “hallucinated” case citation?

A citation that looks formatted correctly but does not point to a real case, or points to a real case that does not say what the citation claims. General-purpose AI models can produce these in legal research output. They can be hard to spot without pulling the case in a real database.

Can specialized legal AI tools be trusted for citations?

Some of the legal-specific platforms, including ones connected to legal research databases, have stronger guardrails because the citations come from indexed sources. The verification burden is lower than on a general model, but the lawyer still has to pull the case before filing. The duty of supervision does not transfer to the tool.

What about using AI to check citations after I find them?

That is closer to a defensible workflow. Pull the case in a real database, then ask the AI to summarize or check consistency. The verification still requires reading the case. AI as a second look, not as the source.

Have lawyers actually been sanctioned for AI-generated fake citations?

Yes. There are published opinions from federal and state courts across the country in 2024 and 2025 documenting sanctions and discipline referrals. The factual record on this is established.

Should I disclose AI use to the court?

The rules vary by jurisdiction and by judge. Some courts have standing orders requiring disclosure of generative AI use in filings. Some do not. Check the local rules and the judge's standing orders before you file. When in doubt, disclose.