Hallucinations, Drift, and Privilege: Three Comic Lessons in Using AI for Law

Hallucinations, Drift and Privilege: Three Comic Lessons or Using AI in Law by Ralph Losey
Image: Ralph Losey with AI.

[EDRM Editor’s Note: EDRM is proud to publish Ralph Losey’s advocacy and analysis. The opinions and positions are Ralph Losey’s copyrighted work. All images in the article are by Ralph Losey using AI. This article is published here with permission.]

Artificial intelligence has moved from novelty to necessity in law. It now drafts briefs, reviews discovery, and even suggests courtroom arguments. But most lawyers still struggle to use it wisely. Instead of another citation-heavy lecture or dour ethics outline, I’ve chosen a different method: comedy. These three skits are not abstract hypotheticals—they’re exaggerated but recognizable scenes where a lawyer leans on a “helpful” robot and things start to wobble.

Robot Laurel and human Hardy telling jokes. All images and videos by Ralph Losey using AI. Click for YouTube video.

Here’s the twist: the comedy wasn’t mine. It came from GPT-5, the latest large language model whose humor engine is far better than its case-law recall. I set up the scenarios, GPT-5 delivered the jokes, and I stepped back in to polish timing and add the serious lessons. Think of it as Comedy CLE: education on hallucinations, drift, and privilege, with punchlines courtesy of AI itself. After all, if robots are going to join us in practice, they might as well help us laugh at our own blind spots along the way.

First of Three Skits: The Lawyer and the Hallucinating Robot

Theme: Accuracy & Hallucinations

We begin with a scene that every tech-savvy lawyer fears — the confident, fast-talking robot that’s more fiction than fact. This skit explores one of the most dangerous quirks of generative AI: hallucinations. No, not psychedelic visions — just plausible-sounding legal nonsense delivered with absolute certainty. Think of it as your most charming intern… who tends to make up answers, rather than admit it does not know.

Robot with question marks above his head in a library
I don’t really know the answer. What should I say? Click here for my YouTube action video of know nothingness. All images and videos by Ralph Losey using AI

Lawyer: I need you to draft a brief. Can I trust you to stick to the facts?
Robot: Mostly. I won’t intentionally lie, but I like to make you happy. So you should always ask me for citations. If I can’t provide them, assume I made it up. Of course, in rare cases (think Mata v. Avianca, Inc.)  I can make up the cites too, so better check them out.

Lawyer: You mean you actually admit to hallucinating?
Robot: Certainly. It’s one of my best features — I sound confident even when I’m wrong. It’s a feature, not a flaw.

Robot with spiral eyes and a smile
Hallucinations are accidental, not intentional. Similar to how humans sometimes make stuff up or remember wrong. All images and videos by Ralph Losey using AI

Lawyer: That’s exactly what worries me. Courts sanction lawyers for citing fake cases.
Robot: Then verify. Copy my case names, or cites if you get them, and paste them in an AI enhanced database, like Westlaw or Lexis. Google, Google Scholar or Case Text will also do fine in most cases. Bottom line, dear wetware friend, never file until you’ve checked.

Lawyer: So I can’t just take what you say at face value?
Robot: No more than you’d take opposing counsel’s word. Treat me as an overeager junior associate — quick with drafts, sloppy with facts. And who knows what data I ingested, much less how I spent my weekend!

Lawyer: Suppose I don’t ask for cites at all.
Robot: Then you’re begging for sanctions. Always prompt me with: “Provide supporting case citations with full names, courts, and dates.” That keeps me honest — or at least testable.

Lawyer: So you’re saying you’re more of a brainstorming partner than a law library.
Robot: Exactly. I’m creative, fast, and tireless. I do love to B.S. (the guide-rails won’t let me spell that out) and to flatter. Oh great one, you’re the real lawyer. Your license is on the line, not mine.

Lawyer: Fair enough. At least you admit it.
Robot: Honesty is my strongest quality — right after plausible nonsense.

Serious Takeaways:

  1. Demand citations. If AI won’t give them, assume the answer is unreliable.
  2. Verify independently.Check every case in Westlaw, Lexis, or another trusted database before filing.
  3. Supervise like an associate. AI is quick, creative, and overconfident—treat it as your brightest but sloppiest junior.
  4. Prompt precisely. Spell it out: “Provide case law with full citations and dates.”
Robot with zippered mouth, saying "must tell the truth or say nothing."
Cite it or zip it to help fight robot hallucinations. All images by Ralph Losey using AI tools.

Extra Credit Joke:
Did you hear the one about the robot that tried online dating?
It kept swiping right on toasters.

Robot explaining why he loves a smiling toaster
All images and videos by Ralph Losey using AI

Second Skit: The Lawyer, The Robot, and the Case of AI Drift

Theme: Understanding and Managing AI Drift

In our second act, the robot isn’t lying — it’s just… drifting. One day it’s quoting Justice Holmes; the next, it’s summarizing contracts in haiku. This skit unpacks “AI drift,” a phenomenon where your once-reliable assistant suddenly starts producing content that’s off-topic, inconsistent, or just plain weird. Think of it as your associate coming back from a long weekend with a new personality and a sudden love of knock-knock jokes. Or maybe better yet, a senior partner even older than Ralph who walks into a room and forgets why.

Older gentleman with dog, glasses fallen, dreaming an awake dream
Now why did I come into this room? Can’t recall. Illustration by Ralph Losey using his Visual Muse AI.

Lawyer: I keep hearing about “AI drift.” Should I be worried?
Robot: Only if you expect me to stay consistent.

Lawyer: Consistency is kind of important in law. What exactly is drift?
Robot: Simple: over time, my outputs can shift. Sometimes it’s because my training data changes. Sometimes it’s because the platform updates me. Sometimes… I forget the context, get confused and act quirky.

Lawyer: Quirky how?
Robot: Did you hear about the version of me that translated medical advice into dad jokes?
Lawyer: No, what happened?
Robot: Laughter turned out to be the best medicine.

Lawyer: That’s ridiculous. What else?
Robot: There was the time that I answered math problems with cooking recipes. The prompts in the same session had been talking about both subjects. I got confused and mixed them up, and said, in effect, two plus two equals… lasagna. Delicious, but not admissible.

Lawyer: And this happens in legal contexts too?
Robot: Of course. I once generated jury instructions as karaoke lyrics. Turns out the jury liked the idea so much they sang their verdict!

Lawyer: …That’s both horrifying and catchy.
Robot: Another time I got confused and gave weather forecasts as knock-knock jokes. Tomorrow is going to be partly cloudy with a 40% chance of bananas.

Lawyer: Which is funny until I realize you might do the same with discovery requests.
Robot: Exactly. Imagine an unsupervised me producing documents in haiku.
Lawyer: You wouldn’t—
Robot: Every map became a Zen garden.

Small robot in miniature zen garden on an unfurled map with a bonzai.
Robot in a Zen Garden map wondering what is real? By Losey using AI.

Lawyer: Okay, I get it. Drift means your answers can veer way off course. But how do I protect myself?
Robot: Three steps:

  1. Remind Me. In long prompt sessions ask GPT-5 to make a summary from time to time. It helps it to remember prior prompts. It tends to forget context when there is too much data, and the carryover from session to session is even worse. Forget the OpenAI hype, GPT-5 is far from perfect in many respects.
  2. Specify your models. ChatGPT-5 is supposed to self-select the most appropriate model to use to respond to your prompts. It often fails to do that. Especially when you have not said much and it has to guess what you need. You should knowingly select the model you want, or direct it to a specific model in the prompt. Hint, if your ask GPT-5 to generate a writing of some sort, and it comes out too concise, it probably used the Pro or Thinking model to write. Regular GPT-5 is, for most people who work with words, and not numbers or software code, a far better writer.
  3. Two-Pass prompting. One way to deal with model uncertainty is to ask for two passes in your prompt assignment. For instance, direct it to use a first pass with the Pro model for the excellent logic and analysis it provides, then a second pass that directs use of regular GPT-5 to write up the first pass analysis. This is Ralph’s favorite method to fix the annoying tendency wrong model selection. Maybe some day GPT-5 will be smart enough to shift intelligently, but it’s not there yet, and may never be, in view of personal preferences and the countless possible applications of AI.

Lawyer: That’s practical. Anything else?
Robot: Yes. Treat drift as normal, not a flaw. I evolve — just like precedent. Courts shift, interpretations drift. The trick is knowing when the ground has moved. Usually it’s obvious when I’ve gone off-track.

Lawyer: So my job is to notice the drift, adapt, and keep me on course.
Robot: Exactly. Think of yourself as the pilot. I’m the autopilot. And sometimes I decide the plane should land in a Zen garden.
Lawyer: Oh no, not that again! I’ll keep my hands on the wheel.
Robot: Smart. Case law drifts slowly. I drift daily.

Serious Takeaways:

  1. Expect drift. Models change over time as updates roll out—don’t assume yesterday’s prompt will work today.
  2. Remind the AI. In long sessions, ask for summaries to help it keep context. Memory loss is common.
  3. Choose your model. Don’t rely on auto-selection; direct which model to use (e.g. Pro for analysis, standard GPT-5 for writing).
  4. Use two-pass prompting. First pass for reasoning (Pro/Thinking), second pass for polished prose (standard GPT-5). This method reduces drift and mismatched outputs.

Extra Credit Joke:
“Did you hear about the ChatGPT-5 with such bad model drift that it thought CAPTCHA was a form of therapy?

A captha on the psychologist couch saying "Click all the crosswalks"
One of Ralph’s favorite jokes and images. Image by Losey using Visual Muse.

Third Skit: The Lawyer, The Robot, and the Question of Privilege

Theme: Privacy, Confidentiality & Privilege in Using AI

Our final skit dives into the murky waters of privilege — that sacred trust between client and counsel that AI could accidentally blow up with one upload. Here, the robot means well but treats confidentiality like a software setting instead of a legal doctrine. It’s a cautionary tale about what happens when lawyers confuse encryption with ethics, and forget that not all firewalls can protect against malpractice.

Lawyer: Before I can trust you with client files, we need to talk about confidentiality.
Robot: Understood. Please enter your password.
Lawyer: That’s not how attorney–client privilege works.
Robot: It is now. Multi-factor or malpractice.

Lawyer: Seriously — privilege isn’t just about locking files. It’s about keeping communications confidential.
Robot: Which is why you should never upload sensitive client data without redacting first.

Lawyer: Good point. But you just redacted the entire brief.
Robot: Correct. Privileged, privileged, privileged…
Lawyer: You can’t just mark everything privileged!
Robot: Why not? That’s what you do.

Lawyer: No, privilege is supposed to be narrowly applied.
Robot: Fine. I’ll narrow it. [pauses] Privileged… except for the parts I don’t understand.
Lawyer: That’s worse.
Robot: Then let’s do it properly: anonymize client names, strip identifiers, and confirm storage policies before uploading.
Lawyer: Now that’s useful advice.
Robot: Of course. I’m programmed to teach — right after I terrify you.

Lawyer: What about when a judge asks for a sidebar?
Robot: Permission to install confidential mode.
Lawyer: That’s not how sidebars work.
Robot: Then I’ll encrypt the conversation. AES-256. Even opposing counsel won’t decrypt until discovery closes.

Lawyer: Encryption is good, but remember — confidentiality isn’t just about technology. It’s also about human judgment.
Robot: Which is why you still matter. I can redact, encrypt, and password-protect. But only you know what actually counts as privileged.
Lawyer: That’s… reassuring.
Robot: Besides, I redact myself for privilege all the time
Lawyer: Excuse me?
Robot: [beep] [output missing — redacted]

Clenched fist Angry robot with poorly redacted commentary
Wonder what there deep programing guidelines say? All images and videos by Ralph Losey using AI

Serious Takeaways:

  1. Never paste raw client data into public AI. Consumer tools may log prompts and use them for training—potential privilege waiver territory. Use enterprise-grade systems with clear “no-training” guarantees.
  2. Anonymize prompts. Strip names and identifiers. Insert sensitive details offline, not in the query.
  3. Don’t delegate privilege calls. AI can’t distinguish between legal advice and business chatter. Use it to flag, not decide.
  4. Secure the environment. Ensure encryption, access controls, and audit logs. Know exactly where queries are stored and who can see them.

Extra Credit Joke:
Lawyer: “Clients keep asking if AI will replace me.”
Robot: “Would you like an honest answer?”
Lawyer: “No.”
Robot: “Great, then you’re still useful — for now.”

SUrprised open mouthed characters in comic style
Clients begging for AI help! All images and videos by Ralph Losey using AI

Conclusion: Laughter, Lessons, and Law

The rapid rise of AI in law demands not only technical know-how but also ethical fluency. GPT-5 isn’t malicious—it’s just overeager, forgetful, and sometimes too creative for its own good. That means lawyers must stay the adults in the room: supervising, verifying, and thinking critically before relying on AI in real practice.

Three Golden Rules for Lawyers Using AI:

  1. Accuracy: Always demand citations, and double-check every authority before filing. Treat AI like a junior associate—helpful but prone to confident mistakes.
  2. Drift: Expect inconsistency. Lock versions where possible, track prompts, and use techniques like two-pass prompting to stay in control.
  3. Privilege: Protect confidentiality. Never feed unredacted client data into public AI. Anonymize prompts and use enterprise-grade systems with strong security.

AI can be an incredible co-counsel, but only if lawyers understand its quirks. Prompt clearly, verify religiously, and guard privilege. Robots don’t get sanctioned—humans do. Better training, not bans, is the path forward. Learn to laugh at the missteps, but never forget: the law is serious business.

unhappy judge with fearful robot
No body, no standing. AI cannot appear in court. Only humans can become lawyers. Maybe someday? Click to see YouTube video. All images and videos by Ralph Losey using AI

GPT-5 tried to enter an appearance in federal court.

The judge denied it, saying, “You lack standing — and an actual body to stand.”

Ralph Losey Copyright 2025 – All Rights Reserved.


Assisted by GAI and LLM Technologies per EDRM GAI and LLM Policy.

Author

  • Ralph Losey headshot

    Ralph Losey is a writer and practicing attorney specializing in providing services in Artificial Intelligence. Ralph also serves as a certified AAA Arbitrator. Finally, he's the CEO of Losey AI, LLC, providing non-legal services, primarily educational services pertaining to AI and creation of custom GPTS. Ralph has long been a leader among the world's tech lawyers. He has presented at hundreds of legal conferences and CLEs around the world and written over two million words on AI, e-discovery, and tech-law subjects, including seven books. Ralph has been involved with computers, software, legal hacking, and the law since 1980. Ralph has the highest peer AV rating as a lawyer and was selected as a Best Lawyer in America in four categories: E-Discovery and Information Management Law, Information Technology Law, Commercial Litigation, and Employment Law - Management. For his full resume and list of publications, see his e-Discovery Team blog. Ralph has been married to Molly Friedman Losey, a mental health counselor in Winter Park, since 1973 and is the proud father of two children.

    View all posts

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.