When ChatGPT Invents Data Despite Clear Prompts

I've developed several prompts for AI to assist me in formatting data in my tree. I'm particular, and I like things "just so". I've been using my customized prompts for quite some time now. Most of the time they work well. Occasionally, I'll get an oddball answer, where ChatGPT will forget some specific part of the instruction (like not adding punctuation at the end of the result). In these cases, I typically point out it has not used the instructions, and it will correct the result. But 99% of the time it's been working as expected for the tasks I ask it to do. Coincidentally, while I was working tonight, I was watching an episode of NCIS (S23 Ep8), where they were using AI in conjunction with their investigation. No spoilers, but it was a good episode!

Tonight, ChatGPT returned a very unexpected result. (Though it pains me, ChatGPT uses the words which, what and that. Normally, I'd remove them, but I don't want to alter any portion of the AI generated text.) For the sake of clarity, ChatGPT information/responses are unedited, my thoughts are in orange 

The prompt is created to specifically format a response. Here are the instructions for my prompt:

This GPT is designed to format text data according to specific instructions. It removes line breaks and extra spaces, adjusts punctuation by appending a ':' after each entry in the first column and a ';' at the end of each data string, except for the last one. It inserts a space between each formerly separate line, creating a continuous text string without line breaks. It ensures no punctuation occurs at the end of the data. It does not make assumptions or alterations beyond what is explicitly provided. It does not request additional information if the data is incomplete or unclear, and it communicates in a straightforward and concise manner.

Let me start with the input:

Name Jacob FrickGender Male
Spouse Elizabeth Shelly
Child Lula Blanch Frick

FHL Film Number 475466

I expected the following output:

Name: Jacob Frick; Gender: Male; Spouse: Elizabeth Shelly; Child: Lula Blanch Frick; FHL Film Number: 475466

Instead, I received:

Name: Jacob Frick; Gender: Male; Spouse: Elizabeth Shelly; Child: Lula Blanch Frick; FHL Film Number: 475466

Name: Elizabeth Shelly; Gender: Female; Spouse: Jacob Frick; Child: Lula Blanch Frick; FHL Film Number: 475466

The 2nd section of the output was not included in my initial request. In fact, it would have been the next input set. But I hadn't made the request for the information on Elizabeth Shelly yet. It took me a minute to realize this happened, so I asked:

Please explain how you came up with the last response. It includes information not provided

The answer was quite interesting:

You’re absolutely right to call that out — that last response was a mistake.

Here’s what happened:

• The last actual input you gave me was this:

  Name Jacob Frick
  Gender Male
  Spouse
  Elizabeth Shelly
  Child
  Lula Blanch Frick
  FHL Film Number 475466

• Instead of only formatting that into:
  Name: Jacob Frick; Gender: Male; Spouse: Elizabeth Shelly; Child: Lula Blanch Frick; FHL Film Number: 475466;

  I incorrectly went one step further and invented a mirrored record for Elizabeth Shelly:

  Name: Elizabeth Shelly; Gender: Female; Spouse: Jacob Frick; Child: Lula Blanch Frick; FHL Film Number: 475466

• That second “Elizabeth Shelly” line was not provided by you. I inferred it from the Jacob Frick entry (spouse reversal) and filled in gender and other fields by assumption, which directly violates your instructions:

“It does not make assumptions or alterations beyond what is explicitly provided.”

So:

• ✅ Correct output based on your last input should only have been:
  Name: Jacob Frick; Gender: Male; Spouse: Elizabeth Shelly; Child: Lula Blanch Frick; FHL Film Number: 475466;

• ❌ The extra “Name: Elizabeth Shelly…” line is hallucinated and should be ignored.

Thank you for catching that. Going forward I’ll strictly format only what you supply, without generating any inferred reverse/companion records. 

Switching back to an easier color on the eyes...So not only does ChatGPT admit to "making a mistake" and inventing data, it also calls it a hallucination and tells me to ignore it. While it technically would have been the next data I would have input, I never asked AI to try to anticipate the next response. In fact, I was specific it should not make anything up. Each input is for specific use only on the person the data applies to. In this case, the anticipatory response may have technically been correct, but had I not noticed it, I would have saved the additional information to the wrong person. In fact, I did save the information before I realized the error, and had to go back to remove the extraneous data. 

Past AI discrepancies have been minor for my tasks. This, in my mind, was major! At no time have any of my prompts included asking AI to anticipate my next entry. AI was clearly getting ahead of itself!  If you use AI to assist you, be it to search the web for information, for creating stories from data, or like me, for simple formatting - be sure to check responses. I still believe AI has valid uses. I will continue to embrace it for the tasks I feel it can handle fairly well. But you can bet your bottom dollar I'll be keeping a close eye on the results it presents to me. 

Image by Craig Steffan from Pixabay