You mentioned using Deep Research. I believe that mode on ChatGPT provides "fully documented, with clear citations to sources, making it easy to verify and reference the information". Would you mid sharing some of this documentation for some of the more dubious figures? For example, I'm not able to track down any references to 'Anowen' the Cornish herbalist from Roseland.

@Anaxandron

Hey man, thanks for asking here you go:

https://chatgpt.com/share/6802ac0e-114c-8000-86e7-c358acd1331e

My main criteria to no include someone was about the ammount of figures they sometimes are folk myths or fiction characters.
That can heppens, even with 15 sources like this one.

I am sure 98% of critisism this mod is having is from people who have no idea what is ChatGPT on Deep Search.

But now, I hope this answer you, and you let me know if we should keep or delete Anowen. :)

Best

I'm sorry, but you cannot substitute actual research for what chatGPT claims to be accurate. All three individuals shown in the preview images for the mod are entirely invented, without any textual source for their existence. I understand the difficulty in finding surviving sources from the early medieval period, but allowing ahistorical and fictional individuals in a mod that is specifically called "Historical" is a disservice to historiography and to the history of the period.

You claim to utilize the Deep Research function for your actual job, and I believe you. You say that you instruct the program not to invent characters or provide names incongruent with the culture or region, and I believe you. However. "10 minutes" for "a list of 20 names for a specific heritage and culture" is not research. The mod description states that it has added 640 characters to the game; of the 3 shown in the preview, none are historic. The preview, at least, should show at least one figure with accurate textual basis.

In the example of "Elen ferch Cynyr", she is not actually mentioned in "The Life of St.Cadog." The only Helena/Elen mentioned is the mother of Constantine the Great, who lived in the 5th Century, and would eventually become Saint Elen. The other two individuals, "Naka Jatra" and "Rado Bernhardus," lack any textual evidence at all.

I urge you to reconsider utilizing generative AI to such an extent. Generative AI, ultimately, relies on predictive language to produce an output, any output, that will meet your request as best it can. When it comes to doing historiography, it will never surpass fundamental research methods and human effort. You are trying to bolster the ranks of historic characters in the game, but without investing the proper time and methodology, you will inevitably have more fabrications than actual historical representations. I'm sorry, but doing proper history is hard work, and you may need to resort to more rigorous methods than you are used to. I would strongly encourage you to seek out experts and communities that are versed in the regions and periods you are examining, so that you can receive helpful insight and be directed to greater sources than you would otherwise find.

I am posting in this discussion from a position of defending historical integrity, and from a deep love of the period and of highlighting the lives of lesser-known individuals. If you interpret this critique as "aggressive," then I am unsure what else I can do to assist you. If this is truly a labor of love, and you share the same passion for history that I and many others do, you may be able to find more reliable means of getting accurate research for your mod.

Spikewerks, this is a much better reply—definitely appreciated over a “read a book” comment, especially since you don’t know me, and I’m actually an avid reader. That said, no hard feelings—I just think there’s no need to get personal.

Now, let’s focus on what really matters here: making this mod as historically accurate as possible. I’d love to work together on that. But to do so, we need to establish a few ground rules:

My research method is based on elimination using Deep Search, rather than a curated selection. In practice, that means I pull a large number of potential names, and I rely on feedback to help determine which ones are historically supported and which are not. I will immediately remove any character proven to be non-historical.

I hope it’s clear that I have a surplus of characters—and I have absolutely no problem removing the ones that aren’t real. So if you notice something that doesn’t check out, tell me! I want your help.

What I don’t appreciate is being judged as if I’m trying to mislead people by labeling something “historical.” Historians themselves often debate whether certain individuals truly existed, especially for the years 850–1050. My goal for this game mod was to lean toward realism while still making it engaging, which is why I used Claude to generate a backstory based on what little is known.

For example, in the mod you’ll find a blacksmith. From historical sources, we might only know his name, his region, and that he was a talented blacksmith. That’s it. So sure—I could’ve just written “Name Lastname, a skilled blacksmith from X region.” But we’re making a game, not a textbook. So I asked an AI to build a richer, more immersive background using the known facts as a base. I love reading those results—and I think many players will too. Have you checked them out?

Just to clarify, this mostly affects the earlier time range (850–1050). For later periods, when sources become more reliable, the number of AI-filled backgrounds will drop significantly.

At last, NO character of this mod is a halucination or my creation, they are came from Deep Research with sources, as I just presented here on the other post. What I had to filter was so obviously fictionals.

So… will you help me out?

Best!

Spikewerks is entirely correct, though. The thing is, LLMs don't *like* not being able to give you an answer. When there isn't one to give, they often make one up that sounds plausible, and can even give fictitious sources--but looking into those primary sources shows they don't actually back up the claim the LLM is making. So when you say "NO character of this mod is a hallucination" that's factually wrong. Most or all of them are hallucinations, I'm sorry to tell you--because your sources are not real. You *cannot* use LLMs for serious research, because they will present lies to you as confidently as facts.

It's true that there are not many actual sources for historical figures in these times. That's why the LLM is having such hard times finding them, and thus is inventing those sources because that's what it thinks you want. Deep Research is not Google Scholar. If you want to use AI to create historically *plausible* people, go ahead, but be upfront about the fact they're fictional. Presenting them as historical, when they are verifiably not, is something I cannot condone.

I know you truly thought these sources were legitimate, and I'm not blaming you for it--it's a common misconception. But that's not how LLMs work. They predict the most likely string of words; they're not actually doing the research, no matter how it was marketed to you or your company. Any sources they provide you have to *check* and see if it actually says what it claims it says. Because I guarantee you every single "historical" person in your images has *zero* sources I could find outside of Deep Reasearch... so what research is it pulling from? Nothing. Unfortunately.

@emmaf628

So, why 80% of the time I get no results from my promps?

This is the prompt I use normally, example:

List up to 10, Celebrity or Notable Non-Noble, including poets, theologians, scholars, warriors, soothsayers, explorers, bankers, merchants, artists, physicians, knights, strategists, merchants, heroes, diplomats and authors from IBERIAN heritage and ARAGONESE culture on years 850-950 CE, make sure to do not list anyone that was not alive on given period, that was not listed before on this conversation, and they are clearly from the given culture. Do not use language special characters to write names and always use full names for names. Reflect again when the list is done if there is any person could be fictional or folklore, and remove them.

Thank's for replying @emmaf628, but I truly believe hallucination is not the case we are talking here about if the person exist or not.

What you be getting confused is the backstory, this I am giving a "creative" permission to Claude AI, to fill some gaps, so we don't have a character just as "He was a well know bakery chef." only.

But, my main question: why I get results on 10%-20% of the time?
You say "... LLMs don't *like* not being able to give you an answer..." I understand this, I don't think is the case here.

Best

With all due respect, you're wrong. The prompt you're using is causing the LLM to hallucinate these "historical" figures. Why else do you think you can't find actual data on these people? You get results 10-20% of the time because if there *is* something, it *might* be able to find it. But if there isn't, it'll make something plausible up, because all it's doing is putting the most likely words together.

Let's say I asked it to come up with a list of all 45 current F1 drivers on the 2025 grid, their ages and nationalities, and not to make any up. There are only 20 current F1 drivers, for the record. Try this for yourself--I don't have access to this LLM. But I'm guessing it'll give you a full list of 45. The first 20 will probably be right, but it *can't* be right after that, and doesn't know enough to tell you that because *you're* confident there's 45. Maybe mix and match some real drivers names (eg Lewis Hamilton and Max Verstappen makes it pull out Max Hamilton as no. 21), maybe do some past drivers that aren't on the grid anymore, and so on.

This is what's happening here. You ask it for "up to ten" (which to the AI generally means just ten--at the very least, more than a few) figures from whatever culture, and when the LLM's search finds only two, it feels like it needs to give you more than that. So it gives you the two real ones, then makes up five more. That's how they work. And why so many people (not just you) are led astray by these things.

(And for the record, there's a chance it might pass the Formula 1 test--the grid being 20 drivers is something that's possible for it to know. It *wouldn't* be able to say that there were fewer than 10 people in that place and time, so it's more likely to make something up there. I was mostly using that as an example of the sorts of things it does, when there is a finite number to check on.)

@emmaf628, this feels like one of those classic "agree to disagree" moments—and that's totally fine.
Our experiences, perspectives, and understanding of LLMs differ, and in such a rapidly evolving space like AI, I don’t believe either of us can claim absolute truth.

That said, I genuinely appreciate your input and the time you took to share your thoughts.

Best regards!

ChatGPT's Deep Search obviously doesn't work that well if it generates completely non-existant characters. Get yourself together, man.