zlacker

Simulating History with ChatGPT

submitted by arbesm+(OP) on 2023-09-12 12:34:40 | 175 points 79 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
◧◩
9. benbre+Rt[view] [source] [discussion] 2023-09-12 14:38:30
>>ilaksh+ks
Thank you! Re: gamification, I guess I finally found a use for all the time I spent playing Gemstone III when I was 12. It seems that MUDs or writing about them were well represented in the training data because preventing it from veering off into D&D like fantasy was actually the hardest part (hence the talking rat incident).

Something I wrote about Gemstone years ago:

http://theappendix.net/issues/2014/10/dont-cry-for-me-elanth...

30. ianbic+wD[view] [source] 2023-09-12 15:18:34
>>arbesm+(OP)
This is great... I ended up making a project in this same vein; it's not online but I made a demo video of recent progress [1], and some posts [3...]

I didn't actually start it as a historical game, but just thinking about what it would be like to roleplay an entire life as a series of scenes (like [2]). But while you can roleplay a vaguely "now" moment by not specifying any date, if 60 years passes you have to acknowledge that both the character and the world around the character are changing. And then you have to define a start date, make the roleplaying system aware of the historical context... and why not let the start date be 2000BC, 1700, or 1960? So it quickly became historical.

There's a ton of challenges. General hallucination is one, of course, but ahistorical biases probably bother me more. The author mentions a talking rat appearing in one; I had a simulation where a building was listed as a "character" and so it started interacting with the player [10]. But those are obvious enough that I kind of enjoy the absurdity.

Ahistorical biases really comes out in female characters, where it can be hard to get GPT to fully acknowledge historical gender roles. I think it's super-OK for the player to break those gender norms, but "society" should respond accordingly. For instance playing a young woman from a politically motivated family in the ~200BC Rome, while there's lots of possibilities, become a senator is not one of them... but GPT thought it was.

Also GPT has a high bias towards being friendly and accepting, like in the post with Ea-nāṣir: "He meets your gaze, his demeanor shifting from initial resistance to acknowledgement...." – both the response and the tone of the response are very familiar GPTism. I have a feeling Ea-nāṣir wasn't actually such a conciliatory dude.

Anyway, a challenging scope but it's a lot of fun, especially since GPT knows about all kinds of history that I don't and I get to learn just while playtesting.

[1] https://youtu.be/6ZaTvSucoBA

[2] https://en.wikipedia.org/wiki/Alter_Ego_(1986_video_game)

[3] https://hachyderm.io/@ianbicking/110945869211907827

[4] https://hachyderm.io/@ianbicking/111021491162338640

[5] https://hachyderm.io/@ianbicking/110890095506958475

[6] https://hachyderm.io/@ianbicking/110805239146369379

[7] https://hachyderm.io/@ianbicking/110658807975175746

[8] https://hachyderm.io/@ianbicking/110642068987102144

[9] https://hachyderm.io/@ianbicking/110619939448651143

[10] https://hachyderm.io/@ianbicking/110894806242767625

◧◩◪
39. thepti+JL[view] [source] [discussion] 2023-09-12 15:51:54
>>SamPat+fE
Please share it here or do a Show HN when you are ready, would love to see this.

I commented in this direction elsewhere (>>37482853 ) but interested in your implementation -- do you have the LLM running the whole world-model, or do you have it using function calling to drive a text adventure game engine (which would give stricter guarantees around persistence of the world).

◧◩
42. benbre+2X[view] [source] [discussion] 2023-09-12 16:33:43
>>thepti+HE
"I think it can be taken further; rather than "spot the errors that ChatGPT made", you could flip the script and make it "survive 20 turns of conversation without making a historical error", so you'd need to know things like local traditions, perhaps the geography of the ancient settlement you're studying, contemporaneous history like "who is the emperor and what's the sentiment towards him" and so on."

Yes, exactly. This is where I've been heading with my planning for assignments. For instance, when confronting Ea-nāṣir about his poor quality copper, I'd want my students to actually show some knowledge of the geography and political dynamics of ancient Mesopotamia.

The "Fall of the Ming Dynasty" simulator I link to at the bottom of post is probably the most well developed example of this that I've come up with so far. In that one, I added a "political intrigue minigame" in which ChatGPT is supposed to assess the human player's ability to deploy rhetoric appropriate for a minor courtier in 1640s China (from the prompt: "success depends on your luck score + rhetorical skill, tested via a series of open-ended prompts that HistoryLens will assess and grade; only the highest scoring responses will allow you to succeed in the minigame.")

Here is the full prompt for that one if people want to try it: https://chat.openai.com/share/86815f4e-674c-4410-893c-4ae3f1...

◧◩◪
44. gsuuon+c31[view] [source] [discussion] 2023-09-12 16:57:20
>>thepti+wH
Can you point me to some text-adventure engines? I'm hacking on an in-browser local llm structured inference library[1] and am trying to put together a text game demo[2] for it. It didn't even occur to me that text-adventure game engines exist, I was apparently re-inventing the wheel.

[1] https://github.com/gsuuon/ad-llama

[2] https://ad-llama.vercel.app/murder/

◧◩◪◨
46. thepti+K51[view] [source] [discussion] 2023-09-12 17:08:34
>>gsuuon+c31
Sorry, not my area of expertise, I just know they exist, but I don’t know how the different ones compare.

There are a few: https://en.m.wikipedia.org/wiki/Category:Text_adventure_game...

And z-machine is the one I have seen for the one text adventure I know of: https://en.m.wikipedia.org/wiki/Z-machine, but I would be surprised if that’s the best one for a new project as it’s quite old.

48. koochi+p71[view] [source] 2023-09-12 17:17:05
>>arbesm+(OP)
Check out inquistory an AI platform designed for history learning. https://inquistory.com/

It combines GPT-4 with sources, and relevant images. It also has the ability to chat with historical figures.

◧◩
60. DonHop+JE1[view] [source] [discussion] 2023-09-12 19:23:44
>>thepti+HE
Great idea!

Why leave hallucinations to chance? ;) The prompt could tell ChatGPT to randomly insert several authoritative sounding but verifiably false facts, to give the students debunking challenges! That solves the problem of GPT-5 being too smart to hallucinate, while still leaving open the possibility of talking rats.

What you're envisioning reminds me of Timothy Leary's Mind Mirror, published by Electronic Arts in 1985 for the Apple ][ and other home computers:

>>32578683

https://scalar.usc.edu/works/timothy-leary-software/index

https://www.rockpapershotgun.com/diy-transcendence-with-timo...

>Players answer questions that, when churned by Mind Mirror’s cryptic algorithms, can allegedly help them reveal intriguing new aspects of their psyche. Gameplay predominantly revolves around defining, comparing and then role-playing through different personalities in various text-based life simulations.

https://www.myabandonware.com/game/timothy-leary-s-mind-mirr...

https://store.steampowered.com/app/1603300/Timothy_Learys_Mi...

I extracted all the text from the Apple ][ Mind Mirror floppy disk image:

https://donhopkins.com/home/mind-mirror.txt

  Hello, I'm Timothy Leary.
  Welcome to MIND MIRROR.

  MIND MIRROR (c) copyright 1985, 1986, Futique, Inc.
  Published by Electronic Arts

  MIND MIRROR
    Design and script by Timothy Leary.

  MIND MIRROR
    Program and Design by Peter Van den Beemt and Bob Dietz.

  MIND MIRROR reflects and qualifies your thoughts.

  OPTION 1
    MIND TOOLS
    Enhance Insight, Mental Fitness, Learning Skills and Performance.

  OPTION 2
    MIND PLAY
    SIGNIFICANT PURSUITS.
    Sophisticated Head Games.

  MODE 1 
    MIND MIRROR
    Learn how to Micro-Scope and Map your thoughts.

  MODE 2
    LIFE SIMULATION
    Test your empathy in amusing Role-Play Odysseys.

  SELECT LEVEL
    Beginner
    Intermediate
    Master
    Consultant

  Choose AUTO-PLAY
  or INTER-PLAY.

  Mirror your own thoughts. 
  Compare them with others.

  RETURN begins game.
  SPACE BAR clears text.

  [...]

  "Mirrors should reflect a little before throwing back images." -Jean Cocteau
Also, here are the scales represented as JSON:

https://donhopkins.com/home/mind-mirror.json

Just for laughs, here's ChatGPT's summary of that file, and its answers to questions about Timothy Leary -- I sure hope it's not hallucinating:

https://chat.openai.com/share/044c41a3-fbc5-49cd-a3d1-c42f07...

What's interesting is that game was based on Timothy Leary’s PhD dissertation “The Social Dimensions of Personality: Group Process and Structure”, which he ultimately used to break out of jail.

https://archive.org/details/leary/leary.300dpi/mode/2up

Before he got into LSD, he designed the Leary Interpersonal Behavior Circle personality assessment, which laid the foundations for understanding human personality and interpersonal behaviors.

https://en.wikipedia.org/wiki/Interpersonal_circumplex

http://paei.wikidot.com/leary-timothy-interpersonal-circle-m...

In the 1970s, Leary was arrested for possession of marijuana. As part of the intake process, he was given a psychological assessment designed to gauge the risk of escape or violent behaviors in inmates. This test was known as the "Group Psychological Assessment Test." Leary was familiar with the test – having designed it or at least aspects of it. Understanding the criteria being measured, Leary answered in such a way that he was categorized as someone who posed a very low risk of escape or violence.

As a result, he was assigned to a minimum-security prison. With the lower level of security and his connections, Leary managed to escape prison in September 1970. His escape involved various affiliations, including with the Weather Underground, a radical left-wing organization. After his escape, Leary fled the country and spent time in various locations, including Algeria and Switzerland, before eventually being recaptured in 1973.

◧◩◪
62. ianbic+qG1[view] [source] [discussion] 2023-09-12 19:28:35
>>benbre+5j1
I have had some success in suppressing crazy actions by asking some questions up front, especially "is this socially acceptable?" https://hachyderm.io/@ianbicking/110170158329883997

Depending on how the prompt is phrased it can result in a response like "it would be inappropriate to throw an octopus at this conference," have the character actually attempt it but usually be foiled during the attempt like "as you take the octopus from your suitcase a security guard stops you with a growl, 'you better not try that kid'," or have it happen but immediately get a stiff response like being kicked out. (It's much harder to get the response "there is no octopus in your suitcase"!)

◧◩◪
71. rastap+3I2[view] [source] [discussion] 2023-09-13 00:26:18
>>benbre+2X
What are your thoughts on this update? https://chat.openai.com/share/6178453a-4a8a-430a-ac13-156b54...

Basically, re-iterate the original instructions each time, describe last 2 moves in details, and provide brief summary of all the previous moves. Can have much longer games this way - maybe this deserves to be a python script.

◧◩◪◨⬒
73. cpeter+oM2[view] [source] [discussion] 2023-09-13 01:01:43
>>thepti+K51
Inform is a popular modern programming language for “interactive fiction” (text adventures). The language and its data structures might give you some inspiration for how to model interactive fiction for an LLM.

https://en.m.wikipedia.org/wiki/Inform

◧◩◪◨⬒
76. thepti+8a3[view] [source] [discussion] 2023-09-13 04:48:24
>>benbre+YY2
Perhaps a no-code UI might let you wire up the conversational memory being suggested here.

I haven’t used these but saw a post on them:

https://cobusgreyling.medium.com/flowise-for-langchain-b7c40...

https://cobusgreyling.medium.com/langflow-46563a8af323

[go to top]