So I just came across this and thought it was an interesting use of ChatGPT. Kind of curious to others thoughts on this.

My kids and I just played D&D with ChatGPT4 as the DM - medium.com

I gotta say it's tempting. My wife & I have never played D&D but it sounds really cool. We could use this to get our feet wet and decide if it's something we want to pursue more. None of our friends play and it sounds like games can go on for a while so something like this feels low-stakes to test it out.

Anyone else given it a go?

Wow, that was an interesting read. I've been playing ttrpgs of various sorts for over a decade, and while it's not something my group or I would ever try, it's impressive that the technology can remember the context of the scene, and each character's ability scores.

My honest opinion though, is that ChatGPT is just good enough to be a very boring DM. The original AI Dungeon was nonsensical, and couldn't remember the details of a scene, but that's part of what made it entertaining. It was like a surreal comedy. On the other end of the spectrum, human DMs will add creative ideas to the game, and introduce subtle elements to help with characterization. ChatGPT is competent enough to remember the scene, ruining the comedy, but not creative enough to produce something that would really hook players.

Like, with the exception of the rogue trying to steal a lizard mount, it didn't seem like there was any actual roleplaying in the session. The flow took the form of "You find yourself in a room with a door" "we open the door" "After the door, there's a monster" "we fight the monster" "after the monster there's a trap" etc. The AI even decided the characters' actions during the combat. I'm probably being too harsh. Like I said, the technology is impressive. This would have been impossible even just a year ago. But still, it feels to me like when chess ai went from beating grade schoolers to beating middle schoolers. An improvement for sure, but not good exactly.

Maybe some hardcore simulationists wouldn't mind the lack of story elements introduced by the DM, but I think most players would have more fun with playing with a human at this level of technology. If the can find one, that is.

This can certainly work, way better than shown in the article.

A lot of it boils down to the interface and settings. With something like silly tavern you have more control over the settings and prompts sent to LLM. You can also use built in D20 to reduce positive bias that all language models have. I remember using early gpt4 for a single player text based adventure game. Context size was quite small (8k), so often I had to write and insert summaries so model won't go off track too much. For me it was able to generate various quests, random encounters, handle equipment and economy and (with small help from me) simulate fights with creatures.

Would it work in 2024 ? Probably. To me early GPT4 was and still is the best LLM. It was great at pretty much everything, but it got dumbed down by excessive system prompt and other "muh safety" things