pouët.net

Realtime 3D rendering engine driven by a diffusion model

category: code [glöplog]
I know that GenAI is seen as something extremely controversial in the demoscene.

But the writing has been on the wall for a long time - the technologies used for GenAI are also applicable for realtime rendering.

This is a publication that interactively renders Doom based on a diffusion model.

https://gamengen.github.io/

It's basically the same technology that is used to generate images. But instead of generating an image from noise based in a text-prompt conditioning, it predicts the next image based on an action as input.

Inherently demoable.
added on the 2024-08-28 18:54:11 by Azure Azure
Nice research, but pretty useless IMO, it's only good at learning and reproducing what's already there (here: Doom's first level), not coming up with anything new or creative like a gamemaster.
added on the 2024-08-28 19:59:12 by Kabuto Kabuto
I'm sure some GenAI might be interesting as postprocessing or something.
added on the 2024-08-28 20:14:08 by Preacher Preacher
train it on c64 demos and generate infinite scrollers
(bit pointless, but so is the c64 scene)
...as opposed to the vast utility of the rest of the demoscene?
added on the 2024-08-28 21:08:08 by sagacity sagacity
We’ve come to the point where “GenAI” sounds like “Ed Sheeran”. Or “singularity”. Or “big data”. Or “blockchain”.
added on the 2024-08-28 21:17:58 by 4gentE 4gentE
Preacher: that has already been done, example: https://youtu.be/P1IcaBn3ej0

But I guess that this is less of an active research topic as it‘s pretty difficult to auto-generate hours of both in-game footage and „should look like this instead“ footage for proper training
added on the 2024-08-28 21:21:23 by Kabuto Kabuto
Thoughts before checking the links : it's gonna be tough to get stable visuals. The interesting is that it should be able to generate all kinds of visual styles.

While interesting in itself, I feel that this would be more viable with a low details game + a "genAi" pass to add tons of details, vary the style,... But 🤷
added on the 2024-08-28 22:05:35 by p01 p01
Yep.
added on the 2024-08-28 22:55:26 by p01 p01
I was already wondering why no one had opened a thread here. :]

I found it yesterday on arXiv and I shared the link with some people on Telegram.
Extremely cool tech.

Here is a direct link to the paper.

There are some videos here.
added on the 2024-08-28 23:06:25 by ham ham
The problem with generative AI is not really with the technology itself, after all, it is ju#t technology.

Personally there are two things that I find problematic:

- that some/most of the models it was trained on rndom data found on the internet, without any care for authors right. It is industrial scale stealing someone else's work.
- and maybe more importantly, it has very unreasonable co2 emissions as it uses a crazy lot of computing power.

These things seem important to me. Maybe one or the other or both do not seem important to you. In any case, it's possible to use some of the tech without doing these things, and then, I have no problem with it, and can't wait to see what people would make with it.
size coding uses a crazy lot of computing power as well, just sayin
added on the 2024-08-29 08:53:38 by NR4 NR4
Quote:
- that some/most of the models it was trained on rndom data found on the internet, without any care for authors right. It is industrial scale stealing someone else's work.
- and maybe more importantly, it has very unreasonable co2 emissions as it uses a crazy lot of computing power.


That’s right. These two points should be enough. But there’s more:
- its use is to aid the managerial class to furtherly devalue/bypass creative class
- it uses nearly slave human labour in places like Kenya oftenly resulting in PTSDs
- it puts the artists in a situation where they are forced to furtherly enforce it and end up fighting their own “ghosts”
- what it produces is unoriginal and can be produced with orders of magnitude less resources and collateral damage to the world, it uses all these resources trying to replace human talent and artistic practice

And all of that is collateral. Nominal use is further squeezing of profit.

All this should be more than enough for a boycott.

But they don’t seem to care. All these useful idiots. They evangelically repeat false stories of “historically, this or that tech… ..photography… …blabber… …drool…” …you know the drill by now. Some of them even know it all, some are literal useful idiots. Because of vanity. Because this tech offers desperately mediocre people a glimpse of false greatness. It offers the untalented revenge over the talented. And it gives an unfair advantage to some of the talented willing to stoop that low.
added on the 2024-08-29 09:19:12 by 4gentE 4gentE
Checked the link. The output of this thing is as nonsensical as I expected.
added on the 2024-08-29 09:39:13 by dex46... dex46...
What exactly is the appeal of “…hmm, let’s see what else can be faked by glorified auto-fill in good enough quality for casual observer with fleeting focus to be fooled. Oh boy this is so conceptual it’s bound to be called art.” In facts it looks like throwing crap against the wall, seeing what sticks.
added on the 2024-08-29 09:57:22 by 4gentE 4gentE
Quote:
In facts it looks like throwing crap against the wall, seeing what sticks.

I've dabbled in many forms of making art, from making music to writing stuff to making demos. I think this is a very good summary of how the process of making art (at least for me) is.

That said, the videos linked here are terrible. But give it a few years...
added on the 2024-08-29 10:00:22 by Preacher Preacher
It makes me happy to see a post by Azure in 2024.. :)
added on the 2024-08-29 10:14:49 by arm1n arm1n
Quote:
All this should be more than enough for a boycott.

But they don’t seem to care. All these useful idiots. They evangelically repeat false stories of “historically, this or that tech… ..photography… …blabber… …drool…” …you know the drill by now. Some of them even know it all, some are literal useful idiots. Because of vanity. Because this tech offers desperately mediocre people a glimpse of false greatness. It offers the untalented revenge over the talented. And it gives an unfair advantage to some of the talented willing to stoop that low.


@4gentE: You forgot to mention a very curious use of AI that you seem to love, and that is that its mere existence allows some people to climb onto a supposed moral pedestal from which they can rise above all that bunch of stubborn and vain mediocrities and thus proclaim themselves the only ones who follow the straight path of true art.
added on the 2024-08-29 12:59:02 by ham ham
Ha, I was wondering how quickly the thread will derail to good old preaching.

Anyways, on the topic: the demo is interesting, but let's say not very surprising - this level of fidelity is nowadays expected.

It's also the most inefficient way of rendering Doom (and running game logic) *ever* :-P

Unironically, the most interesting part to me are hallucinations that authors were trying to hide so desperately (you can see in main video they cut them to minimum). I didn't see carefully if it happens, but it would be fun if you go back to the same location, but it's slightly different now or even passage/corridor is missing. Also clearly enemies appear out of nowhere - I would be surprised if their appearance is triggered by you shooting in the air (as causality directions may be confused).
added on the 2024-08-29 13:54:04 by tomkh tomkh
Non euclidean mod of Doom is infinitely more interesting than this. No “AI” required.

@ham
Fortunately I don’t know a lot of “…stubborn and vain mediocrities”. In fact I only know one, vaguely and not IRL at that, fitting the description. Blunt as a sack of hammers, yet acting all profound. I’d add “fake, charlatan, reckless and inhumane in his recklessness.

Back to topic as tomkh suggested.
added on the 2024-08-29 15:35:30 by 4gentE 4gentE
Quote:
I've dabbled in many forms of making art, from making music to writing stuff to making demos. I think this is a very good summary of how the process of making art (at least for me) is.

Of course. Me too. It’s one of the tools (throwing crap against a wall and see what sticks that is) in the arsenal. But I always saw it as an “internal” tool. Meaning after throwing at a wall I decide what should stick. Not just publish the whole wall into the wild and externalise the jurying.
added on the 2024-08-29 15:40:05 by 4gentE 4gentE
Literal crap thrown at a wall entails more art and creativity than any genAI / llm shit.
added on the 2024-08-29 15:54:22 by uncle-x uncle-x
Unlike an actual Doom port, this won't run on my decade old mp3 player, which clearly makes this utterly pointless.
added on the 2024-08-29 16:43:18 by DrClaw DrClaw
Would be fun to play a Doom mod where the visuals occasionally hallucinate and the enemies aren't where you think they are :)
added on the 2024-08-29 16:46:41 by Optimus Optimus
When talking about low details + an AI pass, I had in mind something going the Annihilation way, combining, and recombining, elements in unexpected ways to create an evwr changing world.
added on the 2024-08-29 17:18:18 by p01 p01

login