This Game Is Just Gray Blocks. AI Paints Every Frame Live.

Most games draw every frame themselves. Matrix-Game 4.0 from Skywork splits that job in two. A simple game engine runs a world made of plain gray low-poly shapes and handles the physics, the rules and your controls. An AI video model then repaints each frame of that gray world into something that looks like a finished game. The HUD and the sound stay outside the AI, lined up frame by frame.

The project ships three scenes so far: a medieval castle, ancient Egypt and city racing. You can render recorded gameplay offline, or play in a browser demo that streams from a Linux machine with the model on it.

The news: smaller graphics cards

On October 15 the team added presets for NVIDIA’s RTX 5090. A day later, on October 16, they added presets for the RTX 5080 and RTX 5070 Ti, which cost a lot less and have 16 GB of memory.

Their numbers, at 864 x 480 resolution:

• RTX 5080: about 42 frames a second with the plain preset, up to 67 with an optional speed-up called SageAttention.
• RTX 5070 Ti: about 41 frames a second plain, up to 58 with SageAttention.

The 5070 Ti preset uses 8-bit integer math for part of the model, which the team says runs faster than 8-bit floating point on that card.

What those numbers mean

These figures measure the AI model alone, in steady state, without video input and output. They aren’t the frame rate you’d see in the browser, and they don’t include the delay between pressing a key and seeing the result. Encoding the video, sending it over the network, buffering and decoding in the browser all add delay on top.

Other limits from the README:

• Model inference needs Linux, an NVIDIA GPU and Python 3.12.
• On the 5080 and 5070 Ti, each scene uses a fixed text prompt. The racing scene’s changing descriptions need a bigger card.
• Only one game session can run at a time per server.
• The code is open source (Apache-2.0 for the project). The model files and game assets come with their own licenses.

Who could use this

An indie developer could block out a level in plain shapes and see a painted version of it without making textures first. An AI hobbyist with a new RTX 50-series card can now run the real-time demo at home instead of renting a big data-center GPU.

Workflow: test it the honest way

1. Install it on a Linux machine with the matching preset, rtx5080 or rtx5070ti, and start with the plain preset before adding SageAttention.
2. Use the browser demo’s --reference-only mode first to check controls, HUD and sound without loading the model.
3. Then load the model and measure three things separately: model speed, the frame rate of the stream that reaches your browser, and the delay from key press to screen.
4. Play a long session with fast camera turns and watch whether the painted world stays consistent.

Try this prompt

If you set it up with a coding assistant, this keeps the testing focused:

I'm setting up Matrix-Game 4.0 on Linux with an RTX [5080 or 5070 Ti]. Walk me through installing it with the plain rtx[5080 or 5070ti] preset first, then help me write a short test plan that measures model FPS, browser stream FPS and key-press-to-screen delay separately. Don't add SageAttention until the plain preset works.

Would you play a game an AI paints live?