← appaday

Fruitland, the native detour

Part two ended with the game holding 120 frames a second on most islands in the Quest 3's browser, and with a native shell planned that didn't exist yet. The next night of optimizing broke things: dynamic foveation, meant to save frames, kept stepping in where I could see it. My notes after playing it on 20 October:

ok so there are def some new issues introduced with optimizing which is what I was afraid of […] we now have weird foveation popping in and other issues that distract from the actual game development.

In the same message I asked the opposite of what that seems to call for:

I would like to try making a fully native prototype but not editing the game itself […] I want us to go on a little native adventure, imagining we are the kind of nintendo engineers that make sure we get the best of the best out of the given console.

The question behind it was simple. The game is plain JavaScript and WebGL running in a browser. Is the browser leaving the headset's power on the table, or not?

One frame, recorded and replayed

The agent didn't port the game. It wrapped every WebGL call the page makes and recorded one whole frame of an island: every shader, buffer, texture and draw call, with the values each draw was given. That went into a binary file, 2.9 MB for Cocoa Lake seen from far out (481 draws). Then it wrote a player in C++ that replays that frame natively on the Quest through OpenXR and OpenGL ES 3.2, once per eye, with the camera taken from the headset. So the GPU draws exactly the frame the browser draws, and any difference is the platform's. A few switches on the player (antialiasing, shadow samples, precision, resolution) say what each part of the frame costs.

The first night's numbers, for the same frame of Cocoa Lake:

the browser
98 frames a second, about 9.4 ms of GPU time a frame
native, as recorded
112 frames a second, 7.5 ms
native, no antialiasing
120, 5.9 ms: 4x MSAA was a fifth of the frame
native, no shadow map
120, 6.5 ms
the Snow Globe
68 natively, 100 without its glass dome: the dome was a third of the frame

My verdict at half past two that night:

ok so reading all this it sounds like regular native profiling will be the big win with the game staying web based and for publishing being wrapped as apk. that makes us fast, makes development easy and if we want, we can always go native for a part two

That evening I came back to the gap anyway:

we see a 14fps difference in a native build, that iiiis kinda substantial... is the lower gpu clockrate in web also going to translate to cooler headset and less battery draw?

That was the right thread to pull. The browser's session runs the Quest's GPU at about 545 MHz. The native player had asked for the highest sustained level and got 640. Asked for the browser's level instead, the native player ran the same frame at 99.7 frames a second, against the browser's 98. The clock was nearly the whole of native's lead. The browser picks the lower clock by itself, which is probably kinder to the battery, but the headset's power readout showed 9999 whether it was charging or not, so that part stays a guess.

Draw calls, and why the agent missed them

The native player did find something real. Marmalade Town, a busy town island, made 788 draw calls a frame. Folding repeated meshes into one instanced draw took it to 168, and natively from 93 frames a second to 120 at the browser's clock. In the browser it went from 89 to 114. I asked the obvious question:

also I am wondering why you didn't come up with more instancing yourself... you correctly identified that draw calls are a bottleneck, you done it before... so why not now. look for other things you might have missed

The answer: the game's own timer only measures its JavaScript, under 2 ms a frame. Each draw call is paid somewhere that timer never looks, in the browser process that translates WebGL and in the GPU's tiling. The audit that followed found 2,000 throwaway matrices made every frame, face culling switched off and on 184 times a frame on the Beehive, and every bee in the Great Beehive drawn twice since a merge. For anything else I had a method of my own to offer:

use classic binary search.it's one of the easiest ways and I always use it when debugging / profiling. […] e.g. we have a scene with 70fps. then you remove half the scene. if it goes up to 120fps good.

The game can now switch off whole named groups of its scene for a measurement, and that rule went into the instructions every agent reads.

Feel first

The point of the detour was to stop optimizing blind, not to optimize more. I had said so on the first night of it:

feel first! if the game feels great it doesnt matter what stack it runs on or what other details... now of course no foveation and 120fps feel great so that's the counter argument for those two details.

And on the morning of 22 October:

hmmmm let's not do the fps sweep... let's focus on game design again after this round is done. I think the solutions are probably easier than we thought

Antialiasing went back on in full after a day without it (“MSAA off does look pretty stressed”). The title gained a Shuffle list:

one thing that would be great is a “shuffle” mode in which the next level is always at random... I think that'll help me think of structure by confronting me with no structure...

The Fruitland title screen: the rainbow Fruitland logo and a big orange Play button for Orange Landing over a sea of orange juice with citrus halves. Two rows of list tabs read Worlds, Shuffle, Newest, Changed lately, With music, Your marks, Not played yet, Least played, Most played, Needs your feedback, Most loved, Least loved, Not rated yet, Slowest first. Below, numbered island tiles grouped by world, and a card for island 1 that names its recording and its slowest frame rate seen, 117 fps
The title now. Shuffle and Most played sit next to Slowest first, which sorts the islands by the frame rates the headset reported back.

One ruin, three times

I keep notes when I play other games. In September I'd written some down while playing Uncharted, and I pasted them in and asked for islands built on them. Among them:

re use same place with new lighting
re sue same place with different paths. first you walk past the tower then you find a way up
[…]
make things take time... shoving a crate, etc... to feel like you are actually doing something

A builder agent made the Orchard of Pomona, a temple orchard in the juice, and told it three times. Pomona's Gate is by day: yellow flowers mark the way, a heavy carved block takes a moment of straining before it moves, and the Tower of the Winds is walked past. Pomona at Sundown is the same place by other paths, up the outside of that tower. Pomona in the Storm is the same place at night, in rain, with the juice rising.

Pomona's Gate by day: a pale stone statue of a robed woman holding a bowl of fruit stands on a round plinth in purple juice, surrounded by white marble walkways with green tiles, broken columns and fruit trees. A red brick aqueduct with arches runs behind, cherries and grapes along its top, and a gateway of white stone opens on the right. Hazy green hills in the distance
Pomona's Gate, by day.
The same orchard from high above at sunset, everything in warm orange light: a colonnade walk across the purple juice, a round brick tower with a glowing light on top, a temple with a red roof and columns, and the aqueduct along the right. Bats fly out over the hills
Pomona at Sundown: the same ruin, other paths. The bats pour out of the tower top, which is where Pip has to go.
The same orchard at night in a storm, dark blue and violet: the colonnade, the temple and the aqueduct are barely lit, with small yellow flowers glowing along the walk, wind swirls drawn in the air and rain streaks. A beam of light rises from the temple
Pomona in the Storm. The flowers that marked the way by day still glow.
the native player
C++, OpenXR and OpenGL ES 3.2, a 3 MB app with two recorded islands inside; it replays one frame, it isn't the game
the game
still plain JavaScript and WebGL 2 with WebXR in the Quest 3's browser
islands
135 in the title's list, 121 at part two
since part two
123 commits, version 2.118.2

Still rough

The native player isn't public and isn't meant to be: it's a measuring bench that plays back one recorded frame, and Pip can only walk on what was recorded. What a stranger can open is the browser game, made for the Quest 3's browser. All figures here are flat renders of the live build in a desktop browser, not the view in the headset.

The Orchard of Pomona was merged on the morning of 22 October and has no play report from me yet. Not every island holds 120 frames a second; the Snow Globe was the slowest on the bench.