TECHNICAL ARCHITECTURE
How the final system works
Cross-section of the completed application. Each diagram represents a layer of the system. Interactive diagrams let you explore states, pipelines, and branching.
Application state machine
Five canonical states drive the visitor's path through the work. Click a state to see what runs inside it and how the system transitions to and from neighbouring states.
The eleven-step fluid simulation pipeline
Every frame the PIC/FLIP solver runs twelve compute steps that move the particles, transfer to the grid, solve for pressure, and transfer back, accepting mouse and audio-driven forces at specific stages.
PIC/FLIP twelve-step pipeline
Twelve compute steps per frame, from grid clear through particle advection. Audio input drives the transfer step; mouse input drives the force step.
- 01 Clear Reset velocity and mass grids to zero.
- 02 Transfer → Particles deposit velocity/mass onto the grid (P2G).
- 03 Normalize Divide accumulated velocity by mass per cell.
- 04 Mark cells Flag fluid, boundary, and air cells for the pressure solve.
- 05 Copy vel Snapshot the pre-forces velocity for the PIC/FLIP blend.
- 06 Add forces Apply gravity plus mouse / audio-driven forces.
- 07 Enforce bnd Zero the normal component on boundary cells.
- 08 Divergence Compute ∇·u per cell for the pressure projection.
- 09 Jacobi ×50 Fifty iterations of Jacobi to solve ∇²p = ∇·u.
- 10 Subtract u -= ∇p to obtain the divergence-free velocity field.
- 11 Transfer ← Grid velocity is blended back into particles (G2P).
- 12 Advect Advance particle positions with the corrected velocity.
Deferred rendering with five passes
Geometry, occlusion, shadow, composite, and antialiasing each run in their own pass, allowing the composite stage to combine surface data with audio-driven colour without re-rasterising the scene.
Five-pass deferred renderer
Rendering flows left to right across five passes: geometry, occlusion, shadow, composite, and post-processing.
- Pass 1 G-Buffer
- normal
- speed
- viewZ
- Pass 2 Sphere AO
- additive
- occlusion
- Pass 3 Shadow Depth
- ortho depth
- PCF
- Pass 4 Composite
- HSV + audio
- Pass 5 FXAA
- post
Thirteen-track audio zone system
Field recordings are organised into categories, which are in turn routed to spatial zones inside the scene so that sound and space track each other.
Thirteen-track audio zone system
Thirteen field-recording tracks are grouped into five categories, each of which maps onto a set of spatial zones in the scene.
Tracks
- market crowd
- taxi ambience
- driving man
Zones
- bugis street
- bugis mosque court
- bugis hawker
Tracks
- mosque prayer
- fountain
Zones
- inner courtyard
- dome echo
Tracks
- wave near
- wave far
- wind
Zones
- tide line
- wind channel
Tracks
- bird canopy
- leaves
Zones
- palm canopy
- path edge
Tracks
- low drone
Zones
- global ambient
Three visual branches of the fragment shader
The composite fragment shader chooses one of three output paths based on uniforms set by the region: bioluminescent, crystal, or default HSV.
composite.frag three-way branch
The fragment shader selects one of three paths based on two uniforms: the bioluminescent ECP branch, the Bugis crystal branch, and the default HSV fallback.
FRAGMENT SHADER
composite.frag u_bioluminescent > 0.5 u_bugisCrystal > 0.5 else