Oncomfort - Silva a Virtual Reality experience

When your goal is to build a VR application that runs on a Samsung Galaxy S7 (with today standard this is a low end device) for more than 30 minutes without throttling and with as well as possible graphics, yes, this project was quite a challenge.

It started at the end of 2019, its first release was done by the end of 2020 and a few updates were done afterwards.

This ambitious project was produced for Oncomfort who takes the pain and anxiety away through Virtual Reality. Their application is used in hospitals to avoid an anesthesia. The patients can just relax and voyage with their mind into these environments. The objective was to create a new set of environments and animations based on the theme of passing seasons. The patient could start a predefined session length (such as 5, 10, or 15 minutes) and move through various environments with animated animals and background animations. The project had to run on a Samsung Galaxy S7 for a GearVR headset, which was a significant challenge due to the device's limited CPU capabilities and tendency to overheat, leading to throttling and failure.

For the first version of Silva, we gathered a team of around 15 people that worked on the project. I oversaw the production schedule, managed the project importation and conducted extensive research on the optimization techniques available in Unity 2019.3, which had just been released. There is a lot of settings that are easy to find but after a lot of testing it quickly came to light that the Samsung Galaxy S7 was limited on the CPU side with the draw calls but it was possible to achieve some complex shaders. All that without Bloom, my wish that was never possible on that device because of everything else we did and just an empty scene with bloom had a 40% impact on performance.

Here is a few things I ended up doing:
- Set the target FPS limit to a constant 30.
- Updated CPU & GPU levels to produce less heat thus lowering the performances (we needed to have less heat produce first).
- Created our own Uber shader with our own BRDF, simple lambert lighting with custom soft self-shadowing (I hate so much when we have just a black shadow that could have been softer) and GGX specular. It really helped with the animals as it was possible to customize the light and shadow color per animal material.
- Baked all lighting with Bakery to improve the look of the application on the device.
- Developed a custom sky shader that animated colors, clouds, and sun with multiple versions depending on scene needs.
- Created a custom shader for animated light shaft and water.
- Custom terrain shader that took into consideration the maximal number of textures that were possible to work with.
- Optimized all cutout shaders for the VR screen to be pixel anti-aliased (by default only the topology is anti-aliased).
- Merged assets by shader to limit draw calls.
- Optimized all imported assets with Unity settings and geometry to be view specific as much as possible.
- Imported and refined many assets in the end.

It was by far the most optimization I had to do because of the huge limitations we have had. Every shader was tested in its own scene to ensure that the impact on the device was low. After that we tested the application quite a few times with RenderDoc and the Unity Profiler. It helped to quickly find costly elements.

Despite the significant limitations and once the rendering pipeline was in place, the team managed to produce a new environment in just two weeks, and after the first release, the project was migrated to run on a Pico G2 4k. The optimizations paid off, as the device was powerful enough to support the increased resolution and rendering of the scene twice for a client-specific feature.

Overall, the project was a significant challenge, but the team's hard work paid off, resulting in a polished and successful VR application that could be used to relieve pain and anxiety in hospitals.