Herl: Planet-Sized Terrain Ray Marching

Fast enough for virtual reality, even Faster on normal displays

Less This (low poly scenes), More This (realistic landscapes)

Bottom image is a screen capture from Herl

How do we get more realistic and immersive environments?

**Ray Tracing! (actually Ray Marching)**

Herl is a demonstration of how ray marching can be used to render very large and complicated planet-sized environments, even in virtual reality.

"Are you crazy?"

"Ray tracing is too slow!"

"Virtual reality requires simple scenes with low-poly geometry."

Basically Everyone

"I think I'll try anyway."

Me

TLDR: Download Herl and See for Yourself (No VR Headset Required)

Video Demonstration

Screenshots

Preface

Herl has nothing to do with NVIDIA's RTX (Ray Tracing Extensions) or Microsoft's DXR (DirectX Raytracing). Those technologies are newer extensions to conventional rendering methods that add triangle ray casting to triangle rasterization, and so they only work with conventional rendering methods that render a scene using triangles.

In order to use those extensions, geometry must be tessellated into triangle meshes. Most traditional rendering is done this way, because the meshes for many objects like buildings and vehicles are small, simple, and generated only once.

However, some geometry like terrain is too large, complex, or dynamic to be efficiently tessellated into triangles. There are various methods for dynamically tessellating visible terrain into smaller, more manageable meshes for on-the-fly for rendering, but there are usually trade-offs which result in slower performance, bland terrain, and/or popping details as the terrain is viewed from different perspectives.

Instead of tessellation and traditional triangle rasterization, Herl uses ray marching to render terrain.

Terminology

Ray Casting: Mathematical algorithms used to determine whether and how a ray intersects with the surfaces of geometry.
Ray Tracing: Performing ray casts to determine how light affects the surface of geometry. Often multiple ray casts are performed recursively, from one surface to another, to handle light reflections and refractions.
Ray Marching: Performing multiple iterative ray casts (and/or distance estimations) near complex geometry to find ray intersections, or within volumes to composite samples (clouds, smoke, etc...). Ray marching is necessary when the surface intersection or composite sample function is not easy (or impossible) to solve without numeric iterative methods.

Some people consider ray tracing the superset which can include both the ray casting and ray marching techniques, while others consider ray tracing and ray marching to be separate, and still others consider ray marching the superset which can include ray casting and ray tracing.

I consider ray marching to be the superset, but when talking to the general public or someone who is not interested in the particulars, I will often refer to all three concepts as ray tracing, because that is the term which most people are familiar.

Credits

"If I have seen further it is by standing on ye sholders of Giants."

Sir Isaac Newton

Inigo Quilez

Inigo is clearly a super-genius. He has an astonishingly brilliant library of articles which explain how to find geometry intersections, generate noise, ray march clouds and terrain, blend pleasing color palettes, and many other concepts which are essential to ray marching. I'd highlight the most important articles here, but there are too many.

Ben Golus

Whenever I searched the Unity Forums for a solution to an incredibly obscure and complicated problem that I had with Unity (especially its built-in shader variables and libraries), Ben was usually the one with comprehensive but succinct answers to my problems.

Jasper Flick of Catlike Coding

Jasper has an excellent collection of Unity tutorials, and his implementation of Curves and Splines was especially useful when I scripted ship movements.

Shadertoy

Visiting the Shadertoy website and seeing the techniques shared by other people gave me a lot of ideas and inspiration that I was able to apply to Herl. These shaders were especially influential to me:

Mountains by Dave Hoskins

Geodesic tiling by Thomas Hooper

[NV15] DIY Spaceman Cave by eiffie

Initial Objectives

Procedurally Generated Earth-Size Worlds
- Generated from Single Seed
- Large Draw Distances from Near Orbit
- Terrain Deformation
Realistic Terrain, Water, Clouds, Vegetation
- Shaded, but not Self-Shadowing Terrain
- Water with Waves and Reflectivity, but not Refraction
- Simple Grass and Pine Trees for Vegetation
Real-time Performance
- Instant Generation and Loading of New Worlds
- No Delays, Stutter, or Geometry Popping when Moving
- Virtual Reality Compatible Framerates
Unity Graphics and Physics Integration
- Hybrid Rendering with Rasterized Geometry
- Physics Support for Collisions with Terrain
- Up and Down Axis Remains Fixed
- Non-Spherical World and No Orbital Mechanics Necessary
Hardware Agnostic
- Different hardware should render scene and simulate physics identically

Future Plans and Ideas

Better Quality and Performance with Instruction Level Profiling

Until recently, I only owned NVIDIA GPUs. Unfortunately, NVIDIA's profiler for fragment shaders (Nsight Graphics) does not provide instruction level profiling. Strangely, NVIDIA's compute profiler (Nsight Compute) does, but I cannot use a compute profiler on a fragment shader.

The profiling that I can get from Nsight Graphics is high level information like counters and which shaders or draw calls cost the most. I'm sure that these basic metrics help people with a lot of simple shaders and lots of draw calls. However, ray marching uses a single pass of one extremely large shader, and I need to know which parts of that shader are the slowest (at the instruction level), so I can optimize it.

AMD's GPUOpen Radeon GPU Profiler does provide instruction level profiling, but unfortunately I did not own any AMD GPUs until after the public release of Herl, and I haven't had time to perform profiling analysis.

I wish NVIDIA had instruction level profiling for graphics shaders. If anyone at NVIDIA is reading this, then you guys really should feel embarrassed about this. There is no reason why you shouldn't have this capability already, and it is going to be more and more important as graphical shaders continue to become more complicated.
Nighttime
Better Clouds
Better Shadows
Better Atmospheric Obscuration of Near Orbit Objects
Stereo Rasterization of Meshes in Water Reflections
Procedural Neighboring Planets and Moons
Procedural Nebulas
Planetary Rings
Mesh Colliders using Terrain
Spherical Worlds with Terrain
Microsoft HoloLens Support
Foveated Rendering using Eye Tracking in HTC Vive Pro Eye and/or Microsoft HoloLens 2

Current Situation

I wrote most of Herl during the spare time I had while being a caregiver for my mother, before and during the COVID-19 pandemic. She had a stroke that paralyzed the right side of her body and severely affected her ability to speak, write, and use a computer. A few months after returning home from hospitalization due to the stroke, she fell and broke her hip, which required hip replacement surgery. And then the pandemic happened.

Now that the pandemic is over, her condition is improving with speech and physical therapy, and I'm able to have full-time caregivers help her. I am now dedicating my time and effort toward other projects which are more financially rewarding. Herl is now open source for others to learn from and use.

Contribute

Over the last few years, I've spent countless hours inventing and optimizing novel new rendering strategies to make Herl possible, and I still have a lot of ideas that haven't been explored. I hope that if you like what you see, and would like to see more of it, that you might contribute so that I can continue this work.

Contribute on Patreon

Contribute by Sending Bitcoin

1Mbhh4iyK6gQGF34h2Xei9HoVLQ4B8riMo

Thank you!

Downloads

System Requirements for Herl Windows

OS: Windows 7 SP1+, Windows 8.x, Windows 10, Windows 11
CPU: x86 64-bit
GPU: Direct3D 11 support

Optional (for Virtual Reality Experience)

VR Headsets: Oculus (Rift, etc...) and OpenVR (HTC Vive, Valve Index, etc...) compatible

Herl 0.06 for Windows (24 MiB)

Version Changes

0.01
0.02
0.03
0.04
0.05
0.06

Performance Techniques for Rendering Procedurally Generated Terrain in Virtual Reality using Ray Marching (935 KiB)

Herl GitHub Source Code Repository (GPLv3 License)

Controls

Function	Keyboard and Mouse	Game Controller	Oculus	OpenVR
Menu On/Off	Enter Return Space Bar	Start	Start	Right Menu Button
Quit	Esc	Back
Enable Mouse Steering	M
Invert Mouse	I
Steering	WASD Num Pad	Right Stick	Head Orientation	Head Orientation
Center (VR)	C	A Button	A Button	Right Menu Button
Adjust Iris (VR)	O / P
Free Look	F	X Button	X Button	Left Menu Button
Blaster	Left Ctrl Left Mouse Button	Right Trigger	Right Trigger	Right Trigger
Boost	Left Shift Middle Mouse Button	Right Bumper	Right Grip	Right Grip
Previous/Next World	H / J	Left Stick Up/Down	Left Stick Up/Down	Left Pad Up/Down
Terran/Alien	K / L	Left Stick Left/Right	Left Stick Left/Right	Left Pad Left/Right
Spawn Bots	V
Raymarcher On/Off	R

Frequently Asked Questions and Comments

Is this really ray marching in realtime? How are you able to get such high framerates?

All of the ray marching is done in realtime. Every pixel (except the ones displaying a few meshes like the spaceships) is drawn by casting out a ray and determining what and where it hits. I do at least one additional ray bounce for reflections in water, and shadows cast by the clouds and terrain.

I used several ray marching acceleration techniques invented by other people, and then spent years optimizing them. I also used one additional technique that I invented myself and haven't seen anywhere else. I wrote an unpublished paper on it for my thesis, which is available in the Downloads. Honestly, all of these techniques would seem pretty obvious to you if I were to explain them, but then I guess a lot of good ideas are obvious after someone discovers and exploits them.

If you want to see what is not rendered by ray marching, you can press the R key while running Herl.

Will you license the technology for commercial non-GPL applications?

I would be happy to sell a more flexible license for the technology and source code that I developed for Herl to be used in games and other commercial applications. Please Contact Me if you are interested.

Windows prevents me from installing Herl.
I get a message that "Windows protected your PC" when I try to install Herl.

Windows displays this alert until Microsoft detects that a significant number of installations have occurred without malware being reported or detected, so the first few thousand downloads and installations of Herl are likely to see this message.

To bypass this protection, click on the "More Info" link underneath the message, and then click on the "Run anyway" button that appears.

Does Herl run on macOS or Linux?

As far as I know, the shaders used in Herl should all work on Metal and Vulkan. I have not built a Mac release because I don't own a Mac, and so I'm not able to test the build before releasing it. I have not built a Linux release because, although I can dual-boot to Linux on my development PC, there has not been any real demand for it.

If someone wants to send me a Mac with a GPU that supports Metal, or the money to buy one (at least $250), I can promise a Mac release. Metal requires an Intel HD 4000 series (or newer), AMD GCN-based GPU, or Nvidia Kepler-based GPU (or newer). So, basically a 2012 or newer Mac Mini should meet the requirements.

Some people have suggested that they would beta test for me on their Mac if I send them a Mac build, but that isn't really sustainable. I need my own Mac to test and fix whatever (probably minor) issues are discovered, and then have it available for testing future builds.

Does Herl run on Android (5.x +), Gear VR, Oculus Go, or Oculus Quest (without a PC link)?

Yes, but you'd probably only get about 1 frame every second. I've attempted ray marching on the mobile and stand-alone devices and they don't have enough GPU processing power to do anything beyond simple fractals.

The field of view is too small.
The frame rate is too low.

Manually adjust the iris size using the letter O and P keys on your keyboard. If you make the iris larger, the frame rate will drop. If you make it smaller, the frame rate will increase.

Ideally, Herl would automatically adjust the iris size based on your VR headset and GPU performance, but because I don't have access to all the different types of VR headsets and GPUs, I cannot implement and test automatic adjustment sufficiently. Instead, I made the adjustment manual.

When I run Herl using a virtual reality headset, I feel sick.

Suck it up. Seriously though, I named this project Herl for a variety of reasons, and virtual reality sickness is one of them. Maybe someday we will find some tricks to eliminate it, but for now, if you want an immersive virtual reality experience, you must be willing to pay for the consequences. Personally, my brain and body have gotten used to it, and I'm rarely bothered by it anymore, but I got sick a lot during initial development.

If you're really bothered by it, and willing to take over-the-counter drugs, then Dramamine purports to lessen or eliminate virtual reality sickness.

The keyboard, mouse, and joystick controls suck.
Herl has boring gameplay.
The 3D models are ugly.
The terran and alien bots look the same.
The bots are extremely dumb.

Herl is not finished. It is also not a game. It is an evolving (unfinished) technology demonstration.

Herl is meant to be experienced using virtual reality hardware and steered using head motion, not a keyboard, mouse, or joystick. I added flat screen support and additional control options so that the demo could be seen on other non-VR computers. I know these controls are suboptimal, but I haven't had time to improve them.

The 3-D models were purchased online and modified quickly in Blender to be used as stand-ins. They demonstrate that hybrid rendering of both ray marching and rasterized geometry is possible.

My plan is to continue improving the quality of the graphics, and possibly the "gameplay".

Will Herl become a game?

Possibly. It would probably need to be taken over by a game development team with some idea of how to use the Herl rendering and physics technology in a way that produces engaging gameplay. I'm just one guy with a passion for creating engines and tools. I'm not very good at designing gameplay or the artistic side of things, and I don't have the time to do it for free. I contrived a demo which hints at some of the directions that could be taken with Herl, but there is no definitive Herl game planned yet.

Where are the bots spawning?

The bots spawn in space and then fly down and around their base. Turn on bot spawning and then look around the bases.