I've been working for the past few months on a new rendering technique for the billboard vegetation cover of the terrain engine I'm working on.
What we had previously for billboard trees and grass was a standard technique : a geometry shader, generating a camera-facing quad at each tree position and displaying the billboard texture on it. When applied to what I was aiming for (i. e. 1:1 scale vegetation covers with enough draw distance to simulate real life sceneries) this technique had two major issues :
- The shader has to be fed with the tree/bush/grass positions in some way, which means generating and storing them at high enough distances.
- Even with only four vertices per billboard, the sheer amount of them that have to be rendered for realistically dense and distant vegetation makes it unusable.
I had been thinking for a lot of time about possible alternatives and got especially inspired by this paper : https://www.cg.tuwien.ac.at/research/publications/2007/Habel_2007_IAG/Habel_2007_IAG-Preprint.pdf
It describes a method for rendering grass by ray-casting in the fragment shader inside a shell mesh in a fragment shader, itself inspired by another paper on a very similar fur rendering technique.
While what I developped is very different to what the paper describes, this is the core idea that I ended up using. What I implemented currently only works for one vegetation scale, so the screenshots only show trees. I plan to further develop it in a way that makes it possible to render all scales (i. e. trees, bushes, grass, etc.) in one go. Here are a few screenshots of its current state, with still some issues (mainly the shell mesh being incorrect at the edges of the terrain tiles and for single trees up close). Higher resolution versions can be seen here : http://imgur.com/a/3Mddt
How it works :
- The terrain tiles are drawn a second time using the new vegetation shader.
- This shader raises the terrain vertices according to the vegetation height (stored during the terrain generation in a texture that also contains the material and vegetation index for each point on the terrain). This generates our "shell mesh" for the vegetation.
- The fragment shader ray-casts the fragment from the shell mesh in the tangent space of the terrain tile.
- To do this, a grid traversal algorithm is used (taken from here : http://playtechs.blogspot.fr/2007/03/raytracing-on-grid.html) on a simulated 2D grid with the "spacing" we want for the vegetation (the minimum distance between each tree, I used 6m in the screenshots). Each point on this grid implicitly represents the position of a tree. This is used to predict where the ray might likely intersect with a billboard.
- For each square of the grid that the ray traverses, we do an intersection test against quads placed on the surrounding grid points (with a random displacement applied) and sample the corresponding billboard texture at the intersected UV position. If the alpha test fails, the grid traversal continues. If it succeeds, the grid traversal ends and the fragment shaders returns the sampled color and writes the intersection position in the depth buffer.
Performance :
This runs at about 15 to 30fps on my laptop with a gtx850m, without the vegetation casting shadows. By using the same technique during the shadow caster pass, the forests get self-shadowing which is much needed for realism, but runs at 5 to 10fps on my system. There is still a lot of optimizing left to do on the shader and I will work on it when I have the time.
Advantages of this method :
- No need to generate and store lists of tree positions while generating the terrain. If the terrain tiles have information textures containing the data we need (which vegetation type is at each point, and its height) all we have to do is render the tiles a second time with this technique. The billboards' placement becomes implicit, and vegetation can instantly cover the generated terrain without more generation time.
- The billboards' geometry also becomes implicit, and the complexity of drawing the vegetation moves from the amount of billboards to the amount of fragments that the shell mesh casts on the screen. Which means that for any draw distance and for any vegetation density, the cost stays approximatively the same.
Disavantadges :
- The fragment shader is way too complex currently to be useful if you want to render a low amount of billboards, like seen in today's open world games where the terrains are not realistically scaled. It only becomes useful when you need to render huge amounts of billboards at long distances.
- The rendering cost varies wildly depending on how many pixels the vegetation's shell mesh occupies on the screen.
Future possibilities:
- Alpha blending would be easy to implement, by accumulating intersected colors during the grid traversal until it is opaque, instead of only using the first opaque intersected color and returning it immediatly.
- The algorithm to test the intersection for a specific tree at each grid point could be changed to have volumetric trees instead of only flat billboards, by raymarching inside a depth map texture of the billboard (like for parallax effects). But this would probably be way too heavy for now.