DuskEngine Devlog 8 - Custom Shader Support and More
Here's what's happened this past month or so.
Custom (asset) shaders
This is my favorite new feature, the ability to create custom shaders as part of the project's assets.
It currently only works with OpenGL, and the syntax is essentially GLSL, with some functionality inspired by Godot's custom shaders. I will likely change things here once I have to add support for Metal, maybe by using one of those fancy transpiler things, but this is not a high priority at the moment.
The first user of this functionality is this tornado:
Here's a video.
It's made from three separate parts, the first is the opaque part, which uses vertex displacement and shading based on some UV scrolling on noise textures. The second is a semi-transparent overlay that is displaced the same but by a slightly larger amount, thus adding some blurriness to the main tornado body. Finally, the third is a non-displaced version of the same tornado mesh, that uses a mask (noise) texture and UV scrolling to give the impression of swirls.
In total, three sets of custom shaders. The tornado is still WIP, but this is a good start I think.
New scene to work on battles and missions
The old (town) scene has been getting crowded, I turned it into a sort of hub world for now, and moved all combat to a separate scene.
Here's yet another video.
The video above shows combat between minions, and giving orders to some minions I've summoned off-screen.
Async pathfinding on worker threads
Running all pathfinding requests to completion and all on the main thread quickly started taking a toll on the framerate, now that I actually had more than a few agents in the scene.
I made some changes so that path calculations happen asynchronously (thus requests can find themselves in one of three states, InProgress, SuccessfullyCompleted or Impossible). Then, I set a maximum number of paths that can be calculated each frame, with the requests forming a queue. Furthermore, calculations are only performed up to a fixed number of steps each frame, if the destination is not reached within those steps, the request is paused and re-added to the queue.
This ensures that there's a fixed budget allotted to pathfinding each frame, and the nature of the queue plus the pausing mechanism ensures a little bit of each path is calculated as soon as possible (late comers don't wait until the firsts are fully calculated). In the near future I'll also make it so partial results are reported as soon as they're available.
Pathfinding is now done on worker threads, via the new job system functionality... which is actually Jolt's example job system disguised as my own.
This was a quick way to get a job system that both I and Jolt can use, while I do some research into what I need to do to best write my own.
The job system was also immediately useful on the editor side, in speeding up the navigation grid baking step.
Lots of enhancements and refactoring
There have been some optimizations done on the rendering side, in particular some now-superfluous work involving building the depth information for SSAO has been removed. Some work has also been done in order to be able to implement view-projected decals later.
The way assets get checked to see if there's a need to re-bake has been streamlined also, now using a (custom or default) calculated hash to tell.
Replaced std::unordered_map and set with ankerl::unordered_dense, especially in critical paths.
It turns out running a full lua GC cycle each frame is a bad idea for performance, now doing it in steps 🙂
Added support for LuaJIT's sampling profiler, with a rather primitive for now, way to see the results in the engine's editor.