Firefly in the Dusk🌙

DuskEngine Devlog 4 - Asset Refcounting Edition

A bunch of things happened on the engine side this past month or so, but asset refcounting is by far the biggest and most important of them all so I'll focus just on it.

I'll be working more on memory optimizations going forward because I'm planning to add web support as soon as possible and it would be pretty embarrassing to have, for a tiny scene, a very large bundle size as well as unreasonably large RAM and VRAM usage, even if the engine is in such an early state.

A first step that was easy to get out of the way, was to make sure assets don't stick around in RAM long after they've been used by whichever systems requested them.

A while back I made it so the AssetManager returns AssetHandles whenever loading an asset is requested:

class AssetManager final
{
    ...

public:
    template <typename TAsset>
    AssetHandle<TAsset> LoadAsset(const AssetId& assetId)

    ...
}

Only the AssetManager is allowed to create AssetHandles from scratch.

This extra layer of indirection will allow for things like async loading, for example: the asset handle could behave like a sort of Promise, though I have not yet implemented anything with regards to async loading.

It also allows for simpler error handling (the asset handle can have an error state set on it, in case loading the asset is impossible) and indeed, refcounting.

The matter, at first thought, seems fairly simple: each time an asset handle for a particular asset is generated, increment some counter. Each time it gets released, decrease said counter. When the counter reaches zero, the asset can be ejected from whatever cache contains it (when an asset is loaded, it gets added to a cache so subsequent loads return the cached version).

One thing I'm fairly sure I don't want, is to start ejecting assets whenever the last AssetHandle's destructor gets called. That sounds like a potential recipe for chaos and annoying debugging sessions.

It might also lead to unnecessary loads for the same asset, for example if in the same frame, system A loads an asset, uses it, then releases the handle, and later on system B also needs it, but since the asset has been released, a full reload is triggered.

It also means that the memory taken by assets that are truly not needed anymore in that frame, cannot be reused for other things. This is a problem if the size of the assets loaded in that frame, versus the RAM amount of the system, especially if no swap is in place, is out of balance. I won't treat this as a very likely scenario at the moment, but will keep it in mind as I go.

So for now, let's schedule asset cleanup as part of the engine's main loop, to make things more predictable and easier to change in the future.

First of all, the refcounting part:

class IAssetRefCounter
{
public:
    virtual void IncreaseRefCount(const AssetId& id) = 0;
    virtual void DecreaseRefCount(const AssetId& id) = 0;
};

This is an interface for something that will adjust refcounts for assets. AssetHandles get a pointer to such an instance when created by the AssetManager:

class AssetHandle
{
public:
    AssetHandle(const AssetId& assetId, IAssetRefCounter* refCounter) 
        : mAssetId(assetId), mRefCounter(refCounter)
    {
        mRefCounter->IncreaseRefCount(assetId);
    }

    // copy ctor, assignment operator and a bunch of other things
    // omitted

    virtual ~AssetHandle()
    {
        mRefCounter->DecreaseRefCount(mAssetId);
    }
}

So, increase refcount on construction, decrease on destruction. I am not going to paste the whole class in here because who knows what issues it currently has and I wouldn't want to pass them to someone else 🙂

The concrete refcounter looks like this:

class AssetRefCounter : public IAssetRefCounter
{
public:
    AssetRefCounter();

    // these go through that vector below and alter or insert the
    // refcount for that asset
    void IncreaseRefCount(const AssetId& id) override;
    void DecreaseRefCount(const AssetId& id) override;
    void EraseAllZeroRefCountEntries();

    template <typename TCallback>
    void IterateZeroRefCountEntries(TCallback cb)
    {
        for (const auto& p : mAssetAndRefCount)
        {
            if (p.second == 0)
            {
                cb(p.first);
            }
        }
    }
private:
    std::vector<std::pair<AssetId, unsigned int>> mAssetAndRefCount{};
};

The AssetManager creates one at initialization time.

The vector of pairs of asset IDs and refcounts is likely not the fastest implementation, but I don't think it's going to be an issue anytime soon.

Then, at the end of each frame, the engine calls AssetManager::OnFrameEnd which does this:

void AssetManager::OnFrameEnd()
{
    mRefCounter.IterateZeroRefCountEntries([this](const AssetId& toRelease) {
        mAssetCaches.EraseFromAnyCache(toRelease);
        });

    mRefCounter.EraseAllZeroRefCountEntries();
}

I had to go back and fix some sloppy coding here and there, where instead of loading an asset, getting just the info the system needed and storing it for later, I was (re)loading the asset each time that info was needed. This manifested itself as a lot of asset reloads in a short amount of time.

Some extra functionality that I omitted from the refcounter class is iteration over non-zero refcount assets, which is used to list the assets that have active handles, in the editor, in order to spot any leaks.

And that's about it. I think this is a good start to build upon.

I have a feeling my next memory-related optimization will be to finally compress textures, it's long overdue 🙂