UE4 Code: Fixing Hot Reload of AutomationTests


Issue UE-25350 “Hot reload does not compile changes to automation tests”, opened through a question on the answerhub, has been around for quite a while now and may be one of many blockers keeping you from writing unit tests in Unreal Engine. Which you should.

I was so annoyed by this that I decided to get to the root of the problem. I found an easy workaround, which — if you are not interested in the problem — is described further down below in the “TL;DR” section.

The problem

AutomationTests are registered from the constructor of your FAutomationBase subclass (that you create via IMPLEMENT_SIMPLE_AUTOMATION_TEST). The test will only be registered, if it hasn’t been before. Therefore, when a test with the same name is registered, it would skip the registration, because FAutomationTestFramework contains a map of test name to test instance, counting same names (keys) as the same test, even if it’s a different class.

They would be unregistered in the destructor, which would be called when the DLL is unloaded, but for my case, the DLL is never unloaded, because I have a separate module for my unittests. It doesn’t contain any other classes, i.e. UCLASS-es, in which case the module is treated as not hotreloadable and is “abandoned” during hot reload instead, leaving the DLL loaded.

After that the newly compiled DLL is loaded, which does try to register your changed test, but there is still the one with the same name hanging around that you registered previously, so it won’t actually register that test!

Solution Attempts

Sounds simple, just define a UCLASS and problem solved? For some reason if you add a UCLASS, your module is then reloaded as a “UPackage” which goes a different path and doesn’t even reach the aforementioned code, but is instead just abandoned directly.

Instead, we could define a subclass of FAutomationTestBase, which in its constructor binds a “PreUnloadCallback” on the module, which would unregister the test. That doesn’t work though, because the constructor is called before the module is fully registered in the ModuleManager. This leaves us no way to get a reference to the module.

A decent fix would probably be to have Unreal Engine prefix the test names with the module name and unregistering them on shutdown of the module. If somebody wants to pay me to fix UE-25350 properly, DM @squareys on twitter or something, I’d be happy to do it 😛 , otherwise try the following workaround:

TL;DR — The Workaround

If you have this problem, you likely also don’t have any UCLASS-es in your test module. Workaround would therefore be to just create an empty UCLASS in the module that contains the tests.

I add this to my the main module source file `<Your Game>Tests.cpp` to make the test “F<Your Test Type Name>Test” hot reloadable:

class FTestModuleImpl : public FDefaultGameModuleImpl {

    void ShutdownModule() override {
        /* Workaround for UE-25350 */
        FAutomationTestFramework::Get().UnregisterAutomationTest("F<Your Test Type Name>Test");
        // ... for every test you defined.
    }
};

IMPLEMENT_GAME_MODULE(FTestModuleImpl, "<Your Game>Tests" );

 

That’s it! Happy AutomationTesting/GameTesting/UnitTesting everybody! 🙂

See you in the next blog.

 

March 2nd 2018 8:42 PM | by Squareys | posted in Coding, General, Unreal Engine 4