Blend4Web beats Unity on all fronts! Great!!!!
Blend4Web vs Unity: WebGL Performance Comparison
2016-10-21
Editor's Preface
This article was originally published on Habrahabr, a popular Russian website for IT professionals. Its topical theme sparked the interest of thousands of readers who left dozens of comments. We are glad to present the translation of this highly intriguing research on the performance of Unity WebGL and Blend4Web, with reported issues taken into account as well as benchmarks updated for the latest builds of both engines.
Intro
The developers of the Unity engine announced “official support” of WebGL in December 2015. From this point, Unity can be considered a competitive player in the field of 3D Web. Thus, it can be interesting to compare the functionality, the maturity and the efficiency of both Blend4Web and Unity.
I’ve created three test applications of varying complexity as benchmarks. These benchmarks were created for both engines at the same time and are completely symmetrical. Their scenes, objects, textures and even material settings are almost identical. No handicaps and no specific optimizations were made for any of the engines. Whatever was offered “out-of-the-box” was used by default.
Although in some cases I had to improvise. Particularly, a line of code present in the corresponding Blend4Web files was added to Unity HTML files in order to equalize screen resolution on mobile devices (we want to be fair, right?):
<meta name="viewport" content="initial-scale=1.0, user-scalable=no, width=device-width">
All scenes and models were prepared in Blender, then textured and exported to the engines. In some cases, I’ve used Empty objects to place cameras and light sources (that Unity does not import from blend files) precisely where they should be.
The comparison was based on several criteria: loading time, FPS (aka frames per second) and memory consumption. Moreover, the loading time was measured twice: first, as a cold start with browser cache cleared, and then as a hot start with most of the data retrieved from the cache. This helps us to determine which engine can start a scene in the browser faster and with better efficiency. You know, people usually don’t like to wait a long time...
First Test
This and subsequent tests were carried out in Google Chrome 53 browser on PC (Intel I5-3570, 8RAM, GTX 650, screen resolution 1680х1050), Motorola Nexus 6, Samsung Galaxy S6 Edge and Apple iPad 3 (in Safari 10 browser).
Here is a very simple scene that resembles a frame out of a game. There are several models, a couple of light sources and basic animation. No advanced technologies are used here.
Test #1. Loading time, cold start (in seconds, the less the better).
The outcome of the testing is already very interesting. We can see that Blend4Web's scenes were loading much faster than Unity's.
This is because when building a WebGL project, Unity, by default, compresses its (rather big) files. Then, after downloading the archive, the application is extracted on the client side. This, of course, requires additional time. And don’t forget about the loading screen, which adds a few more seconds, and also about the initialization of WebGL itself. The second table (hot start) shows the difference in the efficiency of the two engines particularly well:
Test #1. Loading time, hot start (in seconds, the less the better).
This is the “clear” time that the engines need to show a loaded scene. Blend4Web does it almost in a flash, while its competitor takes about six times longer. Of course, you can shorten the loading time a bit by removing the splash screen which is absent in the paid version of Unity but this is unlikely to make the initialization itself any faster.
Test #1. Frames per second (the more the better).
Everyone’s favorite FPS is a very important criterion as well. No one likes freezes or hold ups.
Here, Blend4Web and Unity perform almost neck and neck, but only on PC. On mobile devices, Blend4Web applications were “hitting the ceiling” of 60 FPS. Unfortunately, there doesn’t seem to be a way to disable FPS limit on mobile browsers, while the desktop Google Chrome was running with the “--disable-gpu-vsync” option.
Unity shows a little different result. Only Galaxy S6 was able to achieve solid results. Apple iPad 3 was able to only score 9, while the Motorola device was unable to even run the test scene.
Test #1. Memory consumption, Mb (the less the better).
Memory consumption data were retrieved from the Google Chrome web browser Task Manager, from the desktop version. As you can see, both engines show very good results.
To tell the truth, I was expecting a different picture that would explain browser crashes caused by Unity apps. But the tests have not shown any major difference in memory consumption, at least for this simple scene. This makes the browser crash on Nexus even more surprising.
Second Test
So, here is the “Island in the Ocean” scene. It is unoptimized, heavy and very big. Trust me, I didn’t do anything to adapt it to run on mobile!
The most complicated thing about this scene was to make it look the same in both engines. It was easy enough in Blend4Web – the models and the landscape were created in Blender, the water material and environment were also done there, nothing too complex about it.
But with Unity, there were problems. First of all, I decided not to use Unity's native terrain. It’s easy and fun to sculpt it and plant it with trees, but this solution, as convenient as it is, might not work fast enough. So, I simply exported a scene directly from Blender, with a landscape model and already placed objects. This allowed me to make the two scenes as similar as possible.
Secondly, basic Unity shaders are unable to work with two-sided materials, which made tree leaves transparent from the backside. I tried to use other ready-made shaders like “tree leaves”, but for some reason it didn’t help. So I decided not to investigate it any further and instead found a third-party shader online that turned out to be perfect for my task.
But let us get to the results of the testing, as they were truly stunning!
Test #2. Loading time, cold start (in seconds, the less the better).
Test #2. Loading time, hot start (in seconds, the less the better).
The first thing was, once again, the Nexus browser crashing when running the Unity application. And Unity also exhibited a very long, almost obscenely long, loading time. What’s interesting is that in this engine there is little difference between the “cold” and “hot” start times. It seems that Unity spends an incredible amount of time preparing the scene for showing.
Test #2. Frames per second (the more the better).
This test also showed interesting results in terms of performance. Galaxy device shows almost identical FPS numbers for both engines but on PC the difference was very significant. What’s interesting is that changing the scale of the scene influences FPS differently in different engines. For example, Unity got the highest values when the camera was zoomed out from the island, while Blend4Web showed the best results when it was as close as possible. It looks like the engines use different optimization approaches. The table shows the maximum possible FPS under the circumstances that is most beneficial for a specific engine.
Now, about the unexpected Apple iPad 3 crash… Yes, as you can see, this device has joined Nexus. Unlike the Motorola, though, it put up a fight to the utmost, and crashed after starting the scene. So we were able to measure the loading time, but working capacity of the Unity application in this case turned out to be zero.
And maybe the reason for this is hidden in the following table.
Test #2. Memory consumption, Mb (the less the better).
Third Test
I also couldn’t pass up physics, so the the final test is dedicated solely to this. The scene has a cube with closed inner space filled with several dozens of tiny spheres. The cube rotates slowly, making the spheres roll.
There are no complex shaders or effects. The scene only uses rigid body and simple colliders.
As usual, it took some effort to make physics work in this scene. And I think you can already guess what the problem was.
The spheres were stubbornly trying to leave their cage and simply rolling out of its limits. It happened in both of the engines. Eventually I was able to find an acceptable solution for PC platform, where all objects behaved as expected, but on mobile devices this scene looked ridiculous. So a new testing criteria was introduced: physics stability, or “outsider spheres”, as I also call it. The scene ran for a minute, and then the number of spheres left in the cage was checked. The more of them was left, the better.
And now...
Test #3. Loading time, cold start (in seconds, the less the better).
Test #3. Loading time, hot start (in seconds, the less the better).
Test #3. Frames per second (the more the better).
Take notice of the equal FPS numbers in Blend4Web scene on mobile devices. It is safe to assume that that the engine could have rendered many more frames that 60, if it wasn’t limited by vertical synchronization. That’s because Blend4Web can process physics in a separate thread (worker). As far as I know, this is something Unity cannot do. This is the reason for such high FPS numbers and also, as you will see, high stability.
Test #3. Physics stability (percentage of lost spheres, the less the better).
As I said, both Unity and Blend4Web performed great on the PC platform. All the spheres remained in place.
Galaxy S6 (Blend4Web) turned out to be the leader among the mobile devices, and iPad 3 (Blend4Web) was an outsider. Unity application has failed the test based on this criteria, with an exception of the PC platform.
All and all, Unity physics in WebGL leave an overall bad impression. After loading the scene, the screen froze, and only after a couple of seconds it finally reached the long-expected FPS. This, of course, only regards PC. On mobile devices, everything was much more discouraging. Most of the spheres were lost after just a few seconds, and only then, with an almost empty scene, the application was able to achieve a solid 60 frames per second.
It is possible that the reason for this outrageous behavior of physics in Unity lies in the following memory test results.
Test #3. Memory consumption, Mb (the less the better).
Compared to the previous two scenes, the third one proved to be much more memory-consuming, and this applies to both engines.
The Unity application behaved in a very strange manner. Right after starting the scene, it consumed no less than 700 Mb of RAM, then after several seconds this amount decreased to 400. Needless to say, by this time the underpowered mobile devices already lost all their spheres. So the FPS numbers for them are not accurate, as they were rendering an empty cube with no objects inside of it.
How to explain this behavior, I do not know, but a fact is a fact: physics in Unity WebGL didn’t leave the best impression.
In Closing
The results of the testing turned out to be somewhat surprising. I hope this article will be helpful not only for users, but for the developers themselves.
Here are the links to the benchmarks. Please notice that the Unity links actually work, they just don't display the loader. And bear in mind that loading is much slower in Unity.
Feel free to test and share your own experience!
