How to optimize performance using instancing in a scene full of skinned-mesh?

Question

I’m working on a web tower defense game based on three.js. Now I’m stuck at optimizing the performance.

My game loads two GLTF models as enemy and tower, both have skinned-mesh. When the player creates a tower or the game spawns an enemy, I use THREE.AnimationUtils.clone to clone the loaded model. Then I add this cloned model to the scene. For animation, I use THREE.AnimationObjectGroup to animate all the enemies.

This results in an average of 370 draw-calls per frame in the performance test with the scene loaded with 45 towers and 70 enemies, which is a nightmare for the game.

I think maybe using instancing can optimize the performance because every tower and enemy share the same model and state in each frame, but only rotation and position are different. But after I studied some examples using instancing, there is no example using instancing with skinned-mesh. (There is a discussion here, but the result here doesn't mention any method with instancing.)

Is there any chance that this can be done with three.js, or some other solution for this situation?

Update

After researched more I found some concepts maybe can help me to implement instancing with skinned-mesh.

Concept

The original post here implement skinned-mesh with instancing in Unity. (It's written in Chinese, I translated the main concept in the following.)

After loaded a skinned-mesh, it has an initial state with all vertices (for clarity, each initial vertex denote as PLT in the following). In any frame of the animation, the final position of PLT (denote as PI) equals to a series of matrix multiplication PI = (M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_1 * PLT) + (M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_2 * PLT) + (...)

• M_bind_1 is the bone-binding matrix of bone 1.
• M_m_n means the transformation of bone m relative to it's initial state under the coordinate system of bone n.

For simplify, use M_f_i = M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_i to represent the transformation. M_f_i means bone-binding matrix of bone i after multiplication at frame f, so PI = (M_f_1 * PLT) + (M_f_2 * PLT) + (...) Once we know M_f_i, we can calculate the position of every vertex in frame f.

The process above can be done inside GPU by passing M_f_i which wrap as a texture. (Under the premise that the skinned-mesh needs to animate around 10 animations and less amount of bones, the require memory is about 0.75Mb.). Finally, we can pass different frame number f to each instance to render skinned-mesh with animation in one draw-call.

Implement with three.js

I haven't build an example code yet because I don't know the concept can work on WebGL or not (also I'm not familiar with GLSL), but I think the way to implement it with three.js can done as the following.

1. Follow here to get M_f_i.
2. Use THREE.InstancedBufferGeometry and THREE.RawShaderMaterial.
   • In uniforms pass initial geometry, M_f_i and texture.
   • In vertexShader process PI = (M_f_1 * PLT) + (M_f_2 * PLT) + (...).
   • In fragmentShader process texture (I have no idea how to do it).
3. Pass f and other instance attribute using THREE.InstancedBufferAttribute.

Problem

• Where is M_f and how to get it by THREE.AnimationClip in step 1?
• How to index each PLT (vertex in geometry)?
• How to deal with texture?
• How to deal with hierarchy Object3D (Object3D.children have THREE.Mesh and THREE.SkinnedMesh at the same time)?

I need someone to tell me this idea works in three.js or not, and how to solve the problem above.

Answer 1

我记得有一个几何或网格合并功能在过去确实帮助了我处理这种情况。我建议你朝那个方向搜索。

它的用法有很多对应项，例如失去您使用的每个 3D 对象的个性，但如果可能，您应该将它用于静态元素（例如环境对象），在其他情况下，如果您的单个对象/塔基于许多对象，它也可能很有用。单个 3D 物体成为一个整体......

根据我的经验（根据每台计算机和 3D 视口的大小可能会有很大差异），最后您的可见相机区域前面不应有超过 50 个（简单）3D 对象，并重复使用所有材质、几何图形、网格。 .否则，一旦游戏中发生有趣的事情，您的性能就会非常差。

希望能帮助到你！