Procedurally generated meshes in Unity without garbage

Procedural mesh generation is very demanding both on performance and memory. Hence, it is very useful to know how it works under the hood and every trick on how to improve it to eventually, make better, stutterless experiences. I was motivated to write this post to clear up some misconceptions that I found all over the webs, be it official Unity docs, various posts, rants, and non-webs like things I was yelled at told are "absolutely correct". I will try to add as much as possible of what I have found over the years and present profiler-proof which is, of course, "absolutely correct". By casting this little bit of doubt, we can begin..

About this post

I expected to focus solely on '-without garbage', but I realized that I can't go straight to the topic without introducing the reader to important concepts like the garbage collector and what Mesh class really is, and hence, why is this post at all this relevant! In the process, hopefully, keeping the reader from making other mistakes in between the lines (of code).

In case you are an experienced procgen Unity developer and know what all this is about I should note that this post is filled with [bad] examples you might already know. If you only want to get to the point, then I advise you to just jump to the bottom of the post to the Conclusion chapter, it should be clear enough.

On the other hand, it certainly is a good thing if you already know how to make procedural meshes, because I will not focus on the "how to make" or "what does Mesh Renderer do?" in this post.. But it's not a requirement, since I am providing simple examples.

Additionally, note that this article focuses on memory, and not on performance, which requires definitely a separate post (-s, or actually a book!), so don't hold a grudge against me if this post doesn't show the most optimized way.

Wording: When I say 'mesh data' I mean the collection of vertices, triangles, uvs, normals, tangents, etc. In this post, I will only use vertices and triangles because that's the least amount needed to make a mesh. In case you want to add more "datas", you can just extend the same templates shown in the post to include those too.

A little bit on Garbage

As you should know, memory allocation and deallocation / garbage collection is a big problem in Unity and is usually the primary cause of stutters, and we really don't want to have them as they break the experience. This is true for every game, but especially games with fast paced gameplay where player reflexes play a large role; and VR games, which absolutely must run smooth on high frame rates to prevent motion sickness.

In my case, INFINLAND, which is a rally racing simulator, is an example of a 'reflex game', where I am putting extreme care to making it run very smooth. And seriously... Because the fans would kill me for a single stutter!

Another case is Thought of Train, which is a infinite train landscape watching experience. And since it's both constant-speed and VR, any dropped frame would be extremely jarring. This is the project where my quest for allocation elimination has started!

There is an extensive post about memory management and Garbage Collection (GC) in Unity which you should absolutely read first in case you don't know anything about memory and garbage as I won't repeat it..

But yeah, I know it's a huge post, so here, a naive advice in shortest short:

• Allocating and deallocating memory is slow! Garbage collection can freeze your game if a large amount of objects has recently been dereferenced
• Never allocate memory during active gameplay, and especially not every frame. Instead create objects at the beginning and pool - reference it and modify instead of destroy-create. Of course we can't eliminate allocations completely, the point is to do it once. 
• How do we know we are allocating memory? Look for the "new" keyword! Actually, be allergic to it! (ummm ..it's kinda weird to suggest being allergic to something tho)
• BUT... Don't be allergic to "new" keyword all the time, value types like structs such as Vector3 are not allocated to the same memory as objects.
• Many built in methods are sneaky and will allocate memory 'from within' without you being aware of it. For example, if they return objects or arrays, they most probably create them.
• and yes, arrays are also reference types and allocate memory when created!
• (actually, you should really read the huge post because I will go on pointing out random stuff forever!)

A little about Meshes

What we use to render meshes on screen and generate procedural geometry is the Mesh class. But it's very important to know that what you see as the Mesh class is actually a kind of a middleman (aka wrapper) and the meshes are actually implemented in native - C++ side of the engine. This is especially evident if we peek inside the decompiled C# code.

Mesh.vertices seems like a field, but actually, it is a get/set property! Now it makes sense why we can't just assign a vertex as easy as mesh.vertices[i] = somePosition. We must set it as a whole array!

Additionally, var vertices = mesh.vertices actually returns a COPY of the array, not the array as a reference that we can now change and see the mesh get immediately updated.

In my opinion, this is the major reason why people are confused about how to properly use the Mesh class. Good thing is that in Unity 5.2 Mesh.GetVertices() was introduced, which makes more sense, and by it clearly being a method, it kind of warns "be careful when calling this!" .. But the bad thing is mesh.vertices was still kept for, apparently, legacy reasons.

Lets start solving problems!

A worst possible scenario

As a visually obvious example - lets say we want to create a simple quad and wiggle its vertices every frame. We create a quad which has 4 vertices for each corner, 2 triangles connecting these vertices, and we can use Random.insideUnitSphere to add some random position to our vertices.

You can copy this script into MeshTest.cs and drop it on a GameObject in an empty scene. When you play the game, you should see this in front of you:

Don't be distracted by the beautiful magentas. Your automatic response probably tells you there must be something broken, but everything is fine! This is just because we didn't assign any material to the MeshRenderer, and for this post it is not important.

Now, it looks like it's all fine and dandy and you may proceed to make more complex meshes out of this template. But! we have made a few crucial mistakes!

If you run a 64bit build with code like this, your computer will probably soon begin slowing down and eventually stop responding completely. On the other hand, if you are running a 32bit build, you would soon meet with this nice dialogue:

Woops. Fortunately, this "Fatal!" error did not kill us, but this is obviously very very bad. Our game has taken all the available space in memory, and cannot allocate more. aka: we have a memory leak!

To see what is going on we should profile. Note that I am looking at a profiler attached with the build, and you should too. Running a game in editor and checking the profiler will show spurious values as there is a significant influence from the Editor itself. To do that, you should set Development Build and Autoconnect Profiler to true in Build Settings.

Now, lets start the build and look at the profiler. Scroll down and click on the 'memory' part, and we can notice something:

A steep increase of number of meshes, every frame, there is one more mesh!

Also notice GC Allocations per Frame is constantly 3B and the Mono value at the top is rising. This "Mono" value is how big our heap currently is (the part of memory that will be garbage-collected)

Well looking at the code, obviously, we are creating a new Mesh() every frame in Update. But.. If we did the same with arrays or most other objects, we would not have the same problem. What is going on?!

As I have mentioned at the beginning, when we stop using objects, as in - we don't reference to them anymore, they will be deallocated from memory. But unlike normal objects, Unity Meshes are a special case in that Unity considers them as "assets" and keeps them in memory... FOREVER! (Well, ok, not technically forever as assets will be unloaded if loading another level for example)

There is a few ways to solve this issue:

1. We can manually destroy meshes by calling Destroy(mesh).. Although this needs to happen after we have had submitted it for rendering;
2. We can call Resources.UnloadUnusedAssets(); to manually unload assets which are no longer in use; ..but these are not satisfactory as we still allocate memory every time, and both methods just cost much more than the simple, and best solution of all: 
3. Reuse the same Mesh!

Instead of creating an instance of Mesh class every frame, we create it once at start, and change the vertices and triangles of this one single mesh. Notice that we have also moved assignment to the meshFilter to Start() as there is no need to set the reference to one and only mesh every frame.

Lets look at the profiler again:

Notice the now steady brown line, and, most importantly there is only 1 Mesh! Mission accomplished!

But, also looking at "GC Allocations per Frame" at the bottom, we still allocate 2B every frame, adding to the "Mono" at the very top. Next, we will fix that too.

Pooling data into arrays

Looking at our code once again, it is evident that we are creating arrays of vertices and triangles every frame too, and as arrays are objects too, they are allocating more memory, and we see those 2B every frame. Just like we have referenced the Mesh and created it once at start, we can do the same thing with vertices and triangles

Lets look at the profiler now:

Wow, 0 GC allocations per frame. We can see only a small spike in the first frame, due to the Mesh and arrays being initialized. We also notice the "Mono" stays steady and doesn't rise anymore.

Notice that we, of course, didn't need to assign triangles every frame, as they never change. We could've also micro-optimized it by caching vectors, but after all, this is a memory talk not a performance talk :)

But.. What if we need to change number of vertices?

Now you may be thinking that since we can't change the array size without making a new array, this method fails as we surely in that case MUST create new arrays, hence allocating memory.

However, there is a solution for that, and it is to use Lists instead of arrays..

Pooling mesh data using Lists 

A little bit about lists

Lists? But they are slow and make garbage! (..ppl say)

Calling AddStuff() will indeed allocate more memory. This is because the List is being extended in memory on the moment of adding new objects.

However, if you initialize a list WITH CAPACITY, this will allocate a chunk in memory for enough data to be stored within this capacity. So when creating a List with capacity 10, the List will reserve 10 places and calling .Add() will not extend the list in memory.

The problem here is of course that we need to know in advance the maximum count. In case the list goes over capacity it is still not a fatal issue, the List will simply extend its capacity automatically. ..Except this means more space in memory will be allocated, which is what we are trying to avoid here. You can always put some bigger number to make sure memory doesn't get allocated later

In case we want to regenerate values, lists can also be 'refilled' by calling list.Clear() and adding again. Note that list.Clear() does not actually release this memory, but just empties the List, so GC will not be called.

Now why is this relevant for our mesh-making??

Unity 5.2 has introduced a bunch of methods for setting data using lists: mesh.SetVertices(), mesh.SetTriangles().. etc that can take Lists as an input. While this may look like a redundant feature at first, and you must surely be thinking (as I did initially) that it converts to an array internally, it actually was introduced for the very purpose of preventing unnecessary allocations in case we want to resize meshes.

To demonstrate that resizing data will not allocate additional memory, lets create a random number of quads every frame - in a single mesh.

You should now be seeing something like this:

Now, lets look at the profiler

Absolutely no allocations in update even with flashing quads and Lists and changing vertex numbers!

Now, if lists are more flexible, why not use them in both cases?

Again, not here to talk about performance, but it's good to note that indexing, adding, clearing lists, is a bit slower than the same with arrays. As meshes tend to be large this may have a significant performance impact. But, always profile!

Modifying authored meshes

Sometimes in procgen, we don't want to create meshes from scratch, but modify imported meshes that we modeled externally. For example, to make a road, we can stack multiple hand-modeled meshes and procedurally bend them along a spline. In this case, it is very important to know when we copy and how we handle object meshes.

It is very important to make a distinction between the get properties of meshFilter's meshFilter.mesh and meshFilter.sharedMesh!

• meshFilter.sharedMesh points to our actual mesh asset that we imported, makes sense, since it is the mesh that is shared between all objects using this same mesh.
• meshFilter.mesh actually creates a copy of the mesh, but then saves the reference to this copy in an internal field. This means that unlike our example with making infinite meshes and going out of memory, this one whil not actually allocate more memory so you are safe calling it in update. But you should still call it at least once in Start() to prevent allocating memory later on.

To explain this distinction better, here is an example where we will use .sharedMesh:

Attaching this code on a GameObject with a mesh, we should get see something like this:

And yeah, everything looks as intended, but then we STOP the game mode and see...

WOOPS, we changed the asset itself!! Now it's completely ruined, and I was stupid enough not to have a copy of the file! Nooooo it's gone forever!!! Boohoo!

...Actually.. Don't cry, it's still there! The file is still there and unchanged. We can actually reimport the mesh (right click on the file and "reimport"). The reason we can do that is that we haven't touched the imported file, what we touched was the mesh that Unity made according to that file and import properties. ... Yeah, sounds complicated, I know..

What we really want to do in this case is make a copy of the mesh at start and then modify this copy. Therefore, we want to use meshFilter.mesh in Start() instead.

This will not just make a copy but also assign it to this meshFilter. Taking a look at the inspector while in play mode, and you can see it's name now turned into "-instance". This means we are working on a copy rather than the asset itself.

So, why would you ever use .sharedMesh then?

Well, there could be instances where you for example generate a mesh procedurally but you want multple objects to share this mesh. In this way you could 'pick' this mesh from one meshFilter.sharedMesh and assign it to another meshFilter.sharedMesh, without allocating any additional memory.

Sharing meshes is not only better as you use less memory, but also allows instancing on the GPU, in case you are using a shader that supports it.

.mesh vs .sharedMesh epilogue

As you can see, meshFilter.mesh and meshFilter.sharedMesh suffer from the same problem as mesh.vertices.. They are properties that do things that we don't naturally expect and, as I've shown, it is really important to know the difference. Just think about it: meshFilter.mesh makes a copy, renaming it to "myMesh Instanced", sets it to an internal field, and assigns it to the meshFilter, and in case it has been already called it will return this mesh. That sounds a bit too much for just a property, wouldn't have been better if it was, for example, meshFilter.GetMeshCopy()?

Actually, I personally use meshFilter.sharedMesh 99% of the time, and if I ever need to copy meshes (which is actually rare), I prefer using (Mesh) Instantiate(mesh) and storing it in my own field. This is just to make sure I won't get things confused and generate additional garbage inadvertently.

Some more tips?

What I prefer doing when doing procgen is to make my own custom Mesh class that stores all the data in arrays (or lists) as fields. This helps from keeping the data arrays/lists all over the place and makes access for modifying data much easier. Another benefit is that it is also threadable, which is something that Unity's Mesh isn't. And after you complete assigning and editing data, you can just 'upload' and Unity Mesh will be updated (outside of the thread of course). Although this may sound ridiculous since we are making a wrapper of a wrapper, it makes sense for productivity's sake.

Here is example of my TMesh (standing for threadable mesh)

Obviously, I only have the vertices and triangles here, so it does not seem like much of a benefit to do it this way, but if you have more data like multiple uvs, normals, having a 'wrapper of a wrapper' helps a lot. Plus, you can also add more methods to the TMesh class to make your life even easier.

Conclusion

• Look at the Mesh class as a wrapper to the native side of the engine, hence everything that looks like a 'field' is actually a property. For example, mesh.vertices returns a COPY of the vertices array, allocating more memory in the process.
• Pool everything! Mesh objects and arrays of data should be made once and then modified by accessing those. Creating them during gameplay will allocate memory and call GC.
• In case you won't change the size of data (vertices, triangles, uvs..), pool them into arrays. Set them with 'mesh.vertices =' or mesh.SetVertices().
• Otherwise use Lists initialized with capacity of predicted maximum size and use 'mesh.SetVertices()'
• Don't 'forget' about your Mesh objects, because even when they are not referenced, they are still kept in memory. You'll have a memory leak!
• Know the difference between meshFilter.mesh and meshFilter.sharedMesh.
• .mesh will make a COPY when called for the first time, calling it every following time will return this copy. So, call .mesh once at Start() to prevent it allocating memory later on
• .sharedMesh will return this very mesh, so modifying it will change all other objects that reference this mesh (so in case it points to an imported mesh, you will actually modify the asset, which can be confusing)
• Consider making a custom wrapper for the Mesh class to store all this preinitialized Mesh and data into fields that can be easily accessed.
• Profile! And profile in build!

That's about it..

This is probably a topic I will update regularly and write more about. As I said, performance is another important topic, and with the new job system (that I haven't touched yet) there is more opportunity to analyse how procgen will be done in the future.

If you've liked this post, follow me on twitter, which is the first place you will find any updates of what I am writing and working on!

And have a happy procgen!