ECS is officially the bomb!

Over the last week and a half, I have been working on my own ECS framework. This is a side project away from my main voxel game, but it is something I wanted to do in order to be able to improve my productivity with making games more quickly. Inspired by Phil’s Bomberman tutorial, I have implemented my own Bomberman clone with my own made-from-scratch ECS framework (though some conventions and names were adapted).

My own framework has less code than Phil’s but as long as the whole game still works on top of it, I would prefer that. Because right now, I don’t need two Entities to check if they are exactly the same, or don’t need to serialize them for other purposes. And I certainly don’t need scripts at the moment.This framework has gone through two main iterations. They differed mostly in how Components are stored in the game.

As you may know in an Entity-Component-System framework Components are just simple data containers, which don’t have any game logic, but are mutable by the game logic code, which resides in the Systems. Entities in my framework do not exist at all as classes, but are rather just numbers implied in Components as you’ll soon see.

Framework Structure

The ECS framework in its current state consists of your typical Component and System base classes, which you can build specific Systems and Components from. Here are the important data structures:

  • A Dictionary of Component arrays indexed by enum. Usually a small amount of different types.
  • Component arrays which are accessed by index, so constant time here.
  • Systems take references of important Component arrays. Iterating them takes linear time.
  • List of Systems which are always executed in the order that they were added.
  • Static array of Messages in Systems. For now, they are just being created by the Input System. Could possibly be made non-static.

The main point of setup and entity processing comes from the SystemManager class, which stores a list of all the different Systems, and calls their Draw and Process function in the game loop.

This class also has an instance of an EntityManager class, which is passed to the System constructors. The EntityManager class is where all the Component arrays are stored (as the base Component class), and where the Systems get all the Components they need. Components are pooled at startup, setting each array to a fixed sized X for max number of entities (though in C# it’s straightforward to resize an array if needed).

The arrays themselves are in a Dictionary, using a Enum for Component type as key. They are arrays of base classes, but they added in as derived classes.

public Dictionary<ComponentType, Component[]> components { get; private set; }

// Add Tile Position components
components.Add(ComponentType.TilePosition, new Components.TilePosition[maxEntities]);

This makes it possible to re-cast them back into their derived classes, but fortunately we would only have to do this on startup. Systems get the arrays of Components they need upon initialization, cast to the the proper type.

// Inside a System that uses Collision and TilePosition components

Components.Collision[] collision;
Components.TilePosition[] tilePosition;

public CollisionSystem(EntityManager entityManager)
    : base(entityManager) 
    // Load important components
    collision = components[ComponentType.Collision] as Components.Collision[];     
    tilePosition = components[ComponentType.TilePosition] as Components.TilePosition[];       

No further casting is needed for entire arrays after this point. The only casting that is done while the game is running is for getting certain Components at a given index.

When an ECS is more like a CS

Not dealing with Computer Science, but dealing with Components and Systems only. There are no entities in the framework, or at least not as objects. There is no Entity class, but instead entity IDs are stored in the components themselves and also referred to indirectly by the array indexes. The Components are access sequentially by the Systems and you can be sure that any Components in the same location of their respective array together make up an entity.

The EntityManager also has as an integer variable, TotalEntities, for the total amount of entities active in the game. It tells each System how far into the Component arrays it should iterate. An entity is “removed” by replacing the removed entity’s components with the components of the last active entity in the array. TotalEntities is reduced by 1, and this is the new index marker to tell the EntityManager where it should add Components to make a new entity.

Since arrays are fixed size, the amount of entities should not exceed the size provided in the pool. You can usually easily test and find out what a suitable size is for simpler games. I want improve this in the future by making the EntityManager resize the arrays to a much larger size if it should reach the limit (which should generally be avoided anyways to maintain good performance).

Component Organization

In the first iteration, the framework had arrays of each Component type, as concrete classes. Each derives from a base Component class, but the arrays are set up as the derived Component classes. So you had arrays of different classes named spriteComponents, screenPositionComponents, etc. This was inflexible for two reasons. First, adding a new component type meant also adding code for it to do a type check in the function to “Add” an entity.

// Get proper EntityPrefab method
Type prefabsType = typeof(EntityPrefabs);
MethodInfo theMethod = prefabsType.GetMethod("Create" + templateName);

// Call method to create new template
newTemplate = (EntityTemplate)theMethod.Invoke(null, new object[] { nextEntity });

// Check every array for proper insertion 
foreach (Component component in newTemplate.componentList) 
    if (component is Components.Sprite) 
        components.components[ComponentType.Sprite][nextEntity] = (component as Components.Sprite); 
    if (component is Components.Bomb) 
        components.bomb[nextEntity] = (component as Components.Bomb); 
    if (component is Components.Collision) 
        components.collision[nextEntity] = (component as Components.Collision);

// Etc...

This has been improved since, and now adding Components to an array doesn’t require manually going through every possible Component type.

// Check every array for insertion
foreach (Component component in newTemplate.componentList)
    components[component.type][nextEntity] = component;

Enity Prefabs

Every game using ECS benefits from having pre-assembled entities to use right off the bat. It’s a logical way to plan the rules of your game and what kind of game objects it will have. I use a small class called EntityTemplate which stores a list of Components. A class called EntityPrefab contains different methods (CreatePlayer, CreateSolidBlock, etc.) to return a new copy of a template, and its Components are added to the pool.

You still have to invoke EntityPrefab methods since the methods are dynamically chosen with the “templateName” String parameter. I would like to replace it with just adding prefabs to a List of EntityTemplates, so you just select them from a list. In hindsight this should have been the more obvious approach but I was taking from Phil_T’s approach to making entity prefabs.

Getting into the Game

I will talk about this in the next post, since I’ve probably gone long enough already! Then I’ll be able to go into more detail on how the game uses the framework. But since the first draft of this post and now, I have also made some more improvements on the ECS code and ironed out some game bugs too. The game is getting closer to being playable!

SeedWorld (my voxel world engine) first update

So here’s what I’ve been working on for the past two weeks. It seems that I have wanted to do a procedural voxel style world for some time and now have just started, only 4 years late on the voxel/cube graphics trend 😀

I am still facing a lot of technical issues which is to be expected early in developing something, but still made a lot of progress. I finally have a octave noise function that I am very satisfied with, in creating those very believable rolling hills you see a lot in procedural landscapes. Here is the breakdown of the current technical specs of the voxel world generation.

  • Voxel data is discarded as soon as chunk meshes are made. Chunks store only vertex data at the minimum*
  • Far draw distance (I want to emphasize this in faster PCs)
  • World divided into 32x32x256 chunks, with an area roughly 2000×2000 in size for the visible portion
  • Multi-threaded support for voxel and mesh generation

Future specs include:

  • Material determines the attributes in game, color gradients, and sounds for interaction feedback
  • Persistent storage for voxels only in the player’s immediate surroundings*
  • Different biomes which affect the visuals and interactivity of the materials

*This supports interactivity for making the world destructible, but only where it makes sense (near the player), and keeps the managed memory footprint low.

Voxel World – First Attempt

Eventually, I will want to make some sort of action/adventure type of game with the voxel/cube engine once it is fleshed out well enough. So far it has been a mostly smooth experience to see what it goes into the code. To get a head start I picked out some pre-existing code to make 2D and 3D simplex noise. I quickly learned how the noise functions readily make a continuous texture without being repetitive, as it’s of huge importance when making a procedurally generated world.

I started working on this on Saturday the 6th, and made some decent progress by Sunday night, making a cube world with 3D Simplex noise and some mesh optimization to avoid rendering hidden cubes. The custom shader is very simple and low bandwidth, taking only a Vector3 for local position and a Byte4 for color. The mesh-building code also adds a shade of green to cubes that are visible from above, and the rest are brownish in color. This creates a somewhat convincing grass-and-dirt look. Finally I implemented some basic brute-force culling to avoid rendering invisible chunks. Quick and dirty procedural world!

Posted Image

Posted Image

Some problems I found with this voxel-cube generator were, I frequently ran into out-of-memory exceptions when I decided to go any higher than 192^3 cubes. I was splitting the world into 32^3 sized chunks but it didn’t really help out the memory problem. My guess is that the triple-nested loops used to calculate the noise values are wounded too tight, and it might benefit from single-dimensional arrays. Also, I was storing the chunks in a resizable list, but it makes more sense to have a fixed sized array to be able to keep track of them. Also, while interesting to look at, the shapes produced by 3D noise weren’t very desirable so I switched to 2D to make a more believable height map. From there, I will then experiment with some 3D noise to get some interesting rock formations and caves going on.

Improvements in Terrain Generation

When I was still tweaking with different combinations of noise patterns, I could only come up with very large smooth, round hills, or many little but very bumpy hills. No repetition, but very bland to look at.

I had the basic idea down- combine many layers of Simplex noise of different frequencies, offsetting the X and Y for each of them just a little. But I had a derp moment when I realized I should be reducing the amplitude (effectively, the height variation) as I increase the frequency for best results. JTippets’ article on world generation really helped here.

Here are some screenshots of various builds, in order of progression. Here is “revision 2” as it follows the first build mentioned in my last journal entry:

Posted Image

Already in revision 2 I have added optimized mesh generation to remove hidden faces. The wireframe render shows this well.

Revision 3 shows the vast improvements in terrain generation that I mentioned previously. The draw distance is improved, and noise patterns create much more natural looking hills and valleys. Color is determined by height variation and whether or not the block is a “surface” block. The white patches you see are sides of steep hills that don’t have the top face visible.

Posted Image

Between revisions 3 and 4 I was trying out ways to speed up voxel generation, mostly with octrees. That didn’t work out as planned, for reasons I will state later in this post. So I went back to my previous way of adding voxels. The biggest feature update here is simple vertex ambient occlusion through extensive neighbor voxel lookups.

Posted Image

Posted Image

It is a subtle update but it greatly improves the appearance of the landscape a lot. I applied the AO method that was discussed in the 0FPS blog. The solution is actually simple to do, but the tedious part was combining the numerical ID lookups for all the neighbor voxels so that each side is lit correctly. I should really change those numbers into Enums for voxel locations so the code is less confusing.

Here is a screenshot just showing just the AO effect.

Posted Image

It is around revision 4 when I also made a Git repo for the project, and it has also been uploaded to a private Bitbucket account.

Performance stats, you say? Unfortunately I am not yet counting the FPS in the engine and I believe my stopwatch use of tracking time for chunk updates is wrong, because when it reads 15 milliseconds (about 67 FPS) the program becomes incredibly slow, as if it was updating only twice per second, but at 10 milliseconds or less, the program runs silky smooth without any jerky movement.

What I can tell you, though, is that currently I am sticking to update just one 32x32x256 chunk per frame in order to keep that smooth framerate. At 60 chunks per row, It’s still quick enough for the world generation to catch up to movement up to around 25 blocks/second. This is throttled by a variable that I can change to tell the program how many “dirty” chunks per frame it should update. My processor is a Pentium G3258- a value CPU but still decent for many modern games (as long as they are not greatly dependent on multi-threading), especially since it is overclockable. I have mine overclocked to 4.2 Ghz. If you have a CPU that can run 4 threads, has 4 cores or more, you should be able to update several chunks per frame very easily.

About using octrees- I did not perceive any performance gains from using them so far. I wanted to use octrees as a way to better find potential visible voxels without the brute force option of going through all the voxels in the array. The good news is: I got the octrees to technically work (also did some nice test renders) and I also learned how to do so using Z-curve ordering and Morton encoding. At least I gained some interesting knowledge there. Bad news: reducing the amount of voxel lookups with octrees did not result in being able to quickly update more chunks per frame, which was the ultimate goal. So I am putting aside the octree-related code for now and maybe it will come in handy later.

Persistent local voxel storage concept, and future updates

The persistent storage for local voxels is definitely something I want to implement, and make a key feature in my engine. Keeping voxel data for the entire visible world is usually wasteful and it only makes sense really to know what you will see immediately around you. After all, if you have a pickaxe, you are not going to reach that block that is 500 meters away. This data storage will update as you move around the world, storing at the most 4 chunks worth of voxels.
This can be applied further with other objects that may interact with the world surface. Say you are a mage that can cast a destructive fireball. Upon impact, you want to calculate the voxel data for the area around the fireball so it can make a crater. Or an ice ball to freeze the surface. Obviously you want these calculations to be done very quickly, so it sounds like a good way to stress test the engine with lots of fireballs and who knows what else being thrown around.

Other more features I want to add soon are the creation of pseudo-random rock formations and measuring slope steepness which will help in generating other pseudo-random elements. Probably gonna add those voxel trees first, in order to add more to the landscape.

One Game A Month: Here comes a new game project!

What? Another game already? That’s right, but this one will not be as big as my racing game project, which I expect to be ongoing for several months and likely at least a year. No, this game will be a short-term project, only planned for one month as part of the One Game A Month quest. I want to get in the habit of finishing games quicker. (Maybe then I could rename the blog Electronic Meteor Games! Imagine that) I want a game I can make more quickly and easily, and just as well be leveraged by the coding experience I have gotten so far. So it will re-use some of the code I’m currently working on right now, but refactored to fit the needs of the game.

The game will be a twin-stick top down shooter. The idea may not be original, but carrying it out should be fairly easy. I do not have a name for it yet, only know at least some features in it will include multiple levels and upgradeable weapons, local multiplayer (not sure yet if I can finish online networking code in a month), and a cool lighting atmosphere for the graphics. So basically what one may expect from a top-down shooter. Characters and setting will be fairly abstract and basic. I don’t have much know-how for modeling human characters so it will be robots blasting other robots.

Here are the main goals I intend to follow for the month-long project:

  • Simplistic but nice to look at graphics and setting
  • Multiple weapons and enemy types
  • Controller support (gotta really get a controller for that though 😛 )
  • Learn some more AI programming as I go along
  • Use what I learned from Meteor Engine for the graphics
  • A lighting mechanic to hide/show parts of the map (somewhat of a “fog of war” for a shooter)

I have been mostly inspired by some of the fast-paced games being put up on Steam Greenlight to do a top-down shooter. It’s a genre that is simple fun and engaging for many people, and I believe that a (stripped down) top-down shooter can be another good game for budding programmers, comparable to platform games. So for this month, I will be slowing down progress of the racing game to work on this one.

On the AI side, I have been reading this set of tutorials to create a state machine. Many game programmers may be familiar with the game state switching pattern to code a complete game. These tutorials take it further in applying it to other ways, like setting up rooms for a dungeon crawler or computer-controlled AI characters that follow their own motives. The latter is the one I’m most interested in. I plan to implement the tutorial code for this game to give me a head start on the AI. It won’t be pretty but the functionality is what counts here.

For graphics, I mentioned the Meteor Engine, but I will not be using it as-is. Rather, the game will have its own graphics code that will take several ideas from the engine. It will be a trimmed down, sort of “lite” version of the engine code, using mainly deferred rendering for graphics. The intent is to provide a setting with many moving lights, and most outdoor daytime scenes aren’t good for that. Features include mostly dark rooms, bullets that light up the room along the path they take, reflective surfaces on characters and level objects, and point light shadows. A lot of the visual inspiration comes from the Frank Engine demo, so expect the game to look more or less like that.

I will code this with XNA, as usual, but I will also try to get it portable to MonoGame. I have been researching this for a while but attempts to port any of my code to other platforms haven’t gone well so far. MonoGame (in its current 3.0 version) on Mac seems to be a no-go with Snow Leopard, something to do with the Apple SDK’s not being up-to-date with what MonoDevelop uses so I would have to upgrade XCode to 4.2 which requires a Lion upgrade. Not up to doing that right now. So it will likely be on Linux before Mac 😛 The cross-platform support is not part of the month-long deadline, it’s just something I would like to do to take my game further like online multiplayer.

I would like to get started today with programming the game, if I want to finish it before the 30th. Just for today to use a placeholder model for the character, draw everything with basic graphics and make the character shoot in all directions. At that point it’s not very different logically from a scrolling shoot-em-up. So look forward for more posts related to my month-long game. It’s been a while since I actually release a game and I want this to be the most complete game I’ve released so far.