Recent Updates RSS Toggle Comment Threads | Keyboard Shortcuts

  • Chris 4:38 pm on January 29, 2012 Permalink | Reply  

    Switching gears- time to make a game? 

    Several weeks have gone by without me updating this blog. To explain why, I haven’t put much work into my rendering engine so there wasn’t anything I considered significant enough to write a whole post about. There have been some changes to my SSAO code and I rolled back my shadow mapping code because I still wasn’t satisfied with the results, but that’s mostly about it.

    When I did put some time aside to work on my programming projects, I’ve mostly been doing stuff to take me away from graphics for a while. There’s a point where every game developer working on other projects has to realize- hey, how about making a game for once? So to get me going in a game-making mood, I’ve done other things, like downloading the Platform Game Sample and making my own changes and gameplay tweaks, and brainstorming ideas for my next game, which I will talk about now.

    Initially I was going to work on a Super Monkey Ball like clone, because the graphics and levels for it are easy to make. I chose Super Monkey Ball for inspiration it is more about quick reflexes and less about slow-paced puzzle solving, like for example, SwitchBall. But now that’s not what I’m going to work on first. Lately I’ve been wanting to make a futuristic vehicle racing game- more ambitious, I know, but that’s what I’ve gotten an appeal for lately.

    Here are two reasons that I want to make a futuristic racer. First, I want to make a type of game that’s not found much on the XBox 360 platform. Playstation has its WipEout series, and although F-Zero still seems to be missing in action after Nintendo’s GameCube, there’s still some good, fast racing action to be found in FAST Racing League, on WiiWare. That leaves the XBox 360, which seems to offer nothing similar to those other two platforms. The last time Microsoft had games like that were the XGRA series on the original XBox. XGRA was the successor to N64′s Extreme-G series. While it’s hard to make a futuristic racer without being subject to comparisons to F-Zero or WipEout, they were decent competitors to those games, and had their own twist in the fast and futuristic themes.

    In upcoming posts, you’ll be seeing more talking about the progress of my as-yet-unnamed racing game, and many of the current stages of development and problems I’ll be encountering. I think it would be more interesting to push my blog more in that direction, as I felt that it still hasn’t covered much actual game development as opposed to engine development. However, I will be using my graphics engine and see how well it does when put alongside some real game code. The racing game will be more in style of the Wipeout series, and I will be designing a lot of the graphics myself, but hopefully I can get some other people to help along as I make more progress in my game.

     
  • Chris 4:36 pm on December 26, 2011 Permalink | Reply
    Tags: ambient occlusion, screen-space ambient occlusion, ssao,   

    Applying SSAO to scenes 

    The spinning Buddha (but you can't see him spin here)

    This has been a topic of interest for such a long time for me, but I finally got screen-space ambient occlusion working in my engine. Click here to see it in action! As with most graphics rendering techniques, there are many ways to skin a cat, and SSAO is full of them. I have read through so many articles on SSAO, looking to find something that works for me, and that is easy to understand and refine. Any approach you take may or may not work immediately, based on what already know and what resources you have to handle it.

    Ambient occlusion is an easy concept to understand. To put it simply, concave areas, such as the corners of a room, will trap some rays from any light that shines on it, so the ambient light is somewhat darker than in other areas. Used in graphics rendering, this can really make it easier to see depth in different spaces, and it makes objects “pop” from the scene.

    SSAO render target only

    Original SSAO render target

    The factors involved in computing ambient occlusion are easy to grasp, but some of the math used in applying it is a bit out of my grasp. Admittedly I am not very sharp on integration in math, which comes into play for many rendering techniques. But at least my linear algebra is good enough, so I just need to work in those terms to find the approach that works well for me. So I finally came to this article on GameDev, which, true to its title, was easy to figure out and works well for nearly all situations. It includes an HLSL shader that can be applied with few modifications.

    To avoid repeating much of what the article says, this SSAO technique requires three important sources of data: normals in view space, positions in view space, and a random normal texture. The random normals reflect 4 samples picked from a preset group of textured coordinates (neighboring samples) which are rotated at fixed angles. The formula in the article attenuates the occlusion linearly, but you can choose to put your own formula if you want a quadratic attenuation, or cubic, etc.

    Tweaking the results

    A few changes were made to the original shader code to be compatible with my program. First, I don’t have a render buffer that stores view-space position, so the getPosition function needed to be replaced. We can reconstruct world space position from depth using the inverse of the camera’s view and projection matrix, and to get it into view space coordinates, multiply it with a view matrix:

    float3 getPosition(in float2 uv)
    {
    	float depth = tex2D(depthSampler, uv).r;
    
    	// Convert position to world space
    	float4 position;
    
    	position.xy = uv.x * 2.0f - 1.0f;
    	position.y = -(uv.y * 2.0f - 1.0f);
    	position.z = depth;
    	position.w = 1.0f;
    
    	position = mul(position, invertViewProj);
    	position /= position.w;
    
    	// Convert world space to view space
    	return mul(position, ViewMatrix);
    }
    

    Probably not the fastest way to get view space from depth, but this code is written with readability in mind. The output image should be four different-colored rectangles evenly dividing the screen, which are the float values of the positions as color. What these colors are depend on the coordinate system you’re using (which is important to know as we’ll soon see).

    After this, I still noticed that the ambient occlusion output seemed to be right, but the values are inverted, so I get a grayscale negative of what is expected. So just subtract the final occlusion value from 1, and we’re good to go:

    ao /= (float)sampleKernelSize;
    return 1 - (ao * g_intensity);
    

    But why do we need to do this? The reason is that the coordinate system used in XNA is right-handed, while the coordinate system in Direct3D is left-handed. The Z-axis usually points to the camera in XNA, meaning that the positive Z values are behind you, but in Direct3D they lie in front of you. The article was written with DirectX in mind, so users of XNA (and OpenGL if you choose to port the code) will have to invert the occlusion term when it’s returned. This corrects the output given from the normals flipped the other way in view space.

    Finally, I removed some of the calculations involved in computing the occlusion, which are the bias and occlusion intensity. The width of the bias didn’t really do anything that I can see any change, and the intensity has been moved out of the the occlusion function and done once in the very last line, which gives the same results as repeating the multiplication by the intensity for each sample.

    Final considerations

    Your mileage may vary with this shader. To get the best results you’ll have to experiment in tweaking the parameters. The radius variable would work well between values of 2 and 10, depending on how much you scale your objects. Values much higher than this will be expensive to compute. The occlusion is best seen with the intensity set between 0.5 to 1.5, and the distance scale kept low, between 0.05 and 0.5.

    SSAO comparison

    Left: without SSAO. Right: with SSAO and bloom

    Of course, you may want to apply your own blur filter to remove the noise from the AO render. This noise pattern is from the random normal texture, and it stays fixed to the screen when the camera moves. I was able to get reasonable framerates with a full-screen render and a Gaussian blur applied to the AO. Some light “halos” are visible as a result from the blur, but they are not large enough to really distract from the view. What’s especially important to know is that the normals from your normal map must be correct in order to get good results, otherwise objects will be darkened in odd places. But that goes without saying that we’d already notice strange lighting with incorrect normals.

    Sources

     
  • Chris 2:13 pm on December 6, 2011 Permalink | Reply
    Tags: PSSM,   

    Parallel-split shadow maps 

    Parallel-split shadow maps are here. Had some struggles with getting it to work today and yesterday. It made the Directional Lighting class huge, but I will factor it out later. As with the previous shadow mapping scheme, it uses the depth data of the G-Buffer so no geometry is re-rendered for the shadow map projection phase.

    This method that takes advantage of the G-Buffer in deferred rendering is, perhaps surprisingly, called forward shadow mapping. It compares the depths between the buffer at camera view and the buffer at the light’s view after it’s transformed with its projection matrix. Then it gets multiplied by the light term, and finally the diffuse color. I decided to skip blurring the shadow map but that can be done in an extra pass if needed.

    Forward Shadowing

    (By the way, in the above webpage, the links to the blurred images are incorrect. Add “blur” before the extension to view the blurred examples in full size ie. “main512blur.jpg”.)

    We still need to render all those scenes at different distances for all the frustum splits. I am using 1024×1024 render targets. That took a toll on the busy Sponza scene :( It’s now from 65 fps down to 45. Sparser scenes are still plenty fast, though- the scene in this video usually runs at over 100fps without a screen recorder on.

    At first I decided to split up the shadow renders into several passes. Not actual effect passes, but repeating the same rendering technique several times. For each pass, the shader would take in different parameters for the light’s view matrix, split distances, and corresponding depth map. Initially this rendered all shadow maps at the same starting depth (the near distance). I noticed an overlapping effect in the lighting, because the closest split was very bright and the farther ones were darker.

    This was a side effect of the light buffer accumulating color values, so no, this won’t work. The shadow renders need to be split. A basic depth conversion formula can convert the depth map to linear view space, simply being this one as found on this depth of field tutorial:

    float linearZ = (-camNear * camFar ) / (depthVal – camFar)

    Also, since camFar is always going to be 1, we can just drop the multiplication for the numerator.

    Eventually I was able to split the distances well but the shader still wasn’t clipping out of bound pixels very well. Also it had some strange rendering bug where the closer maps were showing dim shadows over the farther ones and clamping at odd angles. This was more apparent when the camera was completely facing opposite of the light’s direction.

    Finally I just bit the bullet and put all of the depth map rendering in a single pass. This fixed everything as we now don’t have brightness accumulation over the split regions and the view matrices are perfectly lined up. The shader is still disorganized however, with more branching in some places, more constants being loaded, as well as throwing in all four of the depth map textures to sample in.

    Ideally I would like to have reduced the need to select render textures or texel offsets by doing multiple light view matrix transformations at once. One time I was thinking, “it would be great if you could output multiple positions at once on the vertex shader, just as you can with render targets on the pixel shader”. Then I quickly realized that’s what the geometry shader does. Derp. Too bad it’s not available for XNA use. MJP says it brings crappy performance anyways.

    So here’s the start of my somewhat odd parallel-split shadowing function. There are several different ways to get the trick done, and this is how I managed it.

    float shadow = 1.0f;
    if (shadowing >= 1)
    {
        float shadowIndex;
        if (linearZ > cascadeSplits.z)
        {
            shadowIndex = 3;
        }
        else if (linearZ > cascadeSplits.y)
        {
            shadowIndex = 2;
        }
        else if (linearZ > cascadeSplits.z)
        {
            shadowIndex = 1;
        }
        else
        {
            shadowIndex = 0;
        }
    
        float4 shadowMapPos = mul(position, lightViewProj[shadowIndex]);
        float2 shadowTexCoord = shadowMapPos.xy / shadowMapPos.w / 2.0f + float2( 0.5, 0.5 );
        shadowTexCoord.y = 1 - shadowTexCoord.y;
    
        float shadowDepth = 0;
        float occluderDepth = (shadowMapPos.z / shadowMapPos.w) - DepthBias;
    
        if (linearZ < cascadeSplits.x)
        {
            shadowDepth = tex2D(shadowMapSampler[0], shadowTexCoord).r;
            shadow = LinearFilter4Samples(shadowMapSampler[0], 0.3f, shadowTexCoord, occluderDepth);
        }
        else if (linearZ < cascadeSplits.y)
        {
            shadowDepth = tex2D(shadowMapSampler[1], shadowTexCoord).r;
            shadow = LinearFilter4Samples(shadowMapSampler[1], 0.3f, shadowTexCoord, occluderDepth);
        }
        else if (linearZ < cascadeSplits.z)
        {
            shadowDepth = tex2D(shadowMapSampler[2], shadowTexCoord).r;
            shadow = LinearFilter4Samples(shadowMapSampler[2], 0.3f, shadowTexCoord, occluderDepth);
        }
        else
        {
            shadowdepth = tex2D(shadowMapSampler[3], shadowTexCoord).r;
            shadow = LinearFilter4Samples(shadowMapSampler[3], 0.3f, shadowTexCoord, occluderDepth);
        }
    }
    

    This code resides in the same function used to calculate directional lighting, and each light can be set to cast shadows or not. I recommend using a very low number of lights as the depth map rendering makes this process expensive quickly. Besides, unless you have some weird sci-fi setting with several suns, it just looks plain wrong when you have many directional shadows going on.

    As you can probably tell, I am using four different depth maps and four parallel splits for the whole render. There’s some branching involved, unfortunately, as I can’t pass anything but literals to sampler array indexes. However I was able to replace another if-else statement with just adding up booleans as numbers to get the index for the light view matrix. This code:

        float shadowIndex;
        if (linearZ > cascadeSplits.z)
        {
            shadowIndex = 3;
        }
        else if (linearZ > cascadeSplits.y)
        {
            shadowIndex = 2;
        }
        else if (linearZ > cascadeSplits.z)
        {
            shadowIndex = 1;
        }
        else
        {
            shadowIndex = 0;
        }
    

    was condensed into the code you see near the top of the last example:

        float shadowIndex = (3 -
            (linearZ < cascadeSplits.x) + (linearZ < cascadeSplits.y) +
            (linearZ < cascadeSplits.z));
    

    Edit: Turns out that it IS possible to do texture sampling with variable indexes. Just use tex2Dgrad instead of tex2D to use the samplers, and the program will happily compile the code with variables passed for the shadowMapSampler array. We won’t need to apply a rate of change to the geometry, so the last two paramteres are changed to zero.

    This gets rid of all the if-else syntax, and now the entire shadow lookup code is shortened, and looks much better. The code is now almost 1/3 its original size and there are less comparisons to do.

    float shadow = 1.0f;
    if (shadowing >= 1)
    {
        float shadowIndex = (3 -
            (linearZ < cascadeSplits.x) + (linearZ < cascadeSplits.y) +
            (linearZ < cascadeSplits.z));
    
        float4 shadowMapPos = mul(position, lightViewProj[shadowIndex]);
        float2 shadowTexCoord = 
            shadowMapPos.xy / shadowMapPos.w / 2.0f + float2( 0.5, 0.5 );
        shadowTexCoord.y = 1 - shadowTexCoord.y;
    
        float shadowDepth = 0;
        float occluderDepth = (shadowMapPos.z / shadowMapPos.w) - DepthBias;
    
        shadowdepth = tex2Dgrad(shadowMapSampler[shadowIndex], shadowTexCoord, 0, 0).r;
        shadow = LinearFilter4Samples(shadowMapSampler[shadowIndex], 0.3f, 
            shadowTexCoord, occluderDepth);
    }
    
    

    Basically, I grouped the far distances of the first three splits into a Vector3, and then compare them to the linear depth output for that pixel. X is closest, and Z is the farthest. For four different splits, 3 will be the maximum index. It follows that if linearZ is closer than the first split, the same is also true for the second and the third, so we start by adding up the total true statements and then subtracting that total from the maximum. If all statements are false, then the last split and light view matrix will be used, so the index stays at 3.

    Everything else is mostly standard shadow mapping work, and a simpler to read branching statement follows that determines what depth map to compare and sample from. But if there’s a way to clean it up some more, I’d like to know. The ComputeShadow4Samples is an adaptation of the manual linear filtering function available here. It is necessary for filtering these shadow maps since they are a Single 32 bit float format, and thus can only be interpolated after the shadow comparison has been determined. “0.3″ is just a way to attenuate shadow darkness so they don’t appear completely dark.

    From here on the the resulting pixel color just gets multiplied with the brightness output of the directional light that casts the shadow. I don’t know whether it’s more accurate to replace brightness with the shadow coefficient instead of multiplying brightness with shadow, but it still looks fine either way. So there you have it- directional lighting with the G-Buffer and shadow mapping in one fell swoop.

     
  • Chris 2:46 am on December 4, 2011 Permalink | Reply
    Tags: C#, garbage collection, memory allocation,   

    Cutting down on garbage collection 

    Another big update for the Meteor Engine – now it does almost zero garbage creation at runtime. The only exceptions are getting the current mouse and keyboard input- these always allocate memory- but since these are PC-only inputs and the PC hardly hiccups with a single 1MB collection, these exceptions are a non-issue. I figured that my engine needed a good tuning up in order to avoid any unexpected stalls, especially if I plan to port it to Xbox 360. And better to do it now before the engine becomes any more complex.

    Figuring out how to optimize for garbage collection has really helped me in writing more efficient C# code. Now, I’ll tell you that prior to using XNA, I have never ever coded in C# before. I’ve mainly been a C++ guy. And I only really got serious with trying out XNA this July, which means I have just about 5 months in seeing how the C# language works, with all its peculiarities of value types, reference types, and memory allocation. So with that said, I am far from the best guy to talk about how everything about C#, its CLR and .NET components work. Still, I have learned a lot so far, thanks to the XNA veterans at the App Hub Forums and also some of their respective blogs, and I will be learning a lot more in the time to come.

    As far as reducing the creation of garbage goes, it wasn’t actually too difficult. There were a few cases where I had to write my own functions to circumvent others that really had no way of avoiding memory allocation, but it served me to understand how to do these things on my own. I have tracked down several major causes of garbage collection in my engine:

    • String creation (usually for debug output)
    • Updating mesh transformation matrices for rendering
    • Creating arrays and lists immediately as needed instead of storing them for later
    • Calculating BoundingBoxes with CreateFromPoints() on each frame for culling and rendering
    • Using BoundingBox.GetCorners()  to update the view frustum for directional lights

    So not much to do, though of course some involved more work than others to fix. I had to discover these issues one by one, and I started with the one that’s the most obvious for causing this sort of problem.

    Creating strings

    This one was pretty straightforward, as string manipulation is one of the most reported causes of memory allocation in XNA’s realtime applications. Best to use the SpriteBatch.DrawString more carefully, but luckily with all the stern warnings on using Strings and StringBuilder objects, there are a few existing code bases that you can use to help you out. I eventually took to using Gavin Pugh’s garabage-free StringBuilder extension for formatting numerical values. To make sure I’d get rid of all the string problems, I stopped rendering and updating all other areas of the program. Then I simply put the class into my engine code, re-wrote a few lines in the debug display function, and it was ready to go.

    Mesh bone transformations

    Now came the first challenge, making the rendering code garbage-free. Here’s where GC.GetTotalMemory was going nuts for, as 1 MB of trash was being scooped up almost every second. As I said before, this didn’t create any noticeable stalls on the PC, but I’m not gonna take any chances with the memory-limited Xbox. So with paring down and commenting out code here and there, I found out that copying the bone transforms to a new Matrix array was not the best way to go. Instead of creating new matrices, I pre-allocated a Matrix array for all the bones in the mesh, and updated them there. Here’s the before code:

    		/// <summary>
    		/// Draw all visible meshes for this model.
    		/// </summary>
    
    		private void DrawModel(InstancedModel instancedModel, Camera camera, string tech)
    		{
    			// Draw the model.
    			Matrix[] transforms = new Matrix[instancedModel.model.Bones.Count];
    			instancedModel.model.CopyAbsoluteBoneTransformsTo(transforms);
    
    			foreach (ModelMesh mesh in instancedModel.VisibleMeshes)
    			{
    				foreach (ModelMeshPart meshPart in mesh.MeshParts)
    				{
    					Matrix world = transforms[mesh.ParentBone.Index] * instancedModel.Transform;
    
    					/* .... */
    				}
    			}
    			// End model rendering
    		}
    

    Here’s the improved version:

    
    		private void DrawModel(InstancedModel instancedModel, Camera camera, string tech)
    		{
    			// Draw the model.
    			instancedModel.model.CopyAbsoluteBoneTransformsTo(instancedModel.boneMatrices);
    
    			foreach (ModelMesh mesh in instancedModel.VisibleMeshes)
    			{
    				foreach (ModelMeshPart meshPart in mesh.MeshParts)
    				{
    					Matrix world =
    						instancedModel.boneMatrices[mesh.ParentBone.Index] * instancedModel.Transform;
    
    					/* .... */
    				}
    			}
    			// End model rendering
    		}
    

    The array of boneMatrices is easily allocated after the model has been loaded successfully.

    boneMatrices = new Matrix[model.Bones.Count];

    This allows for better separation of the data and the functions that process it. By the way, foreach loops shouldn’t be causing a problem with the iteration in this case, as the newer version of the CLR runs through foreach loops much better, as explained in this article about memory profiling. Nothing really should have to move to the heap here.

    List and array creation

    This one was just plain dumb on my part. Most of the garbage-creating arrays had to do with the fact that my modular rendering system depended on arrays to pass around render targets as inputs and outputs. As one shader component passes the completely drawn render targets to the next (usually one but the GBuffer needs to pass several), I was initializing a brand new array for the render targets to be returned by the OutputTargets property, on every frame. To my surprise, this wasn’t making the GC memory output tick as fast as others, but it still was a very obvious fix.

    All shader components are derived from the BaseRendere class, where OutputTargets comes from, but I kept overriding that property. Then I realized, well I just have base class to work with right there, why didn’t I just use that? So now I pre-assigned all the outputs to keep always them ready.

    		// GBuffer example
    
    		public override RenderTarget2D[] OutputTargets
    		{
    			RenderTarget2D[] rtArray =
    			{
    				normalRT, depthRT, diffuseRT
    			}
    			get
    			{
    				return rtArray;
    			}
    		}
    

    Now with no garbage:

    		// In the BaseRenderer class
    
    		public virtual RenderTarget2D[] OutputTargets
    		{
    			get
    			{
    				return outputTargets;
    			}
    		}
    
    		// In constructor for GBuffer shading
    
    		outputTargets = new RenderTarget2D[]
    		{
    			normalRT, depthRT, diffuseRT
    		};
    

    Also, passing multiple render targets as a series of parameters was also not playing well with memory. When setting them, just stick them all into a RenderTargetBinding structure instead.

    Bounding boxes and mesh culling

    Here were more unnecessary creations of new objects and referencing other ones for calculations. In creating temporary BoundingBoxes to make new transformed ones to go along with the mesh transformations, we are able to cull meshes easily. But those “temporary” boxes can be made less temporary if we just pre-allocated them into the custom model objects. Here is how my code looked like before:

    /// <summary>
    /// Cull meshes from a specified list.
    /// </summary>
    
    private void CullFromModelList(Scene scene, Camera camera, Dictionary<String, InstancedModel> modelList)
    {
    	// Pre-cull mesh parts
    
    	foreach (InstancedModel instancedModel in modelList.Values)
    	{
    		int meshIndex = 0;
    		instancedModel.VisibleMeshes.Clear();
    
    		foreach (BoundingBox box in instancedModel.BoundingBoxes)
    		{
    			BoundingBox tempBox = box;
    			tempBox.Min = Vector3.Transform(box.Min, instancedModel.Transform);
    			tempBox.Max = Vector3.Transform(box.Max, instancedModel.Transform);
    
    			// Add to mesh to visible list if it's contained in the frustum
    			tempBox = BoundingBox.CreateFromPoints(tempBox.GetCorners());
    
    			if (camera.Frustum.Contains(tempBox) != ContainmentType.Disjoint)
    			{
    						instancedModel.VisibleMeshes.Add(instancedModel.model.Meshes[meshIndex]);
    			}
    
    			meshIndex++;
    		}
    		// Finished culling this model
    	}
    }
    

    Now, the InstancedModel class will just keep a second array of BoundingBoxes to complement the first array of pre-transformed boxes, leaving me to just reference the model for culling instead:

    		private void CullFromModelList(Scene scene, Camera camera, Dictionary<String, InstancedModel> modelList)
    		{
    			// Pre-cull mesh parts
    
    			foreach (InstancedModel instancedModel in modelList.Values)
    			{
    				int meshIndex = 0;
    				instancedModel.VisibleMeshes.Clear();
    				
    				foreach (BoundingBox box in instancedModel.BoundingBoxes)
    				{			
    					instancedModel.tempBoxes[meshIndex] = box;
    					instancedModel.tempBoxes[meshIndex].Min = 
    						Vector3.Transform(box.Min, instancedModel.Transform);
    					instancedModel.tempBoxes[meshIndex].Max = 
    						Vector3.Transform(box.Max, instancedModel.Transform);
    
    					// Add to mesh to visible list if it's contained in the frustum
    
    					if (camera.Frustum.Contains(instancedModel.tempBoxes[meshIndex]) != 
    						ContainmentType.Disjoint)
    					{
    						instancedModel.VisibleMeshes.Add(instancedModel.model.Meshes[meshIndex]);
    					}
    
    					meshIndex++;
    				}
    				// Finished culling this model
    			}
    		}
    

     
  • Chris 2:48 pm on November 28, 2011 Permalink | Reply
    Tags: , HLSL, library,   

    Project overhaul 

    Well, I’ve gotten to the point where I’m satisfied enough with my engine to get some actual game coding done. There are still issues related to shadow rendering but those can be worked on alongside everything else. Right now I’m taking a look at everything from a top-down view and planning to trim down the engine to a leaner size. I will talk more about my game project in a later post… just don’t expect it to be groundbreaking or anything like that :P

    For the sake of completion, I don’t want to get carried away with a one-size-does-all engine that has more than what I truly need. This has been an ongoing learning project lasting over two months, and I feel like it’s come pretty far. The engine isn’t immensely huge- I am focusing on a lightweight design- but now that I’ve worked on it for this long, I have gotten a better idea of what works for my upcoming game, and what I can take out so I don’t wind up with something too big and general-purpose.

    The rendering library

    The biggest change I made in the past week or so is moving all the rendering code into its own library project. It can finally be considered something of an engine, or at the least, a rendering library. For so long I had just a MeteorEngineDemo project, which is the “game” that uses the renderer, and MeteorEngineDemo.Content which grouped all the effect files from the renderer, and meshes used by the game. Having looked at other open-source projects focused on rendering engines in the past, I knew that this wasn’t going to work in the long run. This is just modular programming 101 here. The effects and rendering logic are part of the engine and irrelevant to whatever content you’ll use for any program, and I gotta follow the DRY principle.

    After moving the rendering code into its own project, I now have a library project called MeteorEngine, which is referenced by the MeteorEngineDemo, and the library in turn references MeteorEngine.Content. That last one contains all the content that the engine needs to use (effect files, special meshes, etc). That way, all the content can be compiled and added automatically into a separate content folder that’s used by the MeteorEngine.dll at runtime. These projects will probably be renamed from “MeteorEngine” to “MeteorRenderer” since I’m not concerned with building something that handles all sorts of game logic, and to make it clear that it’s for graphics only.

    Now my workflow is greatly improved- I can add a reference to this library with any other project I’m working on, while still being able to update the library’s source, and all the updates are reflected in all the other projects referencing it. Of course, this leads to the downside of making sure that all the code in my projects will still work with the library, but I’m doing a fairly good job in separating the rendering logic from everything else.

    Variance shadow mapping didn’t prove flexible enough for my needs so I rolled back to using a standard shadow mapping technique in orthogonal space. The shadow is manually filtered thanks to some code from XNA Info. Also, the renderer now uses two HdrBlendable render targets, one for light accumulation, and the second one is for compositing the completed image (before post-processing). These gave me a bit of trouble, since they require a Point filter and they threw off my texture sampling states, causing crashes on startup or textures to display incorrectly. To my surprise, they didn’t seem to impact the rendering performance a whole lot. As an added bonus, I also switched to using the Nuclex font importer for drawing better-looking SpriteFonts.

    Performance

    So you’re curious to know how well it does, huh? Well, here’s a quick overview of my specs. My home/testing computer, which has a Radeon HD 4670 series video card, an Intel Core 2 Duo E4500 with 3 GB of ram. So it’s nothing close to being mindblowing and probably somewhere in the mid-range as far as video specs go. I also have a Macbook but unfortunately I can only use the Reach profile on there :-/

    Okay, so this pic isn’t HD resolution, but check out dat framerate:

    Don’t mind the zero triangle count in the readout- I gotta fix that soon. Since it’s light pre-pass, geometry is rendered at least twice. Three times actually, to project the shadow mapping. Post-effects added are FXAA, depth of field, and light bloom in that order.

    At 720p it’s still a respectable framerate at 124fps, but with the effects disabled and switched to deferred rendering mode. With effects enabled, it hovers just below 80fps.

    Rendering an entire Sponza model is slower- between 65 and 70 fps with one 2048 x 2048 shadow render target. This is mostly to do with the fact that it requires a lot more draw calls-  the model is made up of almost 400 different meshes, and culling each brute force also takes time. It may be more efficient to render if I merged some of the smaller meshes together and re-export the model, but I’d rather move on and create my own models.

    Cutting down the code to size

    My design for this library just didn’t spring up from the moment I started. Over a year ago, when I was learning from tutorials on DirectX, I came up with a low-level design for a rendering system. I drew it on a sheet of paper, emphasizing the most important connections with the components (sheet coming later!).

    The diagram includes some non-rendering components, the game state management (the AppState and Statemanager), which I added to better visualize how it integrates with the whole program. I also got too carried away with static classes, which are probably not necessary for the design. But this is the basis of the design for my current rendering engine, and now that I look back on it, I want to reduce it down to these bare components.

    Some of these components will differ from the original design. Effects will be part of a SceneRenderer class, not the Scene class. The entry point of the renderer (Display) will only exist to load up the SceneRenderer and RenderProfiles, and plug in Scenes and Cameras to it. Also, I won’t even need anything like a ResourceMaker in XNA, because the ContentManager and content pipeline take care of everything in there.

    To this end, there will also be a clearer separation of data and data processors. I already have this in place with Scenes and SceneRenderers, as the Scenes are just containers for lights and models. This allows me to make some further refinements to the code:

    • Separation of data and its processing systems
    • Changing data objects from reference types to value types wherever possible (on the other hand, it makes little sense to have value type objects that contain reference types)
    • Reducing allocation of memory as per one of the paths to minimize garbage collection. This would include, for example, less resizing of lists for meshes, visible instances, etc. A lot of this makes more sense on Xbox 360, which I don’t have, so unfortunately I can’t really test this to the fullest.
    • Condensing some related objects to a single, but still manageable system. For my game, I’m set on what specific rendering systems to use, so I could trade away some flexibility for more simplicity.

    Experimenting with more advanced HLSL is also an option. With classes fully supported in Shader Model 3.0 and XNA as confirmed by this forum post, it can make things more interesting for creating shader frameworks based around deferred rendering. With them we have more ways to manage reusable code and abstraction in shader functions. Here’s a really good article thats shows how function pointer behavior can be done- fully compatible with Shader Model 3.0! To those not used to OOP in HLSL it’s a big eye opener in designing shader code.

    So now that it seems like I’m on the final stretch of working on this as a standalone project, it’s time to battle-test it. I already made another project which will be the “new version” of my game, and threw in my rendering library along with it. The screenshots you saw above are taken from that project. For now it’s going to be a matter of adding my own content and whatever game code I could reuse from the old project. As I’m working on my game, the rendering library will also keep changing along the way.

     
  • Chris 2:12 am on November 24, 2011 Permalink | Reply
    Tags: , light pre-pass, skybox   

    Deferred rendering with skyboxes 

    XNA Dude has only one animation: "haters gonna hate"

    There is one simple but important rendering feature that helps make any game world more immersive- skyboxes. I have planned to add it for a while, but for a deferred renderer the solution didn’t come to me immediately.

    You mainly deal with rendered scenes that later get “flattened” to textures projected onto the viewport, so adding backdrops can’t be done as it is with forward rendering. Skyboxes have the problem of not needing to be treated with the same lighting effects as other objects. Also, they cannot interfere with the scene’s 3D space. I’ve read a few topics on the App Hub forums on how people handle backdrops with deferred renderers, and there was a lot of conflicting information. Should I use one depth clearing effect, or should I treat the skybox like a light? Or can two approaches be combined for more flexible usage? Fortunately the solution to was simpler than I expected.

    No special skybox class is used- it’s just another InstancedModel object which is like any other one, except that it’s not placed in the culling list.  So actually, we can just assume that it can be anything, either a box or a sphere or something in between. There’s no need to cull a skybox most of the time, and if drawn after everything else, we can reduce the fillrate since a lot of the screen will already be covered with the scene objects. The skybox can be scaled, positioned and rotated manually so that it will look right with the rest of the scene.

    What did come to me more naturally was the concept of viewport depth. By splitting up the viewport’s depth range, the skybox can be rendered separately from the rest of the scene without having it interfere or block other objects. With this in mind, a separate rendering method was used for only the skybox, which restricts the viewport to a very far distance. No big depth sorting tricks are necessary, we can just keep the GraphicsDevice.DepthStencilState to Default. We would use these steps to draw the skybox with everything else:

    • Clear the GBuffer
    • Set viewport depth range from 0 to n (some value very close to 1)
      If using deferred rendering:

      • Render the scene to color, normals, and depth in the GBuffer
      • Set viewport depth range from n to 1.0
      • Render the skybox with face culling disabled, only to the color of the GBuffer
      • Accumulate all lighting

      If using light pre-pass rendering:

      • Render the scene to normals and depth in the GBuffer
      • Accumulate all lighting
      • Clear the viewport, render the scene with just the color
      • Set viewport depth range from n to 1.0
      • Render the skybox’s color with face culling disabled
    • Combine color with lighting to draw the final image

    Everything looks right, except that the skybox is unlit, so nothing would still show up. We could increase the ambient lighting a fair amount, but that will make everything else too bright and washed-out. So instead a special case is made for depths that are greater than n to set the ambient lighting bright enough for the skybox to show correctly. This was simple to add into one of the lighting shaders, where depth can be sampled. To keep the skybox from moving, we just translate its position along with the camera. We also need to clamp the textures in the skybox so that no visible seams appear.

     
  • Chris 12:25 am on November 13, 2011 Permalink | Reply
    Tags: animation, , ,   

    Skinned animations and shadows now in place 

    The Meteor engine is starting to come into its own with two new important features. I have now intergrated skinned animations and shadows into the rendering. In comparison, adding some good looking directional shadows was fairly easy. There was just a lot of redundant drawing code to clean up and make it easier to adjust the shadows. The mesh animations are a different problem altogether. Both presented some good challenges for me, and still have problems I need to overcome.

    Skinned Animations and Blender

    The skinned animation feature is working but still underdeveloped- for one thing, they only link to the default (first) animation clip, and I only have the Dude model to test it on. This one was easy to import because it was already made for a skinned animation sample. But making another compatible model like this proved to be no easy task.

    I have a greater appreciation for what the 3D artists do, because getting models animated and exported properly has proven to be a huge hassle. Having took a course on it before, I’m decent with 3D modeling tools, including Blender. But actual modeling of props and characters is a huge time suck, so I’m just making small adjustments to models I found elsewhere.

    I have tried binding an armature (a skeleton in Blender) to another test model and have been unable to assign blend weights automatically. So instead I took the much longer route of manually painting blend weights to each bone myself. With some models this became tedious, because some have underlying nooks and crevices that makes it hard to paint properly, and I have to zoom and turn the camera at all sorts of odd angles.

    The issue is that XNA will not accept any skinned models if even one vertex has a weight of zero, meaning that all vertices need to have some influence to be moved by at least one bone. When having to deal with character meshes with several thousands of vertices, assigning weights automatically is a great time saver, but only when it works properly. I decided to just stick with the Dude model for now and make do with a simple flexible tube that’s not complex in shape. Right now I’m busy cleaning and improving the rendering code, but I’ll get back to the animated mesh problem eventually.

    Animating the Models

    J. Coluna’s recent rendering sample helped me with applying bone transformations on the GPU. It’s just a short set of matrix additions to apply all the transformations in the matrix stack. I created two separate vertex shader functions to deal with skinned and non-skinned meshes, the former having the extra matrix calculations. Skinned meshes and static meshes are grouped in different lists to organize the rendering. I suggest that you check his solution out if you want to do something similar with combining animation with your own shaders.

    So the animation code seems manageable, but my implementation is  clumsy. For every effect technique that renders a mesh, I copied that technique, with “Animated” added to the name, and replaced the vertex shader with the one supporting bone animations. This made my already huge GBuffer / uber-effects file a bit longer, and I have to constantly copy more stuff in it if I need to make a shader for another rendering style. Any missed techniques will make the program crash since it fails to find one with the proper name. This is not a big worry since I’m the only one working on the code, but I do want to use a better way to handle skinned models in the shaders and cut down on the code a bit.

    For now, though, my biggest hurdle on skinning is not technical- it clearly lies in the fact that I gotta learn how to use Blender more effectively to skin and animate whatever models I want to.

    Variance Shadow Mapping

    I briefly covered this topic a in the last post, and I was still experimenting with the results that it will produce when integrated into the engine, but after a lot of tweaking parameters, it’s finally starting to look great. To my surprise, the shadows did not fade out with close proximity to the shadow casters, so the objects don’t look like they’re floating. I’m okay with not having very defined shapes for the shadows all the time.

    You could still get some good quality rendering just a single 512×512 map with the Sponza scene, and with enough saturation, the edges look great without a lot of noticeable bleeding artifacts that come from using screen-space blur. With results like these, I could hold off implementing SSAO for a while. Here you can see a screenshot with only the directional light and shadows applied:

    Shadowing only works with directional lights for now. It has a “casts shadows” property which can be set to true or false, and all lights that have this turned on will be considered for drawing shadow maps. A cool feature is that, for artistic purposes, you can have the intensity of the shadows adjusted independently from the brightness of the light that produces them. This means you can dim or darken the shadows as the light angle changes, or even have a “dummy” directional light that is completely colorless (no light) but you can use its directional and distance properties to cast shadows overhead. It is more possible to create visually striking scenes this way.

    A Few Kinks in the Armor

    For now, I am using a separate Camera object to represent the light’s point of view, as with any shadow mapping technique. But the camera’s orientation is not yet tied with the light’s viewing direction. This means that the camera’s orientation is essentially useless and does not factor in at all with the rendering of the depth map and shadow projection. But I need this information to correctly cull the scene objects from the light’s point of view. As it is, shadows pop in and out as the “main” camera moves, because the shadow mapping effect is rendering only the objects as visibly seen from that camera. Once I figure out how to assign shadow casters and receivers for the appropriate light, that problem should go away.

    Another problem I noticed came about when I combined the skinned animation shader with the shadow mapping effect. Those animated models are giving me a hard time! Here, the Dude model is mostly dark, with light spots in random places, even when they’re not supposed to be light. They move a bit and generally stick around the same areas when he’s walking, which just look odd in the final render.

    At first I thought that I didn’t correctly calculate the normals for the shader, but then I realized that the shadow map projection doesn’t take normals into account at all, so there’s definitely something weird going on with how the shadows are being projected onto them.

    What Else I Can Improve

    Other than these major issues, I’m pretty happy with how these features brought more life into the renderings. There are some pending goals that I plan to take on next week, and at least get two of these done:

    • Better implemenation of cameras for directional lights
    • Possible parallel-split variance shadows, for even better quality
    • An option to use more traditional shadow mapping with some screen-space blur
    • Support hardware instancing of meshes
    • A straightforward .obj file splitter program to break up large .obj models into smaller files (I’ll release it to the public)
    • Frustum culling from the light’s point of view

    I’ll be sure to cover at least one of these topics soon, and continue providing samples related to these.

     
    • Neil 7:34 am on November 21, 2011 Permalink | Reply

      I’ve been leaving comments all over your site. As you know, I too am building an engine using Catalin’s tutorial. I would love to help out with this engine, do let me know if you’d like the help. I’m happy to sign any sort of license agreement etc. if needed.

      • Chris 3:49 pm on November 23, 2011 Permalink | Reply

        That sounds possible, but I do have a bit of difficulty juggling several projects. Do you have any place where I could see the progress of your engine? (the source code is not necessary) Also, I’m not yet sure what license I will release this project under, but I won’t completely rule out releasing the entire source, like at CodePlex.

        • Neil 4:33 pm on November 28, 2011 Permalink

          Check out http://project-vanquish.co.cc as this is where I’ll be updating my engine.

        • Neil 11:51 am on December 4, 2011 Permalink

          Hi Chris,
          Have you made a decision on whether you’d like a hand with this or not?

        • Chris 3:33 pm on December 20, 2011 Permalink

          No, sorry haven’t yet. Been also busy as well looking out for a job and wrapping up some freelance work, so I don’t think I’ll be able to take this on right now.

  • Chris 11:22 pm on November 6, 2011 Permalink | Reply
    Tags: variance shadow mapping, variance shadows,   

    XNA 4.0 variance shadow mapping 

    I’ve just updated and refined another code sample from XNA Community, from XNA version 3.1 to 4.0. This one is on variance shadow mapping, which is basically a way to get shadow maps that are filterable- that is, you can apply any kind of texture filter onto the shadow mapping image to give it a smoother look. Optionally, and usually, a Guassian blur filter is applied. Together, variance shadow mapping improves the visual quality of the shadows as well as giving more leeway to the size and number of textures needed to produce good results.

    In the sample code, the program uses one 1024×1024 texture to produce the shadow map, and applies a two-pass filtering technique for Gaussian blur. This blur is not done in screen-space, but because the original shadow map can be filtered, it is almost indistinguishable from a screen-space blur, which will produce leaking artifacts if done with normal shadow maps.. Most of the heavy image computation is done in this step.

    The shadow uses a 2-channel 32-bit texture for the depth map, in contrast to a single floating-point texture format used in conventional shadow mapping. This allows us to store two “moments”, which are simply the depth and the squared depth stored in the depth texture. From here we are able to calculate the mean depth and variance of the given pixel in the map. One noticeable drawback to variance shadow mapping is light bleeding among shadows where the shadow casters are of very different depths. An easy fix to reduce this effect would be to raise the shadow amount by a certain exponent, but raised too high and the shadows dampen too much.

    I plan to use some form of variance shadow mapping to my graphics engine, but in the meantime I’ll try to make improvements on it, in order to remove the light bleeding more effectively. But in the meantime, you can download current sample project here, compatible with XNA 4.0.

    References:

     
    • oldteen 8:59 am on November 7, 2011 Permalink | Reply

      (sorry for my bad english)

      Hello, I am trying to download this sample, but apparently your zip file is corrupted.
      For your sample about paraboloid reflections, it search for tga image in an absolute path, So I can’t launch the project :)

    • mtnphil 2:52 am on November 9, 2011 Permalink | Reply

      How do you find the look of the shadows generated with VSM? I see in your screenshot above that most of the boxes seem to be floating due to the lack of “shadow density” near the caster. I’ve had the same problems with my implementations, and I’ve never been satisfied with the look of VSM.

      • Chris 4:36 pm on November 9, 2011 Permalink | Reply

        I too find the lack of shadow darkness close to the casters rather unsatisfying- in other articles and papers I’ve read, their screenshot examples showed nothing of the sort. In the meantime, PSSM still manages fine for me, although the appeal for VSM for me is its soft edge shadows and perspective projection.

        [Edit] I found that raising the power of the shaded regions has helped in darkening the shadows closest to the casters.

    • David 12:20 am on January 25, 2012 Permalink | Reply

      Could you please reup the file? The Link doesn’t exist anymore…

    • Marc 1:20 pm on January 25, 2012 Permalink | Reply

      Hi

      Is there any chance you can put this sample back online? The link to download it is broken. Thanks

    • Chris 3:38 pm on January 25, 2012 Permalink | Reply

      Looks like the link expired, or it’s unfortunate removal from the recent takedown of file-sharing sites. I’m re-upload it to a more reliable place, maybe SkyDrive.

    • David 1:47 pm on January 29, 2012 Permalink | Reply

      Maybe you could post the new link here?

    • David 2:22 pm on January 30, 2012 Permalink | Reply

      Thank you very much, thats really cool :D

  • Chris 7:26 am on November 1, 2011 Permalink | Reply  

    Depth of field, revisited! 

    Note: This is an older post updated with a lot of new information. Some source code snippets are coming soon.

    I took a short break from my rendering project, for trying to keep my other coding skills in shape by trying to finish a simple game in a day or two. More on that to come. But right now, I just got done improving the depth of field blur, and combined it together with the light bloom effect.

    Also, a change of scenery seems to be in order, don’t you think?

    Spencer's Estate from RE5

    These models are not created by me, of course. I’m just using them for demonstration purposes.

    Applying blur in the depth of field

    The Depth of Field process currently requires two shaders- one to create a gaussian blur copy of the original composite image, and a second one to blend the original and blurred images depending on depth values. This shader takes two important constants, the “focal range” and “sharp focus”. It linearly interpolates between the pixel values of two render targets depending the values of these two constants. The sharp focus is a 0-1 depth value within the viewport range that will produce the image in its sharpest, where there’s no blending done with the blurred image. The focal range is the distance where the image becomes completely sharp to completely blurred.

    The background blur is a 13-tap Gaussian blur on top of it, applied twice. When applied once it looks a bit like a box blur. By decreasing the sampling distance and applying it twice, the blur appears more smoothed out. While this is less accurate to real life, out-of focus backgrounds, aesthetically speaking, look a bit nicer. Together, and with 15 large area lights, these effects keep the program busy at 45-60 fps. Some optimization should be done soon. Can’t imagine then, how much slower it will perform on Xbox 360.

    I improved the Gaussian blur by pre-calculating the steps and blur weights in the CPU. This code is borrowed from the GraphicsRunner variance shadow mapping example. These weights are calculated only once on the Blur Shader component of the rendering chain, during initialization. This adds more pre-shader work, which lets the pixel shader run with less instructions and more constant values to improve performance. There’s only one variable to adjust, which is the number of steps that you want to sample and calculate for the blur kernel. There’s also a constant that you have to change in the shader to the same value. Now, the shader component reads the number of steps directly from the effect’s sampleWeights parameter, so that all arrays are set to the same size automatically.

    Further filter improvements

    There are at least two ways I can think of to improve the look and performance of the depth of field. Apparently, you can also compute a N x N filter by sampling less than N texture lookups. This article on bloom filtering explains it in a bit more detail, but generally it depends on the not-really-expensive hardware linear filter for textures. It also has a nice chart for manually setting blend weights for filter kernels of many sizes. It would be something I might look into if I ever want to get a bit more performance from blur filters, but it’s working fast enough as it is. Notice that there is some bleeding of colors happening between the foreground and background, which is a result of soft focus:

    C. Viper in Spencer's Estate. It's almost like I'm playing UMvC3!

    Soft focus happens simply from the interpolation of the blurred image with the original image. If you ever used Photoshop, it’s similar to applying a Gaussian blur filter and then fading the filter by a certain amount. It is a good enough depth of field effect, but for more accuracy we would apply the depth of field effect by adjusting the sampling distance based on the Z-distance to the camera. That will not only improve the appearance for some cases, but also make it more efficient as we can compute that in only one shader, thereby reducing the need to pass render targets around. However, some people like the ethereal look produced by the soft focus, so how depth of field is applied can be a matter of taste.

    I will also just mention that I am not creating smaller render targets or otherwise downsampling the source images for this effect- there’s just too much texture swimming and flicker for my liking. What you see in the images are all full-size render targets being computed and blended. When done right, the results are pleasing to look at and computed quickly.

    References:

     
    • Neil 8:34 am on November 10, 2011 Permalink | Reply

      When are you looking at posting the source code for this?

      • Chris 4:19 pm on November 11, 2011 Permalink | Reply

        Are you asking for the blur effect source code? That will be in a few days, I just need to improve the syntax and readability of the code. It will be posted as a module, that is, it’s not a standalone project and you’ll have to integrate it into your work.

    • Neil 10:04 am on November 12, 2011 Permalink | Reply

      I was talking about the whole Depth Of Field code :o ) But, I’ll definitely keep checking back to see the blur code. I am currently following the Catalin Zima tutorial as well, but I am only a small portion of the way through. Where did you get the models that you used in your DoF scene?

      • Chris 5:14 pm on November 12, 2011 Permalink | Reply

        For the models, Google “XNALara” and go to the first result, it should be a forum post. Basically, there are a lot game models extracted by various people. Don’t think there’s any harm in using these models just for demos- there’s already lots of machinima out there. I did the work of converting them using their conversion tools and importing them into Blender, then exported them to .obj which is used by my program’s content pipeline.

        • Neil 4:55 am on November 15, 2011 Permalink

          That I will certainly do. I too have followed Catalin’s tutorial on Deferred Rendering but I’ve started with implementing SSAO. It works great. I have a question about your directional light. Are you using the directional light from Catalin’s tutorial, or do you have your own implementation?

          If you are using Catalin’s, could you post the Color that you are using for the daylight scene?

  • Chris 8:10 pm on October 25, 2011 Permalink | Reply
    Tags: bounding box, BoundingBox, ,   

    Bounding boxes for your meshes 

    While making progress with my rendering engine, one of my goals for this week is to finally implement some kind of frustum culling for the meshes. I could have taken the easier route by only using the pre-built bounding spheres with every 3D model loaded in an XNA program, but I wanted to get tighter-fitting bounding boxes instead. They simply work better for selection and picking, and plus more meshes are culled out of the frustum, which means less false positives and less geometry being rendered off-screen.

    Mesh bounding boxes

    Tank model with boxes enclosing all of its parts

    Today I have finally finished the first major step, creating the bounding boxes. Simply figuring out how to perfectly create the boxes proved to be a frustrating chore. It wasn’t really the formula to create a box from points that was the difficult part, but getting the correct set of points from each mesh. This required a proper understanding of the GetData method for the VertexBuffer object of each mesh part. I will show you how I obtained the data to create those boxes.

    There are so many requests online for wanting to figure out how to correctly build a bounding box for a mesh object, and a lot of those queries are answered with outdated information, or they don’t turn out to be the best case for that particular user. I’ve browsed through several solutions and a few code samples of how to create them, but they were not working for me. Sometimes the program crashes with an OutOfBounds exception, and other times, the boxes are obviously misaligned with the meshes, even after double checking that the correct transformations are in place. But I finally came up with a solution that used a combination of a few approaches to read and add the vertex data.

    Building The Box

    Bounding boxes are just simple geometric boxes, and be represented with two three-dimensional points, the minimum and maximum coordinates. The distance between those two points is the longest possible diagonal for the box, and the points can be thought of as the upper right corner in the front of the box, and the lower left corner in the back. These boxes are usually created as mesh metadata, during build time or when resources are initialized. It would be very costly to read the vertices and update the bounding boxes on every frame- besides, you should use matrix transformations to do that. Here is how we would usually initialize a mesh model:

    public MeshModel(String modelPath)
    {
    	/* Load your model from a file here */
    
    	// Set up model data
    	boundingBoxes = new List<BoundingBox>();
    
    	Matrix[] transforms = new Matrix[model.Bones.Count];
    	model.CopyAbsoluteBoneTransformsTo(transforms);
    
    	foreach (ModelMesh mesh in model.Meshes)
    	{
    		Matrix meshTransform = transforms[mesh.ParentBone.Index];
    		boundingBoxes.Add(BuildBoundingBox(mesh, meshTransform));
    	}
    }
    

    This would typically go in the constructor or initialization method of the class used to keep your model object, and all of its related data. In this case, we have a List of BoundingBox objects, used to keep track of all the upper and lower bounds for all meshes the model might have. Possible uses may be to do basic picking and collision testing, and debugging those tests by drawing wireframe boxes on the screen (which I will cover further in this article).

    You may have noticed the BuildBoundingBox method in adding to the BoundingBox list. This is where we will create an accurate, tight-fitting box for every mesh, and to do this we will need to count all the vertex data for all its mesh parts. It requires a ModelMesh object and a Matrix object which is the bone transformation for that particular mesh.

    This method will start out by looping through all the mesh parts to keep track of the maximum and minimum vertices found in the mesh so far, and returns the smallest possible bounding box that contains those vertices:

    private BoundingBox BuildBoundingBox(ModelMesh mesh, Matrix meshTransform)
    {
    	// Create initial variables to hold min and max xyz values for the mesh
    	Vector3 meshMax = new Vector3(float.MinValue);
    	Vector3 meshMin = new Vector3(float.MaxValue);
    
    	foreach (ModelMeshPart part in mesh.MeshParts)
    	{
    		// The stride is how big, in bytes, one vertex is in the vertex buffer
    		// We have to use this as we do not know the make up of the vertex
    		int stride = part.VertexBuffer.VertexDeclaration.VertexStride;
    
    		VertexPositionNormalTexture[] vertexData = new VertexPositionNormalTexture[part.NumVertices];
    		part.VertexBuffer.GetData(part.VertexOffset * stride, vertexData, 0, part.NumVertices, stride);
    
    		// Find minimum and maximum xyz values for this mesh part
    		Vector3 vertPosition = new Vector3();
    
    		for (int i = 0; i < vertexData.Length; i++)
    		{
    			vertPosition = vertexData[i].Position;
    
    			// update our values from this vertex
    			meshMin = Vector3.Min(meshMin, vertPosition);
    			meshMax = Vector3.Max(meshMax, vertPosition);
    		}
    	}
    
    	// transform by mesh bone matrix
    	meshMin = Vector3.Transform(meshMin, meshTransform);
    	meshMax = Vector3.Transform(meshMax, meshTransform);
    
    	// Create the bounding box
    	BoundingBox box = new BoundingBox(meshMin, meshMax);
    	return box;
    }
    

    A lot of important stuff just happened here. First is the setting up of vertexData, which is an array of VertexPositionNormalTexture structures. This is one of several built-in vertex structures that can be used to classify and organize vertex data. In particular, I used this one because my vertex buffer contains position, normal and texture coordinates up front, and no color data. It will help us determine where our position data is located, which is the only data needed to create our box.

    However, this is not enough to assess the alignment and structure of the vertex buffer. We also need to know the vertex stride, which is simply the number of bytes that each vertex element contains. This number will vary depending on how your meshes are created and what data was imported, and it can even vary with each vertex buffer. With this piece of info, stepping through the vertex buffer should now be straightforward, with the vertex stride ensuring that we get accurate data. The vertexData gets sent to an inner loop where we simply examine each vertex, seeing if we have found a new minimum or maximum position. By default the minimum and maximum are set to extreme opposite values.

    After the loop is done, we now have the only two vertex points that matter, and these are transformed by the mesh’s parent bone matrix. Finally, a new bounding box is created and returned from these two points. Optionally you can also choose to create a custom bounding sphere from the bounding box.

    Drawing the boxes for debugging

    Now with our boxes stored in place, let’s put them to some use. We are going to draw the bounding boxes that correspond to the meshes for each model. If they are drawn together with the model, the wireframes should hide behind solid objects.

    Every BoundingBox has a Vector3 array which represent the eight corners of the box. The first four corners are of the front side, and the last four corners are the back. We are going to use a line list to draw the 12 edges representing the box. Each line connects a pair of corners, and the following array will form the edges:

    // Initialize an array of indices for the box. 12 lines require 24 indices
    short[] bBoxIndices = {
    	0, 1, 1, 2, 2, 3, 3, 0, // Front edges
    	4, 5, 5, 6, 6, 7, 7, 4, // Back edges
    	0, 4, 1, 5, 2, 6, 3, 7 // Side edges connecting front and back
    };
    

    Now in the drawing loop, we will loop through the bounding boxes for the model, set the vertices and draw a LineList for those using any desired effect. This example uses a BasicEffect called boxEffect.

    // Use inside a drawing loop
    foreach (BoundingBox box in boundingBoxes)
    {
    	Vector3[] corners = box.GetCorners();
    	VertexPositionColor[] primitiveList = new VertexPositionColor[corners.Length];
    
    	// Assign the 8 box vertices
    	for (int i = 0; i < corners.Length; i++)
    	{
     		primitiveList[i] = new VertexPositionColor(corners[i], Color.White);
    	}
    
    	/* Set your own effect parameters here */
    
    	boxEffect.World = Matrix.Identity;
    	boxEffect.View = View;
    	boxEffect.Projection = Projection;
    	boxEffect.TextureEnabled = false;
    
    	// Draw the box with a LineList
    	foreach (EffectPass pass in boxEffect.CurrentTechnique.Passes)
     	{
    		pass.Apply();
    		GraphicsDevice.DrawUserIndexedPrimitives(
     			PrimitiveType.LineList, primitiveList, 0, 8,
    			bBoxIndices, 0, 12);
    	}
    }
    

    In practice, you should make sure that if you transformed the scale, position, or rotation of your model, to apply the same transformations to all the boxes as well. Also, remember that it is best to move any shader parameters that won’t change outside of the drawing loop, and set them only once.

    That’s all there is to it. This should render solid colored wireframe boxes, not simply around your models, but in all the meshes they contain.

     
    • Dave 11:08 am on December 1, 2011 Permalink | Reply

      Hi, thanks for this, it’s really great. I am having a small issue when I try to use this inside my game. Everything compiles but when it gets to the part of code used to draw the box outlines I get the error:
      ____________________________________________________________________
      An unhandled exception of type ‘System.InvalidOperationException’ occurred in Microsoft.Xna.Framework.Graphics.dll

      Additional information: The current vertex declaration does not include all the elements required by the current vertex shader. Normal0 is missing.
      ____________________________________________________________________

      And here’s the code:

      http://pastebin.com/GL4w2RH3

      Any Ideas as to why this is happening? Any help would be greatly appreciated, and thanks in advance.

    • Dave 11:14 am on December 1, 2011 Permalink | Reply

      And here is the file with the shader in it:

      http://pastebin.com/uKZPzKXG

      • Chris 1:29 pm on December 1, 2011 Permalink | Reply

        I’m just going by the error message mostly, but your vertex shader is expecting normal data for input, and the wireframe box uses VertexPositionColor for its vertex structure, which does not have normals. You’re passing a ‘NORMAL’ semantic, and you’ll probably also get an error due to ‘TEXCOORD0′. Check to see that you’re applying the correct technique in the shader, like ColoredNoShading which only uses the position and color.

        By the way, use Pastebin next time so you don’t have to leave long code posts.

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel
Follow

Get every new post delivered to your Inbox.