Distortion Effects

Hi, it's time for a super rad tech update! Last week, I mentioned something about music...well, I needed some art to really show it off correctly (Art? For music?) so I did something else this week instead: distortion effects!

This update will include a fairly technical overview of my graphics pipeline. If you'd rather just see the cool special effects, you can skip straight to the bottom and enjoy the video.

A popular graphics technique is to warp sections of the screen. This can be used for all sorts of effects, such 

as shock waves, fish-eye lenses, translucent surfaces, heat hazes, etc. Here are some examples, taken from the first few pages of a Google image search for '2d distortion shader':

Step by step, I'm going to go over how I implemented the infrastructure in the engine for Super Obelisk. For clarity, I'm just going to go over the entire graphics pipeline, including the lighting and slabs (layering) components. One nifty little trick--and this is very useful during development--is the ability to save the contents of the current OpenGL frame buffer (i.e., render surface) to a file, so we can see what's being drawn behind the scenes before the final image is presented to the user.

The following image is our final result for this example.

Note the following features:

1. The scene has multiple slabs. The player is rendered on the bottoms slab. Some blobs are on the top slab.

2. Lights from the bottom slabs don't shine to the top slab. Lights from the top slab, however, shine to the bottom slab.

3. A distortion bubble on the lower slab is obscured by the upper slab (overpass). Also, the bubble does not show the contents of the upper slab.

4. The distortion bubble on the upper slab, however, contains the entire scene, including the other bubble on the lower slab! 

5. Two distortions on the right side of the screen represent a pure left shift and a pure right shift. Where the sprites overlap, there is no distortion, since they cancel each other out. This is kind of hard to see without zooming way in and looking carefully at the floor tiles.

And now, how we got to this scene. The engine gathers up all render data and sorts it into separate rendering layers. In this example, there are two slabs being rendered. I've removed the HUD for simplicity. Here's what happens:

1. Lights on both slabs are rendered.

Lights are on a special layer type and are drawn with a very simple shader which converts world position to screen position via a transformation matrix. The results are stored in the lights buffer. Note that the 'clear color' is the ambient color of the section.

Because we want lights from the upper slab to shine down onto the lower one, we render all lights that are 'on or above' the current slab.

 Here is what the lights buffer looks like after this:

2. Distortion sprites on the lower slab are rendered.

Next, we'll render the distortion sprites to the distortion buffer. The shader for these is the same as the lights buffer--a very simple world-to-screen transformation. I've actually slightly modified the output here for demonstration purposes only--since an alpha value of '0' is typical for pixels (keep reading to find out why), you wouldn't even be able to see most of the image if I didn't set set the alpha to '1' for this image. The result of this rendering:

3. The scene on the lower slab is rendered and combined with the lights.

At this stage, the bulk of the rendering occurs. All remaining entities (tiles, players, enemies, particles, etc.) are rendered onto another buffer, which we'll call bufferA. The shader program for this part of the rendering uses the light buffer from step 1 as a source texture. Each pixel multiplies its output with the corresponding pixel of the light texture. Although not pictured, this shader is also responsible for hue shifts like when an enemy gets hit and flashes different colors (or when a bomb is about to explode). Here's what we have so far; keep in mind there's something fishy about how the alpha channel is saved to the file, which is why the shadows look odd in some of these pictures:

4. Distortion is applied

Now, we take the output from step 3 (bufferA, in this case) and copy it to bufferB. The copy will use the distortion map from step 2 to warp the picture. The default case, where eveything (RGBA) is 0, will just do a straight copy of the pixel (no distortion). So, on bufferB, a pixel at (25, 10) would be sampled directly from (25, 10) on bufferA.

But if the pixel isn't pure black, we apply distortion on the X and Y axis

• Red represents a distortion to the left
• Green represents a distortion to the right. 
• Blue and Alpha represent a distortion up and down, respectively.

A popular distortion scheme is to use only the Red and Green channels of the texture for left and right distortion; Red values less than 0.5 (or 128 on a 256 value scale) represent a left shift, while Red values greater than 0.5 represent a right shift. Green controls the Y axis. Here is an image I found online that is used for that distortion algorithm:

The problem with this scheme is that it doesn't work well with additive blend, meaning distortions can't be overlapped with any meaningful result. For example, a solid sprite with RGBA = (0.4, 0, 0, 0) would be a slight left shift. But two of those overlapped with additive blend would be RGBA = (0.8, 0, 0, 0)--which is actually a right shift, instead of a stronger left shift!

My scheme, however, correctly intensifies the shift as sprites overlap--or cancels the shift out entirely if, say, the Red and Green components line up (which is exactly what the example image is showing with the distortion boxes!)

Anyway, here's the output of bufferB:

5. Render the lights on the next slab.

We clear the light buffer from step 1 and render the lights on the upper slab only. Lights from the lower slab do not affect the items we're about to draw!

6. Render the distortion sprites on the next slab.

We clear the distortion buffer and do the same as in step 2. Nothing fancy here. There's only one distortion sprite on the upper slab:

7. The normal scene of slab 2 is rendered onto bufferB, combined with lights from step 5.

Remember, bufferB represents the entire output of the first 4 steps. We're using the "painter's algorithm" here--we're going to draw right on top of all the other stuff we just drew. Note that we do not clear this buffer before this step, which is crucial. Here's what we have now:

8. Distortion from step 6 is applied.

Similar to step 4, we're going to apply distortion. Except this time, we're sampling from bufferB and writing to bufferA. For every slab we render, these two buffers are alternated--we need both of them, but once we're done copying from one to another, we don't need the first one any more, so we reuse it.

Note that the distortion sprites on slab 2 may sample distortion output from slab 1! So, sometimes you can see a ball reflected on another ball.

...and we're done!

So, what can this be used for? A little reflective orb here and there looks cool, but has limited usage. I put together a video of me walking through a few rooms showing different ways this effect can be used--and I've only had it for a couple days, so I haven't gotten around to discovering what other things I can do with it yet. Check out the video! Also, if you like seeing stuff like this in development--including horrible screenshots of when things go wrong--I usually put those on twitter, at https://twitter.com/ericswheeler .

Video:

Thanks for reading/watching and see you soon!