Circular Clipping Masks

Published 2023-06-12

I wanted to create a comprehensive post on how to create Circular Clipping Masks. This post goes together with a video tutorial I made which you can watch here:

But if the video doesn't work or you don't want to watch a long video, I want this post to make sense without it.

Pico-8 has this function called CLIP(). It restricts all drawing to a rectangular area on the screen. A Circular Clipping Mask is that but instead of a square area it's a circular area. The simplest version of this function is drawing the game inside a screen and making everything around it black.

There are many names for this effect. I call this a "Circular Clipping Mask" in this thread. But possible names could be also "Iris Out", "Circular Stencil", "Circular Matte", "Inverted Circle Fill", etc...

One typical application of such function is a cartoony screen transition at the end of a level like in Super Mario World. Another application is making a "dark level". You only render the environment around the player's character in order to simulate an environment with low visibility.

But this sort of functionality can go a long way. Some games hinge their entire game mechanic around it, like in case of independent titles like Closure or Schein.

Many devs find themselves in situations where they want a Circular Clipping Mask but Pico-8 doesn't provide straight-forward tools to make one. So in this post I want to discuss 4 different techniques to pull of this effect. I also want to invite you to post your own solution so we can build a repository of tools to tackle this recurring problem.

I want to show off code in a practical application scenario. So I decided to mod Jelpi's code to allow us to try out different approaches and to compare the results. Here is the workbench cart I will be using for this post. You can just download this or create your own:


NOTEIn the cart above all the Circular Mask code is in tab 1 for your convenience. This is a small change from the video. I hope this won't lead to any confusion.

Changes I've made

To test if this works I draw a 32 x 32 clipping rectangle around Jelpi using CLIP() in BEFOREDRAW(). I also draw a red circle in AFTERDRAW() that perfectly matches the clipping rectangle. Our goal is to find out ways how we can fill in the gaps between the circle and the square. Let's go!

This is the simplest solution for this problem. Just draw a big circle in the spritesheet and render it on top of the clipping rectangle to fill in the gaps. You need to set the black color to opaque using PALT(). You also need to pick some other color to become the transparent color so the center of the circle becomes see-trough. The result is something like this:

This is a small circle. You can make a bigger one but that will start eating into your spritesheet. So it pays off to make sure you're using the sprite efficiently. Instead of drawing the entire circle in one big sprite you can only create a quarter-circle sprite and draw it 4 times flipped vertically and horizontally.

This will give you more bang for your buck and you'll get a nice big circle.

But that circle is still relatively static. We can scale the size of the clipping rectangle and use SSPR() to make sure our sprites scale accordingly. This works fine but does result in some pixelation.

Your AFTERDRAW() function ends up looking like this. MYX and MYY are the coordinates of the center of the circle. MYR is the radius.

And here is the cart for you to download


In order to save precious sprite sheet space and make things look nice and smooth, how about we draw the circles procedurally? We can't use the CIRCFILL() function to fill the outside of the circle. But how about we draw a bunch of circles with CIRC() and increase the radius as we go? Well, this will get you something like this:

As you can see, this doesn't quite work. You get some ugly gaps between the circles. The algorithm that draws the circles doesn't create circles that neatly fit into each other. And you can't draw more circles in-between to fill in the gaps since the radius needs to be an integer number.

But you can fill the gaps by drawing more circles. Every time you draw a circle you just draw a second, identical circle shifted by one pixel in any direction. This results in a complete coverage.

Something you need to pay attention to is how many circles you draw. As the clipping rectangle gets bigger there are more pixels to cover.

So you need to increase the number of circles to do the job. Drawing around RADIUS/2 number of circles seems to be about right. The result is going to look like this:

Your AFTERDRAW() function ends up looking like this.

And here is the cart for you to download


Ok, time to put our big boy pants on. It's time for Math! This method is something that some devs would maybe consider "the proper way" of doing things. We are going to fill in the corners of the square by drawing the circle manually using a bunch of LINE() functions. We fill only in the areas we need line by line. However, in order to do that we need a mathematical function that tells us how to slice up a circle into individual lines. This sounds like it would use trigonometry but it's actually fairly simple. The following GIF / Cart examplifies:


I explain the Math more in detail in the video. But in short, we draw a circle with the center at coordinate 0,0 with the radius of 1. By applying the Pythagoras Theorem we can plug in 1 as the length of the hypotenuse (c). We can then solve by a or b to derive a function that descirbes that kind of circle on a 2D grid. For any given X we can calulate the Y using this formula:

This equasion is going to be our "reference circle". All we need to do now is do a for loop that will iterate along one of the edges of the clipping rectangle and keep drawing lines into the rectangle. We need to translate the screen coordinates into a value between -1 and 1 to look up where on the reference circle any given line is. We will get a number between 0 and 1 as a return. This number needs to get multiplied with the radius to calculate how long each line has to be to form a nice circle. Here is what this looks like in "slow-mo".

Of course you then need to also draw a second line on the other side of the rectangle but that's just using the same result, no need to go through the calculations twice. This method requires quite a bit of fiddling with the math to get it to work. But when you get it to work you'll arrive at something like this:

Your AFTERDRAW() function ends up looking like this.

And here is the cart for you to download


The true power of Method 3 becomes apparent if we explore it's versatility. Since we are drawing everything manually we have access to some stunning effects. For instance, we can create an effect where the outside of the circle isn't just a solid black color but a dimmer version of the image - fake alpha transparency so to speak. To pull this off we are going to take advantages of the Video Remapping functionality in Pico 0.2.4.

Here is how the effect works. Instead of filling in just the corners of the clipping rectangle, we will fill the ENTIRE SCREEN with our black lines. And instead of black lines we are going to use SSPR to draw thin, line-shaped sprites. We will use Video Remapping to use the screen as the spritesheet. So we will redraw the entire screen line-by-line back onto itself, apply a palette change and leave a circular opening out. This is what the effect looks like (first in "slow-mo" and then in real-time):

This is a lot to take in but here is a commentated AFTERDRAW() function that hopefully steps you through the process

And here is the cart for you to download


So far, our clipping masks haven't been "real" clipping masks. We just filled the outside with a solid black. A true clipping mask like the one you'll get with the CLIP() function should also allow you to combine two unrelated images. The previous step already paved the way to make this happen.

Finally it's time to set our DRAWMYBG variable from the Workbench to TRUE so the program will draw a cute heart background before drawing the Jelpi game. Our goal is now to combine the two images using a circular mask.

Here is how that will work.

This is what the effect looks like (first step 5 in "slow-mo" and then everything in real-time):

The code for this is actually deceptively similar than the previous one so instead of posting the entire AFTERDRAW() function again, I will just focus on important lines. You can look at the final code in the cart posted below.

And here is the cart for you to download


Method 3 was complicated. Method 4 sure SOUNDS complicated but as you will see the code is fairly simple and compact. Method 4 is all about exploiting 0.2.4 functionality to Video Remap and copy around screen contents to make Pico-8 do the heavy lifting for us. Ultimately what we're aiming to achieve is to use the regular CIRCFILL() function to draw a solid circle and "punch out" that circle by setting the circle's color to transparent while drawing it to the screen.

Let's begin with the simple black background version of it. Here is the plan:

This sounds complicated but this is what the entire AFTERDRAW() function looks like:

And it looks like this. Just a nice and clean Circular Clipping Mask.

Now this method has one disadvantage in that it takes a lot of processing power. We are copying and drawing large amounts of data for Pico-8 standards. So this may look like a mediocre trade. However, one huge advantage of this method is that we can have as many overlapping circular masks as we want at no additional cost. We achieve this my simply drawing a few more circles onto the spritesheet. Things can get pretty wild.

The advantages here should be obvious. This can be a useful technique if we have a game where we need more than just one circular mask at any given time. For example: a dark level illuminated by multiple light sources.

Here is the cart for you to download:


We can use similar techniques to the ones discussed in Method 3 to achieve the Fake Alpha effect with Method 4. Here is the plan:

This is essentially the same procedure as the basic Method 4 except instead of filling the spritesheet with black we fill it with the contents of the screen. We also shift the palette darker when we draw the spritesheet back onto the screen. The advantages of Method 4 persist. We can still use multiple intersecting circular masks.

Besides the already mentioned cases this could be potentially used for all sorts of decorative effects such as this canopy shadow effect in A Link to the Past.

Here is the cart for you to download:


And finally here is how to achieve a true Clipping Mask with Method 4. Which means we once again set DRAWMYBG to TRUE so the program renders the heart background before it renders the game. Our goal is to combine the two using the circular mask. Here is the plan:

If you've been following this post none of the methods should be new to you at this point. This is what the result looks like.

And here is the cart for you to download and look at the code


Here is the 30FPS CPU load I got on the above carts.

Method 3 appears to be the most efficient one. Method 4 is flexible and simple but the high CPU load means it's prohibitive to use in 60FPS unless it can be effectively optimized for it's use case. Generally, using the Fake Alpha and True Clipping Mask trick is heavy on the CPU and needs to be used with caution.

None of the methods I've described here are my own ideas. So at this point I wanted to give big thanks to all the people that I've learned those techniques from. The people I'm listing here aren't necessarily the original creators. They are just the people I've learned them from myself.

A good example for this is Method 1. I'm sure others had a similar idea before. But I've first seen it in use by one of my own students! @xCoraNil used this effectively in their game Night of the Worm Slayer.

Method 2 is something I've recently seen in a Twitter post by @sticky.

Method 3 I've first seen in in this excellent post on this subject. It was @freds72 who suggested doing it like this and I was pretty surprised about how simple the core math formula was.

Method 4 is something @NMcCoy suggested in a Twitter conversation not long ago.

This was a long post. Now it's your turn!

Let's build a repository of tools to tackle this recurring problem!