Open Hexagon

Open Hexagon

Not enough ratings
Shader tutorial #1
By VindowVipre
Shaders are easily the coolest feature added to Open Hexagon since its release. But they are hard to learn, so I thought I would create a little tutorial on how to add them to your levels. Enjoy! :)
   
Award
Favorite
Favorited
Unfavorite
What is a shader???
In the simplest sense, an Open Hexagon shader just consists of some code that runs on every pixel and returns an RGBA color for that frame. They use x and y coordinates, but also can use custom values passed in by your Lua script (such as the level time) to do cool stuff like animation. If you're already lost, don't worry, this guide will have plenty of examples to study.
Hooking a shader up to your level
Shaders come with their own collection of jank to deal with, so even getting them to work in your level can be tough to figure out. The first thing you need is to get is the shader's "ID" which is a large integer that the game uses to keep track of different shaders under the hood.
Let's suppose that you've written a shader and called it gradient.frag (you will do this in the next section of the guide):
gradientShaderId = shdr_getShaderId("gradient.frag")
The variable gradientShaderId now represents the shader that you are trying to add to your level. In order to make it start doing something, we need to tell the game which part of the screen to run it on, using a different function:
shdr_setActiveFragmentShader(0, gradientShaderId)
The first number in that function determines where your shader applies. Here's a table that shows you all the options. If it's not already in your utils, make sure to start using the utils.lua file from the base pack as a dependency.
RenderStage = { BACKGROUNDTRIS = 0, -- all background segments WALLQUADS3D = 1, -- wall 3d layers PIVOTQUADS3D = 2, -- hexagon 3d layers PLAYERTRIS3D = 3, -- player 3d layers WALLQUADS = 4, -- walls CAPTRIS = 5, -- hexagon PIVOTQUADS = 6, -- hexagon border PLAYERTRIS = 7 -- player }
Let's go apply our shader to the entire screen*, because why not! And if you are wondering where to put this code, go ahead and use onInit(), but it doesn't matter.
gradientShaderId = shdr_getShaderId("gradient.frag") shdr_setActiveFragmentShader(RenderStage.BACKGROUNDTRIS, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.WALLQUADS3D, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.PIVOTQUADS3D, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.PLAYERTRIS3D, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.WALLQUADS, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.CAPTRIS, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.PIVOTQUADS, gradientShaderId) shdr_setActiveFragmentShader(RenderStage.PLAYERTRIS, gradientShaderId)
* you unfortunately can not apply shaders to the timer, player trail, or other sections of the HUD.
Writing a simple gradient
That's enough Lua for now! But you will have to revisit it one more time later in this section (sorry). The programming language we will now be using to write shaders is GLSL. Start by going to your pack folder and creating a new subfolder called Shaders. Inside that folder, create a new text file called gradient.frag and open it in your text editor of choice.
Inside gradient.frag, create a function called main(). This will consist of the code that runs every time the shader is called. The variable gl_FragColor is what the output color of the pixel will be, so make sure it exists. And since colors in GLSL go from 0 to 1 instead of 0 to 255, an example shader that turns your entire screen blue would look like this:
void main() { gl_FragColor = vec4(0.0, 0.5, 1.0, 1.0); }
The output color should always take the form of a vector with 4 components -- red, green, blue, and alpha (transparency). In case you need a variable with less, you can use float, vec2, or vec3.

You should probably try doing this in your own level just to see if it's working. If it doesn't work, make sure your Lua file matches the earlier section, and restart the game. Also make sure that all of your numbers are specifically floating point numbers instead of integers (0.0, 0., ,.0, instead of 0).

Currently, the shader is outputting the same color no matter where it is located on the screen. In order to make a gradient from left to right, we need an important piece of information -- the width and height of the game window. You see, shaders use a Cartesian coordinate system, and each pixel knows exactly where it is located on the x and y axes. If we compare the pixel location to the overall resolution, we can condense our coordinate system down to just values from 0 to 1.

Go back to your level's Lua file. We're going to declare a uniform variable -- a variable that stays the same for all the pixels. It's going to be a vector with two components: the width of the screen and the height of the screen. Let's put it in onUpdate so that it is constantly updated.
function onUpdate(mFrameTime) shdr_setUniformFVec2(gradientShaderId, "u_resolution", u_getWidth(), u_getHeight()) end

Now, let's go back to the GLSL shader, and declare that value at the top.
uniform vec2 u_resolution; void main() { gl_FragColor = vec4(0.0, 0.5, 1.0, 1.0); }

Time to create the coordinate system we talked about earlier. Let's call it uv, and give it the form of vec2. And remember, the coordinates are normalized, so they go from (0, 0) in the bottom left to (1, 1) in the top right.
uniform vec2 u_resolution; void main() { vec2 uv = gl_FragCoord.xy/u_resolution.xy; gl_FragColor = vec4(0.0, 0.5, 1.0, 1.0); }

The x coordinate of uv gradually increases to 1 as you get farther along the screen... so what's stopping us from just setting one of the color channels to it?
uniform vec2 u_resolution; void main() { vec2 uv = gl_FragCoord.xy/u_resolution.xy; gl_FragColor = vec4(uv.x, 0.5, 1.0, 1.0); }
After refreshing the level, it looks like this:

We finally made a gradient! And a pretty cool looking one too. We can make the transition more pronounced by removing the 0.5 from the green color channel. Now it's pure [0,0,255] to [255,0,255].


If your level looks something like this, congrats! You've got past the hardest part. Here's some examples of what you can do without learning anything new:
Useful functions + resources
Now that you know how to work with a shader in your level, there's an infinite selection of options to pursue from here. The only limit is your imagination (and GPU).

Firstly, what if you want to make a gradient between any two colors, not just colors that have simple channel values? There's a great way to do just that, and it's called mix.

Mix is what GLSL calls the linear interpolation function, which returns an interpolated value between two values that you specify.

mix(20.0, 40.0, 0.0) would return 20, mix(20.0, 40.0, 0.5) would return 30, and mix(20.0, 40.0, 1.0) would return 40. It also works with vectors. Here's that same gradient from before, but now it smoothly transitions between any two colors with no effort on your part:
uniform vec2 u_resolution; void main() { vec2 uv = gl_FragCoord.xy/u_resolution.xy; vec4 color1 = vec4(0.000, 0.640, 0.486, 1.000); vec4 color2 = vec4(0.168, 0.247, 0.750, 1.000); gl_FragColor = mix(color1, color2, uv.x); }


It's cool how uv.x smoothly increases across the entire screen, but what if you want a bit more control? Allow me to introduce you to probably the most important overloaded function, smoothstep.
Smoothstep transitions between 0 to 1 over the span of two edges that you specify. It always returns a value between 0 and 1, no matter the input. And because it's cubic, the transition is noticeably smoother than a linear transition. I use it somewhere in almost every shader I create nowadays.

Here's a shader that creates 2 anti-aliased circles:
uniform vec2 u_resolution; void main() { vec2 uv = gl_FragCoord.xy/u_resolution.xy; vec4 color1 = vec4(0.136, 0.137, 0.145, 1.000); vec4 color2 = vec4(0.274, 0.735, 0.068, 1.000); vec2 location1 = vec2(0.4, 0.5); vec2 location2 = vec2(0.7, 0.7); float mixer = 0.0; mixer += smoothstep(0.18, 0.15, distance(uv, location1)); mixer += smoothstep(0.12, 0.11, distance(uv, location2)); mixer = clamp(mixer, 0.0, 1.0); gl_FragColor = mix(color1, color2, mixer); }

Because the first input of smoothstep is bigger than the second, the smoothstep curve gets flipped around and will output a transition from 1 to 0 instead of from 0 to 1. If you set the first two inputs of smoothstep to be the same number, the circle will not be blurred at all (and fun fact, this is exactly what the step function is).

If you try this out in your own level, however, you'll find that the circles aren't very circular -- they are stretched horizontally! But the distance function isn't broken, so what's happening? It's because the coordinate system we are using always has 0 on the left and 1 on the right, not taking into account the aspect ratio of monitors. This is great for a simple gradient, but it's awful for anything else.

One solution is to stop dividing by the x resolution, and instead have the x coordinates scale in proportion to y.
This will keep things in proportion just fine. The y values range from 0 to 1, while the x values range from 0 to whatever, on a 16:9 screen the maximum value of uv.x will be 1.777. This solution is just fine if your shader infinitely tiles the screen, but it's annoying if your shader is finite. For our two green circles, we definitely want them to be centered, to allow ultrawide screens the same experience. We could manually find the center of the screen with vec2(0.5 * u_resolution.x/u_resolution.y, 0.5), but there's a better way. We need to have the coordinate (0, 0) be the exact center of the screen. and center our shader around it.
vec2 uv = gl_FragCoord.xy/u_resolution.xy; uv -= 0.5; uv *= u_resolution.x/u_resolution.y;
A simplified one-line solution:
vec2 uv = (gl_FragCoord.xy - 0.5 * u_resolution.xy)/u_resolution.y;
This is what I start with in all of my shaders.

If you ever need a random number in your shader, well, it isn't quite that simple. But you can get pretty close with a function that has wildly ranging outputs for close-together inputs. That way it will seem random while still being repeatable on the next frame. Here's an example of a two-input, one-output rng function that I created on the spot:
float random(vec2 px) { return fract(sin(px.x * 1837.577 + px.y * 1949.897) * 9479.679); }

I wouldn't recommend using a function like this. It varies from computer to computer and you will start to notice repeating patterns as you use bigger numbers. Here's a link to a better set of hash functions written by Dave Hoskins: Hash without Sine[www.shadertoy.com]. But keep in mind that these require whole-number inputs instead of any random decimals.

Randomness and noise is it's own rabbit hole to go down, so to learn more I strongly recommend reading the last four pages of Patricio Gonzalez Vivo's online shader tutorial[thebookofshaders.com]. Or read the entire thing if you have time. It's a lot better than this guide.

Here's a glossary[thebookofshaders.com] and explanation (from the same website) for all the overloaded math functions you can use in GLSL.

Here's a YouTube playlist that I learned a lot from. Although it's pretty scuffed at first.

Here's a list of the written articles from Inigo Quilez[iquilezles.org], basically Mr. Shader himself. Most of it goes over my head but there are some crazy things to learn if you have the mind for it. And here's his YouTube channel.

And, of course, it's always fun to explore ShaderToy[www.shadertoy.com] to see the ridiculous art pieces that programmers around the world have created.
Rotation and Skew
Before I let you go, there's one last piece of critical information that is required to make some really cool Open Hexagon levels. How tf do you make the shader move with the level?? If you're using the shader for the background, it has to move exactly like the background panels or it will be incredibly bothersome.

Let's start with rotation. We can use a rotation matrix to rotate our entire coordinate system to the same angle as our level. In case you forgot high school linear algebra like me, the form of a rotation matrix looks like this:


We need the current rotation of the level to be passed into the shader, in radians. Back in our Lua code, we can place a line like this in onUpdate, onInput, or onRenderStage.
shdr_setUniformF(shaderID, "u_rotation", math.rad(l_getRotation()))

And here's how the matrix can be implemented into a shader. Keep in mind, this will rotate the coordinate system around (0, 0), so make sure that is located in the middle of the screen.
uniform vec2 u_resolution; uniform float u_rotation; void main() { vec2 uv = (gl_FragCoord.xy - 0.5 * u_resolution.xy)/u_resolution.y; float s = sin(u_rotation); float c = cos(u_rotation); mat2 rot = mat2(c, s, -s, c); uv = uv * rot; ...

If you're making a level without any skew, you can stop there. But don't make levels without skew. We can do better.

However, due to one of the many quirks of Open Hexagon, we can't just get the current skew of the level using s_get3dSkew(). The amount that the camera is tilted is basically the 3D_skew multiplied by a value that switches between 3D_pulse_min and 3D_pulse_max at a rate of 3D_pulse_speed. You could try to reconstruct what it would be at the current time, but honestly it's much easier to just manually control the skew in your Lua file, so that's what I'll show here.

In your style file, set both 3D_pulse_min and 3D_pulse_max to 1. If they aren't already defined, just add the lines somewhere as so:
"3D_pulse_min": 1.00, "3D_pulse_max": 1.00,
Now just use s_set3dSkew() in your Lua file to control the skew, and you can pass that value into your shader. Here's an example of an implementation that uses a sine wave instead of a triangle wave:
skewMin = 0.40 skewMax = 0.65 skewSpeed = 1.7 skewPhaseShift = 0 function onUpdate(mFrameTime) SKEW = (skewMax - skewMin) * 0.5 * math.sin(skewSpeed * l_getLevelTime() - skewPhaseShift) + (skewMin + skewMax) * 0.5 s_set3dSkew(SKEW) shdr_setUniformFVec2(gradientShaderId, "u_resolution", u_getWidth(), u_getHeight()) shdr_setUniformF(gradientShaderId, "u_skew", SKEW) end

Now, back in our shader, we need to know how exactly that skew value relates to the squished-ness of the y coordinates. I went and compared some screenshots a few months ago and found out that it's just a multiplication of the y values by 1/(skew + 1). A skew of 0 returns 1, and the output grows smaller as skew increases. Perfect.

Doing the multiplication before or after the rotation won't work, so we need to incorporate it directly into the matrix. We just divide the second and fourth values by that 1/(skew + 1) factor.
uniform vec2 u_resolution; uniform float u_rotation; void main() { vec2 uv = (gl_FragCoord.xy - 0.5 * u_resolution.xy)/u_resolution.y; float skewmult = (1.0 / (u_skew + 1.0)); float s = sin(u_rotation); float c = cos(u_rotation); mat2 rot = mat2(c, s / skewmult, -s, c / skewmult); uv = uv * rot; ...
And that's it! Hopefully you have learned enough to start creating some cool visuals now. If you still have any questions, let me know of them, and I will try to help.