Standalone How the XNB shaders work

mrkite77

Terrarian
Someone asked me how the XNB shaders worked, so I figured I'd post here, just to help anyone else who needed it.

A while ago I wrote a quick little command-line program that can disassemble xnb shaders. You can download the source here.

If you disassemble the TileShaders.xnb, you'll see a lot of definitions that describe techniques and passes.
If you scroll down you'll see multiple Resources. Each Resource is a shader for a different paint color.

Near the bottom, you'll see the shader for Pass2, which is the Red paint. We'll use that one as an example.
It's small, so I'll put it here:

dcl v
dcl t.xy
dcl_2d s
texld r, t, s
max r1.x, r.x, r.y
min r1.y, r.y, r.x
add r1.w, -r.z, r1.x
mov r1.z, r.z
cmp r2.x, r1.w, r1.x, r1.z
cmp r2.y, r1.w, r1.z, r1.x
min r.yz, r2.y, r1.y
mov r.x, r2.x
mul r, r, v
mov oC, r

This shader (like all pixel shaders) is called repeatedly for each pixel in the texture.

The first 3 lines define variables passed in from the program. v is most likely the lighting color (we'll ignore it),
t.xy is the XY coordinate for the current pixel we're shading in the texture, s is the texture.

texld sets the vector 'r' to be the RGBA value of the current pixel in the texture.

Now the rest of the code refers to components of 'r' as 'r.x' and 'r.y' and 'r.z' but since we know that r is actually an RGB value,
those are misleading. r.x = red, r.y = green, r.z = blue (and r.w = alpha if the shader actually cared about the alpha channel)...
so in all future references I'll translate "r.x" into "red"... and so on.

Converting the rest into pseudocode, we basically have:
r1.x = max(red, green)
r1.y = min(red, green)
r1.w = r1.x - blue
r1.z = blue
if (r1.w >= 0) r2.x = r1.x else r2.x = r1.z
if (r1.w >= 0) r2.y = r1.z else r2.y = r1.x
blue = green = min(r2.y, r1.y)
red = r2.x
r *= v // apply the lighting.. we're ignoring this, but i translated it for you anyway
outputPixel = r // oC = the final RGB for this pixel

Shader code always assumes that red/green/blue are values from 0.0 to 1.0.
Which is why you can apply the lighting by multiplying two vectors together.

It's then best to go back through that pseudocode and clean up the variable names to make it clearer what is actually happening.
You might then realize that those if statements basically just check to see if blue is greater than the max or lower than the min
and then assign the appropriate output.

The whole thing boils down to:
out_red = max(red, green, blue)
out_green = out_blue = min(red, green, blue)

and that's how red paint is applied to each pixel of a tile.
 

Jeckel

Terrarian
I've been looking at the decompiled shaders and this thread was very helpful. I do have one question though, each resource definition starts with a comment string that is, mostly, composed of hex values. Do those values represent something that wasn't decompiled or do they serve some purpose in the form they are in?
// "CTAB\x1c\x00\x00\x00K\x00\x00\x00\x00\x02\xff\xff\x01\x00\x00\x00\x1c\x00\x00\x00\x00\x00\x00 D\x00\x00\x000\x00\x00\x00\x03\x00\x00\x00\x01\x00\x00\x004\x00\x00\x00\x00\x00\x00\x00s0\x00\xab\x04\x00\x0c\x00\x01\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00ps_2_0\x00Microsoft (R) HLSL Shader Compiler 9"...
 

mrkite77

Terrarian
I've been looking at the decompiled shaders and this thread was very helpful. I do have one question though, each resource definition starts with a comment string that is, mostly, composed of hex values. Do those values represent something that wasn't decompiled or do they serve some purpose in the form they are in?

They do represent stuff that wasn't decompiled. Comments that start with "CTAB" are special comments that describe the structure of various registers. When I was
wring the decompiler, I had intended on decompiling the CTAB sections, but after I got the rest of it working, I realized that it wasn't really necessary to understand the
shaders. (This decompiler wasn't really ever intended for a general public release, but as a personal project to help me implement paint support in Terrafirma).

I may eventually add CTAB support when I get some time. If you're curious, however, you can see exactly what that data represents. The hex values are described by the _D3DXSHADER_CONSTANTTABLE struct at the bottom of d3dx9shader.h

Here's a copy of the header you can look at:
https://github.com/hrydgard/minidx9/blob/master/Include/d3dx9shader.h
 

Jeckel

Terrarian
Thank you very much, mrkite, that answers my question thoroughly. After I've had some sleep, I'll take a look at the linked file for curiosity's sake. :)
 

Kazzymodus

Moderator
Staff member
Moderator
I've been looking at the PixelShader (animated dyes and the Pillar forcefield) and I've noticed that sometimes Constant Float registers are used that aren't defined. Now, I'm assuming that c0 is the same as c, but for example the ArmorGel pass defines c14 to c18 and then starts doing arithmetics with c5 and c6, among others. Is that a decompiling artifact or am I just misunderstanding how HLSL works (probably that)?
 

mrkite77

Terrarian
I've been looking at the PixelShader (animated dyes and the Pillar forcefield) and I've noticed that sometimes Constant Float registers are used that aren't defined. Now, I'm assuming that c0 is the same as c, but for example the ArmorGel pass defines c14 to c18 and then starts doing arithmetics with c5 and c6, among others. Is that a decompiling artifact or am I just misunderstanding how HLSL works (probably that)?

Not all defines are used.. I think it's because they were defined and used, then someone changed the shader and didn't realize that the HLSL compiler doesn't remove unused constants.
 

Kazzymodus

Moderator
Staff member
Moderator
Not all defines are used.. I think it's because they were defined and used, then someone changed the shader and didn't realize that the HLSL compiler doesn't remove unused constants.
Sorry for the late reply, I revisited this thread and now see what you meant.

The problem isn't that there are defines that aren't used. The problem is that some of the constants used aren't defined. So, to extend my example, the shader would perform arithmetics with c14 and c18, which are never defined in the first place.

Do these get a default value or something?
 

Puppeteer

Official Terrarian
Hey, is your programm still working? I've tried using it with wsl ubuntu but when I decompile the generated txt file is empty.
 

mrkite77

Terrarian
Hey, is your programm still working? I've tried using it with wsl ubuntu but when I decompile the generated txt file is empty.

Which shader did you try it on? It should still work since XNA hasn't changed in like a decade... but also, not every .xnb is a shader, a ton of them are textures (I have other tools that will convert .xnb textures into .pngs)

edit: I suppose terraria could've updated to dx10 shaders, which probably have a different format. I haven't checked in years.
 

Puppeteer

Official Terrarian
Which shader did you try it on? It should still work since XNA hasn't changed in like a decade... but also, not every .xnb is a shader, a ton of them are textures (I have other tools that will convert .xnb textures into .pngs)

edit: I suppose terraria could've updated to dx10 shaders, which probably have a different format. I haven't checked in years.
I tried it with the PixelShader. I also have a tool for all the other formats (images, sounds and fonts).
 

Sir Exilon

Skeletron Prime
I don't think there is more to it than the tool linked in mrkite77's post.
I meant a version of that tool that is accessible to the "general public".(that practically anyone can easily use with little to no programming experience)
 
Last edited:

Puppeteer

Official Terrarian
I meant a version of that tool that is accessible to the "general public".(that practically anyone can easily use with little to no programming experience)
If you don't have any programming experience you probably won't understand the xnb shaders either.
 
Top Bottom