Unity HDRP custom pass stencil buffer not working in build mode

Question

I've been attempting to create this stencil/mask buffer using Unity's HDRP C# Custom Pass feature and I've been having trouble getting it to work ONLY in build mode. Here is my code:

In my Setup function, I create the mask buffer

this.maskBuffer = RTHandles.Alloc(
    Vector2.one,
    TextureXR.slices,
    dimension: TextureDimension.Tex2D,
    colorFormat: GraphicsFormat.R16G16B16A16_SFloat,
    useDynamicScale: true,
    name: "MaskBuffer"
);

and in my Execute function, I create a Renderer List to build the stencil using a layer mask:

CoreUtils.SetRenderTarget(ctx.cmd, maskBuffer, ClearFlag.Color, Color.black);

var passes = new[] { new ShaderTagId("Forward"), new ShaderTagId("GBuffer"), new ShaderTagId("SRPDefaultUnlit") };

var mat = new Material(Shader.Find("HDRP/Unlit"));
mat.SetColor("_BaseColor", Color.white);
var desc = new RendererListDesc(passes, ctx.cullingResults, ctx.hdCamera.camera)
{
    layerMask = maskLayer,
    renderQueueRange = RenderQueueRange.all,
    overrideMaterialPassIndex = 0,
    overrideMaterial = mat
};
var list = ctx.renderContext.CreateRendererList(desc);
ctx.cmd.DrawRendererList(list);

And then I'm passing the buffer to a shader graph shader that I made.

Let me be clear: this code does work in the editor. The only thing that does not not work is the mask buffer when I build the game. In the build, the shader itself works because I can output a solid color just fine. I've also tried copying the camera's color buffer into the mask buffer temporarily and that works just fine too. But for some reason, I absolutely cannot get the stencil itself to draw. It always just renders as completely black.

What I've tried:

Removing the layer mask (maybe layers didn't get transferred to the build mode of the game)
Checking the build/runtime logs (there are no errors)
Using a different setup which, once again, works in the editor, but not in build mode:

CoreUtils.SetRenderTarget(ctx.cmd, this.maskBuffer, ClearFlag.Color);
CustomPassUtils.DrawRenderers(ctx, this.maskLayer);

Upgrading Unity versions (I tried two different versions of 6.2 and the newest 6.3)
Using many different variations of the colorFormat and disabling useDynamicScale in the RTHandles.Alloc function
Changing the Injection Point to different values in the Custom Pass Volume

Any help would be greatly appreciated. I'm nearly convinced that this is either some sort of config issue with the default HDRP project, or I've found a bug in the engine.

Thank you in advance!!

P.S. Some of this code may be slightly inefficient but I'm just using it for testing purposes

When you post a technical question about one of the render pipelines, it's a good idea to clearly state what version of Unity you're using and what version of the render pipeline you're using. Features of the engine and render pipelines can change over time, and without clarifying you might get answers that are for a different version than you're using, or future readers might find your question and be uncertain whether it's relevant to the version they are using. — Kevin
– Kevin, Commented 8 hours ago

Critical Angle Studios · Accepted Answer · 2025-12-05 19:44:06Z

I figured it out for anyone who stumbles across this post:

In my mask buffer allocation, I have to add a depth buffer size, like so:

this.maskBuffer = RTHandles.Alloc(
    Vector2.one,
    TextureXR.slices,
    dimension: TextureDimension.Tex2D,
    colorFormat: GraphicsFormat.R16G16B16A16_SFloat,
    useDynamicScale: true,
    name: "MaskBuffer",
    depthBufferBits: DepthBits.Depth8 // Add here
);

From what I understand, the engine expects a depth map because I didn't specify one? Maybe somebody could help me understand a little bit more behind it but I hope this helps somebody!

Another note: The object that I was trying to apply the mask to has a custom Shader Graph shader that I created, which has a specific ShaderTagId that you must add to the shader tag ids list that you can see in the original question.

The way you find that for shader graph is by clicking on the shader and clicking "View Generated Shader" in the inspector and finding the line that says "LightMode" = "<Shader Tag ID name>". In my case, the ID was "ShadowCaster", which I did not have added to the original array, but now that I added it, everything works fine.

Stack Exchange Network

Unity HDRP custom pass stencil buffer not working in build mode

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Unity HDRP custom pass stencil buffer not working in build mode

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions