Opengl Es Stencil Operations

In reference to the problem diskussed in article OpenGL clipping a new question arises. I am implementing the clipping for a node based 2D scene graph. Whenever the clipping boxes

Solution 1:

glStencilFunc(GL_ALWAYS, ref, mask); says that:

• the first argument says that the stencil comparison test always succeeds
• the second and third are hence ignored, but in principle would set a constant to compare the value coming from the stencil buffer to and which of the bits are used for comparison

glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE); has the effect that:

• when the stencil test fails, the new value replaces the old (per the first argument)
• when the depth test fails but the stencil test passes, the new value replaces the old (per the second argument)
• when the stencil and depth test both pass, the new value replaces the old (per the third argument)

So you're setting things up so that everything you try to draw will pass the stencil test, and in any case its value will be copied into the stencil buffer whether it passes the test or not.

Probably what you want to do is to change your arguments to glStencilOp so that you perform a GL_INCR on pixels that pass. So in your stencil buffer you'll end up with a '1' anywhere touched by a single polygon, a '2' anywhere that you drew two successive polygons, a '3' anywhere you drew 3, etc.

When drawing the user-visible stuff you can then use something like glStencilFunc set to GL_GEQUAL and to compare incoming values to '1', '2', '3' or whatever.

If you prefer not to keep track of how many levels deep you are, supposing you have one clip area drawn into the stencil, you can modify it to the next clip area by:

1. drawing all new geometry with GL_INCR; this'll result in a stencil buffer where all pixels that were in both areas have a value of '2' and all pixels that were in only one area have a value of '1'
2. draw a full screen polygon, to pass only with stencil fund GL_GREATER and reference value 0, with GL_DECR. Result is '0' everywhere that was never painted or was painted only once, '1' everywhere that was painted twice.