Tessellation

DirectX 11 provides three new stages to the graphics pipeline that allows us to increase the complexity of objects by generating additional polygons. For this example we will look at tesselating the triangles of an icosahedron to generate a sphere. Our vertex shader for this example simply passes the vertex position to the hull shader.

HSInOut VertShader(VSInput input )
{
 HSInOut Out;
 Out.Position  = input.Position;
 return Out; 
}

Like the geometry shader, the hull shader takes a draw primitive (in this case a triangle) as input is called once for each control point(vertex). The hull shader has two components. A patch constant function and the shader function. The patch constant function is responsible for calculating the tesselation factors for the edges and inside of the triangle before passing it along to the tesselation unit. The main shader can optionally perform additional computations. In this example, it only passes each vertex to the tesselator.

HSConst PatchConstantsHS( InputPatch< HSInOut, 3 > p )
{
    HSConst output = (HSConst)0;
    output.Edges[0] = output.Edges[1] = output.Edges[2] = 4;
    output.Inside   = 4;
    return output;
}

[domain("tri")]
[partitioning("integer")]
[outputtopology("triangle_cw")]
[patchconstantfunc("PatchConstantsHS")]
[outputcontrolpoints(3)]
HSInOut PassThroughHS(
 InputPatch< HSInOut, 3 > inputPatch, 
 uint uCPID : SV_OutputControlPointID )
{
    HSInOut output = (HSInOut)0;
    output.Position = inputPatch[uCPID].Position;
    return output;
}

The real action happens in the Domain shader. Every additional vertex that is output from the tesselator is passed to the domain shader. It provides the original primitive that was tesselated, as well as the coordinates of the newly generated vertex in relation to the original. For a triangle patch, this is the barycentric coordinates of the vertex. While this sounds really special, it is really just the weights with respect to the original triangle vertices. It can easily be transformed by multiplying each weight with the original vertex coordinate as seen in the snippet below. The newly generated vertices are still in the same plane of the original triangle. In order to provide a better looking sphere the domain shader will also push out these new vertices so that they lie on the surface of our sphere before doing the transformation to screen space.

[domain("tri")]
PSInput DS( 
 HSConst input, const OutputPatch< HSInOut, 3 > TrianglePatch, 
 float3 BarycentricCoordinates : SV_DomainLocation )
{
 PSInput output = (PSInput)0;
 
 // Interpolate local space position with barycentric coordinates
 float3 position = BarycentricCoordinates.x * TrianglePatch[0].Position 
     + BarycentricCoordinates.y * TrianglePatch[1].Position 
     + BarycentricCoordinates.z * TrianglePatch[2].Position;
 position = normalize(position);
 output.Position = mul( float4(position, 1.0), WorldViewProjection );
 output.Normal = mul( position, (float3x3)World );
 return output;
}

When creating the hull and domain shader it is important to specify shader model 5.0.

type Geometry(device, shape) =
...
    let hullShader =   
        let byteCode = ShaderBytecode.CompileFromFile(
                                "Simple.hlsl", 
                                "PassThroughHS", 
                                "hs_5_0", 
                                ShaderFlags.None, 
                                EffectFlags.None)
        new HullShader(device, byteCode)  

    let domainShader = 
        let byteCode = ShaderBytecode.CompileFromFile(
                                "Simple.hlsl", 
                                "DS", 
                                "ds_5_0", 
                                ShaderFlags.None, 
                                EffectFlags.None)
        new DomainShader(device, byteCode)

Something extremely important. The primitive topology should be set to a patch list with 3 control points. Failing to do this will result in several hours of pulling out your hair to figure out why things don't work.

type Geometry(device, shape) =
...
member this.Prepare(transforms) =
        do deviceContext.InputAssembler.PrimitiveTopology <- 
            PrimitiveTopology.PatchListWith3ControlPoint
        do deviceContext.HullShader.Set(hullShader)
        do deviceContext.DomainShader.Set(domainShader)
        do deviceContext.DomainShader.SetConstantBuffer(
            vsConstBuffer.Update(transforms),0)
...

Note also that the DomainShader now requires access to the transforms shader constants.