glTF rendering

Notes

glTF loading

The cgltf library is a single-header/single-source C library for reading glTF files.

Normal Maps

Normal maps are typically authored in linear space. Most other textures – albedo, metallic/roughness, AO, etc. – are given in sRGB. It is also best to load the textures with the right format as opposed to using a linear RGB/A format for them and then performing a Linear->sRGB conversion in shader code. [discussion].

It is also not clear what a texture is used for when looking at the textures, images and samplers sections of a glTF file. See the DamagedHelmet sample:

"images" : [
    {
        "uri" : "Default_albedo.jpg"
    },
    {
        "uri" : "Default_metalRoughness.jpg"
    },
    {
        "uri" : "Default_emissive.jpg"
    },
    {
        "uri" : "Default_AO.jpg"
    },
    {
        "uri" : "Default_normal.jpg"
    }
],
...
"samplers" : [
    {}
],
...
"textures" : [
  {
      "sampler" : 0,
      "source" : 0
  },
  {
      "sampler" : 0,
      "source" : 1
  },
  {
      "sampler" : 0,
      "source" : 2
  },
  {
      "sampler" : 0,
      "source" : 3
  },
  {
      "sampler" : 0,
      "source" : 4
  }
]

Intead, we only know their purpose when we parse the materials of a mesh primitive. Below, each index refers to a texture in the snippet above:

"materials" : [
{
    "emissiveFactor" : [
        1.0,
        1.0,
        1.0
    ],
    "emissiveTexture" : {
        "index" : 2
    },
    "name" : "Material_MR",
    "normalTexture" : {
        "index" : 4
    },
    "occlusionTexture" : {
        "index" : 3
    },
    "pbrMetallicRoughness" : {
        "baseColorTexture" : {
            "index" : 0
        },
        "metallicRoughnessTexture" : {
            "index" : 1
        }
    }
}
],

My implementation loads textures lazily. It first scans all textures up front, but instead of uploading them to GPU memory, which requires knowing the format, it instead returns a list of “load commands”. These commands describe how to read the texture (disk or memory) and what parameters to use, with a default format of sRGB. Later, when loading materials, if a normal texture is detected, the relevant load command is patched to use a linear colour space instead. The textures are then uploaded to GPU memory also during material loading.

Tangents

The DamagedHelmet sample has a normal map but no tangent vectors. In such cases, the glTF spec currently says:

“When tangents are not specified, client implementations SHOULD calculate tangents using default MikkTSpace algorithms with the specified vertex positions, normals, and texture coordinates associated with the normal texture.”

However, the MikkTSpace documentation also says:

// Note that the results are returned unindexed. It is possible to generate a new index list
// But averaging/overwriting tangent spaces by using an already existing index list WILL produce INCRORRECT results.
// DO NOT! use an already existing index list.

In other words, if the model has vertex indices like DamagedHelmet, then we should not compute tangents with the model as is. At the very least, we should unindex the model, compute the tangents, and then re-index it. [discussion]

What the glTF Sampler Viewer implementation does instead is to approximate the tangent in screen space when the model has no normals: [source]

vec3 uv_dx = dFdx(vec3(UV, 0.0));
vec3 uv_dy = dFdy(vec3(UV, 0.0));

vec3 t_ = (uv_dy.t * dFdx(v_Position) - uv_dx.t * dFdy(v_Position))
        / (uv_dx.s * uv_dy.t - uv_dy.s * uv_dx.t);

References

glTF Sample Viewer

glTF Sample Viewer code