xrCreateHandMeshSpaceMSFT - Create a space for hand mesh tracking

Parameter Descriptions

Description

A hand mesh space location is specified by runtime preference to effectively represent hand mesh vertices without unnecessary transformations. For example, an optical hand tracking system can define the hand mesh space origin at the depth camera’s optical center.

An application should create separate hand mesh space handles for each hand to retrieve the corresponding hand mesh data. The runtime may use the lifetime of this hand mesh space handle to manage the underlying device resources. Therefore, the application should destroy the hand mesh handle after it is finished using the hand mesh.

The hand mesh space can be related to other spaces in the session, such as view reference space, or grip action space from the /interaction_profiles/khr/simple_controller interaction profile. The hand mesh space may be not locatable when the hand is outside of the tracking range, or if focus is removed from the application. In these cases, the runtime must not set the XR_SPACE_LOCATION_POSITION_VALID_BIT and XR_SPACE_LOCATION_ORIENTATION_VALID_BIT bits on calls to locateSpace with the hand mesh space, and the application should avoid using the returned poses or query for hand mesh data.

If the underlying HandTrackerEXT is destroyed, the runtime must continue to support locateSpace using the hand mesh space, and it must return space location with XR_SPACE_LOCATION_POSITION_VALID_BIT and XR_SPACE_LOCATION_ORIENTATION_VALID_BIT unset.

The application may create a mesh space for the reference hand by setting handPoseType to HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT. Hand mesh spaces for the reference hand must only be locatable in reference to mesh spaces or joint spaces of the reference hand.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to calling createHandMeshSpaceMSFT

• handTracker must be a valid HandTrackerEXT handle
• createInfo must be a pointer to a valid HandMeshSpaceCreateInfoMSFT structure
• space must be a pointer to an Space handle

Return Codes

Success

• SUCCESS
• SESSION_LOSS_PENDING

Failure

• ERROR_INSTANCE_LOST
• ERROR_SESSION_LOST
• ERROR_RUNTIME_FAILURE
• ERROR_HANDLE_INVALID
• ERROR_POSE_INVALID
• ERROR_VALIDATION_FAILURE
• ERROR_FUNCTION_UNSUPPORTED
• ERROR_FEATURE_UNSUPPORTED

See Also

HandMeshSpaceCreateInfoMSFT, HandTrackerEXT, Space

A convenience wrapper to make a compatible pair of calls to createHandMeshSpaceMSFT and destroySpace

To ensure that destroySpace is always called: pass bracket (or the allocate function from your favourite resource management library) as the last argument. To just extract the pair pass (,) as the last argument.

xrUpdateHandMeshMSFT - Update hand mesh buffers

Parameter Descriptions

Description

The application should preallocate the index buffer and vertex buffer in HandMeshMSFT using the maxHandMeshIndexCount and maxHandMeshVertexCount from the SystemHandTrackingMeshPropertiesMSFT returned from the getSystemProperties function.

The application should preallocate the HandMeshMSFT structure and reuse it for each frame so as to reduce the copies of data when underlying tracking data is not changed. The application should use indexBufferChanged and vertexBufferChanged in HandMeshMSFT to detect changes and avoid unnecessary data processing when there is no changes.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to calling updateHandMeshMSFT

• handTracker must be a valid HandTrackerEXT handle
• updateInfo must be a pointer to a valid HandMeshUpdateInfoMSFT structure
• handMesh must be a pointer to an HandMeshMSFT structure

Return Codes

Success

• SUCCESS
• SESSION_LOSS_PENDING

Failure

• ERROR_INSTANCE_LOST
• ERROR_SESSION_LOST
• ERROR_RUNTIME_FAILURE
• ERROR_HANDLE_INVALID
• ERROR_VALIDATION_FAILURE
• ERROR_TIME_INVALID
• ERROR_FUNCTION_UNSUPPORTED
• ERROR_SIZE_INSUFFICIENT
• ERROR_FEATURE_UNSUPPORTED

See Also

HandMeshMSFT, HandMeshUpdateInfoMSFT, HandTrackerEXT

XrHandMeshSpaceCreateInfoMSFT - The information to create a hand mesh space

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshSpaceCreateInfoMSFT

• type must be TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• next must be NULL or a valid pointer to the next structure in a structure chain
• handPoseType must be a valid HandPoseTypeMSFT value

See Also

HandPoseTypeMSFT, Posef, StructureType, createHandMeshSpaceMSFT

XrHandMeshUpdateInfoMSFT - The information to update a hand mesh

Member Descriptions

Description

A runtime may not maintain a full history of hand mesh data, therefore the returned HandMeshMSFT might return data that’s not exactly corresponding to the time input. If the runtime cannot return any tracking data for the given time at all, it must set isActive to FALSE for the call to updateHandMeshMSFT. Otherwise, if the runtime returns isActive as TRUE, the data in HandMeshMSFT must be valid to use.

An application can choose different handPoseType values to query the hand mesh data. The returned hand mesh must be consistent to the hand joint space location on the same HandTrackerEXT when using the same HandPoseTypeMSFT.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshUpdateInfoMSFT

• type must be TYPE_HAND_MESH_UPDATE_INFO_MSFT
• next must be NULL or a valid pointer to the next structure in a structure chain
• handPoseType must be a valid HandPoseTypeMSFT value

See Also

HandPoseTypeMSFT, StructureType, https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XrTime, updateHandMeshMSFT

XrHandMeshMSFT - The data of a hand mesh

Member Descriptions

Description

When the returned isActive value is FALSE, the runtime indicates the hand is not actively tracked, for example, the hand is outside of sensor’s range, or the input focus is taken away from the application. When the runtime returns FALSE to isActive, it must set indexBufferChanged and vertexBufferChanged to FALSE, and must not change the content in indexBuffer or vertexBuffer,

When the returned isActive value is TRUE, the hand tracking mesh represented in indexBuffer and vertexBuffer are updated to the latest data of the time given to the updateHandMeshMSFT function. The runtime must set indexBufferChanged and vertexBufferChanged to reflect whether the index or vertex buffer’s content are changed during the update. In this way, the application can easily avoid unnecessary processing of buffers when there’s no new data.

The hand mesh is represented in triangle lists and each triangle’s vertices are in counter-clockwise order when looking from outside of the hand. When hand tracking is active, i.e. when isActive is returned as TRUE, the returned indexBuffer.indexCountOutput value must be positive and multiple of 3, and vertexBuffer.vertexCountOutput value must be equal to or larger than 3.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshMSFT

• type must be TYPE_HAND_MESH_MSFT
• next must be NULL or a valid pointer to the next structure in a structure chain
• indexBuffer must be a valid HandMeshIndexBufferMSFT structure
• vertexBuffer must be a valid HandMeshVertexBufferMSFT structure

See Also

https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XrBool32, HandMeshIndexBufferMSFT, HandMeshVertexBufferMSFT, StructureType, updateHandMeshMSFT

XrHandMeshIndexBufferMSFT - The index buffer of a hand mesh

Member Descriptions

Description

An application should preallocate the indices array using the maxHandMeshIndexCount in SystemHandTrackingMeshPropertiesMSFT returned from getSystemProperties. In this way, the application can avoid possible insufficient buffer sizees for each query, and therefore avoid reallocating memory each frame.

The input indexCapacityInput must not be 0, and indices must not be NULL, or else the runtime must return ERROR_VALIDATION_FAILURE on calls to the updateHandMeshMSFT function.

If the input indexCapacityInput is not sufficient to contain all output indices, the runtime must return ERROR_SIZE_INSUFFICIENT on calls to updateHandMeshMSFT, not change the content in indexBufferKey and indices, and return 0 for indexCountOutput.

If the input indexCapacityInput is equal to or larger than the maxHandMeshIndexCount in SystemHandTrackingMeshPropertiesMSFT returned from getSystemProperties, the runtime must not return ERROR_SIZE_INSUFFICIENT error on updateHandMeshMSFT because of insufficient index buffer size.

If the input indexBufferKey is 0, the capacity of indices array is sufficient, and hand mesh tracking is active, the runtime must return the latest non-zero indexBufferKey, and fill in indexCountOutput and indices.

If the input indexBufferKey is not 0, the runtime can either return without changing indexCountOutput or content in indices, and return FALSE for indexBufferChanged indicating the indices are not changed; or return a new non-zero indexBufferKey and fill in latest data in indexCountOutput and indices, and return TRUE for indexBufferChanged indicating the indices are updated to a newer version.

An application can keep the HandMeshIndexBufferMSFT structure for each frame in a frame loop and use the returned indexBufferKey to identify different triangle list topology described in indices. The application can therefore avoid unnecessary processing of indices, such as coping them to GPU memory.

The runtime must return the same indexBufferKey for the same HandTrackerEXT at a given time, regardless of the input HandPoseTypeMSFT in HandMeshUpdateInfoMSFT. This ensures the index buffer has the same mesh topology and allows the application to reason about vertices across different hand pose types. For example, the application can build a procedure to perform UV mapping on vertices of a hand mesh using HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT, and apply the resultant UV data on vertices to the mesh returned from the same hand tracker using HAND_POSE_TYPE_TRACKED_MSFT.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshIndexBufferMSFT

• indices must be a pointer to an array of indexCapacityInput uint32_t values
• The indexCapacityInput parameter must be greater than 0

See Also

HandMeshMSFT

XrHandMeshVertexBufferMSFT - The vertex buffer of a hand mesh

Member Descriptions

Description

An application should preallocate the vertices array using the maxHandMeshVertexCount in SystemHandTrackingMeshPropertiesMSFT returned from getSystemProperties. In this way, the application can avoid possible insufficient buffer sizes for each query, and therefore avoid reallocating memory each frame.

The input vertexCapacityInput must not be 0, and vertices must not be NULL, or else the runtime must return ERROR_VALIDATION_FAILURE on calls to the updateHandMeshMSFT function.

If the input vertexCapacityInput is not sufficient to contain all output vertices, the runtime must return ERROR_SIZE_INSUFFICIENT on calls to the updateHandMeshMSFT, do not change content in vertexUpdateTime and vertices, and return 0 for vertexCountOutput.

If the input vertexCapacityInput is equal to or larger than the maxHandMeshVertexCount in SystemHandTrackingMeshPropertiesMSFT returned from getSystemProperties, the runtime must not return ERROR_SIZE_INSUFFICIENT on calls to the updateHandMeshMSFT because of insufficient vertex buffer size.

If the input vertexUpdateTime is 0, and the capacity of the vertices array is sufficient, and hand mesh tracking is active, the runtime must return the latest non-zero vertexUpdateTime, and fill in the vertexCountOutput and vertices fields.

If the input vertexUpdateTime is not 0, the runtime can either return without changing vertexCountOutput or the content in vertices, and return FALSE for vertexBufferChanged indicating the vertices are not changed; or return a new non-zero vertexUpdateTime and fill in latest data in vertexCountOutput and vertices and return TRUE for vertexBufferChanged indicating the vertices are updated to a newer version.

An application can keep the HandMeshVertexBufferMSFT structure for each frame in frame loop and use the returned vertexUpdateTime to detect the changes of the content in vertices. The application can therefore avoid unnecessary processing of vertices, such as coping them to GPU memory.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshVertexBufferMSFT

• vertices must be a pointer to an array of vertexCapacityInput HandMeshVertexMSFT structures
• The vertexCapacityInput parameter must be greater than 0

See Also

HandMeshMSFT, HandMeshVertexMSFT, https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XrTime

XrHandMeshVertexMSFT - The vertex of hand mesh

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandMeshVertexMSFT

See Also

HandMeshVertexBufferMSFT, Vector3f

XrSystemHandTrackingMeshPropertiesMSFT - System property for hand tracking mesh

Member Descriptions

Description

If a runtime returns FALSE for supportsHandTrackingMesh, the system does not support hand tracking mesh input, and therefore must return ERROR_FEATURE_UNSUPPORTED from createHandMeshSpaceMSFT and updateHandMeshMSFT. The application should avoid using hand mesh functionality when supportsHandTrackingMesh is FALSE.

If a runtime returns TRUE for supportsHandTrackingMesh, the system supports hand tracking mesh input. In this case, the runtime must return a positive number for maxHandMeshIndexCount and maxHandMeshVertexCount. An application should use maxHandMeshIndexCount and maxHandMeshVertexCount to preallocate hand mesh buffers and reuse them in their render loop when calling updateHandMeshMSFT every frame.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using SystemHandTrackingMeshPropertiesMSFT

• type must be TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• next must be NULL or a valid pointer to the next structure in a structure chain

See Also

https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XrBool32, StructureType

XrHandPoseTypeInfoMSFT - Describes what hand pose type for the hand joint tracking.

Valid Usage (Implicit)

• The @@ extension must be enabled prior to using HandPoseTypeInfoMSFT

• type must be TYPE_HAND_POSE_TYPE_INFO_MSFT
• next must be NULL or a valid pointer to the next structure in a structure chain
• handPoseType must be a valid HandPoseTypeMSFT value

See Also

HandPoseTypeMSFT, StructureType

XrHandPoseTypeMSFT - Describe type of input hand pose

Enumerant Descriptions

The HAND_POSE_TYPE_TRACKED_MSFT input provides best fidelity to the user’s actual hand motion. When the hand tracking input requires the user to be holding a controller in their hand, the hand tracking input will appear as the user virtually holding the controller. This input can be used to render the hand shape together with the controller in hand.

The HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT input does not move with the user’s actual hand. Through this reference hand pose, an application can get a stable hand joint and mesh that has the same mesh topology as the tracked hand mesh using the same HandTrackerEXT, so that the application can apply the data computed from a reference hand pose to the corresponding tracked hand.

Although a reference hand pose does not move with user’s hand motion, the bone length and hand thickness may be updated, for example when tracking result refines, or a different user’s hand is detected. The application should update reference hand joints and meshes when the tracked mesh’s indexBufferKey is changed or when the isActive value returned from updateHandMeshMSFT changes from FALSE to TRUE. It can use the returned indexBufferKey and vertexUpdateTime from updateHandMeshMSFT to avoid unnecessary CPU or GPU work to process the neutral hand inputs.

See Also

HandMeshSpaceCreateInfoMSFT, HandMeshUpdateInfoMSFT, HandPoseTypeInfoMSFT

XrHandTrackerEXT - Represents a tracker for a hand joints.

Description

An application creates separate HandTrackerEXT handles for left and right hands. This handle can be used to locate hand joints using locateHandJointsEXT function.

A hand tracker provides accurate fidelity to the user’s actual hand shape. When the hand tracking input requires the user to be holding a controller in their hand, the hand tracking input will appear as the user holding the controller. This input can be used to render the hand shape together with rendering the controller in the hand.

See Also

createHandMeshSpaceMSFT, createHandTrackerEXT, destroyHandTrackerEXT, locateHandJointsEXT, updateHandMeshMSFT