Using occlusion

Enabling occlusion
Depth awareness
Using Hand occlusion with earlier versions of Varjo Base
Improving ranged occlusion
Improving hand masks for Hand occlusion

Overview

Occlusion combines real and virtual worlds by utilizing depth sensing capabilities. It allows you to show real-world elements such as tools, hands, or people in the virtual world, creating a more seamless and immersive experience.

Occlusion is available with Varjo Base release version 4.16 onwards with a Varjo Base Pro license. Hand occlusion has been available from Varjo Base 4.12 onwards. See, Using Hand occlusion with older Varjo Base versions.

Enabling occlusion

Enable the setting from Varjo Base Settings > Mixed reality > Mixed reality effects > Occlusion. The setting is disabled by default.

Note that when this setting is enabled, mixed reality will be turned on automatically, and the camera render position may change slightly. Read more about the Camera render position.

The following options are available:

All content in range

Brings everything within a defined distance range into the virtual environment.

Use the range slider to set the distance at which real-world objects appear

Hands

Brings your hands into the virtual environment.

People (Beta)

Brings people from the real world into the virtual environment.

Note that this feature is currently in beta due to ongoing work on segmentation quality and the absence of depth-based occlusion for people.

Depth awareness

Application provides depth information

If the application provides data about the spatial layout of the virtual environment, the system can show content from the real world in relation to virtual objects. For example, hands can appear behind or in front of virtual objects depending on their real-world position.

Read more about Rendering to Varjo headsets.

Application does not provide depth information

If the application does not provide data about the spatial layout, the system defaults to showing the real-world objects as always visible, regardless of their position relative to virtual objects. This ensures that you can still see the content, even if the experience is less spatially accurate.

Note that People (Beta) does not yet support depth awareness.

Using Hand occlusion with earlier versions of Varjo Base

Hand occlusion has been available with Varjo Base release version 4.12 onwards with a Varjo Base Pro license.

 If you are using Varjo Base version 4.12–4.15, you can enable the setting from Varjo Base Settings > Headset > Hands > Show real hands in applications. The setting is disabled by default.

Improving ranged occlusion

The sensors used for All content in range occlusion work best in controlled lighting conditions with non-reflective surfaces. You can try the following things to improve occlusion:

Prefer matte white surfaces
Avoid reflective materials such as mirrors, glass, and glossy surfaces
Avoid black and dark materials
Avoid other sources of infrared illumination in the environment

Improving hand masks for Hand occlusion

If there are holes in the masks or your hands are not outlined correctly, you can try the following things to improve the masks:

Improve your room lighting. See Preparing your environment for tips on lighting and environment setup.
We recommend that you set your hand tracking setting to either 'Automatic' or 'Varjo hand tracking'.
Add objects to the part of the room that serves as the background for your hands, and make sure those objects are distinctly different in color from your hands.
Dim any displays or lights that are located behind your hands.
Remove any gloves, jewelry, bright nail polish, or watches if possible.

Note that some tattoos might also affect hand occlusion quality.