Skip to main content
Version: Reality 5.5 SP1

Tracked Billboard

In this section, you'll discover how to use the Tracked Billboard pipeline with Reality 5.5 SP1. For our demonstration, we will be using TRAXIS Talent Tracking as a provider.

What Is Tracked Billboard?​

Tracked Billboard enables you to seamlessly integrate your keyed talent into your Reality scene, achieving a photorealistic appearance. Furthermore, it can be utilized in a variety of creative scenarios, including Teleportation and Virtual Fly Over, also known as Fly Cam.

Process​

Dragging & Dropping Engine into Nodegraph Canvas

  • Launch your project, go to the Engines section, drag and drop the UE5 process into Nodegraph canvas.

Node Details Panel

AJAIn and AJAOut Nodes

  • Create AJAIn and AJAOut nodes, then select your device details.

alt text

AJAIn and AJAOut Nodes

  • Connect the Scene output pin of the UE5 node into Source input pin of the AJAOut node.
  • Create your camera tracking node. In our example, we use Xync node.
  • Enable the Xync data flow by clicking on the Enable checkbox and defining a UDP Port.
  • Connect the Output pin of the AJAIn node to the Sync input pin to the Xync node.

Breaking Track​

  • Create a Break node and remane it Xync Break for better organizing.
  • Connect the Track output of the Xync node to the Input input pin of the Break node.

Cyclorama Connections​

alt text

Cyclorama Connections

  • Add a Cyclorama node to the canvas. Make necessary changes based on your studio cyclorama and take clean plate.
  • Create an Undistort node.
  • Create two Distort nodes. Rename them as Cyclorama Render Distort and Cyclorama Mask Distort for better organizing.
  • Connect the Lens Distortion output of the Xync Break node to the Distortion input pin of the Undistort node.
  • Connect the Lens Distortion output of the Xync Break node into Distortion input pins of Cyclorama Render Distort and Cyclorama Mask Distort nodes.
  • Connect the Output pin of the AJAIn node to the In input pin to the Undistort node.
  • Connect the Render output pin of the Cyclorama node to In input pin of Cyclorama Render Distort node.
  • Connect the Mask output pin of the Cyclorama node to In input pin of Cyclorama Mask Distort node.
  • Connect the AJAIn node's Track output to Track input of the Cyclorama node.

Adding RealityKeyer​

Tracked Billboard pipeline requires keyed image of your talent; therefore, you have to use Reality Keyer.

Reality Keyer Connections

  • Create a RealityKeyer node.
  • Connect the Out output pin of the Cyclorama Render Distort node to the Clean Plate input of the RealityKeyer node.
  • Connect the Out output pin of the second Distort node to the Clean Plate Mask the input of the RealityKeyer node.
  • Connect the Output pin to the AJAIn node into Input pin of the RealityKeyer node.
  • Create an Undistort node, and rename it as Reality Keyer Undistort.
  • Connect the Output output pin of the RealityKeyer node to the In input of the Reality Keyer Undistort node.
  • Connect the Lens Distortion output of the Xync Break node to the Distortion input pin of the Reality Keyer Undistort node.

Talent Location​

In order to place your talent inside the Reality scene, you have to provide talent location data via a tracking system. In our example we utilized TRAXIS Talent Tracker.

Talent Data Connections

  • Create a corresponding talent tracking data node. In our example we use FreeD node.
  • Enable the FreeD data flow by clicking on the Enable checkbox and defining a UDP Port.
  • Add a Break node.
  • Connect the Track output of the FreeD node into Break node's Input pin, then rename it as FreeD Break.
  • Create a vec3 to vec3d node and rename it as FreeD Location.
  • Connect the Location output pin of the FreeD Break node to vec3 input pin of the FreeD Location node.

Engine Connections​

Engine Connections

  • Connect the vec3d output pin of the FreeD Break node to UE5 node's Talent Location input pin.
  • Connect the second Undistort node's Out output pin to Video input of the UE5 node.
  • Connect the Xync node's Track output pin to the Reality Camera Track input pin of the UE5 node.
  • Create a Composite Passes node.
  • Connect UE5 node's Scene output pin to Composite Passes node's Render input.
  • Connect the Xync Break node's Lens Distortion output to Distortion input of the Composite Passes node.
  • Connect the Output output pin of the RealityKeyer to the Video input of the Composite Passes node.
  • Connect the Output pin of the Composite Passes node to Source input pin of the AJAOut node.

alt text

Final Nodegraph

Your nodegraph is ready for Tracked Billboard scenario.