Green Screen Virtual Studio

Real-time Green Screen Virtual Studio is a specialized production setup used in film, television, and other setups. It involves shooting a subject or scene against a cyclorama, which is replaced with your real-time rendered image by Reality 5.5 SP1 using RealityKeyer.

Process

• Select the Example Project from the Renderers section inside the Reality Hub Launcher module.

• Select Reality 5 in the Launch With section as illustrated above.
• Click on the Launch button.

Engines on Node Details Panel

• Go to the Engines tab, drag and drop the UE5 node into the Nodegraph canvas.

Engine Functions

• Select the UE5 in the nodegraph canvas.
• Click on the UE5 in the Nodes section.
• Click on the Spawn Reality Camera and Spawn Reality Projection Cube buttons as illustrated above.

AJA Nodes

• Add AJAIn and AJAOut nodes to the Nodegraph canvas.

AJAIn Node Properties

• Select your AJAIn node properties based on your device.

• Select your AJAOut node properties based on your device.
• Add a Tracking Node (in our example it is the Xync node) into canvas.

• Go to the Xync node property, click on the Enable checkbox.

Changing the Transform Mapping

Xync Node | Transform Mapping Property Group

• Depending on your studio setup, you may be required to change the Transform Mapping property group settings. In our example, we have to change the NegateX, NegateZ, CoordinateSystem, and NegatePan properties, as illustrated above.

Breaking Track

In order to get Lens Distortion data from your track output, you need to utilize dynamic node approach. To do that:

Track Breaking

• Left click and drag the Track output of the Xync node, then locate and select the Break nos.fb.Track node, as illustrated above.

Track Breaking Completed

• Breaking track is complated, as illustrated above.
• Add an Undistort node to nodegraph canvas.
• Connect the Lens Distortion output of the Break nos.fb.Track node into Distortion input of the Undistort node.
• Connect the AJAIn node's Output pin to Undistort node's In pin.

Node Context Menu

• Optionally, you can right click on the Break nos.fb.Track node and select the Hide unused ping from the Node Context Menu, as illustrated above.

Node Network

Your node network should look like in the image above.

info

Undistorting the image taken by your studio camera is necessary because almost every camera lens has some degree of optical distortion. On the other hand, the Cyclorama node creates a 3D shape that isn't affected by lens distortions. To make sure the Cyclorama and camera image match up correctly, it's crucial to use the undistortion process.

• Add a Cyclorama node to nodegraph canvas.
• Connect the Out output pin of the Undistort node to Video input of the Cyclorama node.
• Connect the Track output of the Xync node into Track input of the Cyclorama node.
• Add two Distort nodes to canvas.
• Connect the Render output pin of the Cyclorama node to In input pin of the first Distort node.
• Connect the Mask output pin of the Cyclorama node to In input pin of the second Distort node.
• Connect the Lenst Distortion output pin of of the Break nos.fb.Track to Distortion input of the two Distort nodes.

Node Network

Your node network should look like in the image above.

• Add RealityKeyer node to nodegraph canvas.
• Connect AJAIn node's Output pin to RealityKeyer node's Input pin.
• Connect Out output pin of the first Distort node to Clean Plate input of the RealityKeyer node.
• Connect Out output pin of the second Distort node to Clean Plate Mask input of the RealityKeyer node.
• Create an Undistort node.
• Connect the Output pin of the RealityKeyer into In input pin of the Undistort node.
• Connect the Lenst Distortion output pin of of the Break nos.fb.Track to Distortion input of the third Distort node's Distortion input pin.

Node Network

Your node network should look like in the image above.

• Connect the Out output pin of the Undistort node into Video Input pin of the UE5 node.
• Create a CompositePasses node.
• Connect the Scene output pin of the UE5 node into Render input of the CompositePasses node.
• Connect Out output pin of the RealityKeyer node into Video input of the CompositePasses node.
• Connect the Lenst Distortion output pin of of the Break nos.fb.Track to Distortion input of the CompositePasses node.
• Add a Resize node to nodegraph canvas.
• Connect the CompositePasses node's Output pin into Resize node's Input pin.
• Add a Mixer node to nodegraph canvas.
• Connect the Resize node's Output pin to the Channel1 input pin of the Mixer node.
• Connect the Multiviewer putput pin of the Mixer node into Source input pin of the AJAOut node.

Node Network

Your node network is complated, as shown above.

Examples

Image Courtesy of Brand Mind Live Virtual Studio

Image Courtesy of BETV - Dreamwall