Green Screen Virtual Studio
Real-time Green Screen Virtual Studio is a specialized production setup used in film, television, and other setups. It involves shooting a subject or scene against a cyclorama, which is replaced with your real-time rendered image by Reality 5.5 SP1 using RealityKeyer
.
Process
- Select the Example Project from the Renderers section inside the Reality Hub Launcher module.
- Select Reality 5 in the Launch With section as illustrated above.
- Click on the Launch button.
- Go to the Engines tab, drag and drop the
UE5
node into the Nodegraph canvas.
- Select the
UE5
in the nodegraph canvas. - Click on the
UE5
in the Nodes section. - Click on the Spawn Reality Camera and Spawn Reality Projection Cube buttons as illustrated above.
- Add
AJAIn
andAJAOut
nodes to the Nodegraph canvas.
- Select your
AJAIn
node properties based on your device.
- Select your
AJAOut
node properties based on your device. - Add a Tracking Node (in our example it is the
Xync
node) into canvas.
- Go to the
Xync
node property, click on theEnable
checkbox.
Changing the Transform Mapping
- Depending on your studio setup, you may be required to change the
Transform Mapping
property group settings. In our example, we have to change theNegateX
,NegateZ
,CoordinateSystem
, andNegatePan
properties, as illustrated above.
Breaking Track
In order to get Lens Distortion data from your track output, you need to utilize dynamic node approach. To do that:
- Left click and drag the Track output of the
Xync
node, then locate and select theBreak nos.fb.Track
node, as illustrated above.
- Breaking track is complated, as illustrated above.
- Add an
Undistort
node to nodegraph canvas. - Connect the Lens Distortion output of the
Break nos.fb.Track
node into Distortion input of theUndistort
node. - Connect the
AJAIn
node's Output pin toUndistort
node's In pin.
- Optionally, you can right click on the
Break nos.fb.Track
node and select the Hide unused ping from the Node Context Menu, as illustrated above.
Your node network should look like in the image above.
Undistorting the image taken by your studio camera is necessary because almost every camera lens has some degree of optical distortion. On the other hand, the Cyclorama
node creates a 3D shape that isn't affected by lens distortions. To make sure the Cyclorama and camera image match up correctly, it's crucial to use the undistortion process.
- Add a
Cyclorama
node to nodegraph canvas. - Connect the Out output pin of the
Undistort
node to Video input of theCyclorama
node. - Connect the Track output of the
Xync
node into Track input of theCyclorama
node. - Add two
Distort
nodes to canvas. - Connect the Render output pin of the
Cyclorama
node to In input pin of the firstDistort
node. - Connect the Mask output pin of the
Cyclorama
node to In input pin of the secondDistort
node. - Connect the Lenst Distortion output pin of of the
Break nos.fb.Track
to Distortion input of the twoDistort
nodes.
Your node network should look like in the image above.
- Add
RealityKeyer
node to nodegraph canvas. - Connect
AJAIn
node's Output pin toRealityKeyer
node's Input pin. - Connect Out output pin of the first
Distort
node to Clean Plate input of theRealityKeyer
node. - Connect Out output pin of the second
Distort
node to Clean Plate Mask input of theRealityKeyer
node. - Create an
Undistort
node. - Connect the Output pin of the
RealityKeyer
into In input pin of theUndistort
node. - Connect the Lenst Distortion output pin of of the
Break nos.fb.Track
to Distortion input of the thirdDistort
node's Distortion input pin.
Your node network should look like in the image above.
- Connect the Out output pin of the
Undistort
node into Video Input pin of theUE5
node. - Create a
CompositePasses
node. - Connect the Scene output pin of the
UE5
node into Render input of theCompositePasses
node. - Connect Out output pin of the
RealityKeyer
node into Video input of theCompositePasses
node. - Connect the Lenst Distortion output pin of of the
Break nos.fb.Track
to Distortion input of theCompositePasses
node. - Add a
Resize
node to nodegraph canvas. - Connect the
CompositePasses
node's Output pin intoResize
node's Input pin. - Add a
Mixer
node to nodegraph canvas. - Connect the
Resize
node's Output pin to the Channel1 input pin of theMixer
node. - Connect the
Multiviewer
putput pin of theMixer
node into Source input pin of theAJAOut
node.
Your node network is complated, as shown above.