TH-AB-204-04: On the Feasibility of a Multi-3D Kinect V2 Camera System for Monitoring Radiation Therapy Treatment Setup and Improving Patient Safety.

PURPOSE: To employ a multi-3D camera system for patient and treatment equipment tracking using the Kinect v2 camera system.

METHODS: The system has two cameras, a color camera (RGB) and a time-of-flight infrared camera (IR). The IR is used to measure the distances between the camera and the subject for each IR pixel producing a point cloud of fraxels (fragment pixel, our terminology for a measured location + color generated by a 3D camera). We intend to use multiple Kinect cameras to generate a real-time digital model of the treatment room. To enable this, each camera's fraxels need to be quantitative and registered to the linac coordinate system. We developed a calibration system consisting of a large flat board with IR and visible markers at measured locations. Images of the board were acquired with both cameras and the relative geometry of the board to the camera physically measured. The board images were used to characterize the geometric lens distortion and the marker locations used to calibrate the fraxel depth scales. The cameras were placed in the same room and pointed to a localization jig consisting of a checkerboard aligned to the room coordinate system. The camera rotations and translations to the board were determined for each camera and were applied to the camera fraxels to generate a single point cloud of the room.

RESULTS: The transformations between the cameras yielded a 3D treatment space accuracy of < 2 mm error in a radiotherapy setup within 500mm of isocenter. A novel-computing framework using multiple intel-NUC systems and a core dual-E6 processors allowed multiple cameras to be used simultaneously, greatly reducing occlusions.

CONCLUSIONS: The proposed 3D camera system has the potential for providing real-time access to the treatment room from outside and remote locations, and automate processes such as collision detection.