For the depth-camera version of Cameo, let's create a new class, CameoDepth, as a subclass of Cameo. On initialization, a
CameoDepth class creates a CaptureManager class that uses a depth camera device (either CV_CAP_OPENNI for Microsoft Kinect or CV_CAP_OPENNI_ASUS for Asus Xtion, depending on our setup). During the main loop in run(), a CameoDepth function gets a disparity map, a valid depth mask, and a normal color image in each frame. The normal color image is used to estimate facial rectangles, while the disparity map and valid depth mask are used to refine the estimate of the facial region using createMedianMask(). Faces in the normal color image are swapped using copyRect(), with the faces' respective masks applied. Then, the destination frame is displayed, optionally with debug rectangles drawn overtop it. We can implement CameoDepth in cameo.py as follows:
class CameoDepth(Cameo):
def __init__(self):
self._windowManager = WindowManager('Cameo',
self.onKeypress)
device = depth.CV_CAP_OPENNI # uncomment for Microsoft Kinect
#device = depth.CV_CAP_OPENNI_ASUS # uncomment for Asus Xtion
self._captureManager = CaptureManager(
cv2.VideoCapture(device), self._windowManager, True)
self._faceTracker = FaceTracker()
self._shouldDrawDebugRects = False
self._curveFilter = filters.BGRPortraCurveFilter()
def run(self):
"""Run the main loop."""
self._windowManager.createWindow()
while self._windowManager.isWindowCreated:
self._captureManager.enterFrame()
self._captureManager.channel =
depth.CV_CAP_OPENNI_DISPARITY_MAP
disparityMap = self._captureManager.frame
self._captureManager.channel =
depth.CV_CAP_OPENNI_VALID_DEPTH_MASK
validDepthMask = self._captureManager.frame
self._captureManager.channel =
depth.CV_CAP_OPENNI_BGR_IMAGE
frame = self._captureManager.frame
self._faceTracker.update(frame)
faces = self._faceTracker.faces
masks = [
depth.createMedianMask(
disparityMap, validDepthMask, face.faceRect)
for face in faces
]
rects.swapRects(frame, frame,
[face.faceRect for face in faces], masks)
filters.strokeEdges(frame, frame)
self._curveFilter.apply(frame, frame)
if self._shouldDrawDebugRects:
self._faceTracker.drawDebugRects(frame)
self._captureManager.exitFrame()
self._windowManager.processEvents()To run a CameoDepth function instead of a Cameo or CameoDouble function, we just need to modify our if __name__=="__main__" block, as follows:
if __name__=="__main__":
#Cameo().run() # uncomment for single camera
#CameoDouble().run() # uncomment for double camera
CameoDepth().run() # uncomment for depth cameraThe following is a screenshot showing the CameoDepth class in action. Note that our mask gives the copied regions some irregular edges, as intended. The effect is more successful on the left and right sides of the faces (where they meet the background) than on the top and bottom (where they meet hair and neck regions of similar depth):
