Let's look at two approaches to integrating face tracking and swapping into Cameo. The first approach uses a single camera feed and swaps face rectangles found within this camera feed. The second approach uses two camera feeds and copies face rectangles from one camera feed to the other.
For now, we will limit ourselves to manipulating faces as a whole and not subelements such as eyes. However, you could modify the code to swap only eyes, for example. If you try this, be careful to check that the relevant subrectangles of the face are not None.
For the single-camera version, the modifications are quite straightforward. On initialization of Cameo, we create a FaceTracker and a Boolean variable indicating whether debug rectangles should be drawn for the FaceTracker. The Boolean is toggled in onKeypress() in response to the X key. As part of the main loop in run(), we update our FaceTracker with the current frame. Then, the resulting FaceFace objects (in the faces property) are fetched and their faceRects are swapped using rects.swapRects(). Also, depending on the Boolean value, we may draw debug rectangles that reflect the original positions of facial elements before any swap.
import cv2 import filters from managers import WindowManager, CaptureManager import rects from trackers import FaceTracker class Cameo(object): def __init__(self): self._windowManager = WindowManager('Cameo', self.onKeypress) self._captureManager = CaptureManager( cv2.VideoCapture(0), self._windowManager, True) self._faceTracker = FaceTracker() self._shouldDrawDebugRects = False self._curveFilter = filters.BGRPortraCurveFilter() def run(self): """Run the main loop.""" self._windowManager.createWindow() while self._windowManager.isWindowCreated: self._captureManager.enterFrame() frame = self._captureManager.frame self._faceTracker.update(frame) faces = self._faceTracker.faces rects.swapRects(frame, frame, [face.faceRect for face in faces]) filters.strokeEdges(frame, frame) self._curveFilter.apply(frame, frame) if self._shouldDrawDebugRects: self._faceTracker.drawDebugRects(frame) self._captureManager.exitFrame() self._windowManager.processEvents() def onKeypress(self, keycode): """Handle a keypress. space -> Take a screenshot. tab -> Start/stop recording a screencast. x -> Start/stop drawing debug rectangles around faces. escape -> Quit. """ if keycode == 32: # space self._captureManager.writeImage('screenshot.png') elif keycode == 9: # tab if not self._captureManager.isWritingVideo: self._captureManager.startWritingVideo( 'screencast.avi') else: self._captureManager.stopWritingVideo() elif keycode == 120: # x self._shouldDrawDebugRects = not self._shouldDrawDebugRects elif keycode == 27: # escape self._windowManager.destroyWindow() if __name__=="__main__": Cameo().run()
The following screenshot is from Cameo. Face regions are outlined after the user presses X:
The following screenshot is from Cameo. American businessman Bill Ackman performs a takeover of the author's face:
For the two-camera version, let's create a new class, CameoDouble, which is a subclass of Cameo. On initialization, a CameoDouble invokes the constructor of Cameo and also creates a second CaptureManager. During the main loop in run(), a CameoDouble gets new frames from both cameras and then gets face tracking results for both frames. Faces are copied from one frame to the other using copyRect(). Then, the destination frame is displayed, optionally with debug rectangles drawn overtop it. We can implement CameoDouble in cameo.py as follows:
Note
For some models of MacBook, OpenCV has problems using the built-in camera when an external webcam is plugged in. Specifically, the application may become deadlocked while waiting for the built-in camera to supply a frame. If you encounter this issue, use two external cameras and do not use the built-in camera.
class CameoDouble(Cameo): def __init__(self): Cameo.__init__(self) self._hiddenCaptureManager = CaptureManager( cv2.VideoCapture(1)) def run(self): """Run the main loop.""" self._windowManager.createWindow() while self._windowManager.isWindowCreated: self._captureManager.enterFrame() self._hiddenCaptureManager.enterFrame() frame = self._captureManager.frame hiddenFrame = self._hiddenCaptureManager.frame self._faceTracker.update(hiddenFrame) hiddenFaces = self._faceTracker.faces self._faceTracker.update(frame) faces = self._faceTracker.faces i = 0 while i < len(faces) and i < len(hiddenFaces): rects.copyRect( hiddenFrame, frame, hiddenFaces[i].faceRect, faces[i].faceRect) i += 1 filters.strokeEdges(frame, frame) self._curveFilter.apply(frame, frame) if self._shouldDrawDebugRects: self._faceTracker.drawDebugRects(frame) self._captureManager.exitFrame() self._hiddenCaptureManager.exitFrame() self._windowManager.processEvents()
To run a CameoDouble instead of a Cameo, we just need to modify our if __name__=="__main__" block, as follows:
if __name__=="__main__": #Cameo().run() # uncomment for single camera CameoDouble().run() # uncomment for double camera