Hand gestures are analyzed by the HandGestureRecognition
class, especially by its recognize
method. This class starts off with a few parameter initializations, which will be explained and used later:
class HandGestureRecognition: def __init__(self): # maximum depth deviation for a pixel to be considered # within range self.abs_depth_dev = 14 # cut-off angle (deg): everything below this is a convexity # point that belongs to two extended fingers self.thresh_deg = 80.0
The recognize
method is where the real magic takes place. This method handles the entire process flow, from the raw grayscale image all the way to a recognized hand gesture. It implements the following procedure:
img_gray
) and returning a hand region mask (segment
):def recognize(self, img_gray): segment = self._segment_arm(img_gray)
segment
). Then, it returns the largest contour area found in the image (contours
) and any convexity defects (defects
):[contours, defects] = self._find_hull_defects(segment)
num_fingers
) in the image. Then, it annotates the output image (img_draw
) with contours, defect points, and the number of extended fingers:img_draw = cv2.cvtColor(img_gray, cv2.COLOR_GRAY2RGB) [num_fingers, img_draw] = self._detect_num_fingers(contours, defects, img_draw)
num_fingers
), as well as the annotated output image (img_draw
):return (num_fingers, img_draw)