Features are extracted, matched, and tracked by the FeatureMatching
class, especially by its public match
method. However, before we can begin analyzing the incoming video stream, we have some homework to do. It might not be clear right away what some of these things mean (especially for SURF and FLANN), but we will discuss these steps in detail in the following sections. For now, we only have to worry about initialization:
class FeatureMatching: def __init__(self, train_image='salinger.jpg'):
self.min_hessian = 400 self.SURF = cv2.SURF(self.min_hessian)
self.img_obj
), or print an error if it cannot be found:self.img_obj = cv2.imread(train_image, cv2.CV_8UC1) if self.img_obj is None: print "Could not find train image " + train_imageraise SystemExit
self.sh_train
) for convenience:self.sh_train = self.img_train.shape[:2] # rows, cols
For reasons that will soon be evidently clear, we will call the template image the train image and every incoming frame a query image. The train image has a size of 480 x 270 pixels and looks like this:
self.key_train, self.desc_train = self.SURF.detectAndCompute(self.img_obj, None)
We will do the same with each incoming frame and compare lists of features across images.
FLANN_INDEX_KDTREE = 0 index_params = dict(algorithm = FLANN_INDEX_KDTREE,trees = 5) search_params = dict(checks=50) self.flann = cv2.FlannBasedMatcher(index_params, search_params)
self.last_hinv = np.zeros((3,3)) self.num_frames_no_success = 0 self.max_frames_no_success = 5 self.max_error_hinv = 50.
Then, the bulk of the work is done by the FeatureMatching
method match
. This method follows the procedure elaborated here:
FeatureMatching._extract_features
.FeatureMatching._match_features
. If no such match is found, it skips to the next frame.FeatureMatching._detect_corner_points
. If any of the corners lies (significantly) outside the frame, it skips to the next frame.frontoparallel
plane. This is done in FeatureMatching._warp_keypoints
. If the result is significantly different from the result we got recently for an earlier frame, it skips to the next frame.In the following sections, we will discuss these steps in detail.