Now that we know how to add a virtual pyramid, let's see if we can add some movements. Let's see how we can dynamically change the height of the pyramid. When you start, the pyramid will look like this:
If you wait for some time, the pyramid gets taller and it will look like this:
Let's see how to do it in OpenCV Python. Inside the augmented reality code that we just discussed, add the following snippet at the end of the __init__ method in the Tracker class:
self.overlay_vertices = np.float32([[0, 0, 0], [0, 1, 0], [1, 1, 0], [1, 0, 0], [0.5, 0.5, 4]])
self.overlay_edges = [(0, 1), (1, 2), (2, 3), (3, 0),
(0,4), (1,4), (2,4), (3,4)]
self.color_base = (0, 255, 0)
self.color_lines = (0, 0, 0)
self.graphics_counter = 0
self.time_counter = 0Now that we have the structure, we need to add the code to dynamically change the height. Replace the overlay_graphics() method with the following method:
def overlay_graphics(self, img, tracked):
x_start, y_start, x_end, y_end = tracked.target.rect
quad_3d = np.float32([[x_start, y_start, 0], [x_end, y_start, 0],
[x_end, y_end, 0], [x_start, y_end, 0]])
h, w = img.shape[:2]
K = np.float64([[w, 0, 0.5*(w-1)],
[0, w, 0.5*(h-1)],
[0, 0, 1.0]])
dist_coef = np.zeros(4)
ret, rvec, tvec = cv2.solvePnP(quad_3d, tracked.quad, K, dist_coef)
self.time_counter += 1
if not self.time_counter % 20:
self.graphics_counter = (self.graphics_counter + 1) % 8
self.overlay_vertices = np.float32([[0, 0, 0], [0, 1, 0], [1, 1, 0], [1, 0, 0],
[0.5, 0.5, self.graphics_counter]])
verts = self.overlay_vertices * [(x_end-x_start), (y_end-y_start),
-(x_end-x_start)*0.3] + (x_start, y_start, 0)
verts = cv2.projectPoints(verts, rvec, tvec, K, dist_coef)[0].reshape(-1, 2)
verts_floor = np.int32(verts).reshape(-1,2)
cv2.drawContours(img, [verts_floor[:4]], -1, self.color_base, -3)
cv2.drawContours(img, [np.vstack((verts_floor[:2], verts_floor[4:5]))],
-1, (0,255,0), -3)
cv2.drawContours(img, [np.vstack((verts_floor[1:3], verts_floor[4:5]))],
-1, (255,0,0), -3)
cv2.drawContours(img, [np.vstack((verts_floor[2:4], verts_floor[4:5]))],
-1, (0,0,150), -3)
cv2.drawContours(img, [np.vstack((verts_floor[3:4], verts_floor[0:1],
verts_floor[4:5]))], -1, (255,255,0), -3)
for i, j in self.overlay_edges:
(x_start, y_start), (x_end, y_end) = verts[i], verts[j]
cv2.line(img, (int(x_start), int(y_start)), (int(x_end), int(y_end)), self.color_lines, 2)Now that we know how to change the height, let's go ahead and make the pyramid dance for us. We can make the tip of the pyramid oscillate in a nice periodic fashion. So when you start, it will look like this:
If you wait for some time, it will look like this:
You can look at augmented_reality_motion.py for the implementation details.
In our next experiment, we will make the whole pyramid move around the region of interest. We can make it move in any way we want. Let's start by adding linear diagonal movement around our selected region of interest. When you start, it will look like this:
After some time, it will look like this:
Refer to augmented_reality_dancing.py to see how to change the overlay_graphics() method to make it dance. Let's see if we can make the pyramid go around in circles around our region of interest. When you start, it will look like this:
After some time, it will move to a new position:
You can refer to augmented_reality_circular_motion.py to see how to make this happen. You can make it do anything you want. You just need to come up with the right mathematical formula and the pyramid will literally dance to your tune! You can also try out other virtual objects to see what you can with it. There are a lot of things you can do with a lot of different objects. These examples provide a good reference point, on top of which you can build many interesting augmented reality applications.