C H A P T E R 9
Kinect Remote-Controlled Vehicles
In Chapter 5, you learned how to hack a remote control in order to control different devices via your body movements. In this chapter, you are going a step further in the control of devices by hacking an RC vehicle's electronics. You're going to use a RC car, but the same logic can be applied to any RC vehicle (helicopter, boat, etc.).
You are going to learn how to use an H-Bridge to control motors. Using the Kinect, you are going to produce an easy interface that connects body movements with the vehicle's movements (Figure 9-1) and you'll apply what you learned from the XBee technology to do it wirelessly. Finally, you are going to create an automated routine that will avoid crashes by using a proximity sensor and certain programmed behaviors. Figure 9-2 shows the parts you'll need for this chapter and Table 9-1 lists the specifics.
Figure 9-1. Kinect and the RC vehicle
Figure 9-2. RC vehicle components
Electrical Motors and the H-Bridge
Before hacking and programming the car, let's talk about electrical motors and how you are going to drive them using an H–Bridge.
An electric motor converts electrical energy into mechanical energy. Most electric motors operate through the interaction of magnetic fields and current conductors to generate the force. The kind of motor in most toys is a DC motor (Figure 9-3). A DC motor is an electric motor that runs on direct current (DC) electricity, usually from batteries.
Figure 9-3. RC motor and H-Bridge
To control a motor, you can turn it on and off using only one switch (or transistor). But controlling the direction is a bit more difficult. It requires a minimum of four switches (or transistors) arranged in a clever way. This can be simplified using an H-Bridge.
H-Bridges are four switches (or transistors) arranged in a shape that resembles an H. Each H-Bridge can control two motors. Each H-Bridge has 16 pins (eight per side), as shown in Figure 9-4. Figure 9-5 shows the schematic of an H-Bridge.
Figure 9-4. H-Bridge pins
Figure 9-5. H-Bridge schematic
You are going to use the H-Bridge to drive two motors. Each pair of channels is equipped with an ENABLE input (pins 1 and 9). A separate supply input is provided in pins 8 and 16, allowing operation at a lower voltage. Pins 4, 5, 12, and 13 are connected to ground. Each side of the H-Bridge has two transistors (pins 2 and 7, and pins 10 and 15). One is responsible for pushing this side to HIGH and the other for pulling this side to LOW. When one side is pulled HIGH and the other LOW, the motor spins in one direction. If you want the motor to spin in the opposite way, you just have to reverse the transistors (the first side LOW and the latter HIGH).
Pins 3 and 6 are connected to the poles of one motor, and pins 11 and 14 are connected to the poles of a second motor. According to the direction of the current, the motor will spin in one direction or the opposite one.
Hacking a Car
Now that you understand these basic concepts of the motor, you can start to build your RC vehicle. The first step is to open up and observe what's inside (Figure 9-6). You should see a circuit and a series of cables. Looking at the circuit, you can see a RC receiver that controls the behavior of the motors. If you follow the cables, there are two that connect the circuit to the batteries. Another two connect the circuit to the front motor, and another two connect the circuit to the back motor. RC cars normally have two motors. The front one turns the vehicle left and right. The back one provides traction, which is the moving of the vehicle frontwards and backwards.
You're not going to use the existing circuit because you're going to build your own. So cut the cables that connect the actual circuit to the motors and the cables that connect to the batteries. Unscrew the board and leave the car ready for your new circuit (Figure 9-7).
Figure 9-7. RC car, just motors and electrical cables
You're going to drive the car using your Arduino. As a first step, you will be using serial data coming from a simple Processing sketch. Later, once everything is up and running, you will use the Kinect and a XBee module to drive the car with your body movements.
Start by joining the power and ground cables coming from the battery pack to a power jack so you can plug your Arduino using just the external batteries (Figure 9-8).
Figure 9-8. RC car with the electrical connection to battery pack
So far, you've had an overview of electrical motors and the H-Bridge. You have opened an RC vehicle, looked inside, and extracted the circuit in order to start to building a new one.
Building the Circuit
The next step is to prepare your Arduino board and connect it to an H-Bridge in order to control the two motors (the back one that provides the traction and the front one that provides the direction). In this project, you're going to use a prototype shield instead of fusing the strip panel. A prototype shield is a platform for building a new project; it makes it easier to connect an Arduino to all the external bits and pieces. Shields are normally designed to sit on top of an Arduino, facilitating the connections. There are some pins that connect to the Arduino pins and there's a prototyping area where you solder your external components.
Start by soldering your breakaway headers in order to plug them into your Arduino board. Then solder two cables to 5V Arduino and ground and plug your H Bridge (Figure 9-9).
Figure 9-9. Prototype board and the H-Bridge
Next, solder the H-Bridge and start making all the connections. The H-Bridge chip is going to be used as two bridges controlling two separate motors. Connect the H-Bridge's pins 8 and 16 to 5V voltage. You can power the motors and the H Bridge using any external power device (connecting it to these pins), but remember to share the ground with the Arduino board. In this case, everything is powered through the Arduino (Figure 9-10).
Figure 9-10. H-Bridge connected with power
Then connect the H-Bridge pins 4, 5, 12, and 13 to ground. The next set of cables that you are going to solder are the transistors to Arduino pins. Link Arduino pins 11 and 5 with H-Bridge pins 10 and 15, and Arduino pins 6 and 10 with H-Bridge pins 2 and 7. The circuit will look like the one in Figures 9-11 and 9-12.
Figure 9-11. H-Bridge with inputs for power and ground
Figure 9-12. H Bridge with inputs for power and ground (back)
The next step is to connect the ENABLE inputs. Use a cable that connects H-Bridge pins 1 and 9 with Arduino pins 9 and 8. Remember to have a line in your code that allows them to be HIGH in order to activate the H-Bridge. The last step is to solder the H-Bridge OUT pins (pins 3, 6, 11, and 14) to the motor poles. Figures 9-13 and 9-14 show the circuit from the front and the back.
Figure 9-13. H-Bridge ready for connecting motors
Figure 9-14. H-Bridge ready for connecting motors (back)
The diagram in Figure 9-15 shows the whole circuit.
Testing the Circuit
It's time to test the circuit. You are going to use a simple Processing sketch that will activate each motor according to the position of the mouse, using serial communication with Arduino (Figure 9-16).
Figure 9-16. Processing sketch for testing
The code imports the Serial library in order to create serial communication with Arduino. After that, a series of variables are defined in order to store the values that are going to be transmitted. Finally, other variables are used for building a little car simulation that shows a visual description of what are you transmitting. These variables are used inside a car() function.
import processing.serial.*;
Serial myPort;
boolean serial = true;
int Val01, Val02, Val03;
float temp01, temp02, temp03, temp04;
//car variables
int carwidth= 70;
int carheight =120;
int wheels=30;
float angle;
float speed, wLine, wLinePos;
Setup Function
The setup() function just declares the size of your sketch and defines the port used for transmitting the data.
void setup() {
size(300, 300);
smooth();
if (serial) {
String portName = Serial.list()[0];
myPort = new Serial(this, portName, 9600);
}
}
Draw Function
The draw() function defines the arrow from a series of PVectors. One is defined from the mouse position (end of the arrow) and the other from the center of the canvas. Once you know these positions, you create another PVector that is the subtraction of the first two. After that, you call the function drawVector() that is defined by two PVectors (and will be explained later).
void draw() {
background(255);
PVector mouse = new PVector(mouseX, mouseY);
PVector center = new PVector(width/2, height/2);
PVector v = PVector.sub(mouse, center);
//Draw vector between 2 points
drawVector(v, center);
noFill();
The next part of the code defines the values sent via serial communication. You map the X coordinates in a range of values from 0 (left) to 255 (right). In order to do this, you define a temporary value as the difference between the mouse's X position and the canvas' center. This value is remapped in the range in a temporary float. Then you introduce a series of if conditionals so the final value that you to transmit is 0 (no movement), 1 (turn left), and 2 (turn right). These values activate an action in your virtual car simulation.
For the Y coordinate you also use a temporary value that refers to the mouse's Y coordinate and the center of the canvas. You remap this value between -255 (bottom area), 0 (center), and 255 (top area). This is the value that you pass.
The back motor is going to have two values: one that marks the direction (Val02) and other that marks the speed (Val03). The Val02 variable respond to 0 (don't activate the motor), 1 (activate it forward), and 2 (activate it backward). So according to the position of the mouse, it will change; if it's closer to the center, the motor won't be activated; if it's far, the car will go forward or backwards.
Another condition that you want to introduce is speed; this value depends on the distance of the mouse to the center. At the moment, just two speeds are defined: one of 140 and another of 200.
temp01 = mouseX-width/2;
temp02 = height/2- mouseY;
temp03= int(map(temp01, -width/2, width/2, 0, 255));
//turn left
if (temp03<100) {
Val01=1;
angle=-PI/4;
}
//turn right
else if (temp03>150) {
Val01=2;
angle=PI/4;
}
//no turn
else {
Val01=0;
angle=0;
}
// decide where to move
temp04= int(map(temp02, -height/2, height/2, -255, 250));
//move front
if (temp04>0 && temp04>50) {
Val02=1;
}
//move back
else if (temp04<0 && temp04<-50) {
Val02=2;
}
//don't move
else {
Val02=0;
speed=0;
}
//decide speed
if (temp04>100 && temp04<150) {
Val03=140;
speed= speed+0.5;
}
else if (temp04>150) {
Val03=200;
speed++;
}
else if (temp04<-100 && temp04>-150) {
Val03=140;
speed= speed-0.5;
}
else if (temp04<-150){
Val03=200;
speed--;
}
The next lines of code just draw a couple of circles to define the motor area where the motor won't work. This area is introduced in case you want to stop the car and not keep it constantly in movement. Then you call the car() function that represents a virtual car on the screen. The last line of the draw() function just sends serial data.
ellipse(width/2, height/2, width/3, width/3);
ellipse(width/2, height/2, width/6, width/6);
car();
if (serial){
sendSerialData();
}
}
DrawVector Function
The drawVector() function represents an arrow starting from the center of your screen towards your mouse position (an arrow between two points). You mark the points using PVectors, but you also want to represent an arrowhead that points toward the direction that the car will move.
The arrowhead is represented by two lines. But you have to do a bit of math in order to have the rotation. You translate locally the coordinate system using pushmatrix and translate. These functions locate a new local origin to be the reference for the rotation of the arrowhead.
The angle that you rotate your coordinate system is defined by the PVector function heading2d. This function returns a float that defines the angle of rotation for the vector. So once your coordinate system has been rotated, you define the lines that form your arrow: a vertical line (remember that you have just rotate your UCS) that goes from 0,0 to the length (or magnitude) of your PVector, and for the head, you just define two lines from the extreme to both sides.
The last step is to use popmatrix to close the translations and rotations.
void drawVector(PVector v, PVector loc) {
pushMatrix();
float arrowsize = 4;
translate(loc.x, loc.y);
stroke(0);
rotate(v.heading2D());
float len = v.mag();
line(0, 0, len, 0);
line(len, 0, len-arrowsize, +arrowsize/2);
line(len, 0, len-arrowsize, -arrowsize/2);
popMatrix();
}
Car Function
The car() function represents a diagramatic verison of your car. The central rectangle defines the body and the four small ones represent the wheels. Select a blue color and use the functions pushMatrix and popMatrix to represent each rectangle on the screen, allowing rotation for the front wheels. So, according to the values received (or the mouse position), the wheels turn in one direction or the other. Finally, to represent the speed, you define a red line that crosses the back wheels, simulating the rotation of the wheels.
void car() {
rectMode(CENTER);
//body
noFill();
stroke(30, 144, 255);
rect(width/2, height/2, carwidth-wheels/2, carheight);
//front wheels
pushMatrix();
translate(width/2-carwidth/2+wheels/4, height/2-carheight/2+wheels);
rotate(angle);
rect(0, 0, wheels/2, wheels);
popMatrix();
pushMatrix();
translate(width/2+carwidth/2-wheels/4, height/2-carheight/2+wheels);
rotate(angle);
rect(0, 0, wheels/2, wheels);
popMatrix();
//back wheels
pushMatrix();
translate(width/2, height/2);
rect(-carwidth/2+wheels/4, carheight/4, wheels/2, wheels);
rect(carwidth/2-wheels/4, carheight/4, wheels/2, wheels);
//line simulating speed
stroke(255, 0, 0);
line(-carwidth/2, wLine, -carwidth/2+wheels/2, wLine);
line(carwidth/2, wLine, carwidth/2-wheels/2, wLine);
wLine=(carheight/4+wLinePos+speed);
if (wLine<carheight/4-wheels/2) {
wLine=carheight/4+wheels/2;
speed=0;
wLinePos=wheels/2;
}
else if (wLine>carheight/4+wheels/2) {
wLine=carheight/4-wheels/2;
speed=0;
wLinePos=-wheels/2;
}
popMatrix();
}
SendSerial Function
The sendSerial() function is the same as in previous chapters; it just sends data with a letter before it in order to recognize and avoid any loss of data.
void sendSerialData() {
myPort.write('S'),
myPort.write(Val01);
myPort.write(Val02);
myPort.write(Val03);
}
With this code, you send four values of serial data to drive your motors. The first value is just to identify the car, the second value is a range for moving the front motor, the third value defines a direction, and fourth value defines a speed. Now you have a sketch that sends serial data to your Arduino according to your mouse position. You are really close to your first test but first you need to program your Arduino.
Arduino Testing Sketch
The code starts by declaring floats and the Arduino pins that are connected to your H-Bridge. Note that for the back motor you use analog pins.
float Val01, Val02, Val03;
int motorFrontLeft = 11; // H-bridge leg 10
int motorFrontRight = 5; // H-bridge leg 15
int motorBackUp = 6; // H-bridge leg 2 (PWM)
int motorBackDown = 10; // H-bridge leg 7 (PWM)
int enablePinFront = 8; // H-Bridge leg 9
int enablePinBack = 9; // H-Bridge leg 1
Setup Function
The setup() function just declares the start of serial communication and the Arduino pin mode; in this sketch all pins are outputs.
void setup() {
// initialize the serial communication:
Serial.begin(9600);
// Set the pin modes
pinMode(motorFrontLeft, OUTPUT);
pinMode(motorFrontRight, OUTPUT);
pinMode(motorBackUp, OUTPUT);
pinMode(motorBackDown, OUTPUT);
pinMode(enablePinFront, OUTPUT);
pinMode(enablePinBack, OUTPUT);
}
Loop Function
Start by reading the serial data. If any data is available, the first piece of information should match the letter S. If this is the case, continue reading and store the values as floats.
After reading the values, you introduce a series of if conditionals that will drive the motors according to the values obtained. For the front motor drive, you get values 1 (turn left), 2 (turn right), and 0 (no action in case you want to drive in a straight line). If the values are 1 or 2, the H-Bridge's ENABLE pin is active, meaning that the motor starts to work and the direction is defined by the functions TurnLeft() and TurnRight().
For the back motor, you receive two values (Val02 and Val03); one defines the direction and the other defines the speed.
Variable Val02 defines the direction. If its value is 1, it puts the H-Bridge ENABLE pin as HIGH and calls the MoveUp() function that is defined by a value (speed). If variable Val02 is 2, the ENABLE pin is HIGH and calls the function MoveDown() runs, moving the car in the opposite direction. Finally, if variable Val02 is 0, the ENABLE pin is LOW, meaning that this motor is disconnected and the car won't move.
Variable Val03 defines the speed of the motor. Potentially, you could have values from 0 to 255, but according to the Processing sketch you just have two speeds, normal and fast, with values of 140 and 200.
void loop() {
// check if data has been sent from the computer:
if (Serial.available()>4) { // If data is available to read,
char val = Serial.read();
if(val == 'S') {
Val01 = Serial.read();
Val02 = Serial.read();
Val03 = Serial.read();
}
}
// turning left and right
if (Val01==1) {
enable(enablePinFront);
TurnRight();
}
else if (Val01==2) {
enable(enablePinFront);
TurnLeft();
}
else if(Val01==0)disable(enablePinFront);
if (Val02==1) {
enable(enablePinBack);
MoveUp(Val03);
}
else if (Val02==2) {
enable(enablePinBack);
MoveDown(Val03);
}
else if(Val02==0) {
disable(enablePinBack);
}
}
Turning Functions
The functions for turning right and turning left are similar. Each function sets one pin to HIGH and one to LOW. The direction of the current flow, and therefore the direction of the motor, is determined by which pin is in which state. The function is straightforward: a digitalWrite is applied to the Arduino pins.
//function to turn Right
void TurnRight(){
digitalWrite(motorFrontRight,HIGH);
digitalWrite(motorFrontLeft,LOW);
}
//function to turn Left
void TurnLeft(){
digitalWrite(motorFrontLeft,HIGH);
digitalWrite(motorFrontRight,LOW);
}
Move Functions
The move functions are similar to the turning functions; the main difference is that you use analogWrite with an external value defined in the void. This value marks the speed of rotation of the motor. So instead of using HIGH and LOW, you use speed: 255 as a maximum value and 0 to indicate the car has stopped.
//function to move
void MoveUp(int speedD){
analogWrite(motorBackUp,speedD);
analogWrite(motorBackDown,0);
}
//function to move reverse
void MoveDown(int speedD){
analogWrite(motorBackDown,speedD);
analogWrite(motorBackUp,0);
}
Enable/Disable Functions
The enable and disable functions just write HIGH or LOW to the pin marked when they are called. You use them for the H-Bridge ENABLE pins, activating or disconnecting the motors.
void enable(int pin){
digitalWrite(pin, HIGH);
}
void disable(int pin){
digitalWrite(pin, LOW);
}
Now you're ready to test your car for the first time! Upload your Arduino code (leaving the USB cable plugged into the car; you need it for the serial communication). Run the Processing sketch and move your mouse. The car should move and turn according to your mouse position!
Proximity Sensors
Now let's introduce a bit more complexity. You are going to use a proximity sensor in the front part of the car to check the distance from another object. Using the values it returns, you are going to introduce a routine of “stopping the car” inside your Arduino code. The idea is to avoid collisions. The code will let you control the car but if it is going to crash with an object, it will override your movements and automatically stop (and actually move backwards a little).
The routine will check the value coming from the proximity sensor and it will compare it with a threshold (a value that we have defined previously in the code).
If this value is higher than the threshold it will mean that an object is really close. If that is the case, the back motor will start to turn backwards for a short period of time and after that it will check again the value coming from the sensor, repeating this routine if the value is still bigger than the threshold.
If the value from the sensor is smaller than the threshold it means that there is not any object close enough, and there is no risk of crashing, so the car will move normally.
An analog infrared (IR) distance sensor is a really simple component (Figure 9-17). It shines a beam of IR light from an LED and measures the intensity of light that is bounced back using a phototransistor. If you film this sensor with a cellphone or a camera and you look at the sensor through the screen, you will see the IR LEDs glowing.
Figure 9-17. Infrared proximity sensor
To make it work, just connect +5v and ground, and an analog signal will be returned (Figure 9-18). This signal is in a voltage proportional to the distance between the sensor and an object in front of it. The sensor you are using has a range of 80 cm (0.4 volts) and 10 cm (3 volts).
Figure 9-18. Proximity sensor connected to Arduino
You can work with the values that come from the sensor if you choose an appropriate maximum value to act as threshold in order to activate your stop car function.
The IR proximity sensor has been used in many interactive applications. Even if you are just using the outcome voltage coming from the sensor, it's worthwhile to know how the sensor works and how you can measure real distances with it. (You can skip this explanation if you want.)
The voltage returned is not linear, as you can see in Figure 9-19.
Figure 9-19. Proximity sensor's voltage/distance graph
However, you can convert the voltage returned to distance using an equation. Depending of the model of sensor, this equation will vary. For the Arduino code, the equation for transforming inputs to distance is
float distance = 12343.85 * pow(analogRead(sensorPin),-1.15);
Before adding it to your circuit, do a quick test by connecting a sensor to your Arduino board using a pigtail or an infrared sensor jumper. Use this code for the test:
int sensorPin = 0;
void setup(){
Serial.begin(9600);
}
void loop(){
int val = analogRead(sensorPin);
//float val = 12343.85 * pow(analogRead(sensorPin),-1.15);
Serial.println(val);
delay(100);
}
Use Arduino analog Pin 0 for receiving the values, and start the serial communication. Next, choose which values you want out of the Arduino: the voltage (int val) or the distance (float val). Finally, just add a delay to slow down the output.
Figure 9-20. Proximity sensor's output values
The values that you obtain in the Serial Monitor, depending on the code you use, are between 650 (really close) and 20 (object far away), as shown in Figure 9-20. If we comment the int val and uncomment the line float val, we will use the equation previously described in the code. If we upload this “modified” code to Arduino we will obtain values between 70 and 7, meaning real distance in centimeters. Using this equation is great because we have real values, but it also gives us a big error when the object is really far away, due to the math that you applied.)
Enough theory! Let's get back to the RC vehicle. To keep things simple, use the voltage values. In your case, just knowing the threshold value that indicates that your car is close to an object is enough to activate your stop car routine.
After testing, add the sensor to your car circuit. Just solder the cables to power, ground, and analog pin 0 in your prototype board. The circuit looks like the one in Figure 9-21.
Figure 9-21. Proximity sensor connected
Fix the sensor on the front area of the car using some removable adhesive (in case you want to use the sensor for another application), as shown in Figure 9-22.
Figure 9-22. Proximity sensor on car's front
Now change the Arduino code in order to get an automated response according to the data coming from the sensor. As mentioned, the idea is to stop the car and reverse it for long enough to avoid a collision. If the data coming from the sensor is high (meaning an object is really close), the back motor will automatically reverse.
Add a few variables to the beginning of the code. A new variable (Val04) activates or deactivates the automatic response. The sensor is incorporated with an analog IN pin (pin0), an initial value, and a threshold value to mark the distance from which the automated response will be active.
float Val01, Val02, Val03, Val04;
(....)
int sensorPin = 0;
int sensorValue=0;
int Value =500; //threshold for detecting object
Setup Function
The setup() function remains the same.
Loop Function
The loop() function incorporates this new value from the serial data. According to the value of Val04, Arduino reads the data coming from the sensor (automatic response on) or just marks this value as 0 (automatic response off).
Then you add a general if before all the functions, marking the value coming from the sensor and comparing it to the threshold. In other words, if no object is close, you drive the vehicle with your Processing sketch, but if an object is nearby, the code jumps directly to the new function called stopCar().
void loop() {
if (Serial.available()>5) {
char val = Serial.read();
if(val == 'S'){
Val01 = Serial.read();
Val02 = Serial.read();
Val03 = Serial.read();
Val04 = Serial.read();
}
}
if (Val04= 1)sensorValue = analogRead(sensorPin);
else sensorValue=0;
// turning left and right
if (sensorValue<Value) {
if (Val01==1){
enable(enablePinFront);
TurnRight();
}
else if (Val01==2) {
enable(enablePinFront);
TurnLeft();
}
else if(Val01==0)disable(enablePinFront);
if (Val02==1) {
enable(enablePinBack);
MoveUp(Val03);
}
else if (Val02==2) {
enable(enablePinBack);
MoveDown(Val03);
}
else if(Val02==0) {
disable(enablePinBack);
}
}
else {
enable(enablePinBack);
stopCar();
}
}
Move Functions
All move functions remain the same. You just add a new function stopCar() that exists only to call the previous function MoveDown() and disable the back motor once it has moved backwards. This function acts as a brake, avoiding collisions.
void stopCar(){
for(int i=0; i<2000; i++){
MoveDown(200);
}
disable(enablePinBack);
}
Once you have connected the proximity sensor, you can test your car. Upload this new Arduino code (you are still not wireless, so you need to leave the USB cable plugged to your Arduino board). Run the previous Processing sketch again and move your mouse. The car should move as before, but if you put an object in front of it, it should stop and then move backwards.
XBee and Wireless
The last step is to add a wireless module. You learned how to use XBee modules for wireless communication in pervious chapters. (Have a look at those chapters for a refresher on the concepts and how to install drivers in case you haven't done it before.)
As you have seen, you can use an XBee shield or just connect the cables that you need. In this case, you want to everything as compact as possible, so solder the cables coming from the XBee Explorer as described in Table 9-2 and shown in Figure 9-23.
Figure 9-23. XBee Explorer
If you solder the cables on your prototype shield, the circuit will look like that in Figure 9-24.
Figure 9-24. XBee on the circuit
Now just tidy up the cables and fix them to the vehicle to avoid disconnections when the vehicle is in movement. The final vehicle will look like Figure 9-25.
Figure 9-25. The complete hacked RC car
The very last step is to prepare a Kinect sketch and start driving your car. But first, test the car. Plug your XBee Explorer USB to your computer, run the Processing sketch, and make sure that data is getting to your Arduino. You should see a flashing Arduino LED RX. If that's happening, you should be able to drive your car just using your mouse.
Kinect RC Interface
Now you're going to use the movement of your hand to drive the vehicle.
- According to its relative position to the center of the screen, you can drive the car, moving it towards this direction.
- If your hand is in the central position, the vehicle won't move.
- If you draw a circle with your hand, you activate a virtual switch, turning on and off the automatic response, as shown in Figure 9-26.
The automatic response marks if Arduino is going to read the data coming form the sensor, and therefore it will react automatically when an object is really close using the function stopcar().
In the case this automatic response is off, the data coming from the sensor will be overwrite to 0, meaning that the car will be “blind” respect the objects that are in front and won't have any automatic reaction.
Note that this circle should be drawn close to the center of the screen where the car is not in movement so the gesture of doing a circle won't drive the car over the place.
Figure 9-26. Safety mode on and off
Similar code has been used in the previous chapters. You're just creating a few new functions and putting some text on the screen in order to display live data. The code is based on hand and circle recognition. Start by importing the libraries.
import SimpleOpenNI.*;
import processing.opengl.*;
import processing.serial.*;
SimpleOpenNI kinect;
Serial myPort;
Next, declare the NITE objects explicitly, using the NITE functions for hand tracking, gesture recognition, and circle detection. You declare the session manager (that deal with NITE gesture recognition), control points (that deal with hand points), and circle detector.
// NITE
XnVSessionManager sessionManager;
XnVPointControl pointControl;
XnVCircleDetector circleDetector;
You're going to work with text on the screen so you declare a font and some strings. (You declare them here in order to avoid null values once you start your Processing sketch. They update their value once the hand has been detected.)
// Font for text on screen
PFont font;
String Ctext;
String dir;
The other variable that you declare is handTrackFlag, which refers to a hand being detected. The same principle is applied to the boolean circleTrackFlag. Other variables that you declare are PVectors that represent the positions of the hand on the screen, and the position of the hand in the real world. Then other PVectors will represent the center of a circle when we do the gesture and again we use two different variables: one representing the center on the screen and another representing the center in the real world. Other variables are radius and a counter t, that will serve for changing the state of a Boolean activating or deactivating the automated response. PVector v is the difference between the screen center and your hand position but in 2D. Boolean automated sends data about using the proximity sensor or not, and Boolean serial allows you to send serial data. The last set of values is a set of temporary ones that are remapped before being sent via serial communication to Arduino.
// Variables for Hand Detection
boolean handsTrackFlag = false;
boolean circleTrackFlag = false;
PVector screenHandVec = new PVector();
PVector handVec = new PVector();
int t = 0;
float rad;
PVector centerVec = new PVector();
PVector screenCenterVec = new PVector();
PVector v = new PVector();
boolean automated=true;
boolean serial = true;
int Val01, Val02, Val03, Val04;
float temp01, temp02, temp03, temp04;
Setup Function
In the setup() function, you initialize all the objects that you will be using later on, starting from the Kinect ones and then calling the NITE session manager where listeners will be added. As explained in previous chapters, you need to call methods create, destroy and update, so the methods XnVControl() and XnVDetector() invoke your callback functions in order to create, destroy and update one object (a circle or a hand detection).
void setup() {
// Simple-openni object
kinect = new SimpleOpenNI(this);
kinect.setMirror(true);
// enable depthMap generation, hands + gestures
kinect.enableDepth();
kinect.enableGesture();
kinect.enableHands();
// setup NITE
sessionManager = kinect.createSessionManager("Wave", "RaiseHand");
// Setup NITE.s Hand Point Control
pointControl = new XnVPointControl();
pointControl.RegisterPointCreate(this);
pointControl.RegisterPointDestroy(this);
pointControl.RegisterPointUpdate(this);
// Setup NITE's Circle Detector
circleDetector = new XnVCircleDetector();
circleDetector.RegisterCircle(this);
circleDetector.RegisterNoCircle(this);
// Add it to the session
sessionManager.AddListener(pointControl);
sessionManager.AddListener(circleDetector);
After calling the Nite methods, add the size of your canvas and initialize the font (which should be added to the data folder). You also initialize the Strings that you defined before and finally start your serial communication (depending on a Boolean, so it's easy to connect and disconnect).
// Set the sketch size to match the depth map
size(kinect.depthWidth(), kinect.depthHeight());
smooth();
// Initialize Font
font = loadFont("SansSerif-12.vlw");
Ctext="Automated mode ON";
dir = "-";
//Initialize Serial Communication
if (serial) {
String portName = Serial.list()[0]; // This gets the first port
myPort = new Serial(this, portName, 9600);
}
}
Draw Function
The purpose of this sketch is to send data to Arduino, according to the hand position and the gesture that you perform in front of the Kinect camera. You start by calling a black background and a PVector to mark the center of the screen, as shown in Figure 9-27.
Figure 9-27. Driving the car
After that, you update the Kinect object so you can see what your Kinect is seeing.
void draw()
{
background(0);
PVector centerL = new PVector(width/2, height/2);
// Update Kinect data
kinect.update();
// update NITE
kinect.update(sessionManager);
// draw depthImageMap
image(kinect.depthImage(), 0, 0);
Next, you calculate the difference in 2D (just XY coordinates) between the projection of the hand and the center of the canvas. This operation should be familiar; remember your Processing test sketch? You are doing the same thing but using the data coming from the Kinect.
//displacement between the centre and the hand in 2D
v.x = screenHandVec.x-centerL.x;
v.y = screenHandVec.y-centerL.y;
Next, call a series of functions depending on whether the Kinect is tracking a hand. If so, it draws a hand and an arrow and then drives the car. If the Kinect detects a circle, it draws on the screen. A series of functions is running and updated continuously: some text on the canvas and the data that you are sending via serial.
if (handsTrackFlag){
drawHand();
drawArrow(v, centerL);
controlCar();
}
if (circleTrackFlag){
drawCircle();
}
textDisplay();
if (serial){
sendSerialData();
}
}
Other Functions
The function controlCar() maps all the data in just a few values in order to transmit them to the Arduino. This function is similar to the one used previously in the Processing sketch. It just remaps the arrow vector according to the coordinates X and Y and then stores these values. It uses temporary values to remap them with a series of if conditions in order to have a clean output.
void controlCar() {
temp01 = screenHandVec.x-width/2;
temp02 = height/2- screenHandVec.y;
temp03 = int(map(temp01, -width/2, width/2, 0, 255));
if (temp03 < 75) {
Val01 = 1;
}
if (temp03 > 175) {
Val01 = 2;
}
if ((temp03 > 75) && (temp03 < 175)) {
Val01 = 0;
}
temp04= int(map(temp02, -height/2, height/2, -255, 250));
if ((temp04 > 0) && (temp04 > 50)) {
Val02 = 1;
}
else if ((temp04 < 0) && (temp04 < -50)) {
Val02 = 2;
}
else {
Val02 = 0;
}
if ((temp04 > 100) && (temp04 < 150)) {
Val03 = 140;
}
if (temp04 > 150) {
Val03 = 200;
}
if ((temp04 < -100) && (temp04 > -150)) {
Val03 = 140;
}
if (temp04 < -150) {
Val03 = 200;
}
//println(Val01 + " " + Val02+ " " + Val03+ " " +Val04 );
}
The Other function represents the hand on the screen. Once the hand is detected, it appears as a red dot that makes the same movements as your hand. There's also a function for representing a circle in case the gesture is detected. With the counter t, you can change the Boolean automated every time the new circle is produced. So the automated behavior is activated and deactivated with the same function. If you draw a circle, the automated behavior is turned off, and if you repeat the gesture, it's on again. These gestures also change the color of the circle and the text referring to the automated mode as well as Val04, which is sent via serial.
// Draw the hand on screen
void drawHand() {
stroke(255, 0, 0);
pushStyle();
strokeWeight(6);
kinect.convertRealWorldToProjective(handVec, screenHandVec);
point(screenHandVec.x, screenHandVec.y);
popStyle();
}
void drawCircle() {
if(t == 1)automated=!automated;
if (automated == true){
Val04 = 0;
noFill();
strokeWeight(6);
stroke(0, 255, 0);
ellipse(screenCenterVec.x, screenCenterVec.y, 2*rad, 2*rad);
textAlign(LEFT);
Ctext = "Automated mode ON";
}
if (automated==false){
Val04 = 1;
noFill();
strokeWeight(6);
stroke(255, 0, 0);
ellipse(screenCenterVec.x, screenCenterVec.y, 2*rad, 2*rad);
textAlign(LEFT);
Ctext = "Automated mode OFF";
}
// println(automated);
}
The function drawArrow represents an arrow between two PVectors. It was used at the beginning of the chapter and it is pretty straightforward.
void drawArrow(PVector v, PVector loc){
pushMatrix();
float arrowsize = 4;
translate(loc.x, loc.y);
stroke(255, 0, 0);
strokeWeight(2);
rotate(v.heading2D());
float len = v.mag();
line(0, 0, len, 0);
line(len, 0, len-arrowsize, +arrowsize/2);
line(len, 0, len-arrowsize, -arrowsize/2);
popMatrix();
}
textDisplay represents live text (meaning that the values update automatically) on the screen. The function text asks for a value or string plus X and Y positions. In this sketch, you are showing the car's direction, speed of the back motor, and whether the automated mode is activated or not. According to the values you are sending to Arduino, you make the rules for representing these values using a series of if conditionals. The last function is sendSerialData, which as usual uses a letter to identify each message.
void textDisplay(){
text(Ctext, 10, kinect.depthHeight()-10);
int value;
if(Val02 == 0) {
value=0;
}
else {
value = Val03;
}
text("Speed: "+value, 10, kinect.depthHeight()-30);
text("Direction: "+dir, 10, kinect.depthHeight()-50);
if ((Val02 == 1) && (Val01 == 0)) {
dir ="N";
}
if ((Val02 == 2) && (Val01 == 0)) {
dir="S";
}
if ((Val02 == 0) && (Val01 == 1)) {
dir="W";
}
if ((Val02 == 0) && (Val01 == 2)) {
dir="E";
}
if ((Val02 == 1) && (Val01 == 2)) {
dir="NE";
}
if ((Val02 == 2) && (Val01 == 2)) {
dir="SE";
}
if ((Val02 == 2) && (Val01 == 1)) {
dir="SW";
}
if ((Val02 == 1) && (Val01 == 1)) {
dir="NW";
}
}
void sendSerialData() {
// Serial Communcation
myPort.write('S'),
myPort.write(Val01);
myPort.write(Val02);
myPort.write(Val03);
myPort.write(Val04);
}
The last part is the NITE callbacks. As shown in previous chapters, they refer to the events of creating an object (hand or circle) once the gesture is detected, destroying it when the hand or the circle has gone off the screen, and updating them. Hand events and circle events have different callbacks. The only lines that are new are the ones that refer to the counter t. Each time a new circle is created, the counter starts to add, meaning that just once it will have the value of 1 (which is necessary for changing a Boolean). Each time that circle is destroyed, this counter comes back to 0.
// XnVPointControl callbacks
void onPointCreate(XnVHandPointContext pContext) {
println("onPointCreate:");
handsTrackFlag = true;
handVec.set(pContext.getPtPosition().getX(),
pContext.getPtPosition().getY(),
pContext.getPtPosition().getZ());
}
void onPointDestroy(int nID) {
println("PointDestroy: " + nID);
handsTrackFlag = false;
}
void onPointUpdate(XnVHandPointContext pContext) {
handVec.set(pContext.getPtPosition().getX(),
pContext.getPtPosition().getY(),
pContext.getPtPosition().getZ());
}
// XnVCircleDetector callbacks
void onCircle(float fTimes, boolean bConfident, XnVCircle circle) {
println("onCircle: " + fTimes + " , bConfident=" + bConfident);
circleTrackFlag = true;
t++;
centerVec.set(circle.getPtCenter().getX(),
circle.getPtCenter().getY(), handVec.z);
kinect.convertRealWorldToProjective(centerVec, screenCenterVec);
rad = circle.getFRadius();
}
void onNoCircle(float fTimes, int reason) {
println("onNoCircle: " + fTimes + " , reason= " + reason);
circleTrackFlag = false;
t = 0;
}
Now, test this code with your vehicle. The Arduino code hasn't changed, so just plug the Kinect and the XBee USB Explorer. Run the new Processing sketch and start to drive, as shown in Figure 9-28!
Figure 9-28. Driving your RC vehicle
Summary
If you use this code (make sure that the Boolean serial is set to true), you will be sending serial data to drive a hacked RC car. Using the position of your hand, you will be able to drive it; using a gesture, you can activate a simple anti-collision system.
You have learned several new things in this chapter. Regarding hardware, you learned about RC motors and how to drive them using a simple H-Bridge. You opened a remote control car and, using the components, reprogrammed its behavior using Arduino. You also learned how proximity sensors work and the kind of data you can obtain from them.
This application is only the principle; the same logic can be applied to any RC vehicle. Next time, hack a helicopter or a boat! Just open it up, investigate what's inside, and start to have fun!