As Intel Galileo has a different processor, Intel Quark, from regular computers, we need to recompile and build any libraries with the cross-compiler and toolchain provided with Yocto Project. A successful cross-compile process will create necessary binaries for us to run on Intel Galileo.

In the Chapter 1, Getting Started with Intel Galileo, we initialized a build environment for Yocto Project in order to create a Linux filesystem and toolchain for Intel Galileo. Yocto Project allows you to build a full Linux filesystem or any other software without rebuilding the whole Linux filesystem. During the build environment initialization process, many Yocto Project recipes have been downloaded but not been used as they have not been included in the image. OpenCV recipe is one of the recipes downloaded during this process along with many others.

You can locate the OpenCV Yocto Project recipe in $BUILD_DIRECTORY/meta-clanton_v1.0.1/meta-oe/meta-oe/recipes-support/opencv directory. opencv_2.4.9.bb is the file that includes instructions and rules to download the OpenCV source from an upstream repository and build. When you open the opencv_2.4.9.bb file, you will see that the line starts with DEPENDS; it is the variable that includes the OpenCV library, which needs to be built. v4l2-utils is the library that includes the kernel modules, so when we start building, OpenCV v4l2 modules will be built together.

$ cd $BUILD_DIRECTORY/meta-clanton_v1.0.1
$ cd source poky/oe-init-build-env yocto_build
$ bitbake opencv

OpenCV ipk packages are listed under $BUILD_DIRECTORY/meta-clanton_v1.0.1/yocto_build/tmp/deploy/ipk/i586. The v4l2 kernel module packages are in the $BUILD_DIRECTORY/meta-clanton_v1.0.1/yocto_build/tmp/deploy/ipk/clanton directory.

In order to install related packages, you can copy OpenCV, the v4l2 packages, and their dependency module packages to the Intel Galileo filesystem and then install them with the help of opkg tool. In addition to installing the packages to Intel Galileo, you also need to install development files to the Intel Galileo toolchain directory on your host to compile and build your OpenCV application. In Chapter 1, Getting Started with Intel Galileo, we installed SDK to the /opt/clanton-tiny/1.4.2/ directory. You need to install development files into the /opt/clanton-tiny/1.4.2/sysroots/i586-poky-linux-uclibc directory.

However, instead of using a copy and install process, you can add OpenCV to the image recipe and that will automatically install the required packages to the filesystem image.

Edit $BUILD_DIRECTORY/meta-clanton_v1.0.1/meta-clanton-distro/recipes-core/images/image-full.bb, add a new line as IMAGE_INSTALL += "opencv", and rebuild the image and toolchain. Then follow the instructions from Chapter 1, Getting Started with Intel Galileo, to copy the required files to the SD card and re-install the toolchain into your host machine supporting OpenCV.

In order to work with images or video coming from your security camera, you need to know the basics of the OpenCV library. OpenCV is an open source cross-platform computer vision and machine learning library. OpenCV provides hundreds of algorithms for analytics on images, mostly for real-time use cases, and that's why it is used in robotics appliances.

Visual analytics algorithms require a vast amount of processing power. Therefore, OpenCV requires a powerful CPU and GPU for complex analytics. However, we will only use some basic functionality from OpenCV just for surveillance purposes. Intel Galileo's CPU is powerful enough to do basic image manipulation such as saving a captured image or video from a camera. It will be enough for home surveillance. If you would like to do more complex analytics with your home automation application such as face recognition, Intel Galileo may lack the required computational power.

Let's briefly overview the functionality we will use from OpenCV to capture frames from a network camera or USB camera.

We will have two sample applications to capture images from a USB camera and the D-Link DSC-930L network camera and save them to Intel Galileo.

We started with creating a new C Project, as described in Chapter 1, Getting Started with Intel Galileo, when we created a C file as capture_from_usb.c. We need to add OpenCV libraries to Makefile in order to build application with OpenCV libraries. Makefile.am should look like this:

bin_PROGRAMS = capture_from_usb
capture_from_usb_SOURCES = capture_from_usb.c
AM_CFLAGS = @capture_from_usb_CFLAGS@
AM_LDFLAGS = @capture_from_usb_LIBS@ -lopencv_core -lopencv_imgproc - lopencv_legacy -lopencv_ml -lopencv_highgui -lm
CLEANFILES = *~  

While capturing from USB devices, we need to provide the index of the video file. In our case, the video file is /dev/video0, and so we put the index as 0.

#include <stdlib.h>
#include <stdio.h>
#include "opencv/cxcore.h"
#include "opencv/cv.h"
#include "opencv/highgui.h"
#define USBCAM 0
int main(void) {
        //Get USB Cam Device
        CvCapture *pCapturedImage = cvCreateCameraCapture(USBCAM);
        //Get Frame from Capture Device
        IplImage *pSaveImg = cvQueryFrame(pCapturedImage);
        //Save Image to Filesystem
        cvSaveImage("capture.jpg", pSaveImg, 0);
        //Release Image Pointer
        cvReleaseImage(&pSaveImg);
        return 0;
}

When we run the application capture_from_usb binary inside the Intel Galileo, it will create the capture.jpeg file.

root@clanton:~# ./capture_from_usb 

Let's create a new project to capture from a network camera. In the new project, we created capture_from_network.c and build with Intel Galileo toolchain.

While accessing streams from a network camera, we need to provide the IP address of the network camera. In our case, it is 192.168.2.141 in our local network. In most cases, network cameras are secured with user credentials. We also provided the username and password as admin and 123 respectively with the link with IP_CAM, as seen in the sample code given here:

#include <stdlib.h>
#include <stdio.h>
#include "opencv/cxcore.h"
#include "opencv/cv.h"
#include "opencv/highgui.h"
#define IP_CAM "http://admin:[email protected]:80/mjpeg.cgi?user=admin&password=1 23&channel=0&.mjpg"
int main(void) {
  //Get Capture Device assign to a memory location
  CvCapture *pCapturedImage = cvCaptureFromFile(IP_CAM);
  //Frame to Save
  IplImage *pSaveImg = cvQueryFrame(pCapturedImage);
  //Save Image to Filesystem
  cvSaveImage("capture.jpg", pSaveImg, 0);
  //Release Memory Pointer
  cvReleaseImage(&pSaveImg);
  return 0;
}

When we run the application capture_from_network inside Intel Galileo, it will create the capture.jpeg file:

root@clanton:~# ./capture_from_network

You can copy the .jpeg file to your host machine to open and view the frame. Just like copying files to Intel Galileo from your host machine using the SCP tool, you can use SCP to transfer files to your host machine using your host machine's credentials.

Let's follow up from the previous section with saving video, instead of only one frame, to Intel Galileo. We will get the video from a network camera and will save it to Intel Galileo.

We have created a new project and a new .c file save_video.c in the project. We will use the same libraries that were used in capturing an image. Our aim is to record 5 seconds of video to Intel Galileo, as shown here:

#include <stdlib.h>
#include <stdio.h>
#include "opencv/cxcore.h"
#include "opencv/cv.h"
#include "opencv/highgui.h"
#include <time.h>
#define IP_CAM "http://admin:[email protected]:80/mjpeg.cgi?user=admin&password=1 23&channel=0&.mjpg"
#define FPS 15
#define DURATION 5 //seconds
int main(void) {
  //Capture Device and Captured Frame
  CvCapture *pCapture = cvCaptureFromFile(IP_CAM);
  int width = (int) cvGetCaptureProperty(pCapture, CV_CAP_PROP_FRAME_WIDTH);
  int height = (int) cvGetCaptureProperty(pCapture, CV_CAP_PROP_FRAME_HEIGHT);
  //Get Size of Video
  CvSize S;
  S.height = height;
  S.width = width;
  //Frame Structure
  IplImage *frame;
  //Create VideoWriter
  CvVideoWriter* videoWriter = cvCreateVideoWriter("record.avi", CV_FOURCC('X', 'V', 'I', 'D'), FPS, S, 1);
  //Get Current Time
  time_t now;
  struct tm *tm;
  now = time(0);
  //Start Recording for Given Time
  printf("Recording....
");
  if ((tm = localtime(&now)) == NULL) {
    printf("Can't Get Time
");
    return -1;
  }
  int start = tm->tm_sec;
  int progress = tm->tm_sec;
  while ((progress - start) <= DURATION) {
    frame = cvQueryFrame(pCapture);
    cvWriteFrame(videoWriter, frame);
    now = time(0);
    if ((tm = localtime(&now)) == NULL) {
      printf("Can't Get Time
");
      return -1;
    }
    progress = tm->tm_sec;
  }
  printf("Saved....
");
  //Release Pointers from Memory
  cvReleaseCapture(&pCapture);
  //This is needed to make recorded video container to be closed properly
  cvReleaseVideoWriter(&videoWriter);
  return 0;
}

When we have run the save_video binary, it will create a record.avi file. You can copy the file to your host machine and view the recorded video with a video player:

root@clanton:~# ./save_video
Recording ....
Saved ....

$ sudo mount /dev/sdc1 /media/sdcard

$ sudo fsck.ext3 –f /media/sdcard/image-full-galileo- clanton.ext3

$ sudo resize2fs /media/sdcard/image-full-galileo-clanton.ext3 1638400

Accessing the video camera in your home remotely is essential for home security. If you want to connect your Intel Galileo to a cloud system and access a video stream through your Intel Galileo, you need to stream video from Intel Galileo. You may also want to stream video from a USB device and access it remotely; for this you would need to create an application, or a worker thread in your application, to stream the video.

Since Intel Galileo doesn't have GPU or any other supported video encoders/decoders, it will be wise not to try any transcoding to change the video format. A suggested method is to stream video from Intel Galileo, writing the JPEG frames to a network socket, and have another device read the frames.