Getting Started with Facial Recognition in ROS

Introduction:


Facial Recognition works in three phases.

• First, the computer must collect data of who you are. It receives frames from your visual input device and uses the pixels to create eigenfaces and run its preprocessing algorithms.
  
• Secondly, the computer then trains on that data set to recognize the most common features that make you up. You are described mathematically and labelled with a given label. We call this supervised learning.
  
• Lastly, the computer is able to recognize you by the label given in step 2. It can then monitor the visual input stream and process the faces in the video stream to compare it to what it has in its memory. If there is a match, it can let you know there has been a match.
  

Brown University was kind enough to make this public for everyone. It is a facial recognition package implemented in ROS and written in Python. Here is how one would go about doing this if they already have ROS set up.

---

Install the Brown University package:

• ~/ros_workspace:
  
  
• $ cd ~/ros_workspace
  
  
• $ svn checkout ​http://brown-ros-pkg.googlecode.com/svn/trunk/ brown-ros-pkg-read-only
  
  Update the ROS package list:
  
  
• $ rospack profile
  
  Check if everything is working by typing the following command:
  
  
• $ roscd ros_scratch
  
  Finally, compile the ros_scratch node:
  
  
• $ roscd ros_scratch $ rosmake
  
  

---

Install the dependencies for gscam

• $ rosdep install gscam
  
  Compile gscam.
  
  
• $ rosmake gscam
  

---

Setup

• $ ls /dev/video*
  
  Run the gscam ROS Node. Open several terminals. In the first terminal, start roscore.
  
  
• $ roscore
  
  In the second terminal window, start gscam. gscam must be run from its bin directory.
  
  
• $ roscd gscam $ cd bin $ export GSCAM_CONFIG="v4l2src device=/dev/video0 ! video/x-raw-rgb ! ffmpegcolorspace" $ rosrun gscam gscam
  
  With the gscam node running, open a new terminal and execute:
  
  
• $ rostopic hz --window=1000 /gscam/image_raw

---

More setup

• $ rosrun image_view image_view image:=/gscam/image_raw
  
  
• Download procrob_functional-master.zip from ​https://github.com/procrob/procrob_functional/downloads
  
  
• Move the face_recognition package into your ros directories. The face_recognition is nested inside of the zipped download.
  
  
• Do rosmake face_recognition when its in the place you want it to be. (Your ROS workspace preferably)
  
  
• In separate terminals run the face recognition server and client as follows:

---

Once that is all done and set up, begin with the demo:

• In a new terminal, start your roscore.
  
  
• $roscore
  
  Open a new terminal. This will feed the webcam data into the gscam package
  
  
• $roscd gscam $cd bin $export GSCAM_CONFIG="v4l2src device=/dev/video0 ! video/x-raw-rgb ! ffmpegcolorspace"
  
• $rosrun gscam gscam /gscam/image_raw:=/camera/image_raw
  
  Change directories so you're at ~/your_workspace/face_recognition (Make sure you're in the face_recognition package when you run these or else it won't work!)
  
  
• $rosrun face_recognition Fserver $rosrun face_recognition Fclient
  
  Acquire data to learn from:
  
  
• $rostopic pub -1 /fr_order face_recognition/FRClientGoal -- 2 "Your_name_here"
  
  Tell the package to train from the images you have just acquired. In the name of data to train, put whatever you put in the last command.
  
  
• $rostopic pub -1 /fr_order face_recognition/FRClientGoal -- 3 "The_name_of_data_to_train"
  
  Recognizing and classifying your face. This will recognize the first face it sees that is inside its database.
  
  
• $ rostopic pub -1 /fr_order face_recognition/FRClientGoal -- 0 "none"
  

---

What the commands do:

• $rostopic pub -1 /fr_order face_recognition/FRClientGoal -- someNumber someString
  
  where someNumber = {0,4} with
  
  
• someNumber = 0, (recognize_once) acknowledge the first face recognized in the video stream. When the first face is recognized with a confidence value higher than the desirable confidence threshold, the name of the person and the confidence value are sent back to the client as result.
  
  
• someNumber = 1, (recognize continuous) to continuously recognize faces in the video stream. Every face recognized with confidence value higher than the desirable confidence threshold and its confidence value are sent back to the client as feedback. This goal is persuaded for infinite time until it is canceled or preempted by another goal.
  
  
• someNumber = 2, (add_face_images) Goal is to acquire training images for a NEW person. The video stream is processed for detecting a face which is saved and used as a training image for the new person. This process is continued until the desired number of training images for the new person is acquired. The name of the new person is provided as "order_argument"
  
  
• someNumber = 3, (train) The database is (re)trained from the training images
  
  
• someNumber = 4, (exit) the program exits.
  
  

---

Modifications of commands:

• You can modify the base command with these parameters:
  
  
• "add_face_number" (int, default = 25): a parameter for the "add_face_images" goal which determines the number of training images for a new person to be acquired from the video stream
  
  
• "show_screen_flag" (boolean, default = true) : if output screen is shown
  
  
• "confidence_value" (double, default = 0.88) : a face recognized with confidence value higher than the "confidence_value" threshold is accepted as valid.
  

Problems I ran into:

• if you get an error saying something about /home/chrisclick do a gedit Makefile and edit the name of chrisclick to be whatever
  
  your user directory is. to find out just type pwd if you are unsure.
  
  
• In the command above, the export command uses /dev/video0. Be sure to use the actual port your webcam is on if it is different.
  
  If you get these errors:
  
  
• /usr/bin/ld: CMakeFiles/Fclient.dir/src/face_rec_client.o: undefined reference to symbol 'vtable for boost::detail::thread_data_base'
  
  Then add this to your CMakeLst.txt
  
  
• rosbuild_link_boost(Fserver thread) rosbuild_link_boost(Fclient thread)
  
  after the line
  
  
• rosbuild_add_executable(Fclient src/face_rec_client.cpp)