ABSTRACT
Effective face detection in real-time is an essential procedure for achieving autonomous motion in telepresence robots. Since the procedure demand high computation power, using it to create autonomous motion in low-cost robots is a challenge. This paper addresses this issue and making three contributions.
First, the process to enabling the real-time face detection on Raspberry Pi’s graphical processor is presented. Second, the development of an autonomous pan-tilt telepresence robot to follow an interlocutor face using two Raspberry Pi-1 model B is demonstrated. Third, the evaluation on resource requirements when operating the robot in various scenarios is described.
The face detection module ran in average at 16.7 Quarter VGA frames per second, while mediating real-time video conversation remotely between two parties. The results confirmed that vision-based autonomous motion can be added to a low-cost telepresence robots with acceptable performance. Thus, making secure telecommunication via robots is viable with less budget constraint.
REAL-TIME FACE DETECTION ON RP-1B COMPUTER
Fig. 1 Process of the real-time face detection model and the real-time video communication with the connected smartphones.
Real-time face detection has been implemented on a telepresence robot which has two RP computers, model 1B. As shown in Fig.1, the face detection module is allocated on the RP#1 computer, while the real-time video conferencing is handled by the RP#2 computer. The robot has two camera, i.e., the RP camera module for face detection and the web cam for video conferencing.
Existing research has shown the benefits of using LBP algorithms and offloading the face detection to the RP’s GPU. Thus, the LBP algorithm from the Open Source Computer Vision (OpenCV) library has been used for face detection. To offload the computation of the face detection process to the CPU, the multi-media abstraction layer (MMAL) application program interface is used.
AUTONOMOUS PAN-TILT TELEPRESENCE ROBOT
Fig. 2 The telepresence robot (left) prototype of the robot hardware design (right) the robot’s screen when testing face detection module.
The real-time face detection unit has been added to the development of a telepresence robot. This is to allow the robot to have autonomous PTU to follow the interlocutor face. The robot consists of a monitor, a web cam, an RP camera module, a ServoBlaster library and two servo motors, a power supply (as the prototype shown in Fig.2).
As described, two RP computers model 1B have been used for processing the face detection and video conferencing. The specification of the robot is listed in Table 1. The total cost of the robot prototype is approximately 550 USD as of today currency value. For safety reason, the robot will be packaged using Acrylic cover which push the cost up to 600 USD.
EVALUATION AND DISCUSSION
Fig. 6 Increased memory requirement when using face following during the video conference with different modes of content (reference with baseline).
Fig. 7 Increased CPU loads when using face following during a video conference (value shown in times higher than the baseline).
Fig. 8 Increased bandwidth requirement when using face following during a video conference (value shown in times higher than the baseline).
The third step of the experiment quantified the resource requirement while the autonomous face following feature is turn on, during a video conference session. The memory requirement range from 120 – 125 MB in all content mode, increased from 25 – 34 percent over the baseline (Fig. 6).
The CPU load was push to peak when using face following feature, increased quite stably from 15.7 – 19.4 times over the baseline (Fig. 7). Similar to the second experimental step, the sending bandwidth was dominate, however, shown less variation. The send bandwidth range from 760 Bps – 56 KBps, increased from 43.8 – 95.5 times over the baseline (Fig. 8) during a video conference session.
DISCUSSION
The internal memory of two RP computers (1GB in total) is enough for all telepresence robot operations including the autonomous face following feature. Adding face detection module pushes the CPU load up to its maximum. Thus, when adding a new feature, like mobility, to the robot a separate processing resource is required. Using the ServoBlaster library for interfacing with servo motors added overhead on the CPU load, thus, causing slower response from the robot. In the future work, this hardware will be replaced with an Arduino controller.
CONCLUSION
This paper presents a technique to enable real-time face detection on the RP’s GPU for making autonomous face following in an affordable, pure RP telepresence robot. The design of the remote steering technique by using a dedicated RP computer to support the real-time video conference has been described.
The proposed techniques have been evaluated by measuring the memory, processing time, and bandwidth requirements. The results showed that the proposed face detection implementation exploited full CPU loads with little extra memory required, and has no impact on the quality of the real-time video conferencing. This showed that vision based autonomous motion can be added to a low-cost, pure RP telepresence robots with acceptable performance.
Source: Thammasat University
Authors: Krit Janard | Worawan Marurngsith