Image recognition software for driver monitoring systems

 Since 2013, we have been developing image recognition software for driver monitoring systems (DMS). Since 2018, FaceU (R) for DMS has been adopted as a DMS for mass-produced vehicles by major auto parts manufacturers and auto parts manufacturers in Japan and overseas, and a total of approximately 70,000 units have been shipped to three car manufacturers and four models. We are proud of our achievements (as of the end of August 2019).

In order to reduce traffic accidents, there is a need for a driver monitoring function that detects and alerts the driver's condition such as looking away, distracting, falling asleep, and losing posture. In addition, DMS is expected to enter into a full-fledged period after 2020 in response to the assessment program represented by Euro NCAP and the flow of automated driving.

FaceU (R) for DMS has functions to detect human head position, face orientation, line of sight, eyelid open / closed degree, drowsiness, etc. from IR camera and RGB camera images, and to authenticate individuals. Image recognition software that can be used for functions (judgment such as looking away, distraction, falling asleep, posture collapse, and driver authentication).

The license for FaceU (R) for DMS Ver.2.8 has been started from October 1, 2019.
Based on more real-world data, it achieves the same low processing load and memory saving size as the previous version while further improving robustness and recognition accuracy. In addition, when used in conjunction with our occupant status recognition software, it can recognize dangerous behavior such as mobile phone operation and smoking behavior, contributing to the development of advanced DMS products for customers.

Based on the experience and technology cultivated in the adoption record in mass-produced vehicles, we will continue to improve the performance and functionality of FaceU (R) for DMS and support customers' DMS development.

Features

1. Resource saving and high performance

Combining the traditional statistical analysis method with the newly developed Deep Learning high-speed inference engine, it achieves compactness and high performance. No GPU, FPGA, or NPU is required, and real-time processing is possible with only a relatively inexpensive CPU (eg, ARM Cortex-A7 1GHz x 1).

Resource saving and high performance

Figure 1: Effective detection range

2. Can detect even a wide range of faces


Effective detection range: Yaw -90 to +90 degrees, Pitch -40 to +70 degrees (Figure 1)
The degree of freedom of the camera installation position is high and can be combined with various in-vehicle equipment (Figure 2).

Can detect even a wide range of faces

Figure 2: Examples of devices that can be combined

3. High robustness for low image quality

Recognize blurry image quality such as brightness, low resolution, blur, and noise with high accuracy (Figure 3).
A relatively inexpensive camera and a small number of light sources operate, contributing to system cost reduction

High robustness for low image quality

Figure 3: High robustness against low image quality (brightness, resolution, noise)

Supported platforms

Portable design regardless of CPU and OS, compatible with various platforms. Tuning according to memory size and CPU power is also possible (optional)

Proven platform

・Renesas R-Car E2
・Renesas R-Car M2
・Renesas R-Car V2H
・Renesas R-Car E3
・Renesas R-Car M3
・Renesas R-Car H3
・NXP i.MX7
・NXP i.MX8
・Panasonic GerdaC
・Allwinner V40
・Allwinner V3
・Ambarella A12A55
・RasberyPi 3B

Reference specs

Software size

  • Library: 25MBytes
  • RAM: 24MBytes (when the input image size is VGA)

Function

  • Face ID
  • Face detection
  • Face parts detection
  • Face orientation detection
  • Gaze detection
  • Eye opening / closing
  • Sleepiness
  • Age estimation
  • Gender estimation

Face recognition conditions

Face width: 40 pixels or more
Face orientation: Yaw: within ± 90 degrees, Pitch: within -40 to +70 degrees, Roll: within ± 45 degrees
Shield: Mask / Sunglass face / part detection possible

processing speed

  • 15-20fps (ARM Cortex-A7 1.0GHz x 1Core)
  • 30fps(ARM Cortex-A53 1.0GHz x 1Core)

     * Varies depending on functions and conditions