Image recognition software for driver monitoring systems
Since 2013, we have been developing image recognition software for driver monitoring systems (DMS). Since 2018, FaceU (R) for DMS has been adopted as a DMS for mass-produced vehicles by major auto parts manufacturers and auto parts manufacturers in Japan and overseas, and a total of approximately 780,000 units. We are proud of our achievements (as of the end of December 2021).
In order to reduce traffic accidents, there is a need for a driver monitoring function that detects and alerts the driver's condition such as looking away, distracting, falling asleep, and losing posture. In addition, DMS is expected to enter into a full-fledged period after 2020 in response to the assessment program represented by Euro NCAP and the flow of automated driving.
FaceU (R) for DMS has functions to detect human head position, face orientation, line of sight, eyelid open / closed degree, drowsiness, etc. from IR camera and RGB camera images, and to authenticate individuals. Image recognition software that can be used for functions (judgment such as looking away, distraction, falling asleep, posture collapse, and driver authentication).
The license for FaceU (R) for DMS Ver.2.8 has been started from October 1, 2019.
Based on more real-world data, it achieves the same low processing load and memory saving size as the previous version while further improving robustness and recognition accuracy. In addition, when used in conjunction with our occupant status recognition software, it can recognize dangerous behavior such as mobile phone operation and smoking behavior, contributing to the development of advanced DMS products for customers.
Based on the experience and technology cultivated in the adoption record in mass-produced vehicles, we will continue to improve the performance and functionality of FaceU (R) for DMS and support customers' DMS development.
Features
1. Resource saving and high performance
Combining the traditional statistical analysis method with the newly developed Deep Learning high-speed inference engine, it achieves compactness and high performance. No GPU, FPGA, or NPU is required, and real-time processing is possible with only a relatively inexpensive CPU (eg, ARM Cortex-A7 1GHz x 1).
Figure 1: Effective detection range
2. Can detect even a wide range of faces
Effective detection range: Yaw -90 to +90 degrees, Pitch -40 to +70 degrees (Figure 1)
The degree of freedom of the camera installation position is high and can be combined with various in-vehicle equipment (Figure 2).
Figure 2: Examples of devices that can be combined
3. High robustness for low image quality
Recognize blurry image quality such as brightness, low resolution, blur, and noise with high accuracy (Figure 3).
A relatively inexpensive camera and a small number of light sources operate, contributing to system cost reduction
Figure 3: High robustness against low image quality (brightness, resolution, noise)
Supported platforms
Portable design regardless of CPU and OS, compatible with various platforms. Tuning according to memory size and CPU power is also possible (optional)
Proven platform
・Renesas R-Car E2
・Renesas R-Car M2
・Renesas R-Car V2H
・Renesas R-Car E3
・Renesas R-Car M3
・Renesas R-Car H3
・NXP i.MX7
・NXP i.MX8
・Panasonic GerdaC
・Allwinner V40
・Allwinner V3
・Ambarella A12A55
・RasberyPi 3B
・XILINX ZYNQ7010
・XILINX ZYNQ7020
Reference specs
Software size
- Library: 25MBytes
- RAM: 24MBytes (when the input image size is VGA)
Function
- Face ID (Masked faces are also supported)
- Face detection
- Face parts detection
- Head position detection (x,y,z)
- Face orientation detection
- Gaze detection
- Eye opening / closing
- Blink detection
- Mouth opening / closing
- Sleepiness
- Age estimation
- Gender estimation
Face recognition conditions
Face width: 40 pixels or more
Face orientation: Yaw: within ± 90 degrees, Pitch: within -40 to +70 degrees, Roll: within ± 45 degrees
Shield: Mask / Sunglass face / part detection possible
processing speed
- 15-20fps (ARM Cortex-A7 1.0GHz x 1Core)
- 30fps(ARM Cortex-A53 1.0GHz x 1Core)
* Varies depending on functions and conditions
Demo
DMS with Cluster on R-Car E3
PUX's DMS(Driver Monitoring System) solution and High quality drawing on a meter cluster achieved by "R-Car E3 (1 chip solution)"