SentinEye™  Pedestrian Avoidance

What's the worst thing that can happen to a driver in an autonomous car ?  Most people would say "hit a pedestrian".  If there is a scale of things not to hit it might be pet, bicyclist, motorcyclist, adult, child.  Hit an object or another car, and it's likely that no one is hurt or at least not seriously. Hit a pedestrian and the outcome is almost certainly bad.

Today's driver-assisted and fully autonomous vehicles are running experimental technology and they will be for years.  There are 1000s of "corner cases" they are not yet capable of dealing with.  A pedestrian is waiting to cross the street, but he's motionless, looking down at his phone.  The car can legally turn, but should it wait ?  What would a human driver do ?  First establish eye contact, or maybe a slight tap on the horn, or a shout ?  A bicyclist is crossing a busy street using the crosswalk.  He has the "walking man" signal, so he's legal, but so is a car turning left where he's about to ride.  He holds up his hand -- not much, just a few inches off the handlebars -- effectively saying "I'm going first".  No problem, right ?  A slight hesitation by the driver and both go their separate ways.  Two women are crossing the main loop in a shopping center; they are distracted, looking at stores.  A car waits for them ... finally they establish eye contact and the driver gives the "come on" signal with her fingers, and the women cross safely.

These are common enough scenarios that drivers might encounter a few time times in a week.   But drivers are ultra careful; they make nuanced judgments based on a wide range of sensory input and behavioral context, they know what it's like to be the pedestrian, and they know the consequences.  Can an autonomous vehicle drive this way ?  Can it establish eye contact ?  Can it consider behavioral context ?  Not for many years.  The same goes for other human interaction such as hand signals and horn taps.  State departments of transportation can't bring themselves to require autonomous vehicles to display a visible indicator, much less enforce human interaction requirements.  Something like a light bar indicating fully autonomous, partial, or human driver control, seems like common sense.  At least in that way nearby human drivers, bicyclists, and pedestrians could make safe decisions as to how much space they want to give autonomous cars, and whether to forego human interactions.  Unfortunately, common sense for autonomous vehicles is not likely to prevail until regulation is enacted years from now.


The SentinEye™ Solution

SentinEye™ is an in-car high performance computing (HPC) system dedicated to identifying and warning of dangerous situations.  It is not intended to drive or navigate.  SentinEye™ can be used in both non-autonomous and autonomous cars.

For non-autonomous cars, SentinEye gives drivers an increased level of confidence, and more importantly, lets them know when they're near semi or fully autonomous cars.  For autonomous car drivers, SentinEye is essentially a "redundant system", a dedicated safeguard.  SentinEye focuses exclusively on a limited set of objectives, detecting and warning of the following:

  •  pedestrian and object avoidance
  •  drowsy driving, crossing onto the shoulder
  •  nearby autonomous vehicles

  
The SentinEye™ System -- How it Works

Unlike other small, lightweight systems in GPS-style dashboard packaging, SentinEye is built for high performance, high throughput, reliability, and high availability.  Signalogic is an engineering company, not a high profile start-up; our emphasis is on function and accuracy, not form.

The base SentinEye™ system includes two (2) USB cameras, 150W AC inverter, and a small form-factor HPC server.  The server performs image pre-processing, and object / pedestrian detection and recognition, and stabilization and cancellation of relative motion.  Recognition is performed using advanced deep learning models (several models running concurrently).

Fundamental reasons for this approach include:

  • Cloud compatibility -- ability to run the same algorithms as the cloud, including the most accurate deep learning models based on the latest R&D, continuously tested in the cloud against huge data sets, and continuously downloaded to the vehicle (when connectivity permits)
     
  • Real-time response -- providing drivers with the fastest possible response with no delay
     
  • Smarter decision making -- by running multiple concurrent deep learning models, called the "consensus" approach, SentinEye provides the highest level of recognition accuracy and lowest level of false positives
     
  • Minimal Internet -- reduced need for Internet connectivity

This approach requires an HPC server with size and power consumption reduced by two orders of magnitude, a technology that Signalogic has developed over several years.  This "car supercomputer" forms the basis of the SentinEye system, fully software compatible with its larger counterparts in the cloud.

For debug and monitoring purposes, SentinEye allows optional keyboard, mouse, and display to be connected.  A phone can be connected via USB for display or data purposes.  When available, WiFi is used for Internet connectivity, including upload of new deep learning training data acquired during operational driving.

False Positives

As has become clear from the current state of autonomous car testing, false positives are a major problem.  An autonomous system might well detect a pedestrian or cyclist at the vision layer of software, but higher "decision" layers of software may treat the detection as a "false positive" due to priority placed on a "smooth ride" (i.e. avoiding unnecessary braking).  SentinEye suffers from no such conflict in priorities.  SentinEye warns the driver any time a pedestrian is detected within the current physical range of an accident (given proximity, relative velocities, reaction time, braking time, etc).  Different levels and types of audible warning are used to indicate urgency; no higher layers of decision are involved.  The driver can then make a decision whether to brake.

Warning of Fully Autonomous Vehicles

SentinEye™ allows optional Lidar and radar detectors to be connected in order to detect autonomous vehicles.  Laser and radio waves emitted from partial or fully autonomous vehicles are corroborated with optical recognition to let you know which cars are in partial or fully autonomous mode, and let you decide how much distance to give -- or whether to avoid them entirely.

Status

SentinEye™ is currently under development, with initial availability in 4Q 2018 and initial cost $2799.  We are accepting pre-orders.  For more information, send e-mail inquiries to info (at) signalogic (dot) com.