Integrate Light-Weight Deep Learning Tools with Internet of Things

Today’s smart devices, such as collision avoidance system, rely on detectionof past and current events and then react to mitigate and reduce the severity of the ultimate consequence-The lack of proactiveness often renders such a detection system less useful for emergency situation- The ability to predict the future based on what has been heard, sensed, observed at the current moment is ofutmost importance for real-time systems, such as autonomous driving, safety and security monitoring and warning- It is quite common that a prediction system completely reconstructs (i-e-, predicts) the raw image pixels of the future frame-Howeverit does not necessarily reflect human intelligence where the future is in an abstract format and complete visual reconstruction of a scene is overkill for decision making and tooslow for smart devices, such as autonomous driving, collision avoidance system, and safety monitor- A novel approach, i-e-, Deep Sensor, is introduced in this proposal to execute learning tasks in a contextually-meaningful latent space, generate the abstract representations (embeddings) of future frames, and then perform classification among the learned sequence of embeddings- The objectives in this proposal are to integrate recent successes in neural networks, smart sensors and the Internet of Things (IoTs) and design a cost-effective real-time prediction system that learns from the sequenceof past video frames and generates the abstract representation of frames for up to several seconds in future-