Visual Cortex On Silicon

Visual Cortex On SiliconAn NSF Expedition in Computing


 

This is an example of how the smart glove - part of our Third Eye project - can be deployed for shopping in a retail store for the visually challenged. The demonstrator is Gus Smith - an undergraduate student who has been working on developing the technology. The demo was presented at the 22nd annual Coalition for National Science Funding (CNSF) in April 2016 at Capitol Hill.


--->

Smart Glove Demo


This is an example of how the smart glove - part of our Third Eye project - can be deployed for shopping in a retail store for the visually challenged. The demonstrator is Gus Smith - an undergraduate student who has been working on developing the technology. The demo was presented during the first Glass workshop for STEM teachers held on 28th February 2015 in collaboration with CSATS.


Convolutional Neural Network Structure


This is an example of a Convolutional Neural Network (CNN) running on an image from the Leeds Sports Pose Dataset. We visualize the outputs of the first five layers. Each layer can consist of a convolutional layer, a rectified linear layer, a max-pooling layer and a local-response normalization layer. Only first two layers have a max-pooling layer and a local-response normalization layer. Details of this CNN can be found here.


Visual Co-occurrence Network


This is a Visual Co-occurrence Network that can be used to build relationships in a retail store.


Shopping in a Retail Store


This is an example of a system that can be deployed for shopping in a retail store.


Wizard of Oz



Situation Awareness


This is an example of situation awareness by decoding head movements and video data. This prototype can reliably decode 20 different activities.


AIM Acceleration


This is an accelerated AIM algorithm (Attention by Information Maximization). The acceleration is done on Xilinx ML605 board. The algorithm finds the most salient regions in the input stream.


Accelerated Saliency


This is an FPGA-accelerated Saliency algorithm. The heat map shows the most salient region in the input. The Saliency algorithm used was developed by Laurent Itti's group.


Object Recognition


This video shows object recognition using a sample set of toy vehicles. An algorithm called HMAX running on an FPGA is used to extract the feature vector which is then followed by an RLS classifier.