The product is based on Caffe MobileNet SSD based techniques. So basically the entire project was divided into 5 phases.
- Building a robust Model to carry the entire hardware and also to make it as compact as possible.
- Learning how Raspberry Pi works and Linux Programming.
- Tackling how to use the Intel Movidius stick.
- Delivering a feedback in the form of Audio Feedback
- Using Webcam or Picamera with the same application.
Building a compact Model
The final model which we made was –
Learning how raspberry pi works
- First we dealt with figuring out how to install Raspbian OS into the board.
- All of the commands like – VNC, SSH, SCP and others took us almost a week to figure out.
- We also had to struggle a lot configuring the Audio output and Specially the bluetooth devices!
Tackling how Intel Movidius Stick works
The first attack this device gave us was its price – 10,000 rupees only!
We started to learn how to use this Stick with rapsberry pi. This is when we learned that this chip works in two ways-
The NCS works by using specialized graph files as input and then processing the nueral network. Thus – when we first had our caffe model – we converted it into graph file using the INTEL MOVIDIUS SDK on our computers and then using the NCS API – we coded the program in python to process image frame by frame on a live video feed!
The raspberry pi 3b+ comes with predefined settings to set Audio jack as default audio output. We used the python library E-speak which gives Text-To-Speech output without using the internet. We also had to do bluetooth settings which are tough on raspberry pi – so we had to try out almost every settings given on internet.
Using Webcam and PiCamera
Since we used raspberry pi – we had option to use the Picamera and Webcam as well. To use both of them is pretty easy and configuring how to use them in python code to use input live feed frame by frame was also quite straightforward.