Custom IoT Voice Assistant

Web
Python
Artificial Intelligence (AI)
Database
Operations / Cloud
System Design
Embedded Systems
Machine Learning (ML)
Voice Recognition
HCI / UX / UI
Communication
Hardware
IoT

A custom IoT voice assistant developed as a senior design project in the year 2019 - 2020. This voice assistant is unique since it can interact with a client specified database to help a user query the database with natural language queries. There are two ways to input a query into the system: a cloud GUI application and the physical microphone on the system. A user can hold down the button on the enclosure to speak a natural language query (for example, "How many Google Home Minis were shipped in the past month?"). The system converts this query into text, runs natural language processing on the textual query and uses an Elastic Search database to generate a response to the user's query. The result is then output through the speaker for the user to hear, or output as a table on the cloud GUI application. A more technical description follows: Users are able to speak a query to the IoT device, a Raspberry Pi, which then using Google's Speech to Text API and natural language processing will convert the query (input from the microphone) to text and run the query against an index returning the result through a Text to Speech program, eSpeak. The result is converted back into human recognizable speech and then output through the speaker. The technologies used included PostgreSQL, ElasticSearch, Node.js, Python, NLTK, SpaCy, eSpeak, Google's Speech to Text API, PyAudio, RasPi GPIO, HTML and CSS.

0 Lifts 

Artifacts

Name Description
Front-End Front-end of the application   Download
Project Website Link to the Google Site for this project   Link
Online File Repository A link to our GitHub page   Link
Team Protocols A document outlining our team protocols.   Download
Project Summary Video Link to the unlisted video on YouTube   Link
Enclosure An image of the enclosure   Link
Enclosure Dimensions A link to the enclosure dimensions.   Link
Block Diagram A link to the block diagram for the project.   Link
Feedback