Home Host

Summary

The advent of smart home devices is something that is genuinely exciting and has been forever expanding into more and more uses for your home. Amazon recently just released a new innovation, the Ring Always Home Camera, a drone that literally flies around your home to act as a security camera. We want to emulate this experience without the privacy concerns of having a large company being able to literally see your home and control everything. Especially with the recent controversies with companies such as Eufy which promised what happened in your home stayed in your home and proceeded to do exactly the opposite. It’s hard to trust these companies when time and time again they fail, so,

Plan

We want to build a computer that would be able to somewhat replace what Amazon and Google’s smart home ecosystems is able to do. Hence, this requires heavy compute power in order to power the TensorFlow algorithms we are running, which is an image detection model and a voice recognition model. We want to train these models to largely adapt to a home environment, being able to detect and distinguish faces as well as respond to basic commands. In order to have the simulate the little speakers that Alexa and Google has, we will instead utilize a fleet of old Android devices as well as bluetooth speakers. These Android devices will be utilizing Droidcam in order to redirect both microphone input and camera input to the computer. This was the best solution that we could find to simulate a relatively wireless system within a smart home environment, with the only wire being needed is power. Bluetooth speakers will be deployed alongside these old Android devices in order to emulate the assistant speakers in conjunction with the microphones. For processing this data and pointing it to where it’s going, we will be utilizing TensorFlow and training our own model based on our own needs. This will be running on the 6600 xt utilizing ROCm as our driver to better adapt these machine learning algorithms to our GPU. After setting this up, we’ll be able to put in our commands to the script and figure out what the script should do when it detects things. We will have a list of key words that you can say to the device to trigger things such as lights turning on and playing music. However, for everything else, we will be utilizing ChatGPT’s API in order to have a conversation with the system. The next key part of this project is that on top of the PC handling these devices, the PC will also function as a wireless router utilizing OPNsense, an open source routing software that allows our PC to function as a wireless router. In this way, all the Android devices will wirelessly connect to the PC directly, rather than through a secondary machine. This will decrease latency and increase the speed in which the models will recieve the image/audio, process, and respond. Next, we want to be able to offer something else that these major companies currently do not offer. We will assemble a dual smoke and carbon monoxide sensor in an arduino that can be installed to monitor your home’s safety. Through this system, it will send texts based on when one of the alarm triggers, this way, you can feel safer going in your home. Next, it will utilize the bluetooth speakers in order to still make an alarm noise. Lastly, we thought, where could this smart home management system possibly be placed in a PC? This was simple, we would run Android TV on the system and allow it to function as your home’s smart TV on top of everything else. Through this, it will allow it to be able to solve the many issues of Smart TVs being slow. This single machine will be able to connect to various devices to use it as an alarm system, smart home system, and lastly, an Android TV. Multi-functional systems, here we come!

Ok, that was a lot of words and probably not the clearest wall of text but below we’ll list off the features of this system and what it can do and how it does it. This time, it’ll be in one short and sweet list rather than a messy wall of text

Extra Notes: - This project is extremely technical in nature and requires a lot of jerry rigging, modification to software, as well as custom solutions. - The first inherent problem with this is the designation of a custom LAN. This should in theory work, but we have to ensure that it is isolated from the actual network of the system. We would also have to use our Wi-Fi Adapter in order to create a Wi-Fi network for the phones to connect to - The next part is modifying the current Linux App for Droidcam to support the connection of multiple devices and bind them to different “virtual” cameras as well as microphones. Furthermore, we would have to create a way to link these microphones to the bluetooth speakers that they will be alongside, that way, they will function like “nodes” and respond only in the same speaker. - After this, we then need to work on a way to capture the audio as well as the image that is detected from the phones then place it into a TensorFlow Algorithm. This will require some adaptation to TensorFlow as it does not do real time image detection, only from stills. Additionally, we will also have the develop a script that will listen at all microphones at the same time and detect for speech. After it detects this speech, it will check whether or not it is the activation words. If it is indeed the activation word, it will play a little activation sound and continue listening and actively take input. This way it brings it closer to the design of Amazon Alexa and Google Home. - The next part of this is the image detection. This is currently only being done one at a time by TensorFlow, so we’d have to adapt our script to it and create a solution for it to process multiple videos at the same time from different camera sources. - TensorFlow will all be processed under the GPU through AMD’s ROCm. Whilst it is most definitely possible to downgrade the hardware or even remove the GPU all together, it will increase the processing time of the overall system. Since the goal is to replace what Google Home and Amazon Alexa does, having to wait a minute for a response from the system is simply outrageous. Hence, we picked both the 6600xt and the 12700K in order to run these operations both in parallel as this system will require a lot of processing both for images and voice since we need to process 3 voice detection scripts and 3 image detection scripts. - Another fundamental part of our system is the Wireless Carbon Monoxide, Smoke, and Gas Alarm. This will be assembled into a Arduino or a Raspberry Pi 4b where we will develop a little script to let us know when any of these are triggered. It will wirelessly connect to the system through either a Particle Photon or a Raspberry Pi 4. - The most important part of the project is the website. Through this website, you’ll be able to add custom commands for the voice assistant, indicate what to do when people are detected, and view the cameras anytime. It will also handle the notification system. - Lastly, the whole system will be running Android TV which will be emulated.

ProductSupplier/LinkCost
RX 6600 XT (Refurbrished)https://www.amazon.com/PowerColor-Radeon-Gaming-Graphics-Powered/dp/B09BWR9J1M/$314.97
CPU and Motherboardhttps://www.microcenter.com/product/5006215/intel-core-i7-12700k,-asus-z690-m-plus-prime-ddr4,-cpu-motherboard-combo$353.98
RAMhttps://www.amazon.com/dp/B07PV7GL6C$65.97
SSDhttps://www.amazon.com/dp/B07Q37V1C9$92.99
Coolerhttps://www.amazon.com/Thermaltake-Motherboard-Addressable-Universal-CL-P105-AL12SW/dp/B09XFKW431/$24.99
Analog Rotation Potentiometerhttps://www.dfrobot.com/product-87.html$2.90
Analog Gas Sensorhttps://www.dfrobot.com/product-681.html$6.90
Analog LPG Gas Sensorhttps://www.dfrobot.com/product-684.html$6.90
Analog CO Sensorhttps://www.dfrobot.com/product-686.html$9.85
Particle Photonhttps://www.dfrobot.com/product-1324.html$19.00
MPSA13 transistor (Darlington pair)https://guitarpcb.com/product/mpsa13/$0.49
Arduinohttps://www.amazon.com/Arduino-A000066-ARDUINO-UNO-R3/dp/B008GRTSV6/$28.50
CaseOwnedOwned
750w Power SupplyOwnedOwned
Taxes$84.65
Shipping$9.45
Total$1021.54