A dust collection system in a woodshop often serve more than one tool. In that case they need “blast gates” to open the hose path for the tool in use and close off other hoses to other tools. These blast gates are usually manual.
Better systems would have blast gates controlled by microcontrollers.
For tools on their own circuits, these electric blast gates could be controlled by detecting current flow when a tool is turned on and then send a signal to turn the system on and open/close the appropriate blast gates.
Some smaller electric tools are equiped with dust bags that can be removed and connected to the dust collection system. They often share a circuit at a workbench. Other bench tools like drills don’t really need dust collection.
project description
Create a machine learning pipeline to train a microcontroller to identify a particulatr power tool by its sound and/or its current draw profile.
If the tool detected uses the dust collection system, turn it on and open its blast gate while closing any open blast gates
data collection
Set up a microphone and recording system that saves short segments of sound data whenever the sound level is greater than a threshold.
labeling data
A wood worker familiar with the sounds of their tools listens to the recordings and labels those with dust collection ports.
feature extraction/ model design
Decide on the temporal or frequency features that are most illustrative. Start with MFCC and micro_conv.
motivation
I am beginning to realize that tinyML is not just its own little nitch isolated from the rest of IOT. It has the potential to augment existing procedurally programmed IOT systems.
The proposed system will start out with procedurally programmed microcontrollers that each have a job but that communicate with other IOT devices.
One device will sit at the electric panel and react to current sensors in the panel sending signals to…
a server or pi or microcontroller that decides who needs to know.
microcontroller blast gates that run the servo’s that open and closes each gate.
smart switch that turns on the big vacuum when told to.
an automated sound sample collection system that sends data to a server
Once a bunch of data is collected and labeled, the ML system can be designed. Eventually it will be brought online adding very cool functionality.
Thanks for that reference. A few people use the shop. I am thinking of bugging the shop. Maybe a web app on an old tablet that listens and records short samples in the shop whenever the sound goes above a certain level. Later I will listen to the little segments and label them.
I had exactly the same idea here are some additional thoughts I had:
Some of my tools have a completely different sound depending on setup. E. g My vintage Delta table saw sounds differently when the blade is lowered than when it is raised. This led me to think about audio signatures when various tools actually contact wood, but then there will be hardwood, softwood, MDF and blades differences to deal with. This is a little late for perfect dust collection, but better than no automation.
I then wondered about the RF signature of the various motors watching the minute sparking that happens when the motors start up. There must be some RF emitted and it could be very distinctive at the right frequency sampling rate.
Any approach that brought all of the dimensions together would be pretty effective.
The side effect of providing an accurate log of machine usage is interesting too.
If you’re open to collaboration, I’d be interested.
Hey @PNFoley, absolutely, I would love to collaborate. Once I clear some time to work on it, I think I will start by coding up some automatic recording setup to collect short segments of sound data and save them to a server. Then I want to have a way of preprocessing those samples in multiple ways, time-series, fft, mfcc … and then visualizing them. Determine what features might be best to train on.
I have looked at maybe doing data collecting as a web app on an old tablet using the audio api and maybe the web-speech api but after playing with some code it may not be a good fit to try to force a browser to ‘wake on sound’.
So that would send me back to C++ and microcontrollers maybe an esp32 since it can be a client that sends data up to the server when it wakes up.
What areas would you want to explore to start out?
If folks don’t mind, I’d encourage you to use Edge Impulse toolchain to automate your process a bit for these things. It is very helpful. If you folks run into any limits etc., please let me know. I know those folks well and they are great champions of our course and want to support learners and explorers. Good luck!
Real world data collection as you’re describing sounds like a great way to start, since as we’ve learned, training data is so important. I think we should work out a simple file naming convention and recording quality standard so that the library that build is easy to use and share. How do you think an iPhone App like AVR Pro would do (M4A, 48K sample rate)?
I don’t know anything about the Edge Impulse toolchain so I’ll learn about it. I’m also going to research the RF angle.
What’s the correct protocol to exchange email addresses so that we can communicate more effectively?
@PNFoley,I am used to sending data to a server from esp8266 using mqtt and maybe I can get the esp32 to build little data packets of short sound samples and send them to edge impulse as an https post.
I just got blink to work on esp32 using esp-idf instead of arduino toolchain. Supposedly better since that is what the tensorflow people used when they ported to esp32. I am running using the esp-idf extension on vscode remote win 10 wsl2 running ubuntu 20.04. Hope the couple of hours to get it working was worth it.
Check out the following link from Edge Impulse. Now that you know what is going on under the hood you can easily use Edge Impulse’s toolchain to accelerate your productivity.
@vjreddi, edge impulse is rather impressive. I have been modeling the data acquisition piece for the project on their data ingesting service and their use of the CBOR Binary payloads (e.g. for audio) format.
@PNFoley figured I should get the data in and out of some standard format first on my laptop. Though I have written a lot of C++ for the esp8266 I have had to jump through a couple of hoops to get g++ setup on WSL2 and vscode to compile on intel. Once I can write data to CBOR (and decode it) I will move that code to the esp32. Then the plan is to send packet via POST to the server first from the laptop, then from the esp32. After that I need to think about sound threshold detection and how to kick off a recording-posting event.
Based on this discussion, I was thinking of building a general python application to help out with data collection and maybe a supporting program for the Arduino Nano BLE sense sometime down the line.
The intention is to enable data collection for longer intervals (edge impulse already takes care of data collection for short intervals).
I’m thinking of adding configurable options like sensors selection(mic/accelerometer etc.), time period, output format(json/cbor etc.).
Maybe down the line, i can add some modules to directly connect it with the edge impulse api as well.
@DanielDsouza , The problem with the nano for data collection is that you need to stay connected to a laptop and get the data over the serial interface. I suppose that could be piped to a networked board and sent out over the internet that way. That’s what pushed me toward the esp32 which has the internet stack built in and communicates over wifi.
I have been looking at CBOR implementations for decoding with node.js thinking if edge impulse doesn’t work out as a long term data store I could put those CBOR json-like packets either right into Mongodb or flatten it into mysql on one of my cheap vps servers.
There is also a python implementation there that might be useful for you in going the serial route for devices like the nano.
It would be great to supplement existing tools for data acquisition.
Your idea makes me think that there are things to be done to improve the opposite end of the pipeline as well. What if we could extract a record of every prediction that these little devices make along with our comment to the device ‘yo nano, you are wrong’ From that database we could construct an ROC curve so we could get all that false positive like data out and keep track of our drift.
I was a bit apprehensive about moving away from the Nano BLE Sense since sensor variations might cause an effect on the model. I hadn’t considered using the esp32 as an intermediate device for transfering the data to the net, but yes this suggestion brings up a lot more ideas on making it more portable, I am thinking of the A6 GSM module as well now.
This is really useful, I wasn’t aware that CBOR was well ported to a lot of platforms.
Sounds interesting! I was focusing only on data collection, but adding this would give a good coverage as well to the entire pipeline.
@mcktimo Sorry for the delay. I captured a few sound samples including ‘silence’ and ‘general shop noise’ using Edge Impulse and then ran them through Impulse toolchain but the accuracy was 0% with a very disappointing confusion matrix. @vjreddi I need to work through the Keyword Spotting link you provided a little more deeply. I will also get more samples,
@mcktimo A small question: how did you break the barrier between Linux in WSL2 and windows to flash the code? Since the WSL2 doesn’t support com port, the current solution I found is to use JLink Remote Server.
@dzgr, Yes, straightening of the twisted path to making it work is as follows. Install it in linux not windows, set it up from a remote ubuntu 20.04 window in vscode
installed WS2 and put Ubuntu 20.04 on it
made sure that all the prerequisites worked. It turns out ubuntu 20.04 does not include venv (it includes virtualenv)
install Remote-WSL extension in vscode. Connect to your WSL2 through the remote exlplorer button the extension puts in the left bar.
go to your /home/folder. The extension defaults to putting the compile tool in /home/you/.espressif and asks where you want the esp-idf (I put it in /home/tim/esp)
Install the Espressif IDF extension. and for the config process installing the latest release version of the esp-idf
When I hit the flash button that gets added to the bottom of vscode it calls Executing task in folder blink: powershell.exe -Command "python \\\\wsl$\\Ubuntu-20.04\\home\\tim\\esp\\esp-idf\\components\\esptool_py\\esptool\\esptool.py -p COM4 -b 460800 --after hard_reset write_flash --flash_mode dio --flash_freq 40m --flash_size detect 0x8000 partition_table\partition-table.bin 0x1000 bootloader\bootloader.bin 0x10000 blink.bin" <
There are also buttons to set the com port and they use the windows names COM4
esp-idf is straight-up C and is rather tedious. Supposedly you can use “Arduino.h” as a component and then use all those libraries but some versions (my version) don’t work right now.
Hope this helps and that I have not made you yet another captive of the microsoft universe.
here are some additional thoughts I had: