First of all, what have I been up to?
I’ve been spending my time at robotics, by making the experience for new students on programming joining the team better, as well as working on and applying my leadership skills in other extracurriculars.
The New Plan
Ok, so… things have switched up a bit. I’m now going to be using TensorFlow with a Raspberry Pi to process audio in real time. I’ve been working on an audio model (my first AI model ever!) on Google Colab that I can run on the Raspi.
Model Progress
This model currently takes a collection of .wav sound files that consist of various people saying “yes”, “no”, “down”, “up”, “left”, “right”, “stop”, and “go”.
I first learned how to train a model that can classify these files correctly when it processes them. The file starts by creating a plot visualization for a couple of audio waveforms, to get a feel for what the code should be doing. Then, using FFT (Fast Fourier Transform) we can convert these waveforms to spectrograms, which show frequency changes over time. Now with frequency and volume, these files can be represented as 2D images and we can analyze them. Then we set up basic model building specifications, and train the model with the files, using the metric accuracy.
Lastly, we plot the accuracy and loss curve, allowing us to evaluate the model performance and predict using the model. I found that this model was often incorrect in its analysis of a file, but it did have its’ fair share of successes.
Next Steps
Now that I’ve started on the model, there are a couple of things to do.
First, I need to try this code out in my terminal, rather than using GPU in Google Colab.
Secondly, I need to try this with a larger file. Processing a longer 5-minute file will consist of taking overlapping subsets of the audio file, and individually running the model on each to predict the outcome.
After I get this working, then comes the hardest part… analyzing REAL TIME audio…
That’s a wrap for now, I’ll come back with more progress later