A colleague (@Danimp94) and I are working on a TinyML project to take part ideally in the HardvardX competition. We find ourselves a little bit stuck and we would appreciate to receive some help from this community. We also consider our questions are possibly common questions and discussing about them might be helpful to other TinyML students in this forum.
First of all, let us introduce what is our project about. We would like to append snippets of code for a better explanation, but because this post is long enough, we attach the GIT repo directly:
In this project we are aiming to develop an intelligent alarm using TinyML that wakes up the user at the best moment (i.e. the REM stage). The idea behind this project is to develop a device capable of identifying the different sleep stages (Wake, Non-REM, REM), and then predict which moment is the most suitable to wake up the user within a given boundary.
For example: the user wishes to wake up at around 8 am and it is okay if the system could wake him 15 minutes earlier or later. The user will then define a threshold of 15 minutes and the system will know that it can wake the user up at the most appropriate time between 7:45 and 8:15.
For this project we have planned to use the IMU integrated in the Arduino Nano BLE 33 Sense and an external Heart Rate sensor, obtaining a total of 4 different measurements (3 for the axes of accelerometer and 1 for the HR) in a time series format.
For data acquisition, we have chosen the following dataset: Motion and heart rate from a wrist-worn wearable and labeled sleep from polysomnography v1.0.0 which has been labelled with professional equipment and personnel.
Because of the way this dataset comes, it needs some pre-processing to be functional. The data it contains is the following:
Accelerometer data: 3 axes with 50-150 samples per second.
Heart rate sensor data: 1 sample per 15 seconds.
Labels: Stages (0-5, wake = 0, N1 = 1, N2 = 2, N3 = 3, REM = 5) N1, N2 and N3 are the 3 different Non-REM sleep stages. These have been taken every 30 seconds.
For further data processing, we have done the following: we take the peak value (for each axis) per second of the accelerometer and combine it with the heart rate sensor sample.
Because the resulting dataset is not too large, we use AutoFeat to add derived features (e.g. lineal combinations) so that the model has more features to learn from.
Note that we are not including references for this description but we can provide them if someone wishes.
Now that we have described the project by and large, here come the questions.
Dataset related questions
Is the processing we have used a good idea or would it be better to use a “rawer” data? This question comes because after some pre-processing, our dataset turned out to be very compact and with not too much information for training.
The dataset contains too much N2 labelled data. This makes the dataset to be biased causing the model to mainly predict this label. What options are there to solve it?
Here there is the distribution of labels (4 and -1 are non-valid labels, which do not count for the dataset).
Model related questions
Would it be possible to process the data with CNN (e.g. converting the time series to spectrograms as in the course examples)?
Otherwise, is it better to use a DNN Keras model applying (or not) time series?
If none of the above, should we use a more traditional Machine Learning model? Could we make it tiny and convert it to .tflite?
- After tinkering with different variations of our dataset and different types of neural network models, we found that the validation set (both loss and accuracy) was never going alongside the training set. Incidentally, this happened for many different combinations, no matter what. The picture illustrates this behaviour:
Is this due to overfitting? Or are we overlooking something vital?
Please feel free to ask any questions if you have and to answer any of our questions posted. We would be only too happy to hear from anyone in the community!