How machine studying and wearables can change health
TL;DR: Scroll to the underside part (Outcomes) to see the venture outcomes and accuracy of the machine studying fashions.
I not too long ago accomplished a machine studying venture the place I used to be in a position to monitor bicep curl type of a consumer. If the bicep curl kind was right, the mannequin would point out right kind. If the bicep curl kind was incorrect, for instance, if the consumer swung their elbow an excessive amount of, the mannequin would point out that the shape was incorrect and state why it was incorrect.
Word: “Form” refers back to the motion carried out throughout one repetition of a bicep curl. Having right kind when weightlifting is necessary with a view to stop damage and goal the right muscle tissue.
The objective of this venture was to develop proof of idea that I might construct a machine studying mannequin with sensor information from smartwatches to trace delicate variations in actions throughout sure weightlifting workout routines. Sooner or later, I’ll broaden on this venture with a view to develop a full fledged app.
Right here’s how I did it:
I’ve robust pursuits in information science and health, and am at all times searching for methods to mix the 2 matters. I observed there was an rising functionality for wearables to trace cardio train, however not a lot for weightlifting. That’s once I got here up with the thought to make use of wearables to trace weightlifting kind. Trying again, I’d say it is necessary that you just select a subject that genuinely pursuits you, as a result of while you run into hiccups and setbacks (and you’ll), it provides you with motivation to get previous these moments.
Previous to this venture, I didn’t have any expertise working with sensor information, so I needed to conduct lots of analysis. Lately, there have been many papers masking Human Exercise Recognition (HAR), the place researchers primarily checked out utilizing sensor information from physique sensors or smartphones to determine when customers are conducting varied actions resembling standing, sitting, operating, strolling, and climbing stairs. The commonest sensors utilized in these papers had been an accelerometer and a gyroscope.
I checked out sort of knowledge used, the processes used to gather information, and the way fashions had been constructed and examined. I additionally took be aware of among the important variations between these HAR papers and my analysis. These variations included extra delicate variations in weightlifting kind in comparison with the actions recognized within the HAR papers, and my analysis was solely going to make use of a wrist sensor as a substitute of a waist sensor or a number of sensors connected all through the physique.
Based mostly on my analysis, the fashions I ended up selecting to check for this venture had been a a Lengthy Brief Time period Reminiscence Community (LSTM), a 1-D Convolutional Neural Community (CNN), and a CNN-LSTM, which is a mix of the 2 earlier fashions. An LSTM is a sort of Recurrent Neural Community (RNN). RNNs are widespread when coping with sequence information, resembling sensor information. An LSTM is a sort of RNN that’s good at retaining data all through the entire sequence of knowledge. A 1D-CNN can extract options from the uncooked sensor information, just like how the extra widespread 2D-CNN is used when classifying picture information. A CNN-LSTM makes use of the options extracted by the CNN and feeds that into the LSTM.
To gather information, I designed an experiment by utilizing individuals to carry out Three variations of a bicep curl: one right curl and two various kinds of incorrect curls. The wrong curls had been curls with the consumer’s elbow swinging an excessive amount of and a curls the place the consumer doesn’t undergo a full vary of movement for the train. I collected the info by attaching a smartphone to the individuals wrist and asking them to finish 250 repetitions of every of the three variations of curls (with breaks in between and them being monitored so the info wouldn’t be defective). I used the Androsensor Android app to gather the accelerometer and gyroscope sensor information. The info was collected at a fee of 100Hz and I used a time interval of two seconds for every bicep curl sequence, which suggests that there have been 200 information factors in every sequence of knowledge. It is very important retain data from the start of the curl because the community makes its method all through the sequence of knowledge.
The info cleansing and information wrangling was carried out utilizing Python, Numpy, and Pandas. A window of two seconds was used for every curl, with a sliding window of 0.5 seconds to create extra information factors. As a way to create much more information factors, I used beginning factors at 0.1, 0.2, 0.3, and 0.four seconds right into a curl to create extra 2 second home windows.
The fashions had been examined utilizing a leave-one-out take a look at. This take a look at was used reasonably than a randomized cut up as a result of the mannequin ought to work on somebody if it has by no means seen their information earlier than. For instance, in a gaggle of three individuals, if the mannequin was skilled on particular person 1 and particular person 2’s information, the mannequin must be correct on particular person 3’s information with out having been skilled on it earlier than.
The glossary for the under mannequin structure desk is on the finish of this submit.
All three fashions had been skilled and examined utilizing varied changes in parameters for every mannequin. The mannequin architectures will be seen within the desk above. The coaching and testing had been carried out utilizing the Tensorflow and Keras machine studying libraries and on Google Colab. Google Colab was chosen as a result of it has free uninterrupted GPU entry for as much as 12 hours at a time. The outcomes of every of the fashions will be seen within the desk under. The very best performing mannequin structure for every mannequin is in daring.
Beneath you’ll be able to see the ultimate outcomes for the perfect performing architectures for every mannequin utilizing accuracy and F1 score as measures. The ultimate scores had been averages of utilizing every of the three individuals because the take a look at set for every mannequin. Mannequin dimension was additionally necessary to trace for this venture as a result of cellular and wearable units solely have a restricted quantity of assets and processing energy. Due to this fact, the smaller the mannequin dimension, the higher it’s to make use of on cellular units (greatest follow is often having the info despatched from a wearable to the cellular system and the cellular system does all of the processing as a result of it has extra processing energy than a wearable system).
Conv1D(filters, kernel dimension) — 1-Dimensional Convolutional layer; filters refers back to the dimensionality of the output area; kernel dimension refers back to the size of the convolution window.
LSTM(items) — LSTM layer; items refers back to the dimensionality of the output area.
Dropout(dropout worth) — Dropout layer; dropout worth refers back to the share of items to randomly take away within the specified layer.
MP(window dimension) — Max Pooling…