Today’s article is about a topic that has intrigued me for the best part of 20 years. That of the use of wearable sensors to monitor performance.
Terminology
For the uninitiated, Artificial Intelligence is a general term for a variety of mathematical techniques to evaluate data that has been generated from some data capture device (video, photograph, GPS, Accelerometer, etc).
One important and quite useful subset of Artificial Intelligence (AI) is that of “Machine Learning” (ML).
ML uses statistical models to evaluate data and try to make sense of what looks like just a jumbled set of numbers. This could be more generally called “Pattern Recognition” (PR).
How are both ML & PR useful in the evaluation of sport performance (data being collected from a wearable device for example)?
History of Artificial Intelligence in Sport
It is worthwhile reviewing the history of the implementation of sport wearables to understand the value and power of the implementation of machine learning algorithms to better evaluate and understand the mountains of data these new devices are now generating.
In the early 2000’s when my company, “GPSports Systems” introduced the first Sport GPS system to the world market – we didn’t provide much data analysis – largely relying on the sports themselves to process the data and work out what they wanted from the data (to be honest, we didn’t know what to look for either, we just instinctively knew that the information would be useful – we allowed the market to determine this use by providing them with the raw data to evaluate).
I remember meeting up with an AFL club and asking how the new system was going and he showed me a pile of spreadsheet printouts with 1000’s of cells of information and saying that this is what this technology had done to him!!!! Spending countless hours data mining trying to find useful/meaningful information from the data!!
Jump forward a dozen years and the advent of accessible machine learning engineers & code has resulted in much of the data crunching being replaced by smart machine code with some platforms now generating “Answers” rather than just “Data”.
Where there is particular value in the use of ML algorithms is in making sense of the large data sets generated from the widely used IMU’s (Inertial Measurement Units – Accelerometers & Gyroscopes).
An Accelerometer is a sensor that measures acceleration/deceleration (often in 3-planes – X, Y & Z).
A Gyroscope is a sensor that measures angular velocity (rotational speeds/accelerations).
Unlike GPS which often has the data processed with the output being of immediate value to the end user (eg position on a map, speed, direction, altitude, etc), these IMU sensors typically just generate a lot of raw data with only a small number of practical applications to date, including:
- Screen rotation on your phone.
- Steps taken
- Sleep quality, etc
Raw IMU data when processed using PR algorithms, can detect a stride and provide a total count at any time during the day or detect when you are still whilst asleep in bed and generating a sleep quality based on this amount of movement you are doing when in bed.
With the appropriate code, valuable information can be gleaned from these sensors making the wearable devices that they are contained within very useful for exercise or training purposes.
For example, wrist based IMU’s are now available to detect the total number of forehands, backhands & serves completed during a tennis match and there are companies that state that they can auto detect up to 400 gym exercises all based on the IMU outputs from a wearable worn on the arm.
Like all patterns, some activities are easier to detect than others. For example, Cyclical movements are much easier to detect than non-cyclical activities.
Walking, running, cycling, swimming, rowing – are examples of cyclical activities whereby the repeated nature of the moment allows for ML/PR to be quite accurate in detecting these activities (also if you review the market, you will see most wearables target these types of sports/activities).
The image above is an example of an athlete completing a series of sprint run throughs. Each spike indicates a single run through and we can see on the right the G-forces being generated for each effort. (The blue graph is a sum of the X, Y & Z data).
In this instance, the accelerometer is sampling at 100Hz (100 samples of the X, Y & Z axis per second) and so it is possible to zoom in and look at more finite data:
This above image is a zoomed in view of first of the 4 above sets (looking at individual sprint repeats) and whilst not at step by step view yet, we can see the sort of deceleration forces this athlete is producing each sprint repeat and we can now start to compare sprint repeat to sprint repeat as see how similar they are to each other.
If we zoom in again (on the first sprint repeat) – we then get a glimpse of more valuable information.
We are now able to see individual steps completed by the athlete as well as what their body was doing in the X & Z planes also (green is Y – vertical force), X-red is side to side and Z-yellow is front to back.
Even at this level of granularity, it is difficult for the human eye to detect any variations in the pattern, but this is what MI/PL has been designed to do and through the application of these algorithms to this sort of data, you are able to pick out things from the data that are not visible by viewing the athlete training or looking at the raw data above.
During my time at GPSports, we used a clever combination of GPS & IMU data to determine running symmetry.
As you can see from the image above, (an output of Accelerometer data (Y-Axis) from an athlete showing the accelerations and decelerations from each step taken by an athlete during running), the downward pointing data is the deceleration of each step whilst the upward pointing data is the acceleration of each step.
What was noticeable was that the LEFT contact was always of less G-force than the RIGHT contact when reviewing the deceleration component (a measure of how hard the athlete was hitting the ground with each step).
We then developed an algorithm that would evaluate this data at different running speeds (Using GPS for the speed aspect) and providing a “Running Symmetry” metric for the athlete/coaching staff to review.
This algorithm was shown to accurately detect the potential for injury if the asymmetry was >15% b/w ground contacts. Here is an example of using wearable sensors to “PREDICT” an outcome!!
Again cyclical movements are much easier to evaluate at this point in ML development. One of the reasons smart watch companies got into trouble with their products a few years ago (claims of inaccuracy) was because trying to analyse non-cyclical sports is a huge challenge. Adding to this the fact that the sensor was being worn on the wrist of one arm added to this challenge (Not a true representation of what the total body is doing).
HOW WILL MACHINE LEARNING BECOME MORE USEFUL FOR SPORT IN THE FUTURE?
Firstly, the sensors being used will become more accurate & be able to remove “junk” data that is generated (through better filtering technologies).
Secondly, the IMU’s will become smaller, more power efficient and will more easily be able to be placed on multiple body parts providing a truer picture of what the athlete is doing during training/games.
Thirdly, with the rapid improvement in integrated sensors with clothing, it won’t be far away that reliable, affordable sensor infused clothing will be available allowing for the user to capture a 3D images of their performance.
Add to this 3D image machine code that can evaluate good v’s bad technique and you have a potential “VIRTUAL COACH” that would be able to not only provide performance metrics (Distances, speeds, heart rates, etc) but also how you are performing your task and providing live feedback to the wearer of the garment to improve their technique in real time!!!
SUMMARY:
- All sport/fitness wearables use some combination of sensors (Accelerometer, Gyroscope, GPS, Heart Rate).
- Currently the data analysis is still quite basic with only core metrics generated (distances, Number of strides, etc).
- With the improvements in Artificial Intelligence Machine Learning and Pattern Recognition algorithms, we will likely see steady improvements in performance analysis and getting closer to the holy grail of analysis – that of “PREDICTIVE PERFORMANCE”.
- For a great review on the use of AI and elite sport – check out this article (AI In Sports)
