During my time at the Machine Perception and Cognitive Robotics Lab (MPCR) at FAU, I helped to lead a team of students to develop an application that could track posture in real-time. We called this the Good Posture Application (GPA). My team members were Henry Herzfeld (GitHub) (LinkedIn), and Paul Morris (Github).

I came up with this idea with my physical therapist at the time, Mary Pengelley. Because of my cerebral palsy, I have a tendency to lean more to my right side instead of sitting straight. It's an unconscious thing, one I try to be more aware of, but something that I often need reminders to fix. So I thought: "What if I could make something that could give me a subconscious reminder?" As I proceeded along this line of thought, I realized that such an app could be used for anyone that sits at a desk most of the time, to keep them from developing long-term postural issues. So the GPA team was born.

In the beginning, we used Microsoft's Kinect 2.0 technology.

This camera draws a 2D dynamic skeleton around a person when it sees one, and follows the person until they leave the camera feed. We realized that we didn't need all of the points of the Kinect skeleton for our purposes, so we removed all points except the ones representing the spine and the hips. Using these points, we were able to give the user a score representing the quality of their posture. This was later changed to a grade for younger audiences. An example of this demo version of our application in action can be seen below:

We eventually envisioned this as a smartphone application, as shown above, but we haven't been able to accomplish this to this point.

As we continued to work on this project, we realized that support for the Kinect 2.0 was diminishing. We were also acutely aware that this current implementation does not use Machine Learning at all. So, we set out to correct these two issues.

The latest version of the GPA is thus a binary classifier capable of classifying "good" vs "bad" posture. These terms are incredibly subjective and difficult to define. For our purposes, we defined "good" posture as someone who was sitting up straight and "bad" posture as someone who was leaning to the side or slumping over. We collected this posture data from a PC webcam. We asked a few people to sit in a chair in front of the camera and recorded frames of them sitting in these "good" or "bad" positions. An example of each is shown below:

We then annotated each of these images with the skeleton from the Openpose library from CMU. This skeleton served a similar purpose to the Kinect skeleton that we used in the preliminary version of this app-- to track the posture of a participant. The OpenPose skeleton contains the following skeleton points, shown below:

We recognize that this is a very small dataset for a neural network classifier, but we created this dataset only as a proof of concept and did not mean it to be representative of the effectiveness of a posture classifier on a larger scale. Here is a breakdown of the composition of the classifier, and the results on the dataset described before:

As you can see, we used a three-hidden layer neural network to classify these posture images, with promising results. For future work, we wanted to change our network to a convolutional neural network (CNN), and also use a camera that is specialized for collecting pose data.

The GitHub repository for the classifier described in this article can be found here.