The focus of this project was on two important health-related behaviors: physical activity and sedentary behavior. The lack of physical activity and continual sedentary behavior over time can contribute to obesity and morbidity. The purpose of this project was to create an intervention device that uses real-time behavior pattern recognition via machine learning. Once accurate recognition of different types of activities is achieved, we would add a message feedback system that encourages physical activity and breaks from prolonged sedentary time through individually tailored prompts and suggestions. The overall goal was to produce an interactive system that easily integrates into an individual’s daily living. We proceeded through an iterative design and testing process with adult participants.
From this project, we were able to show that machine learning algorithms can be used to classify types of activity from sensor signals on smart phone devices and this information can be processed and used in feedback loops to inform users of their activity levels. This project trained a number of student interns who volunteered their time to assist with programming, recruitment, participant data collection, and data entry.
Results and Outcomes of Project Phases
Specific results and outcomes of the project phases are summarized below.
Phase 1a: Develop a simple app for the Apple iPod Touch and collect lab-based data
We completed our first data collection and analysis on our iPod Touch application with 17 participants. We used this data for initial algorithm testing and to validate the Fitbit’s commercial accelerometer device. For the iPod Touch annotated data, we tested three machine learning algorithms (Decision tree, Naïve Bayes and K-nearest neighbor). We found that by using a leave-one-out training and validating protocol, classification accuracies for the individual activities were poor for the three algorithms (17.14 % to 21.48%). However, when activities were grouped into three broad categories of walking, jogging and sitting, the average accuracies improved substantially to 80.3% for decision tree, 80.6% for KNN, and 98.4% for Naïve Bayes.
- This work was published in JMIR (Wu et al., 2012).
- Data are available as an annotated dataset in the iDASH repository within a public community, called Physical Activity Sensor Data.
Phase 1b: Collect field data with participants for 2 weeks
We collected data on 12 participants wearing Fitbit and Actigraphs in their daily life for 2 weeks. We completed data processing of Fitbit daily and minute-by-minute (steps/energy expenditure) data. This data was annotated from participant activity logs. Energy expenditure was classified into sedentary, light, moderate and vigorous intensity categories. The activity log entries were condensed into activity frequencies and further categorized by posture, gait and intensity (assigned MET values). Work continues to aggregate the three data sources (Fitbit, Actigraph, activity logs) to a common interval of 15-minute increments. We will be using these annotations for further Fitbit validation and to compare sedentary time to non-wear time on the Actigraph.
- Documents for the Phase 1b Actigraph and Fitbit datasets were finalized along with the DUA for sharing these datasets in the Physical Activity Sensor Data Community.
- Paper is in progress on the validity of the Fitbit and Actigraph devices for measuring reported sedentary behaviors
Phase 2a: Develop an Android platform data collector app and collect annotated data of specific activities
We collected data using the Samsung Galaxy (Android) player in a free living setting. The application collected raw data on various activities such as walking, running, sitting, standing, and driving with 20 participants. Locations associated with the activities include front pants pocket, back pants pocket, interacting and not interacting in the hand, desk, armband, and lying flat on a seat (for driving). We collected raw data (acceleration, rotation, orientation and gravity data) to develop the activity classifier for the Android app. We also built an on-board Android activity classifier that displays current activity type and accumulated activities, which became the basis of our intervention feedback app called SenSed.
- Documentation for the Android motion dataset was finalized along with the DUA for sharing these datasets in the Physical Activity Sensor Data Community.
- We presented this work at the iDASH All Hands Symposium in July 2012.
Phase 2b: Develop an activity detection and feedback app and conduct a user study
We developed an activity detection and feedback app, called SenSed. Using the Sensed smart phone application, we conducted a user study with 12 individuals to obtain feedback and test the accuracy of the application in free living conditions. A goal setting feature was added to the application prior to testing. Goals were based on minutes and intensities (METs) of physical activities.
- The SenSed app is available on the iDASH website.
- Initial pilot testing was included in a graduate student’s dissertation project, which used the Fitbit as the data tracker along with PC installed software for prompting real-time smart messages to test the system’s effect on workplace breaks in sedentary time (i.e., prolonged sitting).
Publications & presentations from this project
- Wu, W., Dasgupta, S., Ramirez, E., Peterson, C., Norman, G.J. “Classification accuracies of physical activities using smartphone motion sensors.” Journal of Medical Internet Research. 2012; 14(5), e130. PMID: 23041431. PMCID: PMC3510774.
- Norman, GJ, Kolodziejczyk, J, Ramirez, ER, Hekler, EB. (2014). How to Deliver Physical Activity Messages. In: CR, Nigg (Ed.) ACSM’s Behavioral Aspects of Physical Activity & Exercise.
- Ramirez, E.R., Peterson, C., Wu, W., & Norman, G.J. (2012). Accuracy of the Fitbit Pedometer for self-paced and prescribed physical activity. Medicine and Science in Sports and Exercise, 44(5S), S465.
- Norman, GJ, Wu, W, Ramirez, ER, Dasgupta, S, and Peterson, C. “Accuracy of the IPod Touch for Detecting Self-Paced and Prescribed Physical Activity” presented at Medicine 2.0 Conference, Boston, MA, September 2012.
- Norman, GJ, Wu, W, Ramirez, ER, Dasgupta, S, and Peterson, C. “Wireless Technology for Health Behavior Change Measurement & Intervention” presented at 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, August 30, 2012.
- Norman, GJ. “Wireless Technology for Health Behavior Change Measurement & Intervention” presented at the Workshop on Wireless Healthcare Technologies and Applications, National Central University, Taiwan, March 23, 2012.
- Norman, GJ. “Design & Evaluation of Mobile Technology for Health Behavior Change” presented at the University of Michigan Center for Health Communications Research, Ann Arbor, MI, February 21, 2012.
- Norman, GJ & Movellan, J. “Control Theory Approach to Health Behavior Change Interventions” presented at the NIH mHealth Evidence Workshop, August 16, 2011.
- Norman, GJ & Dasgupta, S. “Approaches to Processing Sensor Data from a Machine Learning Perspective” presented at the NIH 2011 mHealth Summer Training Institute, La Jolla, CA, June 2011.