Artwork by Sami Lee.
For the last 5+ years, I’ve been tracking my various aspects of my personal health using Samsung Health. It helps track weight, calories, heart rate, stress, and exercise and stores all of it in the app.
However, the app only gives some basic high level charts and insights. Luckily, it enables you to export your personal data into CSV files that you can then import into your tool of choice and perform any kind of analytics. In this post, I’m going to show how to export it all, then load it into Zeppelin and some sample Pandas queries that’ll enable you to start building more complex queries yourself.
Use the following links to jump directly to each section:
Table of Contents
Download your Data
First, open Samsung Health on your mobile phone
Scroll down and tap “Download personal data”
Tap the Download button
Login to your account when it prompts you
Wait for it to download
After it completes downloading, tap View files.
Select the file, tap compress, save it as a zip file.
Then upload the file to the computer running Zeppelin. I upload it to OneDrive, then SFTP it to the server, then extract it into a folder
Working with the Data
The export should contain a number of interesting files in the main directory along with the json and files directories. The main files are CSV formatted.
| File Name | Doc Link | Notes |
|---|---|---|
| com.samsung.health.ecg.{ts}.csv | ||
| com.samsung.health.floors_climbed.{ts}.csv | Link | |
| com.samsung.health.food_info.{ts}.csv | ||
| com.samsung.health.food_intake.{ts}.csv | ||
| com.samsung.health.height.{ts}.csv | ||
| com.samsung.health.nutrition.{ts}.csv | ||
| com.samsung.health.sleep_stage.{ts}.csv | Link | |
| com.samsung.health.user_profile.{ts}.csv | ||
| com.samsung.health.water_intake.{ts}.csv | Link | |
| com.samsung.health.weight.{ts}.csv | Link | |
| com.samsung.shealth.activity.day_summary.{ts}.csv | ||
| com.samsung.shealth.activity.goal.{ts}.csv | ||
| com.samsung.shealth.activity_level.{ts}.csv | ||
| com.samsung.shealth.best_records.{ts}.csv | ||
| com.samsung.shealth.blood_pressure.{ts}.csv | Link | |
| com.samsung.shealth.breathing.{ts}.csv | ||
| com.samsung.shealth.caloric_balance_goal.{ts}.csv | ||
| com.samsung.shealth.calories_burned.details.{ts}.csv | ||
| com.samsung.shealth.exercise.{ts}.csv | ||
| com.samsung.shealth.exercise.weather.{ts}.csv | ||
| com.samsung.shealth.floor_goal.{ts}.csv | ||
| com.samsung.shealth.food_favorite.{ts}.csv | ||
| com.samsung.shealth.food_frequent.{ts}.csv | ||
| com.samsung.shealth.food_goal.{ts}.csv | ||
| com.samsung.shealth.goal.{ts}.csv | ||
| com.samsung.shealth.goal_history.{ts}.csv | ||
| com.samsung.shealth.insight.milestones.{ts}.csv | ||
| com.samsung.shealth.library_subscription.{ts}.csv | ||
| com.samsung.shealth.permission.{ts}.csv | ||
| com.samsung.shealth.preferences.{ts}.csv | ||
| com.samsung.shealth.report.{ts}.csv | ||
| com.samsung.shealth.rewards.{ts}.csv | ||
| com.samsung.shealth.sleep.{ts}.csv | Link | |
| com.samsung.shealth.sleep_combined.{ts}.csv | ||
| com.samsung.shealth.sleep_data.{ts}.csv | ||
| com.samsung.shealth.sleep_goal.{ts}.csv | ||
| com.samsung.shealth.social.friends.{ts}.csv | ||
| com.samsung.shealth.social.leaderboard.{ts}.csv | ||
| com.samsung.shealth.social.public_challenge.{ts}.csv | ||
| com.samsung.shealth.social.public_challenge.detail.{ts}.csv | ||
| com.samsung.shealth.social.public_challenge.extra.{ts}.csv | ||
| com.samsung.shealth.social.public_challenge.history.{ts}.csv | ||
| com.samsung.shealth.social.service_status.{ts}.csv | ||
| com.samsung.shealth.stand_day_summary.{ts}.csv | ||
| com.samsung.shealth.step_daily_trend.{ts}.csv | Link | |
| com.samsung.shealth.stress.{ts}.csv | ||
| com.samsung.shealth.stress.base_histogram.{ts}.csv | ||
| com.samsung.shealth.stress.histogram.{ts}.csv | ||
| com.samsung.shealth.tip.{ts}.csv | ||
| com.samsung.shealth.tracker.heart_rate.{ts}.csv | ||
| com.samsung.shealth.tracker.oxygen_saturation.{ts}.csv | Link | |
| com.samsung.shealth.tracker.pedometer_day_summary.{ts}.csv | ||
| com.samsung.shealth.tracker.pedometer_recommendation.{ts}.csv | ||
| com.samsung.shealth.tracker.pedometer_step_count.{ts}.csv | Link | |
| files/ | Contains random files like your profile picture | |
| json/ | Contains the binned data which is higher resolution (e.g. minute level) data from workouts, sleep, etc. |
Notes:
Once the data is loaded and accessible to Zeppelin, create a new notebook. This first block of code provides some basic package imports for Pandas and charting.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.ticker as mtick
from datetime import date, time, datetime
plt.style.use('ggplot')Code language: JavaScript (javascript)I define a few utility methods that help me load data from the different CSV files.
ts = "{timestamp}"
def file_name(name):
return "/shealth/samsunghealth_example_user_" + ts + "/" + name + "." + ts + ".csv"
def bin_file(type, uuid):
return "/shealth/samsunghealth_example_user_" + ts + "/jsons/" + type + "/" + uuid[0] + "/" + uuid + ".binning_data.json"
Code language: JavaScript (javascript)The timestamps stored in time based fields are UTC time, but the time offset is in a separate column. This makes it hard to do certain kinds of investigations because you can’t see things like what time you go to bed on average if you travel around.
The following code provides a load method: load_file(), that can be used to correctly parse the dates.
def date_parser(date, tz):
return pd.to_datetime(date + " " + tz, utc=False, format='%Y-%m-%d %H:%M:%S.fff %Z%z')
def load_file(name, date_cols=None, tz_col=None, **kwargs):
parse_dates_param = dict([('foo' + x, [x, tz_col]) for x in date_cols])
df = pd.read_csv(file_name(name), header=1, index_col=False, keep_date_col=False, parse_dates=parse_dates_param, date_parser=date_parser, **kwargs)
for key, val in parse_dates_param.items():
df[val[0]] = df[key]
df = df.drop(columns=key)
return dfCode language: PHP (php)Analytics
Below I show a few different examples of how to load and visualize interesting data sets.
Sleep Data
Sleep data can be loaded with the following code. This produces two data frames: sleep_data and sleep_data_by_day. If take naps during the day/night, then you’ll get one record per nap + one record per sleep event. If you wake up at night and move around enough, then Samsung Health may create a separate sleep record for when you go back to sleep.
sleep_data = load_file("com.samsung.shealth.sleep", date_cols=['com.samsung.health.sleep.start_time', 'com.samsung.health.sleep.end_time'], tz_col='com.samsung.health.sleep.time_offset')
sleep_data['start_time'] = sleep_data['com.samsung.health.sleep.start_time']
sleep_data = sleep_data.set_index('com.samsung.health.sleep.start_time').sort_index()
# Drop invalid data
sleep_data = sleep_data[sleep_data['efficiency'] > 0]
sleep_data['day'] = sleep_data['start_time'].dt.floor('d')
sleep_data_by_day = sleep_data.groupby(['day']).agg({'sleep_duration': 'sum', 'movement_awakening': 'sum', 'start_time': 'min', 'physical_recovery': 'mean'})
sleep_data_by_day['first_sleep'] = sleep_data_by_day['start_time'].apply(lambda dt: (dt - dt.replace(hour=0, minute=0, second=0, microsecond=0)).total_seconds() / 60 / 60)
sleep_data_by_day = sleep_data_by_day.sort_index()
Code language: PHP (php)With this, you can graph how many hours of sleep you get per night:
ax = (sleep_data_by_day[['sleep_duration']] / 60).plot()
ax.set_ylabel("hours")Code language: JavaScript (javascript)
Sleep data is additionally broken out into a separate file for “sleep stages”. Sleep stages breaks down each night sleep into light, deep, REM, and awake stages. These stages help explain how good of a night’s sleep you’re getting and identify if you’re tossing and turning too much.
The following code will load sleep data into a data frame:
sleep_stages = pd.read_csv(file_name("com.samsung.health.sleep_stage"), skiprows=1, index_col=10)
sleep_stage_name_map = {40001: 'awake', 40002: 'light', 40003: 'deep', 40004: 'rem'}
sleep_stages = sleep_stages[sleep_stages['start_time'] > '2021-01-01']
sleep_stages = sleep_stages.sort_values(by='start_time')
# Data Massaging
# Convert Samsung ids into friendly names e.g. 40001 -> 'awake'
sleep_stages['stage_name'] = sleep_stages['stage'].replace(sleep_stage_name_map)
<meta charset="utf-8">sleep_stages['duration'] = <meta charset="utf-8">sleep_stages['end_time'] - <meta charset="utf-8">sleep_stages['start_time']
<meta charset="utf-8">sleep_stages['day'] = <meta charset="utf-8">sleep_stages['start_time'].dt.floor('d')
stage_by_day = stages.groupby(['day', 'stage']) \
.agg({'duration': 'sum'}) \
.unstack()
stage_by_day.columns = stage_by_day.columns.get_level_values(1)
stage_by_day = stage_by_day.fillna(0)
stage_by_day['total'] = stage_by_day['awake'] + stage_by_day['deep'] + stage_by_day['light'] + stage_by_day['rem'Code language: PHP (php)There’s a couple interesting views of this data that I use. One is to project out the % of time in each stage over a time period to see if my sleep is changing.
stage_by_day['awake%'] = stage_by_day['awake'] / stage_by_day['total']
stage_by_day['light%'] = stage_by_day['light'] / stage_by_day['total']
stage_by_day['rem%'] = stage_by_day['rem'] / stage_by_day['total']
stage_by_day['deep%'] = stage_by_day['deep'] / stage_by_day['total']
data = stage_by_day[['awake%', 'light%', 'rem%', 'deep%']]
ax = data.plot()
ax.set_title("% sleep time spent per stage")
ax.yaxis.set_major_formatter(mtick.PercentFormatter(xmax=1))Code language: JavaScript (javascript)
Heart Rate
Heart Rate data can be found in two locations: the main CSV file and in the binned JSON files. The JSON files contain higher resolution per minute. The main CSV file will contain either hourly summaries in the case of binned data or individual data points. My watch can be configured to collect heart rate data either continuously or intermittently. If it’s continuous, then you’ll see the binned data.
The following block will load the summary data and additionally load any binned data that is associated with an hour block into a data frame.
# Load Heart Rate Data (non-binned)
heart_rate = pd.read_csv(file_name("com.samsung.shealth.tracker.heart_rate"), skiprows=1, index_col=False, dtype={'com.samsung.health.heart_rate.custom': str, u'com.samsung.health.heart_rate.binning_data': str})#, parse_dates=['com.samsung.health.heart_rate.start_time'])
heart_rate['start_time_tz'] = pd.to_datetime(heart_rate[['com.samsung.health.heart_rate.start_time', 'com.samsung.health.heart_rate.time_offset']].agg(' '.join, axis=1))
heart_rate.set_index('start_time_tz')
# Load the binned data
import glob
hr_bins = []
hr_simplified = heart_rate[heart_rate['com.samsung.health.heart_rate.binning_data'].isna()].rename(columns={
"com.samsung.health.heart_rate.end_time": "end_time",
"start_time_tz": "start_time",
"com.samsung.health.heart_rate.heart_rate": "heart_rate",
"com.samsung.health.heart_rate.min": "heart_rate_min",
"com.samsung.health.heart_rate.max": "heart_rate_max"
})[['start_time', 'end_time', 'heart_rate_min', 'heart_rate_max', 'heart_rate']]
hr_bins.append(hr_simplified)
for file in glob.glob(bin_file("com.samsung.shealth.tracker.heart_rate", "*")):
hr_bins.append(pd.read_json(file))
heart_rate_full = pd.concat(hr_bins)Code language: PHP (php)- heart_rate contains just the summaries meaning either hourly buckets if you enable continuous heart rate monitoring or single samples if don’t
- heart_rate_full contains all available heart rate data at the lowest resolution available
Plot min, max, and average daily heart rate per day:
heart_rate['day'] = heart_rate['start_time_tz'].dt.floor('d')
heart_rate \
.groupby('day') \
.agg({'com.samsung.health.heart_rate.heart_rate': ['mean', 'min', 'max']}) \
.plot()Code language: JavaScript (javascript)
Weight
Tracking your weight is easy too. The weight is stored as kilograms. The code below concerts to pounds, but if you want to use kilograms remove the conversion line.
weight = load_file("com.samsung.health.weight", date_cols=['start_time'], tz_col='time_offset')
weight = weight.set_index('start_time').sort_index()
# Convert to pounds
weight['weight'] = weight['weight'] * 2.20462
fig, ax = plt.subplots(2, figsize=(12, 8), sharex='all')
ax[0].plot(weight.index, weight[['weight']])
ax[1].plot(weight.index, weight[['body_fat']])
ax[0].set_xlabel('Time')
ax[0].set_ylabel('Weight')
ax[1].set_ylabel('Body Fat %')
plt.show()Code language: PHP (php)That gives us a plot similar to the below:
Conclusion
Samsung Health gives a great export of data that you can analyze. I’ve given examples of how to leverage a few of the files. In the future, I plan to share some other analytics that I’ve done as I’ve worked to bring data analytics to my personal health.
If you’ve noticed any issues or have any other insights you’ve found, leave a comment below.
Thank you for this article, great work. I was planning to use Pandas for the same purpose and I’m glad I don’t have to start from scratch.
This is great — thank you! One issue, though, is that I think there is an error in your date parser. The time on the watch is already in UTC, and the time_offset column is UTC to the local time. Your parser ends up shifting in the wrong direction. I found that the following parser gives the correct offset:
def date_parser(date, tz):return pd.to_datetime(date, utc=True, format='%Y-%m-%d %H:%M:%S.%f').tz_convert(tz)