Drowsy – Sleep Analytics and Improvement through Technology

Posted by Tanya Kuruvilla, December 11, 2023

18 views (last 30 days) | 0 Likes | 0 comment

Today we are joined by Jonathan Wang, Andrew Fu, Eric Liu, and Suparn Sathya who won the “Best Use of MATLAB” award at HackDavis 2023. Their app tries to make maintaining a healthy sleeping schedule fun and intuitive by tracking and showing comprehensive sleep data. Over to the team to explain more…

From left to right: Jonathan Wang, Andrew Fu, Eric Liu, Suparn Sathya

Inspiration

We were inspired by a common occurrence among college students: sleep deprivation. Staying up late to squeeze in some studying is super common, especially around exam season, but sacrificing sleep for extra study hours is counterproductive. Lack of sleep affects memory, concentration, and creativity, and impacts a student’s academic success and mental health.

To combat this, we thought of creating a sleep tracking game. Our goal was to encourage students to sleep more with in-game rewards. Additionally, we intended to design the app to offer detailed insights into the quality of their nightly sleep and provide advice on enhancing sleep quality for the future.

Breaking down the problem

After we identified sleep deprivation as a common issue that our project could be centered on, we decided to look towards existing products that each had their pros and cons to understand what we needed to develop an effective solution. Based on our research, we found that most existing solutions do very little to ensure consistency and quality of sleep hours but instead rewarded users for waking up daily which doesn’t tackle the core issue of lack of sleep. Even so, another issue was that there was little incentive being provided to users to maintain healthy sleep schedules. To resolve both of these limitations, we decided that the most optimal way to develop a solution would be as a webapp that would be able to access user sleep data. On the webapp, users would be able to view audio analysis of their sleep which may be an indicator of overall quality. Based on the time users go to sleep and wake up, the webapp would “game-ify” their sleep by rewarding them with virtual points that can be redeemed through an online store.

How did we implement it?

Very early into the hackathon, we decided to integrate MATLAB into our project because of its powerful signal processing capabilities. We had no solid pointers on how to get started with designing a sleep analysis algorithm, so the majority of our time was spent on prototyping and testing our data against MATLAB’s various filters and feature extraction functions. The final version of our code is a sleep staging algorithm that takes in an audio recording of someone’s breathing for the duration of their sleep.

The main challenge was identifying every instance of a breath. The solution we came up with involved performing peak analysis on the audio data. Keep in mind the majority of our technical design choices were made either due to simplicity or efficiency; the code below demonstrates some of these aspects.

% Reads audio data. Default sampling rate is 44100 Hz. 

[source, samplingRate] = audioread(“Sleep_2023-5-21.wav”); 

% Reduces audio data sampling rate to 60Hz and truncates total duration to the nearest minute. 

sourceLength = length(source); 

overflow = mod(sourceLength, 3600); 

source = abs(source(1 : samplingRate / 60 : end – overflow)); 

% Evenly splits audio data into 60 second samples. 

source = reshape(source, 3600, []); 

We split the audio data into minute by minute chunks instead of attempting to perform analysis on the entire signal mainly due to the fact that the performance of certain functions such as envelope slows exponentially with increasing input size, but besides this, partitioning the data by minutes allows as discretize the computation and analysis of respiratory rate. To further speed up the program’s performance, we also resampled the data in 60Hz instead of the default 44100 Hz. This reduces the total number of data points by a factor of 735.

% Removes outliers from raw data. 

source = hampel(source); 

% Creates peak envelopes with peak separations of 60 samples. 

source = envelope(source, 60, “peak”); 

In an audio signal, the inhalation and exhalation process is represented by a dense pocket of noise. Knowing this, we settled on running our audio data through the hampel filter, which sparingly removes outliers, generally only targeting data points outside of the noise pockets. After applying the filter, we fit a peak envelope onto the raw signal to extract the shape of the noise pockets; the envelope’s peak separation of 60 samples implicitly assumes that respiratory rate doesn’t exceed 60 breaths per minute (normal respiratory rate while sleeping is 12-20 per minute). Performing these transformations helps minimize the impact of noise and prepares the data for peak analysis.

% Iterates over every minute of audio data. Gets total number of peaks from enveloped data for each minute. 

numCols = size(source, 2); 

respiratoryRate = zeros(1, numCols); 

for i = 1 : numCols 

 pks = findpeaks(source(:, i)); 

 respiratoryRate(i) = length(pks); 

end

% Identifies 15 occurrences of the most abrupt changes in respiratory rate over the full course of sleep, given each change happens at least 15 minutes apart. 

ipt = findchangepts(respiratoryRate, MaxNumChanges = 15, MinDistance = 15); 

rrLength = length(respiratoryRate); 

ipt = [1, ipt, rrLength]; 

values = zeros(1, rrLength); 

for i = 1 : length(ipt) – 1 

 values(ipt(i) : ipt(i + 1)) = mean(respiratoryRate(ipt(i) : ipt(i + 1))); 

end

The number of peaks in the signal every minute is equal to the number of breaths taken that minute, also known as the respiratory rate. Respiratory rate is expected to vary minimally, which is why we incorporated code that averages these values in steps. The findchangepts function flattens the respiratory rate data into 15 steps, with each step covering at least 15 minutes. We chose these parameters for the function semi-arbitrarily with the intent of morphing the result into a textbook-representation of a sleep-cycle graph.

Results

The audio data for the plot below, represented in light-gray, is a 1 minute excerpt of Andrew’s sleep. It demonstrates our program’s peak analysis algorithm.

The envelope, represented by the black lines, wraps around most of the source’s peaks. The peaks that do not fall under the envelope had been filtered out by the hampel filter, which suggests that they were outliers, and most likely unwanted impulse noise. All of the envelope’s peaks are marked with an asterisk. Counting the number of asterisks yields the respiratory rate. For this plot, the respiratory rate also represents a data point in Andrew’s 7 hour sleep during the hackathon shown in the plot below.

Respiratory rate is represented by the light-gray signal, which is visibly very noisy. Because variations are typically ±1, they are likely to be caused simply by sampling error. The black line represents a smoothed version of the respiratory rate signal. With the knowledge that respiratory rate generally changes depending on sleep cycle stages, respiratory rate plots could be used practically to track sleep cycles.

All of us have tested the peak analysis algorithm extensively and found it to be generally tolerant to background noise and low breathing volume while also being highly accurate in correctly identifying the number of breaths taken. Accuracy degrades when the algorithm samples audio longer than a minute, which is another reason why choosing to discretize respiratory rate by a minute-to-minute basis was a good design choice. Our final result is a sleep stage graph, the data for which is exported to our web app, where it is rerendered by its frontend.

Key Takeaways

The main things we learned from our project was to combine various sub-projects into our main project so that we could add as much functionality to our projects. We each played to our strengths whether it was backend, frontend, or the MATLAB and then combined those different parts into our final project. Overall, it was a great experience because we picked up new skills such as NextJS, while also learning about how to integrate files in different languages into our main project.