How accurate is your sleep tracker? Are consumer sleep wearables good enough to used for research and clinical purposes? Join us as we navigate the complex issues surrounding marketing claims, scientific validation, reliability, raw data and why ‘black boxes’ technology is an impediment to research.
In this first instalment of a 2-part episode, we talk about the rapid improvement that consumer sleep tracker have made in a short amount of time, and how, compared to expensive validated technology like actigraphy, could potentially unleash powerful new ways to advance sleep science.
For Part 2 of the discussion: https://sleepjunkies.com/the-great-sleep-tracker-debate-part-2
Prefer to read? Download the full episode transcript here
Skip to sections:
- 2:48 Introducing Jesse Cook
- 4:43 Discussing the review ‘Wearable Sleep Technology in Clinical and Research Settings’
- 6:50 The lack of incentives for rigorous validation of consumer sleep trackers (CST)
- 10:18 CST have great potential but they are ‘black boxes’
- 13:29 Actigraphs and the first generation consumer sleep wearables
- 16:20 Newer generation sleep wearables are getting much more accurate
- 21:52 Is it worth paying $1000 for a clinical actigraph?
- 25:21 Comparing CST measurement techniques to polysomnography
- 31:32 REM sensitivity of CST has improved from 30% to 60% in only a few years
- 32:40 The problem with detecting sleep from ‘motionless wake’
- 37:02 Machine learning – the ‘low hanging fruit’ to improve accuracy of CST
- 40:09 Challenge for doctors when patients report use consumer sleep trackers
- 41:37 Fitbit, NIH and big data initiatives and future potential
This episode’s guest:
Jesse Cook is a doctoral student within the Clinical Psychology program at the University of Wisconsin-Madison under the primary mentoring of David Plante, MD, PhD. Previously, he completed his undergraduate degree at the University of Arizona, whereby he assisted in projects directed by Dr Richard Bootzin.
Jesse’s research primarily focus on the assessment and treatment of persons with unexplained excessive daytime sleepiness. Additionally, he has published multiple papers evaluating the utility and ability of wearable consumer technologies as a sleep assessor, relative to PSG. You can reach Jesse by email at jdcook4@wisc.edu
Resources:
Jesse Cook Researchgate: https://www.researchgate.net/profile/Jesse_Cook3
Jesse on Twitter: https://twitter.com/SleepAndSports
Review: Wearable Sleep Technology in Clinical and Research Settings: https://bit.ly/2K4e9MV
Fitbit/NIH research collaboration: https://bit.ly/2RUzsDa
Orthosomnia: https://jcsm.aasm.org/viewabstract.aspx?pid=30955
Philips Actigraphy devices: https://www.actigraphy.com/solutions/actigraphy.html
Sleep tracking guide: https://sleepjunkies.com/the-ultimate-guide-to-sleep-tracking/
More Episodes:
[powerpress_playlist limit=20]
Full transcript:
Jeff Mann: 02:18 Good afternoon. I’m joined here with Jesse Cook. Hi Jesse and thanks for joining us today.
Jesse Cook: 02:25 Thank you for having this discussion. I’m really excited to be here.
Jeff Mann: 02:29 I’m really excited. We had a brief chat earlier in the week and I could tell that you’ve got the same sort of passion and interest. Jesse’s a a graduate student at the University of Wisconsin, Madison, can you give us a little background to your academic life and how you got involved in sleep?
Jesse Cook: 02:49 Yeah, absolutely. So I was an undergraduate at the University of Arizona in Tucson and I joined the late Dr. Richard Bootzin’s sleep laboratory and spent a couple of years as a research assistant under his guidance and really fell in love with the field of sleep.
I didn’t realize initially how vast the topic was and the implications that it had on so many different aspects of life, whether it’s performance, health, your day to day mood, all sorts of things.
It really just invigorated me to continue on. And so upon graduating, I took a job working alongside Dr. David Plante here in Madison, Wisconsin. That was back in 2013. We’ve been working alongside one another for now, the past six years. wow, that’s crazy.
03:44 And specifically we tend to work with patients who have unexplained excessive sleepiness. The diagnostic terminology is idiopathic hypersomnia. And we do a bunch of neurophysiological evaluations with these patients to try and figure out what’s truly going on.
But at the same time, I’ve found myself over the last few years really seeking out an understanding of the utility and ability of these commercial consumer sleep trackers and how useful they are, not just for research purposes and not just for clinical purposes as well, but just for an everyday user, and an individual in the population and whether they truly have merit.
Jeff Mann: 04:32 Well, this is the topic we’re talking about today, consumer sleep trackers and specifically we’re going to talk about wearables because there’s other types of sleep trackers, as I’m sure everybody’s aware of.
Jeff Mann: 04:44 I first got in touch with Jesse because, you peer reviewed a brand new review that’s come out. Correct me if I’m wrong, but I think this is probably the most comprehensive and certainly the most up to date look, we’ve had yet to do with the role of consumer sleep trackers and how they measure up in terms of accuracy and what the implications are for using these in clinical practice and also in research. And this was a colleague of yours….. um…
Jesse Cook: 05:18 Yeah… Max de Zambotti and his team, SRI International. They do a lot of work, similar to how we do it out here where it’s not their main objective, but they explore the ability of these devices in the context of research designs that evaluate them against the gold standard, polysomnography.
And this comprehensive review was the first to really not just fundamentally organize the existing data in a digestible manner, but also formulate an approach to evaluating these devices systematically going forward, which is a major challenge as various research teams have employed different techniques, different analytical designs, and these inconsistencies just further complicate the interpretation of the abilities of these devices. And as such, I must thank Max and his team for outlining such an appropriate structure and process in that sense.
Jeff Mann: 06:20 I’m not a scientist. I’m definitely a layman, but I read a lot of papers and I’ve got to say it’s quite a good read actually, and it’s very clearly laid out.
But there’s a couple of passages that I’ve highlighted that are very sort of pertinent to the discussion. And the reason I asked you on, Jesse to talk about this, I don’t think there’s a proper conversation going on about this issue of sleep wearables and their efficacy and validation.
06:48 There’s a couple of passages here. One of them says “there is a lack of incentives from both a scientific community and industry to perform dedicated scientific validation of sleep tracking wearables. Thus, the existing validation studies are frequently initiated by the curiosity of isolated researchers or research groups moved by the need to find affordable accurate and reliable alternatives to expensive medical grade devices for measuring sleep in natural contexts”
Jeff Mann: 07:18 And I guess you’re one of those guys, Jesse,
Jesse Cook: 07:22 I seem to have found myself in that camp. Yeah, that’s a major point of complication currently as ideally we want as researchers in the science realm to have a very symbiotic relationship with these companies, these manufacturers, in the sense that we think these products have merit across many different domains.
But we just want to make sure that they’re appropriately described and evaluated. And it does not seem that a lot of these companies are putting in the appropriate efforts to truly characterize their device in the context of other sleep measurement tools that have demonstrated validity and reliability.
And the problem that comes into play here, and as Max pointed out, the isolated researchers in groups, when you start involving the companies in your study designs, ie if they send you their products, if they fund the investigation in its own right, then you’re inviting their ability to suppress the output of the design, the results.
08:38 And so we have this kind of difficult balance beam situation where we want to have them involved, but we want to be transparent with what we find with the devices as well. So it’s just a challenging landscape there.
Doctor Plante and I, as Max pointed out, found ourselves starting to go down this road because of their affordability, because of prevalence in the society as a whole so far, because of their purported ability to classify REM sleep, which is really important for patients that we tend to work with.
These characteristics make these devices very advantageous on many levels. So understanding their true utility is very important in that sense. It’s just complicated to navigate as I mentioned earlier.
Jeff Mann: 09:27 Yeah, absolutely. So we’ll dive in a little bit. This is other short passage that I want to read out. And this for me kind of sums up this tension between the scientific community and the business community. And it says “for the scientific sleep community, the necessity of opening “black box wearable devices” is important for raw data access and standardization. But raw data access and cloud services do not come free. Within this scenario it’s unclear if a line of consumer products and platforms more focused on the needs of researchers and clinicians would fit the consumer wearable companies business model” And I think that really neatly sums up the difficulty that you just expressed.
Jesse Cook: 10:18 Yeah, absolutely and this opens up so many realms of conversation here. That’s currently the biggest challenge, the lack of raw data availability and transparency when it comes to their scoring algorithms, their staging algorithms. What actually these devices are doing.
Their main, I wouldn’t want to to say ‘competition’, but the sleep field relies upon something known as actigraphy. Clinical actigraphs that typically are about $1000. So markedly more expensive than these consumer wearables, that seemingly perform very comparably to these consumer wearables and in certain estimations of sleep, but allow easy access to the data in the sense that researchers can then manipulate the data to produce equations, functions that can help in the diagnostics process.
We call those distinguishing functions, where we can map onto certain diagnoses, disorders based on the data that’s available. And that just is completely inhibited at this current juncture with the consumer wearables. So it really comes down to what purpose these devices are going to have in our society.
11:36 If they’re strictly going to be utilized so that the average individual can have a better sense of how they’re sleeping, what their sleep patterns are like, their bedtime and rise time consistency and that in the context of their overall health, then that’s a wonderful product to be utilized and marked.
But once these companies start endorsing their products as medical grade equipment, which has happened previously, it becomes much more difficult in that regard to have the relationship that would be most optimal in that sense.
Jeff Mann: 12:10 Yeah, it’s a very muddy situation at the moment and there are a lot of complications, but there’s also huge, huge, massive potential as well.
Jesse Cook: 12:19 Absolutely.
Jeff Mann: 12:20 So let’s talk about the sort of brief history and we’re literally only going back about 10, 15 years at the most here, but I know you wanted to make this distinction that even in this short time, there’s been a progression and we’ve moved on from simpler technology to more advanced technology and it’s still on a very steep upward curve.
This technology’s getting better and better every year. So I guess the first thing that spurred a lot of people onto getting into sleep tracking were the apps, so we’ll skip over those. So you know, you can get an app and put it on his smart phone and people would put them under their pillow, and it’s tracking their movement, supposedly translating that into sleep data.
13:07 But where your involvement started, Jesse I believe, was looking at some of these simpler wearable wrist worn devices. The big companies like Fitbit that we’ve heard about. So do you want to explain how they work and you were involved in some proper validation studies of these early devices?
Jesse Cook: 13:29 Yeah, absolutely. I’ll kind of walk through briefly the history as you said over the markedly significant progression that’s occured in a relatively short time, which is seemingly standard for society these days with technology and whatnot.
So I mentioned actigraphy earlier. An actigraph is something that you wear that has a triaxial accelerometer in it and measures movement and from movement counts – it goes through a programmed algorithm that then outputs whether it’s sleep or wake when we’re talking about sleep classification here.
14:07 And the original models of these wearable consumer sleep trackers utilize the same technological principle in that it was strictly accelerometer based. We call those single sensor models.
And these models perform fairly poorly in their estimations of total sleep time. And they were limited in the sense that they couldn’t classify sleep, they couldn’t stage sleep beyond just sleep and wake, the dichotomous classification scheme that actigraphy is confined to as well.
Jesse Cook: 14:43 And a lot of these devices, the results were indicating overestimations of total sleep time around anywhere ranging from 30 minutes to an hour, which is a significant amount of time.
Jeff Mann: 14:58 That’s quite a lot in a night’s sleep,
Jesse Cook: 15:00 Yeah it has a vast complications – I guess it depends on how much you’re sleeping in a night – but it has vast complications when people start interpreting their data as well, which may get into.
But more recently these devices have added additional biosensors. Most have now incorporated heart rate tracking. There’s a fair amount of existing data out there that indicates that you can reliably stage sleep based on heart rate fluctuations throughout the night.
And so these devices are now pairing accelerometer based movement counts with heart rate data to determine the sleep wake classification as well as whether somebody is in light sleep, deep sleep or REM sleep during their sleep periods.
15:50 And what we’ve seen so far is that initially these multisensory models, were not performing any better than their single sensor models, although I will stress that no research investigation has been performed directly comparing a single sensor to a multisensor, so you kinda have to take that with a grain of salt.
But over the last few years, in particular, the data that’s come out of our laboratory as well as Max’s team and some other places as well, seemingly is indicating that these devices are getting much more accurate. Their bias and estimation is reduced quite a bit, especially when considering total sleep time, which may be their most useful attribute.
These devices are still really limited in their ability to classify sleep. And I can go into more detail about how we analyze those types of characteristics, but basically these devices are getting better it seems and the multisensory components are probably most attributal to that as we have no idea whether there’s been algorithmic changes -kind of going back to that black box issue earlier. Right.
17:01 So that’s encouraging, very encouraging in many aspects. And it’s gotten to a point, Jeff, where these devices from my data have actually seemingly suggest better performance than clinical actigraphs.
So you have to start asking yourself. Why pay the almost 10 times more increase in cost for a clinical actigraph versus these wearable sleep trackers? And there’s advantages and disadvantages to both of those measurement techniques that complicate the issue further.
But that’s kind of the branching off point. And so to truly evaluate these devices, going back to your kind of initial question, you want to have what we call concurrent evaluation with the gold standard, polysomnography.
So for my studies people, patients are in our sleep center, Wisconsin Sleep, they’re undergoing a full clinical polysomnographic evaluation. At the same time these individuals are wearing a consumer sleep tracker or wearable on their non dominant wrist. In some of my designs I also have a clinical actigraph as well so we can make comparisons there.
18:11 There’s some additional components whereby these devices need to be synchronized on the same network of computers so that you don’t have issues with time synchronization, things of that aspects. And then you do some analyses and your basic analyses are looking at congruence with PSG, polysomnography more or less in relation to the main outputs of the PSG variables. How do these devices perform?
And then the additional, kind of more important analyses that a lot of research teams advocate for are the epoch epoch comparisons, which directly evaluate the devices’ staging abilities.
What we call an epoch is a duration of time, a window where we stage usually 30 seconds of sleep, wake or some variation of sleep and how those are congruent or not congruent with the PSG staging. And this allows us to determine the sensitivity, the specificity and the accuracy of each device in the context of sleep versus wake classifications and the various stages of sleep.
Jeff Mann: 19:20 Okay. So just to break it down a little bit, there may be some people – I gave you a lot there – Yeah, that’s great, but they’re hearing a lot of terminology that they may not be familiar with.
So we’ve talked about actigraphs a little bit and these single sensor early wearables, let’s just use the generic term FItbit, you know, wrist worn activity trackers.
So initially this accelerometer, it’s a tiny little chip that sits inside these devices and they’re very cheap to produce, it’s tracking movement. So it knows when your wrist is moving, you can track the direction of movement. But these devices, they started out just to monitor and track activity and then the companies decided, okay, well maybe we can use it as a proxy for sleep and weight and this sort of dichotomous, either sleep or either wake.
20:19 And in a sense that’s where actigraphy is. Do we have any more complicated versions of actigraphs or are they all single sensor devices?
Jesse Cook: 20:30 To my knowledge, the most widely utilized actigraphs still exist strictly as a single sensor device. They may be paired in a clinical setting with some other devices to assess breathing and things of that nature as well. But for the most part, the actual actigraphic device is still strictly accelerometer, focusing on those movement counts.
Jeff Mann: 20:55 And how long have people been using actigraphy in research for sleep? It’s quite a while, isn’t it?
Jesse Cook: 21:02 Yeah. It’s got a very robust, longstanding history. Um,
Jeff Mann: 21:07 We’re talking decades here.
Jesse Cook: 21:09 Yeah, a very long time and it’s supported, for the estimation, the assessment of various sleep disorders by the American Academy of Sleep medicine. So it’s generally regarded as a useful tool to assess habitual sleep wake patterns in people.
Jeff Mann: 21:27 So we have a tool, an actigraph and some big companies out the Phillips making these. And as you said, it’s 1000 bucks or so to buy. You can buy one of these activity trackers from China now..
Jesse Cook: 21:42 Right, $30
Jeff Mann: 21:44 Under $10 I’ve seen them. There are just so many of them because they’re so cheap to produce. But in a sense, these really cheap devices, the technology in those is no different to an actigraph which costs 1000 bucks. Is that a fair comment?
Jesse Cook: 22:01 That is a reasonable comment. Certainly and in a lot of ways, even at that lower level, there are still the multisensory options as well. The difference doesn’t necessarily lie in the underlying technology that they’re utilizing – the triaxial accelerometer – but the validation of the scoring algorithm.
Just because it’s collecting the data in a similar fashion, doesn’t mean that it’s utilizing it in an appropriate fashion for either device.
Jeff Mann: 22:37 So this is what researchers such as yourself and other sleep researchers all around the world are paying for when they’re buying a $1000 Phillips Actigraph. Their paying for all the research and development and the ability to dive in, to have access to all that data. But in a sense, the actual thing on your wrist, it’s not worth 1000 bucks just on its own, isn’t it?
Jesse Cook: 23:02 You’re absolutely right in that sense. Just the, the comfort and the sustained relationship and the fact that, in my Philips Respironics manual I have the algorithm that they utilize to compute sleep versus wake. Makes, makes it a lot easier and more useful in that sense.
Jeff Mann: 23:25 So I just want it to, to rewind a little bit for people that, because they may not be familiar with the sort of standard practices that are going on. And it highlights what you’re talking about in the sense that this second generation of multisensory wearables where companies started to introduce heart rate sensors on their devices, holds, potentially a lot more promise in revealing things about people’s sleep behaviors than a simple single sensor, you know, a simple thing like an actigraph or one of the earlier versions.
But obviously these things aren’t validated. People don’t know how they work. It’s a black box. So it’s a frustration for the scientific community because they have something which might be better, but nobody knows if it’s better, but they can’t use it because there’s no way to get the data and, and it’s not tried and tested.
24:20 And the other thing I just wanted to underline a little bit as well is PSG -polysomnography, which is the gold standard, the accepted gold standard for measuring sleep.
And from my viewpoint, we’ve reviewed a lot of consumer sleep technology over the years and I’ve got boxes and boxes stacked on the shelves. And I’ll try them all out and I think, well that’s interesting. And it is really interesting looking at your data and seeing what these things can track.
But the end of the day, after getting over the novelty factor, even thinking about using them on an ongoing basis, my conclusions from using all these devices is, well, I’ve got a score here for my sleep and it’s telling me my sleep duration. Some of these are breaking down into sleep stages as well, but what is it based on?
And this is kind of the whole context of the discussion here. What are these devices actually telling us? And what you want is for these devices to be congruent with the PSG, the polysomnography.
25:32 And I just wondered if you could just explain briefly to people who may not know exactly what those measurements would consist of. So in the experiment you just described where you go into a lab and they’ve got a Fitbit, let’s say strapped on the dominant hand, they’ve got an actigraph on the other hand and they’re doing a PSG test as well.
What would that look like and what would they be measuring to compare to the wearable measurements?
Jesse Cook: 26:00 Sure, absolutely. So, oh your standard polysomnography is going to utilize electroencephalography – EEG as a main measurement technique that’s basically the ability to assess the electrophysiology of the brain during sleep.
And additional to that, we have EOG, EKG, respiratory belts, nasal canular, we monitor the biophysiology across many different domains.
And these data are then provided to certified sleep technicians. In most situations nowadays it’s becoming automated in some areas, but certified sleep technicians who are actively monitoring an individual who’s sleeping through these waveforms and these data and are staging the sleep, whether it’s wake, which I think I’m experiencing right now. And then there are three non-REM stages N1, N2 and your deepest stage, slow wave sleep, and then REM sleep, rapid eye movement, sleep.
27:16 And those five components are the stages that people go through during a night of sleep. And these devices are outputting nowadays the multisensory versions, calculations of your standard polysomnography variables such as total sleep time.
So the amount of time that somebody is actively sleeping during their sleep window, the period that they were in bed. A sleep efficiency calculation, which should be derived based on the total sleep time divided by the time in bed times a hundred calculation. Usually a sleep onset latency, which is how long it takes somebody to fall asleep.
And then potentially some sort of calculation of wake during sleep. We refer to that as WASO, wake after sleep onset. And then the more advanced models, they start trying to break down the actual sleep into light sleep, deep sleep and REM. And as you may have noticed from my characterization of sleep staging in the research and clinical domain, there are no technically, or there’s no one to one relationship between the terms light sleep, deep sleep and then N1, N2 and N3.
So there’s a bit of discongruency there, or incongruency. But for the most part these devices are trying to output similar estimations of sleep quantification and classification that you would get in a sleep centre. Does that answer the question?
Jeff Mann: 28:57 Yeah. So you know, I’m testing, I’m reviewing one of these devices and it gives me all the stats back, and it’s kind of interesting cause, let’s say I’ll go out at night and have a couple of beers, I may notice something, my REM sleep is less than it should be, or my sleep is more fragmented.
So there are lots of interesting things you can get from these consumer wearables. But my point has always been, if I measured that same night’s sleep in a sleep lab, how would that measure up in terms of, in terms of accuracy. And the gold standard, the sleep lab test, even though it may not be perfect and you know, there’s lots of downsides, obviously you’re in an unfamiliar environment, you’re wired up, you’re strapped up to lots of sensors. So your sleep may not be as natural as it should be, But that is the best we’ve got at the moment, isn’t it? That is the gold standard and that’s what we should measure everything by.
Jesse Cook: 29:55 Interestingly enough, Jeff, there’s actually a lot of data out there that suggests that there is no first night effect in a PSG environment. Meaning that people tend to sleep consistently as they would across multiple nights there.
A lot of people actually sleep better than they do at home because there’s no bed partners, there’s no dogs, there’s no kids, there’s none of that additional stuff that can, complicate sleep.
Kind of going back to what your main point here is. How are these devices actually doing when it comes to their light sleep, deep sleep and REM sleep outputs?
30:39 When I first looked at the multisensory device, my 2018 paper, its sensitivity, so the ability of this device to detect true PSG labelled information, whether it be light sleep, whether it be deep sleep, whether it be REM sleep was very, very poor.
To give an example, that device could only reliably detect REM sleep 30% of the time in congruence with PSG. So that’s not very good. That’s worse than a coin toss.
Jeff Mann: 31:10 No, I mean if you bought something you know, a car and it said it was an off-roader, you would expect it to perform reasonably well off road. In the same way, if you buy a sleep tracker that says it detects REM, but it only does it 30% of the time, you’d be a bit disappointed wouldn’t you.
Jesse Cook: 31:26 Yeah, I think I should only pay 30% of what they’re charging me.
Jeff Mann: 31:31 There you go…
Jesse Cook: 31:31 But basically the newer models have gotten better. And that may have to do with more attention to their algorithms. It’s hard to really say. Improvements in integrating the heart rate sensors, who really knows, but that REM sensitivity is now up to about 65% in some of the more advanced models, which is encouraging.
Jesse Cook: 31:53 Again, that’s, that’s still leaves some room for improvement. But it’s at least trending in a direction that is encouraging. So I’m not going to say that these devices are perfect. As you noted, PSG’s not even perfect. Right. But they’re useful and they’re getting better and that’s definitely encouraging.
Jeff Mann: 32:18 Yeah. Okay, so we’ve sort of tracked the history of this, these early, quite primitive devices. They’ve been getting gradually better, introducing more sensors, heart rate, some of them do other stuff like measure your skin temperature.
Jesse Cook: 32:40 One thing that I think is important, Jeff, that I wanted to include. These devices, whether it be anything that relies on movement. So your actigraphy devices, your consumer sleep trackers, whatever you want to look at, they’re inherently limited by their ability to detect wake during sleep.
Because as you can imagine, you can lie perfectly, still, not moving, but be awake. and it will classify you as a sleep. Right. So the performance, the ability of these devices is likely to be vastly different among different characteristics of individuals. For instance, if someone has a disorder like Parkinson’s, a movement related disorder or they share a bed with a dog or they don’t share a bed with the dog or they have a bed partner, these can lead to wide variations in their estimations of sleep night to night because of that component.
Sleep specificity is what it’s called, the ability to detect true wake and right now for most of the models that are wrist worn and fundamentally accelerometer based, even with the heart rate, at best, you’re looking at a 40% ability in that regard. So very poor.
Jeff Mann: 34:01 Great point. Thanks for bringing that up. That’s mentioned in the review as well. One of the limitations of actigraphy is this inability to measure motionless wake.
And we’ve seen that a lot of. People complaining, they’re sitting on the couch and they decided to binge watch three seasons of Game of Thrones ‘Sounds like a good idea’ and they check their starts and it says that they’ve been asleep for 18 hours. But they haven’t. They’ve only been asleep for six hours, but their Fitbit tells them because they haven’t moved for so long that they’ve been asleep for that time.
34:42 So my question to you is, is it possible? And if it is possible, how would one of these wearables, let’s say just doing movement and heart rate, be able to detect motionless wake from sleep. Is that possible with this sort of technology – algorithms measuring heart rate variability,
Jesse Cook: 35:06 At their current state, I think they’re going to be very limited in this regard. I think they’re going to need to advance the current multisensory approach by incorporating more novel techniques. I don’t want to get too Brave New Worldy here, but I can see the potential of having self applied EEG sensors that can Bluetooth sync to these devices that can then provide some information on the brain activity that’s going on in an individual.
And if we can get some information with that, even if it’s crude and very limited that may be able to enhance their current limitations beyond this inability to truly detect wake. We know that when someone goes into the sleeping stages, their EEG, their brain activity fundamentally changes. And so if we can get a window into that to assist with these other components, the movement, the heart rate, you get a much more comprehensive lens of what the individual’s experiencing and thus we can make a better prediction or estimation of what state of being they’re in.
Jeff Mann: 36:19 Right, okay. These devices, because a lot of these are collecting behavioural data and they’re feeding it back into machine learning algorithms and maybe they eventually would be able to learn that you’ve got a pet in bed somehow, or you know you’ve got some kind of condition.
Maybe they could filter that out and detect that you truly are asleep or awake rather, even though you’re motionless. But without that clever artificial intelligence stuff, you’re saying there’s no real way for these devices as yet without any other other types of sensors, EEG, whatever. That’s pretty hard to do at the moment?
Jesse Cook: 36:56 That’s my belief. And fortunately there are much more brilliant people that me that exists that can help remedy that situation. But you brought up a good point there with the machine learning algorithms. I think that’s the most, maybe like the term low hanging fruit that could potentially progress these devices in the quickest amount of time.
You articulated it very well where we can see patterns of behavior or if we can link patterns of behaviour with certain un-normal irregular patterns of movement and heart rate and the algorithms can start adapting based on that individual, we might be able to increase the capabilities and reduce the estimation errors in these devices.
37:42 The problem that I see with that route is that it takes a lot of user involvement. You’re going to have to have the user fill out questionnaires that say, did you share a bed with a partner last night, please document the hours, like what happened during your hours and sleep. Things of that nature.
And although I think there are a lot of individuals out there that would be compelled to do such activities with their time, I also know that a lot of consumers do not want to do any sort of engagement in that regard.
Jeff Mann: 38:15 Yeah. That goes against the whole mass adoption thing isn’t it. People don’t want to, do stuff. People like easy solutions. Yeah. We’ve already gone off track. I’ll try and get on track again.
So we had this tech come in. The tech got better and as we’re all aware of, and I’m sure everyone listening to this podcast is aware there’s been a huge, massive interest in sleep, sleep awareness.
Jeff Mann: 38:43 Every publication you read is putting out stuff about sleep education. It’s a big trending thing. Now people are saying, right, I need to be concerned about sleep as a health factor. There’s also people who are also into fitness thinking they want to optimize themselves on a day to day basis. Exactly.
And all of this has fed into demand for, for wearables to the extent where now we’ve got people taking their Fitbit data to the doctor. We’ve got this term that came out last year -orthosomnia – a sleep tracker-related anxiety.
So it’s quite an important thing isn’t it? That the data that’s actually being reported back to them is, is actually got some, some sort of validation to it.
Jesse Cook: 39:30 Absolutely, and you harped on a lot of really important points there and I’ll do my best to try and navigate them.
First and foremost it’s amazing to see the boom in recognition and attention placed on sleep these days. We’re seeing the health benefits, proper sleep, proper sleep duration, continuity, consistency, depth and various medical ailments such as Alzheimer’s, cardiovascular illness. Simple just day to day optimization.
So I’m very enthused by how society’s latching onto this. And you know, you mentioned the nature of a patient showing up with sleep data to their doctor and how to best integrate there. It’s super challenging. As you mentioned earlier, there’s a myriad of devices available. We have only evaluated a small portion of these in appropriate manner within that regard. These were evaluated in a specific population usually that may not translate or generalize to everybody.
40:39 So what do doctors even do with this information. And that’s a major challenge and actually prompted the American Academy of Sleep medicine to come out with a position statement last year prohibiting the use of wearable sleep trackers, consumer sleep trackers for diagnostic purposes.
They did in that position statement highlight the potential of these devices. But at this current time, just based on what we don’t know about their abilities and how the abilities translate between manufacturers as well as within manufacturers and different models that they make are just the unknowns there, make it impossible to be utilized in a clinical sense.
So yeah, there’s just a lot, of ‘murkiness’ is the word I like to use going on in this domain. And it’s kind of unfortunate in the sense that these devices can be very powerful.
41:38 You know, we talked briefly a couple of days ago and we talked about how Fitbit has aligned with NIH to generate some, some big data. And these devices can be very useful in that sense to help get more information that would otherwise not be viable to obtain.
I mean, getting research participants to come to a a night of sleep in a laboratory. At best I can get 50 people over multiple years and a good amount of grant funding.
But if we’re able to get this real time data through these devices that can help us gain insight into population dynamics, the etiology and causation of certain disorders and complications, they have extreme utility in that regard.
So hopefully in the coming years we can ameliorate the complications burdening their true integration and we can start utilizing them for the purposes that could shed some insight for our issues.
