Technology might teach us a few things, but it also forces us to lose some skills. Our sense of direction was usurped by GPS. We now relegate even the simplest math problems to a spreadsheet or calculator. Spelling fell by the wayside with autocorrect. The list goes on.
One tried and true skill professionals of all sorts have honed over the years is the ability to read a room and gauge the sentiment. Are people paying attention? Are they engaging with your message? Are they happy to be here? As neuroscience and tech join forces, we may even in this case come to rely on technology to measure whether people are attentive and responsive, and ultimately to gauge their happiness.
Paul Zak, is the founding director of neuroeconomics study at Claremont Graduate University and the founder of Immersion Neuroscience. He has spent his adult life immersed in exploring what happens in the body when we are truly engaged in things — movies, advertisements, lectures, schoolwork.
After years of research, looking at the roles of oxytocin and dopamine as a measure of engagement and what Zak calls “psychological safety” or receptiveness to a message, Zak has automated a way to measure engagement. His method: you simply wear a smartwatch (Apple, Google Wear and FitBit are some of the ones supported) and use his Immersion app. It can measure engagement, attention and the emotional resonance of an experience or a piece of content.
Behind the scenes, Zak has mapped the relationships between 150 neurologic systems and what’s happening in the cranial nerves to measurements by a smart watch like heart rate. He explains: “Algorithms let us identify two core components of immersion: attention driven by the brain’s production of dopamine, and then oxytocin driving emotional resonance.” In simpler parlance, the company captures activities in the brain and correlates those with what’s happening in the heart as recorded by the smartwatch. The end result is a simple cloud-based dashboard that shows a timeline of activity, depicting when a user is deeply engaged and when they’ve tuned out.
Measurements can get quite granular. In a five minute speech, for example, was there a moment when everyone tuned out? Was there a part of an advertisement that didn’t resonate? Zak believes, for example, that at conferences you will be able to get an aggregate measure of the audience’s level of engagement just based onj data generated by their wearables. “Relying on self reporting,” he says, “does not work“.
Other measures of engagement that require all sorts of extraneous hardware like eye trackers or facial recognition are also flawed, Zak says. He is using Immersion in the entertainment world, to get feedback on everything from scripts and trailers to talent acquisitions. It’s also being used to help corporate partners create more impactful communications. Watch a video on how the dashboard works with live and prerecorded content and sign up for a free experience for a self discovery look at your own immersion. According to Crunchbase, Immersion Neuroscience just received a million dollars in funding.
Always a bit suspect of any product that makes inferences about my behavior, I had lots of questions for Zak, but it boiled down to this. Can he tell why I am engaged? Is it because I am looking at the cute guy sitting next to me (exciting my brain) or am I really focusing on the lecture? Equally important, are the algorithms that map brain activity to smart watch measurements valid and bias-free?
Moodbeam, a UK based company, also believes that happy people create thriving businesses, but Christina Colmer McHugh, the co-founder and director, takes a lower-tech approach. The company’s Moodbeam One smartwatch allows you to check in, self report and create a timeline of your feelings. The band has two buttons: one yellow, one blue. (It also measures steps and sleep.) Using it’s companion app you’re given prompts to check in with your feelings: press blue if you’re not doing well at the moment. In the workplace, aggregating that data on a corporate dashboard can allow a manager to gauge the prevailing mood in the workplace. (The information is non-personally-identifiable by default.) Yes, it’s a way for bosses to keep tabs on employee happiness.
Colmer McHugh originally designed the bracelet for her seven year old daughter, who was having some issues at school. “It was a way of checking in that was simple but effective. We could discuss her feelings in detail after school, because the feelings were being recorded.” Fast forward, MoodBeam now has its place in the worlds of education, finance, manufacturing and healthcare where companies are using it to check in on their employees’ emotional well-being. “Businesses are trying to stay connected with staff, especially during the pandemic when work from home and high stress levels were prevalent,” says Colmer McHugh. “Here’s a way they can ask 500 members: ‘You ok?’ without picking up the phone and at a relatively low cost of about $3 an employee.”
Moodbeam reminded me of those Happy or Not terminals used at airports for a quick gauge of customer satisfaction. While they might seem simplistic, when systems like Happy or Not have been used, organizational changes resulted. “How happy you are when you work, and do you matter,” says Colmer McHugh, “are, at the simplest level, telling your employees that you care, and at the management level can be used to inform issues about employee retention and talent acquisition.”
Immersion and MoodBeam are not the only high tech mood rings in town. There are plenty of other bodily reactions that can be measured and correlated with mood, including everything from pupil dilation (eye tracking), skin conductance (EDA/GSR), brain activity (EEG, fMRI), and facial expressions. Companies like IMotions use such biosensors across multiple disciplines to garner insight into emotion. But equating facial recognition to emotion can be flawed in many ways, and be affected by cultural bias.
Me, I’ve wished more than a few times that a boss would wear a mood ring so I‘d know whether timing was right to ask for that raise. The quest to measure emotional states through technology will continue. Some predict that the “affect recognition” market could be worth $56 billion by 2024. Measuring engagement will cause major changes in everything from organizational management to healthcare and education.
But every gain we make using biometric data to infer mood and affect needs to be checked against a sane human touch to see if we’re measuring the right things, equally and fairly. Humans misread human emotion, why should computers fare better?
(Extra credit reading: This Atlantic article on AI and emotions).
View editorial post