Microsoft Research: MoodScope, a context-aware smartphone to sense and share your mood

How many times a day do you touch your smartphone? How would you feel if your smartphone were capable of not only knowing your present mood, but of passively sharing it with your social networks? Microsoft is working on a mood-sensing smartphone capable of logging your mood, and would require no tiresome manual input for social networks or chats, like a smiley or frowny face, but would instead sense your mood and automatically share it.

Microsoft Research Asia explained, “MoodScope is a ‘sensor’ that measures the mental state of the user and provides mood as an important input to context-aware computing.” Mood sharing was described [pdf] as when your context-aware smartphone has “mood sensing” and can automatically share a user’s mood with friends. The “MoodScope: Building a Mood Sensor from Smartphone Usage Patterns” paper said the app could:

convert mood states into a single sentence describing how users feel and automatically post single sentences on social networks, such as “I’m excited!” or “A bit upset today…”. Similarly, an instant message application or video-chat application could also allow users to passively share their mood during a chat or voice call conversation, e.g. through a colored icon.

Think of how handy that could be for the NSA, to not only know who you talk with, but your mood at the time as well. Perhaps it can be attached to the metadata? Any text, email or article could be interpreted wrong, as we are highly influenced by our current mood when we read things. With MoodScope, maybe a sarcasm face could be inserted next to NSA and metadata?

A Microsoft MoodScope smartphone could also log your moods so you can keep track of what makes you happy or sad, increasing “user awareness of mood changes and help users recall valuable moments in their life.” While Microsoft researchers Robert LiKamWa, Yunxin Liu, Nicholas D. Lane, and Lin Zhong envision many mood-enhanced apps, one example was mood playlists.

Music players could automatically create mood – based playlists for users, matching how they feel. Smartphones could change their theme, wallpaper or ringtone settings according to user mood. Web, image, and/or video search could filter results to best match the user’s current mood.

Ironically, Microsoft researchers initially used 32 jailbroken iPhone users from China and the USA to test the mood sensing software that they dubbed MoodScope before building a prototype. The mood detection accuracy started at 66%, but improved to 93% after two-months of training.

They wrote that “mood sensing can build an interesting digital social ecosystem as users’ devices automatically share their moods with close friends and family.”

Privacy concerns aside, these moods would enhance social networks by allowing users to share mood states automatically. Users would be able to know better how and when to communicate with others. For example, parents of a son in a bad mood could decide to call to cheer him up. When text messaging an upset boss, a user could be cautious of speaking brashly. Mood sensing can enable users to digitally communicate closer to the way they would in real life. For mood sharing, an automatic mood sensor will not only improve the usability but also more importantly, lower the social barrier for a user to share their mood: we do not directly tell others our mood very often, but we do not try to conceal our mood very often either.

The researchers suggested that in future work they could overcome some of MoodScope’s shortcomings. It is “currently oblivious to factors well known to alter mood, such as face-to-face arguments, stressful traffic conditions or even the weather.” Additionally, smartphone usage can change in situations like “travel or critical work deadlines” that have nothing to do with mood, but currently could be interpreted incorrectly. Who knows, this might be really cool, fun, and otherwise great . . . or it could be a new chapter in privacy hell?

The privacy preserving mechanisms found within the existing MoodScope design are insufficient for a release to the general public. Nevertheless, we take privacy concerns seriously and adopt a variety of data anonymization techniques when capturing user-to-smartphone interactions. However, unintended leakage of information when handling rich high-dimensional personal data can easily occur. Careful study still remains to understand what types of unintended information an adversary might infer from the specific data we utilize.

MoodScope can operate locally on the phone, and so does not need to record or share the sensitive underlying information-but this does not solve the privacy problems faced during model training. In the future we will explore how mood models can be trained using data from multiple people while still providing sufficient guarantees of privacy to each user.

In conclusion, the Microsoft researchers wrote that they foresee “mood inference as vital next step for application context-awareness” and hope to improve “mood-inference from smartphone usage analysis,” noting that “phone calls and categorized applications strongly predicted mood.”

You can read all about MoodScope: Building a Mood Sensor from Smartphone Usage Patterns on Microsoft Research or download the pdf.

Like this? Here’s more posts:

• You might be a terrorist if…you complain about your tap water
• Breaking down latest leaked PRISM slides claiming U.S. ‘bugged EU offices’
• Project Chess helped NSA snoop on your Skype communications
• NSA whistleblower Snowden: Even innocent Americans are ‘being watched and recorded’
• It’s hitting the fan: Anger mounts over PRISM, NSA spying scandals
• Reporters threatened with CFAA, labeled hackers for finding security hole
• Hacking and attacking automated homes
• Former CIA, NSA director sounds off on PRISM, spying tools
• Rule of 7 applied to domestic surveillance

Follow me on Twitter @PrivacyFanatic