Affective Computing

Affective computing is the field of computing that recognizes, interprets, responds to, or simulates human emotion and related states such as stress, engagement, and fatigue. Rosalind Picard of the MIT Media Lab named the field in a 1995 technical report and her 1997 book Affective Computing, arguing that computers that work closely with people need some ability to recognize and express emotion, and her Affective Computing group still publishes in the area. Commercial systems that estimate emotion from faces or voices are often sold as emotion AI.

Signals and Emotion Models#

Affective systems read observable signals: facial expressions, tone and rhythm of voice, word choice, posture and gestures, and physiological measures such as heart rate and skin conductance from wearable sensors. Each signal is ambiguous on its own, so multimodal AI models that combine several channels are usually more reliable than any single one.

Two ways of representing emotion dominate. Categorical models assign discrete labels, often based on psychologist Paul Ekman's basic emotions: happiness, sadness, anger, fear, surprise, and disgust. Ekman and Wallace Friesen's Facial Action Coding System, published in 1978, breaks expressions into numbered action units for individual muscle movements, such as the lip-corner pull of a smile, and many facial analysis systems detect these units first. Dimensional models instead place a state on continuous axes, most commonly valence (pleasant to unpleasant) and arousal (calm to excited).

Public datasets follow the same split. FER2013, released for a 2013 machine learning challenge, contains 35,887 grayscale face images at 48 x 48 pixels labeled with seven expressions for image classification. AffectNet, published in 2017, gathered more than one million face images from the web, about half of them manually annotated with expression categories and with valence and arousal scores.

The field overlaps with several better-defined tasks. Facial recognition establishes who a person is, while facial expression analysis describes what their face is doing. Sentiment analysis classifies opinions in text as positive or negative, a narrower target than the moods and physiological states affective systems estimate. Conversational AI adds the simulation side: a chatbot or voice assistant that mirrors a user's emotional tone simulates empathy, which makes agreement from a model prone to AI sycophancy feel more persuasive and is one of the humanlike traits linked to AI psychosis.

Where Affective Computing Is Used#

Driver monitoring is one of the most widely deployed applications. Under the EU General Safety Regulation, every new vehicle sold in the EU has needed a drowsiness and attention warning since July 2024 and an advanced driver distraction warning since July 2026, and camera-based driver monitoring systems meet these rules by watching for closed eyes, yawning, head pose, and phone use. Ultralytics YOLO26 models can be trained to detect these cues for driver monitoring, and pose estimation adds head and body keypoints that reveal slumping or a head turned away from the road. These systems estimate fatigue and driver attention, observable states with direct safety consequences.

Media research is another long-standing use. Affectiva, spun out of Picard's lab in 2009 and acquired by driver-monitoring company Smart Eye in 2021, built facial coding tools that measure audience reactions to advertisements and film frame by frame, aggregated across many viewers. Contact centers apply speech recognition and voice-based emotion analytics to flag frustrated callers and coach agents in real time, and health researchers use wearable signals to track stress and to detect seizures.

Scientific and Legal Limits#

The step from expression to inner feeling is an inference, and a contested one. A 2019 review led by Lisa Feldman Barrett examined more than 1,000 studies and concluded that facial movements do not reliably map onto specific emotions across people, cultures, and situations: people scowl when angry less often than the stereotype suggests, and they scowl for many reasons other than anger. Systems trained on posed or web-collected faces also inherit dataset bias, with uneven accuracy across age groups, skin tones, and cultures, and neurodivergent people may express emotion in ways the training data does not cover. Microsoft retired the emotion-inference features of its Azure Face service in 2022, citing the lack of scientific consensus on the definition of emotions and the privacy concerns around such capabilities.

Law now draws firm lines. Article 5(1)(f) of the EU AI Act has prohibited AI systems that infer the emotions of people in workplaces and educational institutions since 2 February 2025, except for medical or safety reasons. Emotion recognition elsewhere is classified as high-risk, with those obligations now due from December 2027 after the EU's 2026 Digital Omnibus delayed them, and since August 2026 anyone deploying such a system must inform the people exposed to it. Because affective systems process faces, voices, and physiological signals, data privacy rules for biometric data apply as well, which favors edge AI designs that process video on the device and keep only the derived alert.