Project Overview

In 2017, during the early stages of AI, I recognized its potential to predict, filter, and personalize online content. Amid the escalating information war between Ukraine and Russia, I aimed to highlight how media manipulates emotions and shapes perceptions. The project used AI to analyze news sentiment and visualize predicted human reactions through a 3D video simulation of a Ukrainian family responding to online media stimuli.

Design Challenges & Technical Constraints

Design Challenge: Translating complex human emotions into an immersive multimedia experience.

• I developed a 3D simulation with NPCs (non-player characters) that reacted to eight primary human emotions.

• Characters exhibited behaviors through animations, sounds, and text based on AI-detected emotions.

Technical Constraints: Limited AI capabilities in 2017.

• My team pivoted from analyzing all multimedia content to focusing solely on text-based news websites, adapting to the AI limitations of that time.

• We used pre-generated dialogue based on sentiment analysis of 1,000 news articles to reduce processing costs.

• Initially focused on English-language resources, we expanded to Ukrainian and Russian content using Google Translate, and later, adapted to new challenges by switching to StarDict.

Solution & Key Features

• Emotion Detection: AI analyzed text for primary emotions using the DepecheMood dataset.

• Visualization: The 3D characters displayed behaviors matching the detected emotions. An LED display showcased news headlines while the simulation ran.

Results & Impact

🚀 Engagement: 25% increase at the Pinchuk Art Center.

📸 Media Coverage: Featured in Vogue Ukraine, sparking discussions about AI's role in media.

✨ Audience Feedback: Viewers reported increased mindfulness of their media consumption habits.

Reflections & Future Vision

• Learning: Simplified technology and pre-generated content helped overcome early AI limitations.

• Today’s Approach: I would leverage modern tools like OpenAI or Meta’s models for real-time sentiment analysis of multimedia content, including video.

• Enhancements: Adding interactive features to allow viewers to provide feedback and contribute to AI training.