Bridging the Gap Between Digital and Physical Stages
For the Voka annual event, we pushed the boundaries of live interaction by introducing a real-time AI-powered MetaHuman. Developed in collaboration with Yondr, this project brought a digital entity to the big stage, allowing hosts, presenters, and the audience to engage in natural language conversations with a virtual human that felt both intelligent and present.
The MetaHuman Presence
The core of the experience was a custom-designed MetaHuman, created and refined in Unreal Engine. We focused on achieving a high level of visual fidelity and natural movement to ensure the character felt at home on a massive LED stage. Use of Live Link and real-time rendering allowed the character to respond instantly to the environment and the people on stage.
AI-Driven Natural Language
To make the MetaHuman more than just a visual feat, we integrated a sophisticated AI system. The AI was specifically trained on data provided by Voka and context related to the event's theme. This enabled the virtual host to answer complex questions, provide insights, and engage in meaningful dialogue rather than relying on pre-scripted responses.
Seamless Real-time Integration
The technical backbone of the project involved a complex pipeline of Speech-to-Text (STT) and Text-to-Speech (TTS) systems.
Key technical achievements included:
- Low-Latency NLP: Ensuring the AI could process speech and generate responses fast enough for a live stage environment.
- Emotion Binding: Linking the AI's sentiment to the MetaHuman's facial expressions and body language in real-time.
- Guardrails & Safety: Implementing a robust system of guardrails to ensure the AI stayed on-topic and professional during the live broadcast, protecting the integrity of the event.
A New Standard for Events
The result was a seamless interaction where the line between digital and physical hosting blurred. This project showcased the potential of MetaHumans not just as static content, but as dynamic, intelligent participants in high-stakes live environments.
If you have any questions about this project or want to discuss AI-driven digital humans, feel free to reach out.
