IntroductionBackgroundOverview of SystemUser ScenarioDesign and FeaturesBusiness AnalysisAwards

Actualities - A Multiverse Concert

UX Researcher, Business Analysis
Project Overview
Actualities is a seamless live performance that allows both in-person and online audiences to attend concerts within the multiverse.  Using various sensors, we capture and digitalize signals from the physical venue, transforming onsite elements into virtual scenes. These visuals are then projected for the live audience while simultaneously being broadcast via live stream for online viewers, creating a seamless and immersive experience across both mediums.
Additionally, we conducted a business analysis to explore the potential for expanding the project’s commercial viability.
My Contributions
  • Organized four different scales of multiverse concerts of varying scales, with participant numbers ranging from 10 to over 100.
  • Have a demonstration in the 2023 SIGGRAPH Immersive Pavilion, winning the “Best in Show” prize.
  • Propose a business model to explore commercialization and future growth opportunities for the project.

Role

UX Researcher

Team

1 PM
1 UX Researcher (me)

1 Unity Engineer

TImeframe

2023/02-2023/09
8 months

Tools

Figma
Background
These days, customers tend to pursue more immersive and interactive experiences in the entertainment industry. With the development of technology, metaverse has become a term for such experiences. There are many emerging metaverse concerts these years, such as Travis Scott and Ariana Grande’s concert in Fortnite.
However, most Metaverse concerts are pre-recorded and not cost-friendly for middle- to small-scale artists or bands.
Overview of System
These days, customers tend to pursue more immersive and interactive experiences in the entertainment industry. With the development of technology, metaverse has become a term for such experiences. There are many emerging metaverse concerts these years, such as Travis Scott and Ariana Grande’s concert in Fortnite.
Detect audience's movement and transform into virtual avatars.
Guitar and cajon music visualization into the virtual world.
Send emjois into the virtual world.
User Scenario
Design and Features

1. Onsite audience interaction - movement detection

Using millimetre-waves technology, we can track the movements of the onsite audience. This data is used to map their positions to virtual avatars, allowing the onsite audience to seamlessly become part of the virtual world.
Actual integration of detecting movement.
Demonstration of detecting movement.

2. Performance interaction - music viusalization

By utilizing sensors from the startup HC MUSIC CO. and capacitive sensors, we can detect the sounds produced by instruments like the guitar and cajon. These inputs are then transformed into visual elements, creating an immersive music-visualization environment.
Actual integration of Music visualization.
Demonstration of Music visualization.

3. Online audience interaction - sending emojis and messages

We integrated Audience, a 360-streaming platform, with our system, enabling the online audience to participate in the virtual experience using their phones, computers, or VR headsets. They can also interact by sending emojis and messages to each other.
Actual integration of sending emojis.
Demonstration  of sending emojis.
Awards
  • Publish: Lin, K. F., Chou, Y. C., Weng, Y. H., Tsai Chen, Y., Lim, Z. Y., Lin, C. P., ... & Pan, T. Y. (2023). Actualities: Seamless Live Performance with the Physical and Virtual Audiences in Multiverse. In ACM SIGGRAPH 2023 Immersive Pavilion (pp. 1-2).
  • Publish: Lin, K. F., Chou, Y. C., Weng, Y. H., Chen, Y. T., Pan, T. Y., & Han, P. H. (2023). Exploring Multiple-Display Interaction for Live XR Performance. In ACM SIGGRAPH 2023 Posters (pp. 1-2).
  • Exhibition: ACM SIGGRAPH 2023 Immersive Pavilion - Actualities.
  • Competition: 2024 Chunghwa Telecom Co. 5G Innovative Applications Competition: Student Group 2nd
  • Competition: 2023 Wellness Science and Technology Innovation and Entrepreneurship Competition: Nominated
  • Competition: 2022 National Colleges and Universities Smart Innovation and Cross-domain Integration Creation Competition -Third Place Award, Cross-domain Integration Award.

Other Works