What if a movie could make us feel the wind, perceive scents, or experience vibrations synchronized with the action unfolding on screen? Although it may sound like science fiction, this idea belongs to a rapidly growing research field known as mulsemedia or multisensory multimedia. A recent study carried out by researchers from the Gandia Campus of the Universitat Politècnica de València presents, on the one hand, an architecture proposal for the delivery and consumption of audiovisual content enriched with sensory effects and, on the other hand, a simulator that facilitates the development and validation of this kind of immersive experience without requiring real equipment.
The work, published in the journals Journal of Network and Computer Applications (Elsevier) and Multimedia Systems (Springer), was carried out by members of the Immersive Interactive Media (IIM) R&D Group at the Gandia Campus, led by Professor Fernando Boronat. In the first paper, the authors present an architecture for a web-based platform built on low-cost technologies, which incorporates a three-dimensional simulator. This simulator, described in detail in the second paper, makes it possible to recreate, evaluate, and simulate audiovisual experiences capable of stimulating multiple senses beyond sight and hearing. The research represents an important step toward the design and validation of multisensory multimedia experiences.
Beyond Audio and Video
Traditionally, multimedia systems have been limited to the playback of images and sound. However, numerous studies have shown that incorporating additional stimuli, such as vibrations, wind effects, temperature changes, special lighting, or even scents, can significantly enhance users’ sense of immersion and overall quality of experience.
This multisensory approach is particularly promising in fields as diverse as entertainment, education, virtual tourism, video games, and virtual and augmented reality applications. The goal is for users not only to observe a scene but to experience it in a much richer and more engaging way.
Nevertheless, designing and testing such systems presents significant challenges. Conducting real-world experiments requires dedicated infrastructure and numerous physical devices distributed around the user, including fans, vibration units, scent generators, ambient lighting systems, and other specialized equipment. This infrastructure is costly and reduces flexibility during the development and research phases.
A Virtual Laboratory for Multisensory Multimedia
To overcome these limitations, the Gandia Campus research team not only implemented a prototype platform demonstrating the feasibility of the proposed architecture but also developed and integrated a 3D simulator compatible with the international MPEG-V standard. MPEG-V is a standard that enables the description and inclusion of sensory effects in multimedia applications, facilitating their synchronization with audiovisual content.
The new platform therefore integrates both the real control of devices capable of generating physical sensations during audiovisual playback and their simulation. On the one hand, it allows audiovisual content to be played while users experience in real time the sensory effects associated with that content in the physical environment surrounding them. In this way, effects such as wind, vibrations, lighting, temperature changes, fog, water spray, or scents can be synchronized with the video being played.
On the other hand, the simulator goes a step further. It includes a graphical interface with interactive three-dimensional elements and enables users to verify when sensory effects are activated or deactivated and from which position they originate around the user, without requiring a real system to be deployed. The MPEG-V standard defines up to 45 different locations for the placement of sensory devices, all of which are represented in the virtual simulator. The simulation environment makes it possible to validate configurations, detect synchronization errors, and study the behavior of multisensory systems before investing in physical hardware. It supports up to nine different types of sensory effects originating from any of the 45 spatial positions defined by the standard and also includes a module to manage commercial devices capable of generating these effects in real-world environments.
The proposed solution is based on web technologies and low-cost IoT devices, facilitating deployment in domestic or experimental environments without the need for specialized equipment. Furthermore, it remains fully compatible with the MPEG-V international standard, ensuring interoperability among content, applications, and sensory devices.
Evaluation with Real Users
The platform was evaluated by 40 participants, who confirmed correct synchronization between sensory effects and audiovisual content. The results showed that the incorporation of these additional stimuli increases the sense of immersion, realism, and enjoyment, significantly improving the perceived quality of experience. In addition, the graphical interface received an excellent usability rating according to the System Usability Scale (SUS).
These advances reinforce the Gandia Campus as one of the leading research groups in the field of multisensory multimedia, or mulsemedia. The combination of simulation tools, open platforms, and systems compatible with international standards helps bring closer technologies that may become part of future entertainment, education, virtual tourism, extended reality, and next-generation immersive applications.
Further Information
Boronat, F., Sáez, C., Simó, L., & Pastor, J. (2026). Web-based MPEG-V standard-compliant platform for multimedia content consumption with multisensory effects. Journal of Network and Computer Applications, 253, 104515. https://doi.org/10.1016/j.jnca.2026.104515
Boronat, F., Villashagñay, E., Simó, L., & González, J. MPEG-V compliant 3D simulation tool for multimedia playback with sensory effects. Multimedia Systems, 32, 468 (2026). https://doi.org/10.1007/s00530-026-02553-x