KELLEY, M. Liam (2023) RIR-in-a-Box: Estimating Room Acoustics from 3D Mesh Data through Shoebox Approximation PFE - Project Graduation, ENSTA.
Full text not available from this repository.
Abstract
The paper written during this internship proposes a novel deep acoustic model of 3-dimensional (3D) room meshes via human-interpretable latent codes representing virtual shoe-box rooms that serve as intuitive and modifiable entities for versatile audio signal processing for augmented reality smart glasses. Although 3D room meshes have been used to generate room impulse responses for simulated data generation purposes, the potential of mesh representations remains to be explored, especially for real-world complex rooms with non-exact meshes estimated at run time. We use advanced smart glasses with spatial mapping capability to dynamically represent an actual room as 3D meshes while pinpointing the user’s location within the room. Given this spatial configuration, we train a graph neural network with a physics-based regularization and a differentiable image-source method to infer a virtual shoebox room that has the same reverberation characteristics as the actual room. Our evaluation shows that our model can create such a virtual rectangular room from 3D meshes of an irregularly-shaped room. Additionally, we demonstrate that the latent codes are useful for conditioning a dereverberation system applied to real-world data captured using Microsoft HoloLens 2.
Item Type: | Thesis (PFE - Project Graduation) |
---|---|
Subjects: | Information and Communication Sciences and Technologies Mathematics and Applications Physics, Optics |
ID Code: | 9944 |
Deposited By: | Liam Kelley |
Deposited On: | 26 févr. 2024 18:22 |
Dernière modification: | 26 févr. 2024 18:22 |
Repository Staff Only: item control page