Audio Processing

Distributed microphone networks

The use of low-cost devices for acquisition of audio streams is becoming widespread. In this context, the space-time processing research community is interested to enable functionalities such as source localization and separation, denoising etc. on such devices, to keep at bay the cost of equipment. Our research activity aims at attaining this goal with ad-hoc algorithms to overcome the inherent challenges that pop up when one intends to use multiple independent devices. In particular, the research activity focuses on synchronization of streams acquired with different clocks, self-calibration, and reduction of the computational cost of space-time processing algorithms.

Acoustic scene analysis and rendering

The use of microphone arrays for the analysis of acoustic scenes has been a hot research topic for the last twenty years. Recently, ISPL has developed the plenacoustic framework: a representation of the sound field as a function of time, space, and direction of propagation, in which acoustic primitives of interest (such as sources, receivers, reflectors, etc.) are conveniently represented as linear patterns. It is therefore easier to perform tasks such as source localization, localization of reflectors and obstacles, extraction of acoustic sources from mixtures, etc.

ISPL also works on the rendering of acoustic scenes through loudspeaker arrays. In this context different methodologies have been proposed, from Ambisonics to Wave Field Synthesis, the most recent of which is based on the use of the plenacoustic framework.

Modeling of propagation in complex environments

Accurate and fast tracing acoustic paths from source to listener in complex environments is of great interest to architects and engineers to predict the acoustics of complex environments, such as auditoria, cinemas, theatres, etc. before their building. Moreover, accurate modeling of propagation is crucial for the rendering of virtual environments, e.g. immersive gaming and walkthrough. In the last ten years ISPL has developed methodologies for accurate and fast tracing of acoustic beams based on a pre-computation of the mutual visibility between walls of the environment. Once this information has been computed in advance, and source and receiver locations are specified, the tracing of acoustic paths consist of a simple lookup on the pre-computed data structure. Techniques have been developed for both complex 3D and 2D environments.

Binaural rendering

In order for a binaural stream to deliver the sense of presence, a suitable modeling of the Head Related Transfer Function (HRTF) must be implemented, which models the response of the ear of the listener to sounds incoming from different directions. Every person has its own HRTF, and the use of a wrong HRTF impairs the listening experience. ISPL is developing techniques for accurate and simple personalization of the HRTF, by means of both direct measurement and prediction.

The Musical Acoustics Lab is located in Cremona, in the “Palazzo dell’arte”, where the prestigious Violin Museum is. Founded in 2013, the Lab builds upon the expertise of the Image and Sound Processing Lab (ISPL) of the Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB) of the Politecnico di Milano, with the collaboration of the Department of Mechanical Engineering. The Lab was established with the financial contribution of the Arvedi Buschini Foundation and the CARIPLO Foundation (Progetto Distretto Culturale di Cremona). The Lab closely collaborates with the Arvedi Laboratory of Non-Invasive Diagnostic Analysis of the University of Pavia, and shares some lab space with it. The Lab capitalizes on the experience of ISPL in the areas of audio/sound processing, computational acoustics and machine intelligence, as well as the experience of the Dept. of Mechanical Engineering on vibro-acoustics analysis. The Musical Acoustics Lab is focused on improving the quality of acoustic musical instruments (particularly stringed instruments of contemporary and classical luthiery) as well as listening environments.

Rendering room – 8m x 5m phono-insulated, vibro-insulated semi-anechoic room for timbral and acoustic measurements and acquisitions, as well as experiments of spatial audio rendering. The room rests suspended on vibro-absorbent material and is completely covered with phono-absorbent material over a wide range of frequencies. The inner acoustics can be altered using reconfigurable acoustic reflectors, enabling experiment of room acoustic analysis and reconstruction, and recording sessions in different acoustic conditions. The semi-anechoic behavior enables experiments of olophonic and plenacoustic rendering based on speaker arrays as well as experiments of spatial acquisition/capture of acoustic fields. The rendering room is equipped with a fourth-order ambisonics rendering dome of 32 high-resolution Genelec speakers for immersive musical listening experiences.

Vibro-Acoustic Analysis Lab – equipped with all the instrumentation that is necessary to acquire high-resolution 3D models of musical instruments; to measure, model and analyze its vibrational behavior; to measure/estimate its radiative behavior. As far as 3D modeling is concerned, the Lab is equipped with a laser scanner for large objects, thickness gauges, advanced software for surface modeling (patchwork as well as levelset methods). As far as vibrational analysis is concerned, the lab has a range of calibrated measurement accelerometers, calibrated hammers, etc. and a laser interferometer for non-invasive measurements on historical instruments. The Lab also has a high-resolution capturing system of 64 measurement microphones for holophonic acquisition of acoustic fields in close proximity (non-invasive reconstruction of vibro-acoustic behavior of soundboards and radiative surfaces).

Control room – equipped with a mobile professional-grade high-resolution spatial audio capturing system for live events. In addition to the mixing system and the professional microphone equipment, the lab also has an Eigenmike, an integrated full-sphere acoustic camera with 32 microphonic capsules that can be used for implementing and controlling virtual microphones, and for spatial audio acquisitions of live events.