Digital musical instruments aim to generate realistic instrumental sounds with controllable pitch, timbre and expressivity. In the past, several synthesis methods have been successfully studied and made available to the general public. More recently, the spectacular results obtained in voice synthesis motivated the development of several neural network based instrumental synthesizers. Despite their quality, these deep models remain black box models whose inner synthesis mechanism cannot be easily interpreted.

Designing inherently interpretable synthesizers has the advantage of providing meaningful parameters that can be inferred from examples for sound analysis and/or interpolated for intuitive control for sound synthesis. However, interpretable models typically require the imposition of some domain knowledge-based structural constraints.

In this thesis, we propose to derive these constraints from the equations describing the physics of several musical instruments. This approach is motivated by the results obtained in other scientific fields, where physics-informed neural networks such as port Hamiltonian Neural Networks (pHNNs) have proven their ability to model complex real-world physical systems and to learn their dynamics from data samples.

We hypothesize that coupling strong physics- based priors with neural instrument synthesis will 1) reduce the space of admissible solutions and help to generate high-quality sounds with less data, and 2) provide interpretable physical parameters that will open new perspectives both for sound analysis and synthesis. To validate these assumptions for several musical instruments, different physical models based on port-Hamiltonian systems will be studied and incorporated as neural architectures. The synthesis quality of these physics-informed neural instruments will then be assessed not only via quantitative and qualitative evaluations, but also via usability case studies within the IRCAM's artistic community.

PhD student: Maximino LINARES

PhD supervisors: Dr Guillaume DORAS (Co-advisor), Dr Axel ROEBEL (Director)

Research laboratory: Sciences et technologies de la musique et du son UMR 9912 - Tutelles: CNRS / Sorbonne Université / Ministère de la Culture