By Ruben Ohana, Research Fellow at the Flatiron Institute in New York.

Abstract

Foundation models are very large architectures trained on large-scale datasets, and can be used to transfer knowledge from a domain to another. Scientific data, particularly numerical simulations of partial differential equations (PDEs), presents unique challenges due to its complexity and the need for domain expertise to assess prediction quality, complicating the building of the first foundation models in this field. In this talk, I will develop our approach of foundation models for scientific data, highlighting the requirements and expectations for achieving meaningful results. I will also introduce The Well, a comprehensive collection of datasets encompassing multi-scale simulations of fluid dynamics, astrophysics, and biological systems. The Well serves as a foundation for developing models that generalize across diverse physical phenomena, aiming to accelerate scientific discovery through large-scale learning.

Short CV

Ruben Ohana is currently a Research Fellow at the Flatiron Institute in New York, where he is using modern machine learning approaches (transformers and diffusion models) to tackle scientific problems. Previous to that, he earned a PhD from Ecole Normale Supérieure supervised by Florent Krzakala and worked with LightOn where he worked on the design of ML algorithms using Optical Processing Units with applications to adversarial robustness, differential privacy, kernel methods... He holds a engineering degree from ESPCI Paris, a master from ENS in condensed matter and a master of Statistics from Sorbonne University.