This doctoral project aims to bring out a way to associate the statistical neural approach and the symbolic approach discretized into descriptors/values of knowledge management systems, in order to propose and implement an innovative and hybrid method dedicated to biodiversity identification. The aim is to go beyond the juxtaposition, or succession of methods, and consider how automatic methods can be a guide within methods based on human observation, and vice versa.

The key points of the project are :

• Exploiting species-associated text resources such as weak supervision on image data to identify discriminating features.
• At the textual level, the challenge is to identify named entities and the relationships between these entities. The images will be used as a weak supervision to locate characteristics (characters of species) in relation to each other in order to build annotated images (mapping) and enrich the knowledge bases.
• The last part concerns the explanatory dimension of the identification system, a particularly interesting axis in the wake of the xAI movement. On the image side, the idea was already to focus on explanations centered on identified semantic elements; on the text side, the idea is to produce a short synthesis based on the observation of an image and a knowledge base. The field of application of biodiversity makes it possible to exploit the multitude of resources available to produce new results at the frontier between the VQA -Visual Question Answering- and the generative summary.

The National Museum of Natural History abounds in text, image, and knowledge base resources on living organisms. This project benefits from data from the collections of the Museum (several million images) and from the application contexts of programs such as those of participatory science. It is also based on a knowledge base collaborative platform (Xper3) already used to help identify living organisms. For research and methodological development, the project will focus on botanical data and herbaria for which the resources, knowledge base (texts and images), are particularly well available.

 

PhD student: Maya Sahraoui  

PhD supervisor: Régine Vignes-Lebbe

Research laboratory: ISYEB - Institut de Systématique, Évolution, Biodiversité