Considering the growing number of individual administrative decisions taken on the basis of results delivered by algorithms or AI systems, the Defender of Rights is concerned about the risks that this algorithmization of public services poses for users' rights. She presents several recommendations so that the guarantees provided for by law are fully implemented.
In her report "Algorithms, AI systems and public services: what rights for users?", the Defender of Rights, responsible for defending the rights and freedoms of users of public services, examines the effectiveness of two particularly important guarantees to ensure respect for these rights: human intervention in decision-making and control of systems, and the requirement for transparency with regard to users.
In order to automate, improve or accelerate certain procedures, the administration is increasingly using algorithms or AI systems. Individual administrative decisions taken by administrations can thus be partially automated - if human intervention is planned - or fully automated. In return, European (GDPR) and national legislation (Data Protection Act and Code of Relations between the Public and the Administration, CRPA) provide guarantees to ensure that the rights of users of public services are respected.
When an administrative decision is said to be "partially automated", a public official must take a positive, concrete and significant action based on or alongside the result generated by the algorithm, in decision-making.
The Defender of Rights notes, particularly on the basis of the complaints she receives, that this intervention sometimes proves to be non-existent – ​​as is the case for the Affelnet high school allocation procedure, or Parcoursup – and sometimes inconsistent or biased, when the people involved in individual decision-making tend to endorse the results produced by the system without questioning them.
Faced with these limitations, and their effects on the rights of users of public services, the Defender of Rights recommends issuing mandatory criteria and operating procedures to more precisely qualify the nature of the human intervention required.
When it has made a decision based on algorithmic processing, a public service must provide a certain amount of information to the user concerned but also to the public. This legal requirement for transparency, which stems from a constitutional principle, must make it possible to understand certain decisions in order to be able to debate them, or even usefully contest them.
The Defender of Rights notes that information obligations, although essential, are sometimes poorly or poorly respected and makes several recommendations, including: