Description
“Before proceeding to final deployment of the model built by the data analyst, it is important to more thoroughly evaluate the model and review the model’s construction to be certain it properly achieves the business objectives. Here it is critical to determine if some important business issue has not been sufficiently considered. At the end of this phase, the project leader then should decide exactly how to use the data mining results. The key steps here are the evaluation of results, the process review, and the determination of next steps.”[1]
This phase involves several tasks that raise important data protection issues. Overall, you must:
- Evaluate the results of your model, for instance, whether it is accurate or not. To this purpose, the AI developer might test it in the real world. This test could often be done in coordination with a project related partner from the domain the system should be rolled-out (e.g. LEA)
- Review the process. You shall review the data processingsystem to determine if there is any critical factor or task that has somehow been overlooked. This includes quality assurance issues. This actually is the absolute latest phase to involve potential end-users into the development process. However, you should involve and learn about the needs of the end-user in a really early stage of your project (Business understanding). At this stage, stakeholders and end-users can give insights into the system’s strengths and weaknesses in real-world use.
Main actions that need to be addressed
Processes of dynamic validation
The validation of the processing, including an AI component, must be done in conditions that reflect the real environment in which the processing is intended to be deployed. Thus, if you know in advance where the AI tool will be used, you should adapt the validation process to that environment. This is best done by involving respective partners from the domain at stake. If the tool will be deployed in country x, you should validate it with data obtained from the respective population or, if not possible, a similar one. Otherwise, the results might be utterly incorrect. In any case, you should advise about the conditions of the validation to any possible user.
Moreover, the validation process requires periodic review if conditions change or if there is a suspicion that the solution itself may be altered. For instance, if the algorithm is being fed with data from a specific group of people, you should assess whether or not this changes its accuracy in another part of the population. You must make sure that validation reflects the conditions in which the algorithm has been validated accurately.
In order to reach this aim, validation should include all components of an AI tool, including data, pre-trained models, environments and the behavior of the system as a whole. Validation should also be performed as soon as possible. Overall, it must be ensured that the outputs or actions are consistent with the results of the preceding processes, comparing them to the previously defined policies to ensure that they are not violated.[2] Validation sometimes needs gathering new personal data. In some other cases, controllers use data for purposes other than the original ones. In all these cases, controllers should ensure compliance with the GDPR (see “Purpose limitation” section in “Principles” chapter and “Data protection and scientific research” in “Concepts” section).
Deleting unnecessary dataset
Quite often, the validation and training processes are somehow linked. If the validation recommends improvements in the model, training should be performed again. Once the AI tool has finally been achieved, the training stage of the AI tool is completed. At that moment, you should implement the removal of the dataset used for this purpose, unless there is a legal need to maintain them for the purpose of refining or evaluating the system, or for other purposes compatible with those for which they were collected in accordance with the conditions of Article 6(4) of the GDPR (see “Define data storage adequate policies” section).
In the event that data subjects request its deletion, you shall have to adopt a case-by-case approach taking into account any limitations to this right provided by the Regulation (see Art. 17(3)).[3]
Performing external audit of data processing
Since the risks of the system you are developing are high, an audit of the system by an independent third party must be involved. A variety of different audits can be used. These might be internal or external; they might cover the final product only or be performed with less evolved prototypes. They could be considered a form of monitoring and a transparency tool, which is supposed to be a quality feature as well.
In terms of legal accuracy, AI solutions must be audited to see whether they work well with the GDPR considering a wide range of issues. The High-Level Expert Group on AI stated that “testing processes should be designed and performed by as diverse group of people as possible. Multiple metrics should be developed to cover the categories that are being tested for different perspectives. Adversarial testing by trusted and diverse “red teams” deliberately attempting to “break” the system to find vulnerabilities, and “bug bounties” that incentivize outsiders to detect and responsibly report system errors and weaknesses, can be considered.”[4]The auditing must also comprise the fulfilment of the principle of explicability. “The degree to which explicability is needed is highly dependent on the context and the severity of the consequences if that output is erroneous or otherwise inaccurate.”[5] In view of the very severe consequences for individuals being suspected or convicted of criminal activities, the applied ML technologies must allow for explicability, among further measures required, so that the developed systems respect fundamental rights. The audit should also focus on the measures implemented to avoid bias, obscurity, hidden profiling, etc., and the correct use of tools such as the DPIA, which can be performed multiple times. Implementing adequate data protection policies from the first stages of the lifecycle of the tool is the best way to avoid data protection issues.
References
1Colin Shearer, The CRISP-DM Model: The New Blueprint for Data Mining, p. 17 ↑
2High-Level Expert Group on AI, Ethics guidelines for trustworthy AI, 2019, p. 22. At: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-aiAccessed 15 May 2020 ↑
3AEPD, Adecuación al RGPD de tratamientos que incorporan Inteligencia Artificial. Una introducción, 2020, p.26. At: https://www.aepd.es/sites/default/files/2020-02/adecuacion-rgpd-ia.pdf ↑
4High-Level Expert Group on AI, Ethics guidelines for trustworthy AI, 2019, p. 22. At: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-aiAccessed 15 May 2020 ↑
5Ibidem, p.15 ↑