Machine learning is a powerful tool for businesses. But it can also lead to unintended and dangerous consequences for users of systems powered by machine learning software. The cause of safety issues is linked to the people and data that train and deploy the machine learning software and systems. Everyone involved in creating machine learning based systems should be aware of possible safety risks come when using machine learning technology.
Safety is a multifaceted area of research, with many sub-questions in areas such as reward learning, robustness, and interpretability.
A machine learning driven system can currently only be as good as the data it is given to work with. However you almost can never traceback to the data that was used to train and develop the system. This makes that the safety aspect should be kept in mind when dealing with security aspects for systems that deal direct or indirect with humans.
To avoid dangerous bias or incorrect actions from systems, you should develop machine learning system in the open and make the everything reproducible from the start.
However safety risks will always be there: It is impossible to cover all perspectives and variables for a machine learning system in development before it is released. And the nature of machine learning systems means that the outcome of machine learning is never perfect. Risks will always be present. So not all use cases possible for machine learning are acceptable from an ethical point of view.
The following activities will reduce safety risks and increase reliability of machine learning systems:
- Systematic evaluation: So evaluate the data and models used to train and operate machine learning based products and services.
- Create processes for solid documenting and auditing operations.
- Involve domain experts. Involvement of domain experts in the design process and operation of machine learning systems. Also involve real people in advance who are in the end targeted by outcomes of ml systems especially when decisions about people are made using machine learning applications.
- Evaluation of when and how a machine learning system should seek human input during critical situations, and how a system controlled by a human in a manner that is meaningful and intelligible.
- A robust feedback mechanism so that users can report issues they experience.