Hazard: Difficult To Understand#

A red diamond shaped outline (like a warning sign) with an image of a closed box and a question mark next to it.

Description#

There is a danger that the technology is difficult to understand. This could be because of the technology itself is hard to interpret (e.g. neural nets), or problems with it’s implementation (i.e. code is not provided, or not documented).

Depending on the circumstances of its use, this could mean that incorrect results are hard to identify, or that the technology is inaccessible to people (difficult to implement or use).

Examples#

Example 1: Deep learning is used to perform credit-scoring (i.e. could deny people credit), but it is difficult to understand (and therefore check) what these decisions are based on.

Example 2: Even when journals have a policy of having code and data availability, published researchers can be unaware of what they agreed to and resist sharing it, as this paper surveying Science publications shows.

Safety Precautions#