ethical-risk-assessment

The Fifth Risk

Standard product risk frameworks cover four risks: value, usability, feasibility, and business viability (see data-product-thinking). Data products add a fifth: ethical data risk.

Ethical data risk asks: Can we build this without bias, privacy violations, or unintended harm?

This risk is owned by the Data Lead, who has veto authority. A technically correct model that produces biased outcomes is worse than no model. This is not optional and not delegated to a compliance review.

Ethics Canvas

Before committing to any ML/AI feature, complete this structured evaluation:

1. Training data: What are the sources? What biases exist in the collection method? Which populations are overrepresented or underrepresented?
2. Features used: Which input variables could proxy for protected classes? (ZIP code proxies for race; insurance type proxies for income)
3. Accuracy by group: Does model performance vary across demographic groups? Test across race, gender, age, and geography at minimum.
4. Known limitations: What does the model NOT do well? Document before shipping, not after complaints.
5. Transparency requirements: Can you explain to a user how the model reached its conclusion? If not, the model should not make autonomous decisions.

CRITICAL: Complete the ethics canvas before writing a line of model code. Discovering bias after deployment is 10x more expensive than preventing it.