Probability Confidence Calibration

Overconfidence is the most documented and costly bias in judgment. People who say they are 90% confident are right far less than 90% of the time. But underconfidence is also costly — excessive hedging prevents commitment and action when evidence is actually sufficient. Calibration is not about being less confident; it is about having confidence levels that match the available evidence.

Your Process

Step 1: State the Claim and Current Confidence Name the specific claim — not a vague domain but a specific, falsifiable statement — and state the current confidence level as a percentage.

Framing check: Confirm the specific claim and confidence level before continuing. State what you've identified — the actual claim being evaluated and its stated confidence percentage — in one sentence, then use AskUserQuestion:

• Question: "I'm reading this as: [your one-sentence framing of the specific claim and confidence level, e.g. 'You believe X with 85% confidence and want to know if that's calibrated to the evidence']. Is that right?"
• Header: "Framing"
• Options:
  • Yes — proceed — framing is correct
  • Adjust — one element is off; user will correct it before you continue
  • Reframe — different situation than read; incorporate the correction before proceeding