Write the resolution rule before the forecast

A dated prediction is only useful if two people can agree later on whether it happened.

PriceWire Research · 2026-09-22 · Prepared with AI assistance; calculations checked programmatically.

Consider the sentence: “Business conditions will improve soon.” It gives the writer too much freedom. What counts as improvement? How soon is soon? Which numbers were available when the claim was made?

Make the question resolvable

For a synthetic example, write: “Will the fictional Activity Index exceed 105 in the first published December reading?” Then record the publication you will use, the expected release date, how missing data will be handled, and whether revisions count. The question is narrower, and its result can be checked.

Record a probability, the time of the forecast and the resolution rule together. Preserve the original. If your view changes, add a dated update and explain why.

Keep unresolved outcomes visible

A question without an observed answer is pending. Counting it as a loss is wrong; quietly excluding it from the list of forecasts can also be misleading. Report registered, resolved and unresolved counts separately.

Use a score, then inspect its limits

For binary outcomes, the Brier score averages the square of the forecast probability minus the outcome, where the outcome is either zero or one. Lower is better on the same set of questions.

Three synthetic forecasts of 80%, 30% and 60%, resolved as yes, no and no, score (0.04 + 0.09 + 0.36) / 3 = 0.1633. That is below the 0.25 score of always saying 50%. Three selected examples establish no forecasting skill.

A meaningful comparison uses the same questions, the same information timing and a benchmark chosen in advance. An informed base-rate forecast can be a stronger competitor than 50%.

Keep the record reproducible

The Research Kit includes an empty CSV journal, synthetic examples and a scoring script. It keeps unresolved outcomes separate and rejects malformed probabilities. It cannot prove that a local file was never edited or determine whether two questions are independent. Those limits stay with the results.