PriceWireThe benchmark is part of the experiment
Before celebrating a result, check whether a simpler alternative faced the same dates, universe and costs.
PriceWire Research · 2026-09-22 · Prepared with AI assistance; calculations checked programmatically.
An experiment can answer the wrong question precisely. Comparing a model's return with a convenient index is one way to do it: the two may hold different securities, take different risks and pay different costs.
Start with the decision
If the question is whether ranking adds value within a defined universe, an equal-weight portfolio from that same universe may be informative. If the question is whether active investing was worthwhile, an investable broad-market alternative may also matter. Name the question before choosing its comparison.
Write the shared conditions
- The same decision dates and information cutoff.
- A clearly stated universe, including how exclusions are treated.
- Consistent holding periods and treatment of missing prices.
- Explicit turnover, spread, fees and slippage assumptions.
- The same treatment of distributions and corporate actions.
Fair comparison does not mean assigning identical cost totals to different strategies. It means using a consistent model that reflects the trades each would make.
Keep the simple alternatives
A complicated model should earn its complexity. Compare it with a deliberately simple rule, and disclose how many alternatives you tried. Selecting a winner from twenty experiments is different from evaluating one pre-registered idea.
A useful negative result
If an apparent advantage disappears after consistent costs, report that result. It can retire a weak hypothesis and prevent unnecessary work. It does not establish that every related strategy fails, nor that reversing the model will work.
Our publication follows that standard: state the question, retain the calculation and describe what the evidence cannot establish. The Research Kit's experiment template makes those choices explicit before results arrive.