Skip to content
More Useless

e-value

— data-science

Snippet

e-values vs p-values

The key is to constructing e-values well.

  • A p-variable for testing : a random variable satisfying Realized values = p-values.
  • An e-variable for testing : a non-negative random variable such that Realized values = e-values.

Advantages of E-values

  • Safe under optional stopping (cf. time-to-event data).
  • Naturally suited for sequential testing, online inference, and gradual evidence accumulation.
  • Easily combined across studies or time (e.g., via multiplication).
  • Useful in high-dimensional and composite/irregular models, where computing exact -values is hard.
  • No need for fixed sample size or pre-specified analysis plans.

Weaknesses

  • Generally less powerful than Neyman–Pearson (fixed-) tests.
  • Require more data to reach traditional thresholds like 0.05-equivalent.

Optional Continuation Problem & p-hacking

  • Repeated trials (e.g., groups A, B, C testing a treatment sequentially) usually lead to:
    • Improper reuse or pooling of -values.
    • Inflated Type I error (p-hacking).
  • E-values solve this: test martingales allow continual evidence monitoring without inflating error.

Construction Example

  • Place a prior over the alternative .
  • Define an e-variable via likelihood ratio or Bayes factor-like object (e.g., test supermartingale).
  • Often of the form: where is under the alternative, under the null.

Connections

  • Bayes Factors: a type of e-variable when null and alternative are fully specified.
  • Test Martingales: a sequence where each is an e-variable, and is a non-negative martingale under .
  • FDR Control: e-Benjamini–Hochberg (e-BH) provides valid FDR procedures using e-values.

Notable Contributors

  • Peter Grünwald: "E is the new P" — formalizes safe testing under optional stopping.
  • Aditya Ramdas: key work on sequential testing, e-BH, martingales.
  • Ruodu Wang: connects e-values to game-theoretic probability and robust inference.
  • Brian Scheffer: applied presentations and tutorials.

Applications

  • Sequential A/B testing.
  • Automated hypothesis validation (e.g., POPPER framework).
  • High-dimensional inference (e.g., genomics).
  • Robust meta-analysis.

You can connnect Knockoffs and e-values. You can define e-values based on knockoff statistics. Pull them out of the hat. You can derandomise knockoff procedure by taking the averages.