Comparing Rating Systems¶

This page provides a side-by-side comparison of the rating systems implemented in Elote, helping you choose the right system for your specific use case.

For a problem-first decision guide with a runnable comparison harness, see How to choose a rating system. For the three systems people ask about by name, see Elo vs Glicko vs TrueSkill. For a reproducible, measured comparison across all of Elote’s rating systems on one interface and one split, with the exact command to rerun it yourself, see Rating-system comparison benchmark.

Overview Comparison¶

System	Origin	Complexity	Uncertainty Tracking	Order Independent	Typical Use Cases
Elo	Chess (1960s)	Low	No	No	General purpose
Glicko	Chess (1995)	Medium	Yes (RD)	No	Sparse competitions
Glicko-2	Chess (2000s)	Medium-High	Yes (RD + volatility)	No	Online chess, volatile skill
Glicko-Boost	Chess (2011)	High	Yes (RD + boost)	Within a rating period	Monthly rating periods, colour
TrueSkill	Microsoft (2007)	High	Yes (sigma)	No	Teams and multiplayer
OpenSkill	Microsoft/academic (2011)	Medium	Yes (sigma)	No	Multiplayer fields with a tunable likelihood
ECF	England (1950s)	Low	No	No	English chess
DWZ	Germany (1990s)	Medium	Partial	No	Youth development
Colley Matrix	Sports (2002)	Medium	No	Yes	Bias-free sports ranking
Massey	Sports (1997)	Medium	No	Yes	Margin-scale sports ranking
Keener	Sports (1993)	Medium	No	Yes	Score-based eigenvector ranking
Pythagorean	Baseball (1980s)	Low	No	Yes	Points-based win expectation
Bradley-Terry	Statistics (1952)	Medium	No	Yes	Maximum-likelihood ranking
Whole-History Rating	Coulom (2008)	High	No	Yes	Time-varying strength curves
Ensemble	Meta-system	High	Depends on components	Depends on components	Complex domains

Online systems (Elo, Glicko, Glicko-2, TrueSkill, ECF, DWZ, Pythagorean) update ratings incrementally after each result. Glicko-Boost is period-native: it rates every player in a rating period simultaneously through apply_rating_period, so results inside a period do not depend on their order, though periods themselves are still applied in sequence. Global-fit systems (Colley Matrix, Massey, Keener and Bradley-Terry and Whole-History Rating) fit the whole connected group from the full set of results, which makes them order independent. Pythagorean is order independent too, for a different reason: its state is a running sum of points that ignores the opponent graph entirely. Massey, Keener and Pythagorean are the three that read the optional score payload; the rest use only who beat whom.

Mathematical Formulation¶

System	Expected Outcome Formula
Elo	\(E_A = \frac{1}{1 + 10^{(R_B - R_A) / 400}}\)
Glicko	\(E(A, B) = \frac{1}{1 + 10^{-g(RD_B) \times (r_A - r_B) / 400}}\) where \(g(RD) = \frac{1}{\sqrt{1 + 3 \times RD^2 / \pi^2}}\)
Glicko-2	Same logistic form as Glicko, evaluated on an internal transformed scale with an added volatility term
Glicko-Boost	\(E_A = \frac{1}{1 + 10^{-g\left(\sqrt{RD_A^2 + RD_B^2}\right)(r_A + \eta - r_B)/400}}\) where \(\eta\) is the advantage to white
TrueSkill	\(E_A = \Phi\!\left(\frac{\mu_A - \mu_B}{\sqrt{2\beta^2 + \sigma_A^2 + \sigma_B^2}}\right)\) where \(\Phi\) is the normal CDF
OpenSkill	Same Gaussian expected-score form as TrueSkill; bout updates follow Weng–Lin Algorithms 1–4, with the likelihood selected by `model` (Plackett-Luce, Bradley-Terry full/partial, Thurstone-Mosteller)
ECF	\(E_A = 0.5 + \frac{R_A - R_B}{F}\) where F is typically 120
DWZ	\(W_e = \frac{1}{1 + 10^{-(R_A - R_B) / 400}}\)
Colley Matrix	Ratings solve \(C r = b\); \(E_A = \frac{1}{1 + e^{-4 (r_A - r_B)}}\)
Massey	Ratings solve \(M r = p\) with \(M = D - A\); \(E_A = \frac{1}{1 + e^{-2 (r_A - r_B) / \sigma}}\), where \(\sigma\) is the fitted rating spread
Keener	Ratings are the dominant eigenvector of \(H_{ij} = h(a_{ij}) / g_i + \varepsilon\) with \(a_{ij} = \frac{S_{ij} + 1}{S_{ij} + S_{ji} + 2}\) and \(h(x) = \frac{1}{2} + \frac{1}{2}\mathrm{sgn}(x - \frac{1}{2})\sqrt{\|2x - 1\|}\); \(E_A = \frac{r_A}{r_A + r_B}\)
Pythagorean	\(w = \frac{PF^{k}}{PF^{k} + PA^{k}}\) on prior-adjusted point totals; \(E_A = \frac{w_A - w_A w_B}{w_A + w_B - 2 w_A w_B}\) (log5)
Bradley-Terry	\(P(A \text{ beats } B) = \frac{p_A}{p_A + p_B} = \frac{1}{1 + e^{-(\beta_A - \beta_B)}}\)
Whole-History Rating	Bradley-Terry likelihood per game plus \(r_{t+1}-r_t \sim N(0, w^2\Delta t)\)
Ensemble	\(E_{ensemble} = \sum_{i=1}^{n} w_i \times E_i\) where \(w_i\) are weights

Key Parameters¶

System	Key Parameters
Elo	K-factor (determines rating change magnitude)
Glicko	Initial rating, Initial RD, Volatility, Tau
Glicko-2	Initial rating, Initial RD, Initial volatility, Tau
Glicko-Boost	Initial rating, Initial RD, White advantage (eta), Boost factors (B1, B2, k), RD-increase coefficients
TrueSkill	Initial mu, Initial sigma, Beta, Tau, Draw probability
OpenSkill	Initial mu, Initial sigma, Model variant (`plackett_luce`, `bradley_terry_full`, `bradley_terry_partial`, `thurstone`), Beta, Tau, Kappa
ECF	Delta (max rating difference), Number of periods
DWZ	Initial rating, Development coefficient (age-dependent)
Colley Matrix	Initial rating (default 0.5)
Massey	Initial rating (default 0.0), Expected-score scale
Keener	Initial rating (default 1.0), Perturbation, Expected-score scale
Pythagorean	Exponent (default 2.37), Prior points (default 1.0); no initial rating
Bradley-Terry	Initial rating, Scale, Regularization, Max iterations
Whole-History Rating	Per-day variance (w2), Initial rating, Max iterations, Precision
Ensemble	Component systems, Weights

Strengths and Weaknesses¶

Elo¶

Strengths: - Simple to understand and implement - Widely recognized and used - Works well with sufficient data - Zero-sum in two-player games

Weaknesses: - No uncertainty measurement - Requires many matches for accuracy - Fixed K-factor can be problematic - Doesn’t handle inactivity well

Glicko¶

Strengths: - Tracks rating reliability - Handles inactivity appropriately - More accurate for sparse competitions - Better for matchmaking

Weaknesses: - More complex to implement - Higher computational requirements - More parameters to tune - Less intuitive interpretation

Glicko-2¶

Strengths: - Adds volatility to capture how consistent a competitor is - Retains Glicko’s uncertainty and inactivity handling - Fast, stable convergence for both erratic and steady players - Used by major online chess platforms

Weaknesses: - The most involved of the Glicko family to implement - Sensitive to the tau system constant - Iterative volatility update adds computational cost - Designed around rating periods rather than single games

Glicko-Boost¶

Strengths: - Rates a whole period the way the published system does, with no within-period ordering effect - Second pass judges a player against how their opponents actually performed - RD boost lets a fast-improving player’s rating move further than Glicko allows - Models the advantage of moving first, taken from the order of each result row

Weaknesses: - Only pays off when results are grouped into rating periods - Twelve system constants, fitted on FIDE data - No persistent volatility between periods, unlike Glicko-2 - Four Glicko passes over the population per period

TrueSkill¶

Strengths: - Models both skill and uncertainty as a Gaussian - Naturally extends to teams and multiplayer games - Fast convergence as uncertainty shrinks - Principled Bayesian foundation

Weaknesses: - Most complex inference of the individual systems - Rating is derived from mu and sigma (no direct setter) - Works on a mu/sigma scale rather than the chess scale - Sensitive to beta, tau, and draw-probability settings

OpenSkill¶

Strengths: - Gaussian skill and uncertainty tracking at TrueSkill’s core quality - Approximate Bayesian updates are far cheaper than factor-graph inference - Multi-competitor bouts and ties without extra machinery - Four interchangeable likelihoods behind one model selector

Weaknesses: - Single-member bouts only (no member-level team updates) - Works on a mu/sigma scale rather than the chess scale - Approximation quality depends on the pairing model chosen - Fewer tuning knobs than TrueSkill (no draw probability)

ECF¶

Strengths: - Linear relationship is easy to calculate - Designed for English chess ecosystem - Simple to understand - Long history of use

Weaknesses: - Limited range of effectiveness - Regional focus - Less theoretical justification - No uncertainty tracking

DWZ¶

Strengths: - Handles youth development well - Age and experience factors - Good for tournament play - National standardization

Weaknesses: - Complex calculation - Regional focus - Parameter sensitivity - Less international recognition

Colley Matrix¶

Strengths: - Order independent: depends only on the set of results - Bias free: no margin-of-victory influence - Clean least-squares interpretation with a unique solution - Self-normalizing ratings in [0, 1] that sum to n/2

Weaknesses: - Re-solves the whole connected group after each result - Uses only win/loss/tie outcomes - Ratings are on a [0, 1] scale, not the chess scale - Requires a connected schedule to compare competitors

Massey¶

Strengths: - Order independent: depends only on the set of results - Consumes the optional scores payload, so real margins of victory are used when available - Rating differences are directly interpretable as predicted margins - Clean least-squares interpretation with a unique zero-mean solution - Self-normalizing: the ratings of a connected group sum to zero

Weaknesses: - Re-solves the whole connected group after each result - Needs real scores to be at its best; falls back to unit margins when they are omitted - Ratings are zero mean, so about half of them are negative - Requires a connected schedule to compare competitors

Keener¶

Strengths: - Reads real scores through a bounded, concave transform, so blowouts inform without dominating - Order independent: depends only on the set of results - The eigenvector recursion values a win in proportion to the opponent’s own rating - Perron-Frobenius guarantees a unique positive solution on any connected schedule - Ratings are strictly positive and average exactly 1.0

Weaknesses: - Re-solves the whole connected group after each result - Much weaker on outcome-only data, where the unit-score fallback discards its main input - The stabilizing perturbation is an extra knob that quietly couples competitors who never met - Requires a connected schedule to compare competitors

Pythagorean¶

Strengths: - The rating is already a win expectation in [0, 1], with no scale to translate - The cheapest system here: constant-time updates, no population fit - Order independent, since the state is a running sum of points - No opponent graph to lose, so serialized state restores exactly, continued play included - A strong points-only baseline that decades of sports analytics have found hard to beat

Weaknesses: - No strength-of-schedule adjustment at all: who supplied the points is ignored - Reduces to a smoothed win percentage on outcome-only data - The exponent is sport-specific; the default is an American-football fit - No uncertainty measure attached to the rating

Bradley-Terry¶

Strengths: - Order independent maximum-likelihood estimate - Statistically principled paired-comparison model - Reported on an Elo-compatible scale for easy interpretation - Regularized so undefeated or winless competitors stay finite

Weaknesses: - Re-fits the whole connected group after each result - Uses only win/loss/tie outcomes - The match graph cannot be serialized (only aggregate counts) - More expensive than Elo for very large populations

Whole-History Rating¶

Strengths: - Estimates a queryable rating curve rather than one lifetime strength - Later evidence revises earlier ratings - Time gaps explicitly control how freely strength can move - Lazy, bounded fitting avoids optimizing after every result

Weaknesses: - More computationally expensive than a single-rating model - Requires meaningful timestamps for the time curve - The opponent graph cannot be serialized; continued play starts a fresh history - Provides no separate uncertainty estimate

Ensemble¶

Strengths: - Combines strengths of multiple systems - More robust predictions - Adaptable to different domains - Graceful degradation

Weaknesses: - Most complex to implement - Highest computational cost - Requires weight tuning - Less interpretable

Choosing the Right System¶

Consider the following factors when choosing a rating system:

Data Density: How frequently do competitors face each other? - Sparse data: Consider Glicko or Glicko-2 - Dense data: Elo may be sufficient
Domain Specifics: - Chess in England: ECF - Chess in Germany: DWZ - Youth development: DWZ - Team and multiplayer games: TrueSkill - General purpose: Elo or Glicko
Order Independence: Do you need a ranking that ignores schedule order? - Yes: Colley Matrix, Massey, Keener, Bradley-Terry, Whole-History Rating, or Pythagorean - Yes, and historical strength matters: Whole-History Rating - Yes, and you have real scores: Massey (linear margin), Keener (bounded score share), or

Pythagorean (points ratio, no strength of schedule)
- No: any other online system (Elo, Glicko, etc.)
Computational Resources: - Limited resources: Elo or ECF - Sufficient resources: Glicko, Glicko-2, TrueSkill, or Ensemble
Uncertainty Importance: - Critical to track uncertainty: Glicko, Glicko-2, or TrueSkill - Uncertainty less important: Elo or ECF
Prediction Accuracy: - Highest accuracy needed: Consider Ensemble - Reasonable accuracy sufficient: Any individual system

Code Comparison¶

Here’s a quick comparison of how to use each system in Elote:

from elote import (
    EloCompetitor,
    GlickoCompetitor,
    Glicko2Competitor,
    GlickoBoostCompetitor,
    TrueSkillCompetitor,
    OpenSkillCompetitor,
    ECFCompetitor,
    DWZCompetitor,
    ColleyMatrixCompetitor,
    MasseyCompetitor,
    KeenerCompetitor,
    PythagoreanCompetitor,
    BradleyTerryCompetitor,
    WholeHistoryRatingCompetitor,
    BlendedCompetitor,
)

# Elo
elo_player = EloCompetitor(initial_rating=1500, k_factor=32)

# Glicko
glicko_player = GlickoCompetitor(initial_rating=1500, initial_rd=350)

# Glicko-2
glicko2_player = Glicko2Competitor(initial_rating=1500, initial_rd=350)

# Glicko-Boost (period-native; the first competitor of a row had white)
glicko_boost_player = GlickoBoostCompetitor(initial_rating=1500, initial_rd=250)

# TrueSkill (mu/sigma scale)
trueskill_player = TrueSkillCompetitor(initial_mu=25.0, initial_sigma=8.333)

# OpenSkill (mu/sigma scale; model picks the Weng-Lin likelihood)
openskill_player = OpenSkillCompetitor(initial_mu=25.0, initial_sigma=25.0 / 3, model="plackett_luce")

# ECF
ecf_player = ECFCompetitor(initial_rating=100)

# DWZ
dwz_player = DWZCompetitor(initial_rating=1600)

# Colley Matrix ([0, 1] scale)
colley_player = ColleyMatrixCompetitor()

# Massey (zero-mean margin scale)
massey_player = MasseyCompetitor()

# Keener (strictly positive, mean 1.0; reads real scores)
keener_player = KeenerCompetitor()

# Pythagorean (win expectation in [0, 1]; reads real scores, no initial rating)
pythagorean_player = PythagoreanCompetitor(exponent=2.37)

# Bradley-Terry (Elo-compatible scale)
bt_player = BradleyTerryCompetitor(initial_rating=1500)

# Whole-History Rating (time-aware Elo-compatible curve)
whr_player = WholeHistoryRatingCompetitor(w2=300.0)

# Ensemble (blend of sub-competitors)
ensemble_player = BlendedCompetitor(
    competitors=[
        {"type": "EloCompetitor", "competitor_kwargs": {"initial_rating": 1500}},
        {"type": "GlickoCompetitor", "competitor_kwargs": {"initial_rating": 1500}},
    ]
)

# Usage is the same for all systems: expected_score, beat, tied, lost_to
player = EloCompetitor(initial_rating=1500)
opponent = EloCompetitor(initial_rating=1400)
print(f"Expected score: {player.expected_score(opponent):.2%}")
player.beat(opponent)  # record a win

# Score-based systems additionally accept the optional scores payload
home, away = KeenerCompetitor(), KeenerCompetitor()
home.beat(away, scores=(35, 3))

scorers = PythagoreanCompetitor(), PythagoreanCompetitor()
scorers[0].beat(scorers[1], scores=(28, 14))

Empirical Comparison¶

While theoretical comparisons are useful, the best way to choose a rating system is through empirical testing on your specific domain. Elote makes it easy to experiment with different systems and compare their predictive accuracy.

Here’s a simple approach to compare systems:

Split your historical match data into training and testing sets
Train each rating system on the training data
Evaluate prediction accuracy on the test data
Choose the system with the best performance for your specific use case

Remember that no rating system is universally best - the right choice depends on your specific requirements, data characteristics, and domain constraints.