Arenas

Arenas are objects that manage populations of competitors and their matchups. Currently there is only one type of arena implemented, LambdaArenas

Lambda Arena

Rating periods

A rating period is a batch of results that share one pre-period rating state. Use rating_period() when every prediction in the batch should be made before any result in that batch changes a rating. This differs from calling matchup() repeatedly, where each result immediately informs the next prediction.

Each row is (competitor_a, competitor_b, outcome, scores). The outcome is 1.0, 0.0, or 0.5 from the first competitor’s perspective, and scores may be None. All currently shipped rating systems use apply_rating_period()’s sequential default, so their final ratings match a stream of the same pairwise updates. A period-native system can override that operation to update its population simultaneously while keeping the pairwise API intact.

class elote.arenas.lambda_arena.LambdaArena(func: ~typing.Callable[[...], bool | None], base_competitor: ~typing.Type[~elote.competitors.base.BaseCompetitor] = <class 'elote.competitors.elo.EloCompetitor'>, base_competitor_kwargs: ~typing.Dict[str, ~typing.Any] | None = None, initial_state: ~typing.Dict[~typing.Any, ~typing.Dict[str, ~typing.Any]] | None = None)

Initialize a LambdaArena with a comparison function.

The LambdaArena uses a provided function to determine the outcome of matchups between competitors. This is particularly useful for comparing objects that aren’t competitors themselves.

Args:

func (callable): A function that takes two arguments (a, b) and returns

True if a beats b, False if b beats a, and None for a draw.

base_competitor (class): The competitor class to use for ratings.

Defaults to EloCompetitor.

base_competitor_kwargs (dict, optional): Keyword arguments to pass to

the base_competitor constructor.

initial_state (dict, optional): Initial state for competitors, mapping

competitor IDs either to exported competitor state documents (as produced by export_state()) or to plain keyword arguments for the base_competitor constructor, such as {"initial_rating": 1200}.

clear_history() → None

Clear the history of bouts in this arena.

evaluate_performance(eval_bouts: List[Tuple[str, str, float | None]], progress_bar: bool = True) → None

Evaluate the performance of the competitors based on a list of evaluation bouts.

Bouts naming a competitor the arena has not seen are skipped: evaluation never adds to the population, so unknown identifiers contribute no prediction.

Args:

eval_bouts (list): A list of (competitor_a, competitor_b, outcome) tuples. progress_bar (bool, optional): Whether to display a progress bar.

expected_score(a: Any, b: Any) → float

Calculate the expected score for a matchup between two competitors.

This method returns the probability that competitor a will beat competitor b. It is a read: an identifier the arena has not seen is scored as an unrated competitor without being added to the population.

Args:

a: The first competitor or competitor identifier. b: The second competitor or competitor identifier.

Returns:

float: The probability that a will beat b (between 0 and 1).

export_state() → Dict[Any, Dict[str, Any]]

Export the current state of this arena for serialization.

Returns:

dict: A dictionary containing the state of all competitors in this arena.

get_all_competitors() → List[BaseCompetitor]

Retrieve a list of all competitors in the arena.

Returns:

list: A list of all competitors.

get_competitor_by_id(id_val: str) → BaseCompetitor | None

Retrieve a competitor by their ID.

Args:

id_val (str): The ID of the competitor to retrieve.

Returns:

Optional[BaseCompetitor]: The retrieved competitor, or None if not found.

leaderboard() → List[Dict[str, Any]]

Generate a leaderboard of all competitors.

Returns:

list: A list of dictionaries containing competitor IDs and their ratings,

sorted by rating in descending order.

match_group(participants: Sequence[Any], ranks: Sequence[float] | None = None, scores: Sequence[float] | None = None, attributes: Dict[str, Any] | None = None, match_time: datetime | None = None) → None

Process a single bout between three or more sides (or two sides with rosters).

This is the N-way analogue of matchup(): it creates missing competitors, captures every pre-update prediction, runs one bout-level update over the whole participant set, and records the result as a MultiBout on the same history the two-player entry points append to.

The bout is updated natively – one closed-form pass over the whole participant set – not as a fan-out of pairwise results, which the Weng-Lin family (and any other bout-level model) treats differently.

Args:

participants (sequence): The sides, ordered by finish. Each entry is

either a competitor id, or an (id, roster) pair where roster is a list or tuple of member ids for a team side. A side is a roster side exactly when its entry is a list/tuple pair whose second item is a list or tuple. When ranks is None the given order is taken as the finish order.

ranks (sequence of float, optional): Finishing ranks per participant,

lower is better, equal ranks are ties. Whole numbers are required. Defaults to 0..n-1 (the given order). When both ranks and scores are given, ranks defines the result and scores are recorded only.

scores (sequence of float, optional): One score per participant.

Validated and recorded with the bout; when ranks is omitted the scores define the finishing ranks (descending, ties share a rank). Not otherwise consumed by rank-based models.

attributes (dict, optional): Additional attributes to record with this bout. match_time (datetime, optional): The time when the bout occurred.

Raises:

NotImplementedError: If the arena’s rating system does not implement

a bout-level update (apply_bout). The first shipped consumer is OpenSkillCompetitor.

ValueError: If the participant list is malformed (fewer than two

sides, an empty roster, the same competitor on two sides), if ranks is not one whole number per participant, or if scores is not one finite, non-negative number per participant. Validation happens before any competitor is created or any history is recorded, so a bad bout leaves the arena unchanged.

matchup(a: Any, b: Any, attributes: Dict[str, Any] | None = None, match_time: datetime | None = None, outcome: float | None = None, scores: Sequence[float] | None = None) → None

Process a single matchup between two competitors.

This method handles a matchup between two competitors, creating them if they don’t already exist in the arena. It uses the comparison function to determine the outcome and updates the ratings accordingly.

Args:

a: The first competitor or competitor identifier. b: The second competitor or competitor identifier. attributes (dict, optional): Additional attributes to record with this bout. match_time (datetime, optional): The time when the match occurred. outcome (float, optional): A known result for this matchup, expressed from

a’s perspective as 1.0 (a wins), 0.0 (b wins) or 0.5 (draw). When supplied, the comparison function is not called.

scores (sequence of float, optional): The two scores as (a_score, b_score) –

always in the argument order of this call, regardless of who won. They are forwarded to the competitors’ result methods in whatever order those methods require. Requires outcome to be supplied, since the scores must be checked against a known result.

Raises:

ValueError: If outcome is supplied and is not one of 1.0, 0.0 or 0.5, if

scores is supplied without outcome, or if scores is malformed, negative, non-finite, or disagrees with outcome.

rating_period(matchups: Sequence[Tuple[Any, Any, float, Sequence[float] | None]], *, period_end: datetime | None = None) → None

Process a batch of results against one shared pre-period state.

Every row is validated before the arena is changed. Predictions for all rows are then captured before the batch is applied, so an earlier result in the period cannot inform a later prediction from the same period.

Args:

matchups: (competitor_a, competitor_b, outcome, scores) tuples. Outcomes

are 1.0, 0.0, or 0.5 from A’s perspective. Scores are optional and use (a_score, b_score) caller order.

period_end: The shared activity time for time-aware competitors.

Raises:

ValueError: If any outcome or score payload is invalid. Validation happens

before competitors or bouts are added.

set_competitor_class_var(name: str, value: Any) → None

Set a class variable on the base competitor class.

This method allows for global configuration of all competitors managed by this arena.

Args:

name (str): The name of the class variable to set. value: The value to set for the class variable.

tournament(matchups: List[Tuple[Any, ...]]) → None

Run a tournament with the given matchups.

Process multiple matchups between competitors, updating ratings after each matchup.

Args:

matchups (list): A list of tuples, each unpacked into matchup(). The

first two entries are the competitors; further entries follow that method’s signature, so a tuple may carry (a, b, attributes, match_time, outcome, scores).

validate(validation_bouts: List[Tuple[str, str, float | None]], progress_bar: bool = True) → None

Run a validation set through the arena without updating ratings, only recording predictions.

Bouts naming a competitor the arena has not seen are skipped: validation never adds to the population, so unknown identifiers contribute no prediction.

Args:

validation_bouts (list): A list of (competitor_a, competitor_b, outcome) tuples. progress_bar (bool, optional): Whether to display a progress bar.

N-player bouts

match_group() runs a single bout between three or more sides – or between two sides with rosters, i.e. a team bout. The participants are given in finishing order (or with explicit ranks), and the whole bout is updated natively with one bout-level pass rather than as a fan-out of pairwise results, which is not mathematically equivalent for bout-level models such as OpenSkill.

from elote import LambdaArena, OpenSkillCompetitor

arena = LambdaArena(lambda *args: None, base_competitor=OpenSkillCompetitor)
# Free-for-all: Ada finishes first, Linus last.
arena.match_group(["Ada", "Grace", "Linus"])
# Team bout: the (ada, grace) roster ties the (linus, kurt) roster.
arena.match_group([
    ("red", ["ada", "grace"]),
    ("blue", ["linus", "kurt"]),
], ranks=[0, 0])

ranks gives one finishing rank per participant (lower is better, equal ranks are ties); scores is one score per participant and defines the ranks when ranks is omitted. Every pre-update prediction is captured before any participant is updated, and the bout is recorded as a MultiBout entry.

Only rating systems implementing a bout-level update (apply_bout) support N-way bouts; the first shipped consumer is OpenSkillCompetitor, including per-member team updates whose team strength aggregates the members’ beliefs.

Note

N-way bouts are excluded from the two-sided analytics – History.report_results, confusion_matrix, calculate_metrics, calculate_metrics_with_draws, optimize_thresholds, random_search, accuracy_by_prior_bouts, get_calibration_data and the calibration plot – because those are defined over a winner/drawer/loser comparison of exactly two competitors. They remain recorded in History.bouts for inspection.

Helpers

class elote.arenas.base.History

Tracks the history of bouts (matchups) and provides analysis methods.

This class stores the results of matchups and provides methods to analyze the performance of the rating system.

Two bout types share the bouts list:

• Bout – a two-sided bout between single competitors, recorded by every two-player arena entry point.

• MultiBout – a bout between more than two sides (or between sides with rosters), recorded by LambdaArena.match_group.

N-way analytics note. Every two-sided analysis in this class – report_results(), confusion_matrix(), calculate_metrics(), calculate_metrics_with_draws(), optimize_thresholds(), random_search(), accuracy_by_prior_bouts() and get_calibration_data() – is defined over a winner/drawer/loser comparison of exactly two competitors. N-way bouts have no a/b winner semantics, so they are deliberately excluded from all of them (they are still recorded and remain available for inspection). Generalizing the confusion-matrix machinery to ranked outcomes is a separate, follow-up feature, not something these methods approximate today.

Initialize an empty history of bouts.

accuracy_by_prior_bouts(arena: BaseArena, thresholds: Tuple[float, float] | None = None, bin_size: int = 5) → Dict[int, Dict[str, Any]]

Calculate accuracy based on the number of prior bouts for each competitor.

This method analyzes how accuracy changes as competitors participate in more bouts, properly accounting for draws as a third outcome category.

Args:

arena (BaseArena): The arena containing the competitors and their history thresholds (tuple, optional): Tuple of (lower_threshold, upper_threshold) for predictions bin_size (int): Size of bins for grouping bout counts. Must be a positive integer.

Returns:

dict: A dictionary with ‘binned’ key containing binned accuracy data. Every non-empty bin reports its observed correct / total accuracy, however few bouts it holds.

Raises:

ValueError: If bin_size is not a positive integer.

add_bout(bout: Bout | MultiBout) → None

Add a bout to the history.

Args:

bout (Bout | MultiBout): The bout object to add to the history.

calculate_metrics(lower_threshold: float = 0.5, upper_threshold: float = 0.5) → Dict[str, float]

Calculate performance metrics based on the confusion matrix.

Args:

lower_threshold: The lower threshold for prediction (below this is a prediction for the second competitor) upper_threshold: The upper threshold for prediction (above this is a prediction for the first competitor)

Returns:

A dictionary with metrics including accuracy, precision, recall, F1 score, and the confusion matrix

calculate_metrics_with_draws(lower_threshold: float = 0.33, upper_threshold: float = 0.66) → Dict[str, Any]

Calculate evaluation metrics for the bout history, treating predictions between thresholds as explicit draw predictions.

Args:

lower_threshold (float): The lower probability threshold for predictions. upper_threshold (float): The upper probability threshold for predictions.

Returns:

dict: A dictionary containing accuracy, precision, recall, F1 score, and draw metrics.

confusion_matrix(lower_threshold: float = 0.45, upper_threshold: float = 0.55) → Dict[str, int]

Calculate the confusion matrix for the history of bouts.

Args:

lower_threshold: The lower threshold for prediction (below this is a prediction for the second competitor) upper_threshold: The upper threshold for prediction (above this is a prediction for the first competitor)

Returns:

A dictionary with confusion matrix metrics: {‘tp’: int, ‘fp’: int, ‘tn’: int, ‘fn’: int}

get_calibration_data(n_bins: int = 10) → Tuple[List[float], List[float]]

Compute calibration data from the bout history.

This method extracts predicted probabilities and actual outcomes from the bout history and prepares them for calibration curve plotting.

Args:

n_bins (int): Retained for backward compatibility but unused. Binning is

performed by visualization.compute_calibration_data.

Returns:

tuple: (y_true, y_prob) where:

• y_true: List of actual scores (1.0 for wins, 0.5 for draws, 0.0 for losses)

• y_prob: List of predicted probabilities

optimize_thresholds(method: str = 'L-BFGS-B', initial_thresholds: Tuple[float, float] = (0.5, 0.5)) → Tuple[float, List[float]]

Find the prediction thresholds that maximize accuracy.

Accuracy is a step function of the thresholds: it only changes at the distinct values of Bout.predicted_outcome. This method therefore evaluates every distinct threshold pair exactly, in O(n log n), rather than sampling. The result is deterministic — repeated calls on an unchanged history return identical values.

Args:

method (str): Deprecated and ignored. Retained for backward compatibility;

the search is exact, so no optimizer is selected.

initial_thresholds (tuple): Thresholds to fall back to. They are returned

unchanged whenever they already achieve the optimal accuracy.

Returns:

tuple: (best_accuracy, best_thresholds) where:

• best_accuracy: The accuracy achieved with the optimized thresholds

• best_thresholds: List of [lower_threshold, upper_threshold]

random_search(trials: int = 1000, seed: int | None = None) → Tuple[float, List[float]]

Search for optimal prediction thresholds using random sampling.

This method performs a random search to find the best lower and upper thresholds that maximize the overall accuracy, including draws.

Note:

optimize_thresholds() computes the exact optimum deterministically and should be preferred; this method is retained for backward compatibility.

Args:

trials (int): The number of random threshold pairs to try. seed (int, optional): Seed for the sampling. When omitted the unseeded

global random module is used, so results vary between calls.

Returns:

tuple: A tuple containing (best_accuracy, best_thresholds).

report_results(lower_threshold: float = 0.5, upper_threshold: float = 0.5) → List[Dict[str, Any]]

Generate a report of the results in this history.

Args:

lower_threshold (float): The lower probability threshold for predictions. upper_threshold (float): The upper probability threshold for predictions.

Returns:

list: A list of dictionaries containing the results of each bout.

class elote.arenas.base.Bout(a: Any, b: Any, predicted_outcome: float | None, outcome: Any, attributes: Dict[str, Any] | None = None)

A single bout between two competitors.

Initialize a bout.

Args:

a: The first competitor b: The second competitor predicted_outcome: The predicted probability of a winning outcome: The actual outcome of the bout attributes: Optional dictionary of additional attributes

actual_winner() → str | None

Return the actual winner of the bout based on the outcome.

Returns:

str or None: ‘a’ if a won, ‘b’ if b won, None if it was a draw or unclear

false_negative(threshold: float = 0.5) → bool

Check if this bout is a false negative prediction.

A false negative occurs when the model incorrectly predicts a non-win.

Args:

threshold (float): The probability threshold for a negative prediction.

Returns:

bool: True if this bout is a false negative, False otherwise.

false_positive(threshold: float = 0.5) → bool

Check if this bout is a false positive prediction.

A false positive occurs when the model incorrectly predicts a win.

Args:

threshold (float): The probability threshold for a positive prediction.

Returns:

bool: True if this bout is a false positive, False otherwise.

predicted_loser(lower_threshold: float = 0.5, upper_threshold: float = 0.5) → str | None

Determine the predicted loser of this bout.

Args:

lower_threshold (float): The lower probability threshold for predictions. upper_threshold (float): The upper probability threshold for predictions.

Returns:

str: The identifier of the predicted loser, or None if no loser is predicted.

predicted_winner(lower_threshold: float = 0.5, upper_threshold: float = 0.5) → str | None

Determine the predicted winner of this bout.

Args:

lower_threshold (float): The lower probability threshold for predictions. upper_threshold (float): The upper probability threshold for predictions.

Returns:

str: The identifier of the predicted winner, or None if no winner is predicted.

true_negative(threshold: float = 0.5) → bool

Check if this bout is a true negative prediction.

A true negative occurs when the model correctly predicts a non-win.

Args:

threshold (float): The probability threshold for a negative prediction.

Returns:

bool: True if this bout is a true negative, False otherwise.

true_positive(threshold: float = 0.5) → bool

Check if this bout is a true positive prediction.

A true positive occurs when the model correctly predicts a win.

Args:

threshold (float): The probability threshold for a positive prediction.

Returns:

bool: True if this bout is a true positive, False otherwise.

class elote.arenas.base.MultiBout(participants: List[Any], ranks: List[float] | None, predicted_ranks: List[Any], scores: List[float] | None = None, attributes: Dict[str, Any] | None = None, match_time: Any | None = None)

A single bout between more than two sides, or between sides with rosters.

Recorded by LambdaArena.match_group and appended to the same History.bouts list as two-sided Bout entries.

Unlike a Bout, an N-way bout has no a/b winner semantics: the outcome is a ranking of the participants (with possible ties), not a winner/loser/draw triple. For that reason MultiBout entries are excluded from every two-sided analysis in History (see the note on that class) and carry their own prediction record:

• participants – the side identifiers, in the finishing order the caller supplied (which is only the true finish order when ranks is None).

• predicted_ranks – the pre-update prediction: the side identifiers ordered by pre-bout strength (descending, ties keep input order). This is the N-way analogue of Bout.predicted_outcome; every strength read happened before any participant was updated.

Attributes:

participants (list): The side identifiers in caller (finishing) order. ranks (list | None): The finishing ranks per participant, lower is

better, equal ranks are ties. None when the caller supplied neither ranks nor scores, in which case the participant order was taken as the finish order.

predicted_ranks (list): The side identifiers ordered by pre-bout

strength, strongest first.

scores (list | None): The per-side scores the caller supplied, if any. attributes (dict): Optional additional attributes recorded with the bout. match_time (datetime | None): The time the bout occurred, if supplied.

Initialize a multi-way bout record.

Args:

participants: The side identifiers in caller (finishing) order. ranks: The finishing ranks per participant, lower is better, equal

ranks are ties. None when no ranking was given.

predicted_ranks: The side identifiers ordered by pre-bout strength. scores: The per-side scores, if the caller supplied them. attributes: Optional additional attributes recorded with the bout. match_time: The time the bout occurred, if supplied.