Intro

The value of performance statistics in tabletop wargames.


Asymmetry

You rarely see a modern tabletop game that isn’t Asymmetric. Unlike chess or backgammon, an asymmetric game allows players to leverage a different set of rules or pieces than their opponent. People love the variety and character that asymmetry brings, and it’s not hard to understand why; asymmetry is popular and worth doing well.

However, there is an obvious downside to such flexibility. Before a single die is rolled or decision made, one player can begin the game with an enormous advantage over the other. Clearly, if I have different rules than you, my rules might be better than yours.

Game balance is the area in game design concerned with this issue. Game balance seeks to reduce the disparity in power between asymmetric components of a game; the goal is to achieve some form of parity, some form of fairness, between these varying rules.

But how can parity be possible? Asymmetric rules are different by design, aren’t we simply comparing apples to oranges?

This is where a statistical perspective becomes useful. By collecting many games and measuring outcomes in the Aggregate, we can infer the effect each asymmetric set has on a typical game’s outcome.

🧐 Aperçu

The goal of statistics is to discover reliable patterns within complex, messy systems.

In Kill Team, our sets of interest are called teams, or more popularly known, Factions. Statistically, we can treat factions as a variable and measure their influence on game outcomes. This relationship, when compared across factions, tells us how balanced they are.

Signal vs. Noise

Kill Team is a dice game, and dice certainly impact who wins and losses.

However, dice are random. Some days they are your best friend; other days they lead your plastic toys to certain death.

Dice are Noise; they are random, unpredictable variation within games. Rolling terribly in a single game says nothing about your skill as a player or the strength of your faction rules. Additionally, there are many other forms of noise. Any random or quasi-random effect like differences in terrain setups or rule mistakes can count as noise.

In a single game, noise can overwhelm the outcome. But as we play game after game, random effects start to cancel themselves out. As the noise cancels, we might notice emerging patterns. These patterns represent persistent effects that exist in all games; patterns that keep pushing in a consistent direction.

We call these patterns Signal. Signal would include things like faction rules or player skill. Unlike dice or random mistakes, their influence persists and does not change dramatically game over game.

🧐 Aperçu

Statistics applies the math of probability theory to quantify uncertainty, enabling inference of underlying patterns hidden within noisy data.

So far, I haven’t mentioned the data being used. But even if we had no data at all, these concepts are incredibly useful for describing what game balance is.

When I say:

Faction X is just as strong as faction Y.

I’m essentially saying:

Ignoring skill and noise, faction X should have the same influence on a typical game's outcome as faction Y.

That is what we mean by Balance.

Balance vs. Fairness

It’s worth mentioning that not all aspects of a game’s fairness has to do with asymmetry. Candy Land is a deeply unfair game, intentionally so. But it is not asymmetrical (beyond one player taking the first turn). In the same way, it’s possible for a game to be balanced but still contain unfair mechanics.

Balance, at least in the sense we defined earlier, doesn’t consider whether a specific rule removes agency, feels bad, or is poorly designed; An Empirically balanced game isn’t guaranteed to be fun.

Nevertheless, imbalance can ruin an otherwise great game.

🧐 Aperçu

In other words, empirical balance is one category of fairness, but not all categories of fairness are empirical balance.

Games are inherently Goal-Oriented. The primary goal is usually to win; although, many players are happy if they achieve something they find interesting or meaningful, even in the face of a loss.

Unfortunately, serious imbalance is where goals go to die. Any goal becomes dramatically more difficult for the player with the disadvantage; meanwhile, goals are robbed of their sense of achievement for the player with the advantage. All around, this leads to a very negative experience.

Concern for fairness is deeply embedded in human nature, it's not just a trait found among "the sweats." Anyone interested in developing fun games for humans should be motivated to stay on human nature’s good side.

So no, balance isn’t the most important thing players and game designers should worry about. But it does matter, quite a bit actually; and statistics are well suited to help.

Kill Team's Advantage

Kill Team does possess an advantage over other, typical wargames when it comes to this topic.

Most other war games are List Building games. In these games, a faction functions as a pool of units and rules that players can draw from to build their own personal squad or army.

Factions of this sort can produce a large variety of lists. Even though lists from the same faction might have some rules in common; ultimately, they’re not the same. Different lists are asymmetric from each other, often significantly so.

Consequently, in a list building game, lists become the ultimate driver of asymmetry. Since lists are the ultimate driver of asymmetry, then lists become the ultimate cause of imbalance. After all, in these games, players do not bring their faction to an event, they bring their list.

Kill Team factions do not function like this. In Kill Team, when you head to the game store, you don’t have to leave any rules at home. You are allowed to bring your whole faction with you; you're free to make choices on a tactical, game-to-game basis. Thus, factions remain the ultimate driver for asymmetry and imbalance (One exception would be the Angels of Death Kill Team, which do "lock in" some of their rules for an entire event).

In Kill Team, factions, as a variable, have a stronger relationship to balance than their list building alternatives. That’s not to say there is no relationship between factions and balance in list building games, there certainly is! They just suffer from an additional Confounder, due to being a step removed from the final cause of asymmetry.

Causation and Data

As one final caveat, I’d like to share a quote from the great Richard McElreath:

Causation is not found in the data.

In other words, discovering a statistical relationship does not mean we know why that relationship exists, or if it even matters!

If I can show a particular faction is statistically unbalanced, that doesn’t mean I know why. At the end of the day, it takes Domain Knowledge to provide reasonable explanations for why one faction might be overperforming and another underperforming; Data doesn’t interpret itself.

Our goal should always be to harmonize domain knowledge with statistical evidence, not pit them against each other in some kind of death match. When expectations and data contradict, it’s on us to critically think through why.

Up next, we'll dive into the primary metric used in faction balance (not just here, but virtually everywhere): Win Rate.