This section covered a few of the issues frequently encountered when designing a rating system. The most prominent of these issues is that somewhat depressingly, there will always be players that seek to exploit the system and gain an unfair or unintended advantage over their opponents. Briefly, the issues covered included:

1. Encouraging Activity – Across multiple rating systems, highly rated players may not want to play against lower rated players since the risk of lost rating is greater for the higher-rated player. Common ways to circumvent this is to directly implement time-based rating decay or to employ external incentives like in-game currency or daily quests to drive participation.
2. Luck vs. Skill – A lot of rating systems are built on the assumption that the better player always wins, which is not a valid assumption for most games. If the system doesn’t compensate for it, then higher rated players are actually penalized as they will lose slightly more often than expected and their average opponent is going to be worse than them, resulting in a larger overall penalty to their rating. To compensate, can mathematically approximate the degree to which luck affects the outcome and adjust the awarded/lost amount of rating accordingly.
3. Matchmaking – Rating systems become especially susceptible to exploitation and undesirable trends if players are able to earn rating based on their own matchmaking. The solution in most cases is to implement automated matchmaking. These solutions often have to balance finding the ideal match against the length of the player’s queue time.
4. Rating Inflation/Deflation – Over time, it is possible for the average rating of players to increase or decrease within the system. This inflation/deflation of ratings can produce some undesirable trends over time since ratings become meaningful when contextualized against other players. Rating inflation generally hurts newer players as they need to gain greater amounts of rating points to catch up with long-time players. Rating deflation unsurprisingly harms more experienced players. The simple solution is to create a system that is zero sum, ensuing there can be no net gain or loss within the system. Often times, this is not possible, and a lot of games may periodically reset their ratings to compensate for this.
5. Draws – In some games, a draw between players means they are very evenly matched, and in other games, a draw is very common. This relationship should be considered within the game and used to determine how to handle draws in terms of ratings. For example, for a very lowly rated player to be able to force a highly rated player into a draw while playing chess says a lot more than forcing a draw in Tic-Tac-Toe.
6. Multiplayer Games – A lot of games will feature cases that are more complex than 1 vs. 1. In these scenarios, rating systems will often need to account for relative placement in free for all game modes, teammate ratings, and individual contributions to the outcome of the game.
7. Disconnection – With online games, it can be difficult to tell if a disconnection event is intentional, and how the system handles this case will incentivize different behaviors. For example, if disconnection results in a loss, then players may attempt to DDOS other players. If it is counted as a draw, players may quit before losing to avoid losing rating points. Furthermore, in team games, if even one player disconnects, then for all other players involved, it becomes much more difficult to determine how to interpret player performance up until that point, whether the game should continue, and how those results should affect their rating. This is also an issue of accessibility, as some players may simply not be able to afford stable internet or may not live in places that have stable internet which raises the question of whether these factors should affect a player’s ability to compete.
8. Cheating – It’s gonna happen, and as a competitor, it makes me very sad. That being said, most good rating systems will need to directly design against different ways players may attempt to cheat or exploit the system. In most cases, directly resolving cases by hand is not practical, and discovery of unhandled/unresolved cheating will destroy confidence in the competitive validity of your game’s rating system. Keep an eye out for positive sum sources of points and for ways players can collude to manipulate outcomes.