Weekly WaveShine #6
Rating System Analysis and Unreal Project Update
Hello, and thanks for stopping by for the this week’s entry in my Weekly WaveShine series! In this week’s entry you can look forward to:
- One of my older design breakdowns of a Sojourn, a character from Overwatch and her pain points [Go To]
- An analysis of a rating system and predicted player experience [Go To]
- An update on my Unreal side project [Go To]
Character Analysis: Sojourn from Overwatch
This is a writeup of a character analysis I did of Sojourn from Overwatch 2 for a design exercise towards the end of 2022. My aim with this is to help get across my ability to examine a character and determine how their mechanics relative to other characters will ultimately affect player experience. I also hope to highlight my ability to make suggestions regarding how this player experience can be improved. For those familiar with the game, this was the period after Sojourn was released when Sojourn was an extremely powerful character before she started to get nerfed. For those unfamiliar with the game, have no fear! We’ll be doing a quick breakdown of the character to provide background for this analysis.
Sojourn from Overwatch 2
Everyone’s favorite cyborg from Overwatch 2 and the subject of our analysis! Sorry Genji.
Summary
I feel the current implementation of the character Sojourn has room for improvement because her current design lacks a “story” from a gameplay perspective and doesn’t deliver as much value to the average Overwatch Player due to her high skill floor. Additionally, she is significantly more performant at higher levels of play, which is the current Player audience she caters to. I believe some changes to her weapon and ultimate ability can make her more distinctive from a gameplay perspective, make her more accessible to the average Overwatch Player, and bring her into more reasonable levels of performance.
Character Overview
This is a quick breakdown of Sojourn’s kit as it was back when she first released.
Primary Fire
Sojourn’s primary fire is a fully automatic projectile weapon. It features a high projectile speed, and the projectiles themselves are quite small. It has a (relatively) large clip size. It has a moderately high rate of fire. The first 9 shots are point accurate, meaning they shoot exactly where the player is aiming. Continuing to hold down the fire button causes moderate bloom, meaning the gun begins to fire randomly within a growing radius. The projectiles experience no (or very little) fall off damage at range. Most characters will do less damage as they get further away from their enemy. This rate differs between characters and is a significant contributor to the effective range of a character.
Sojourn’s Primary Fire
You can see Sojourn’s primary fire in the clip above followed by her secondary fire at the end.
Secondary Fire
Sojourn’s secondary fire is a single action projectile weapon. The projectile itself is extremely high velocity and still small. The projectile has a very high maximum damage, but only has a single shot in its clip. The shot itself is charged based on landing hits with Sojourn’s primary fire and is discharged completely upon firing. The damage of the shot scales to a high maximum value as Sojourn charges her secondary fire.
Bracket A – Ordered Seeding
You can see Sojourn firing her rail shot here at the end of this clip.
Power Slide
Sojourn’s movement ability causes the player to slide in a specified direction for a fixed distance/time. At any time while this ability is active, the Player can cancel the slide by jumping. This jump is much faster and higher than Sojourn’s normal jump. The animation for this movement ability causes Sojourn’s hitbox to rapidly accelerate in the specified direction and causes her hitboxes to move lower to the ground. The jump causes an additional quick change in momentum that can be tough to predict. These factors make it particularly effective in reducing the enemies’ ability to land hits on a Sojourn Player. Finally, the ability itself is on a short 6 second cooldown, allowing it to be used liberally during fights. During a gunfight where players shoot until the other one dies, this ability will likely be used once, possibly twice if the gunfight lasts a particularly long time.
Power Slide
Sojourn’s Power Slide ability. Very satisfying!
Disruptor Shot
Sojourn’s combat ability allows the Player to fire a slow moving projectile that turns into an AOE on contact with a surface or enemies or if the maximum range is reached. The AOE lasts 4 seconds, slowing and dealing moderate damage to enemies within the AOE. It is on a relatively long 15 second CD. It may get used a few times in an overall fight, but only once in a direct gunfight.
Disruptor Shot
You can see Sojourn’s Disruptor Shot at the beginning of the clip. It’s the giant green orb and hard to miss!
Ultimate Ability
Sojourn’s ultimate ability causes her secondary fire to charge automatically and to pierce through enemies, making it particularly devastating to enemies that are bunched up. It has an 8 second duration which can be sufficient for a Sojourn player to eliminate most if not all of an entire enemy team if they are accurate.
Sojourn’s Ultimate Ability
You can see Sojourn’s rail shot charging automatically during her ultimate ability.
Sojourn's Gameplay "Story"
When considering something like a particular character or class, I like to think about the character/class’s gameplay “story”. This is the general path of play that I expect most players to follow. Sometimes, there are multiple “stories” to a character or class which may or may not be a desirable quality. Here’s what I saw as Sojourn’s gameplay “story”:
- Operate from a mid-range position to deal consistent chip damage with primary fire before securing key picks using headshots with her secondary fire from any range.
- Use Power Slide to reach stronger sightlines, escape exposed positions, and/or close distance to chase down vulnerable enemies.
- Use Disruptor Shot to deny sightlines, slow enemy approaches, break up enemy formations, and/or to make it easier to land headshots.
- Use ultimate ability to quickly deal large spikes of damage with her secondary fire ability, ideally when not actively taking fire and enemy tanks are unable to mitigate damage.
Problem #1: Lack of Gameplay Identity
I personally saw two overarching issues with Sojourn’s design at the time. The first design problem with Sojourn was that her weapon was nearly as effective compared to multiple other characters with similar mechanics which impairs her “story” from a gameplay perspective.
One direction we can approach this from is to consider the mechanical skills that contribute to the overall task of aiming and accurately hitting a target within an FPS. Broadly, aim tasks can be broken into:
- Tracking – The task of moving your cursor to continuously match the movement of a target on your screen. Typically relevant with fully automatic weapons.
- Click Timing – The task of clicking the mouse once with the correct timing when the cursor is aligned with the target. This task includes what is colloquially called “flicking” and “snapping”, but also includes other techniques like pre-aiming.
This gives us a few insights. First, we can now identify archetypal characters that correspond to these aim tasks. Specifically, Soldier 76 with regards to tracking and Widowmaker with regards to click timing. Second, we can see that Sojourn’s weapon involves both aim types, which makes the comparison to both Solider 76 and Widowmaker particularly important.
Kit Comparison to Soldier 76
Projectile vs. Hitscan
Soldier 76’s weapon is a hitscan weapon. This means that when the player begins firing, a hit is immediately calculated at the point where the player is aiming on their screen. Compared to a projectile weapon, a hitscan weapon simplifies the aim task and improves effective Player accuracy, especially at longer ranges. Sojourn’s primary fire has a high projectile speed which reduces this disadvantage and at closer ranges, this difference becomes negligible.
Soldier 76 Primary Fire
You can see Soldier 76’s hitscan primary fire. While the player holds down the primary fire button, their weapon fires and hits are calculated (nearly) instantaneously.
Differences in Bloom
Soldier and Sojourn both suffer from randomized bloom after a fixed number of shots, reducing their accuracy over long bursts. Soldier’s bloom begins at approximately 4-5 shots while Sojourn’s bloom begins around 7-9 shots. This means that Sojourn’s weapon will behave more consistently when fired initially and in bursts if we consider this difference in isolation.
Soldier 76 Bloom
You can see an example of what bloom looks like indicated by the growing size of the crosshairs.
Damage Drop Off
Sojourn’s projectiles deal 9 base damage and do not drop off based on range. Soldier 76’s bullets deal 18 base damage, but begin to fall off in damage at 20 meters down to a minimum of 5.4. This means Soldier will likely deal more damage than Sojourn at closer ranges, but whether he deals greater damage at longer ranges isn’t as clear. The disadvantages in bloom and hitscan behavior become less apparent at close ranges, which also makes it less clear whether Soldier will deal greater damage at closer ranges.
Soldier 76 Damage Drop Off
In this clip, you can see how in addition to the inaccuracy caused by Solider’s bloom, each individual hit is also doing less damage.
Clip Size/Rate of Fire
Sojourn has a larger clip at 45, while Soldier has a smaller clip at 30. Sojourn fires 14 rounds per second while Soldier fires 9. A larger clip size and higher fire rate are helpful at all ranges, but again these factors tend to provide more consistent performance at close ranges, where burst firing is no longer necessary and Players are more likely to be engaged in committal, full-auto aim duels.
Soldier 76 Clip Size
Well, at least at the time of writing it was 30. In this clip it’s 25, so it’s clearly something that’s been tinkered with over time.
Abilities Comparison
Soldier’s Helix Rockets are similar to Sojourn’s secondary fire. They enable him to deliver high damage spikes, but their slower projectile nature provides him with less range to do so. Additionally, they do not deal headshot damage, so Soldier effectively lacks Sojourn’s capability to quickly secure a kill with a single button press.
Soldier has an AOE heal as opposed to Sojourn’s disruptor shot. Soldier can use this heal to sustain himself, but will gain the most value from this ability by using it to help his team recover from distributed damage.
Soldier is able to sprint with no cooldown or limit. This provides him with a great deal of positioning flexibility, enabling flanks, aggressive pushes as well as retreats or target switches to cover vulnerable teammates.
Sojourn’s power slide is similarly flexible, but gives up horizontal distance and constant uptime for added verticality. Regardless, both characters have powerful movement abilities allowing them to maintain their desired spacing and angles.
Soldier 76 Helix Rockets
This one’s a bit tough to catch, but Soldier 76 fires his helix rockets at the end.
Solider's Gameplay "Story"
I personally saw Soldier’s gameplay “story” to be:
- Operate from a mid-range position, punishing “squishies” for overextending or for unusually predictable movement with helix rockets.
- Ideally, stay close to the team/healers to be able to provide AOE heal to mitigate distributed damage or to strengthen damage exchanges along a key sightline.
- Use sprint to safely go on flanks to break stalled pushes or to reposition to protect the team from enemy flankers.
- Use ultimate ability to deliver consistent damage and eliminate exposed “squishies”, ideally when not actively taking fire and enemy tanks are unable to mitigate damage.
Kit Comparison to Widowmaker
Projectile vs. Hitscan
The key thing to compare here are Widowmaker’s primary, scoped in fire and Sojourn’s secondary fire. Widowmaker’s weapon is hitscan, while Sojourn’s is an extremely fast projectile. Widowmaker’s weapon must be zoomed in for its single firing action mode, which increases her effective and preferred ranges, and greatly limits her peripheral vision. Both characters technically have infinite range, constrained only by the Player’s accuracy, but Widowmaker’s zoom usually places her further from fights than Sojourn. Because Sojourn is closer to her enemies, the impact of her projectile speed will be less apparent.
Widowmaker’s Primary Fire
You can see Widowmaker’s scoped in primary fire here. As soon as the primary fire button is pressed, a hit calculation is immediately performed.
Differences in Bloom
Widowmaker’s fully automatic mode has a massive bloom and is really only useful in scenarios where Widowmaker is forced into closer quarters. Sojourn’s fully automatic mode is vastly superior and outperforms Widowmaker’s fully automatic mode at all ranges.
Widowmaker’s Automatic Fire
At the very beginning of this clip, you can see the player use Widowmaker’s fully automatic firing mode to quickly get rid of the Symmetra turret.
Charging Damage
Widowmaker automatically charges her shot when scoped in, increasing its damage. Sojourn charges her shot more slowly by hitting enemies with her primary fire. Widowmaker’s zoomed shot has a greater headshot multiplier (2.5x as opposed to 2x). Sojourn’s fully charged shot will deal greater damage on the body as compared to Widowmaker’s fully charged shot. Sojourn’s maximum headshot damage with her charged shot is 240, and Widow’s is 300. Both of these amounts are greater than 200, which is the average health pool for most DPS and healers. The result is that Sojourn is able to bring much of Widow’s instantaneous lethality at mid to close ranges.
Widowmaker’s Charging Damage
If you look closely, you can see an indicator for the shot’s charge while Widowmaker scopes in.
Clip Size
Widowmaker’s primary firing mode has 7 shots, while Sojourn’s secondary firing mode only has 1 shot, and must be charged to be fired again. The result is that Widowmaker is able to consecutively fire her high-damaging shots, killing almost an entire team on her own in an ideal case. By contrast, Sojourn would not be able to accomplish this as quickly or independently.
Clip Size
This player exhausts their ammunition towards the end of this clip. This is an old clip from when the character had 10 shots instead of 7.
Abilities Comparison
Widowmaker’s grapple enables her to move into strong high ground positions and to possibly retreat if forced into close quarters combat. It lacks Sojourn’s flexibility due to Widowmaker’s low performance at close quarters.
Widowmaker’s Venom Mine can be placed to protect flanking routes to Widowmakers’s position, or to trap expected lines of attack, giving her vision on enemies hit by it. It also is less flexible than Sojourn’s Disruptor shot because it has limited range, can be seen and destroyed by enemies, deals very little damage, and does not control space after it has been detonated.
Widowmaker’s Venom Mine
You can see an example of Widowmaker’s Venom Mine above. While enemies are affected, they are visible through walls.
Widowmaker's Gameplay "Story"
I personally saw Widowmaker’s gameplay “story” to be:
- Operate from mid to long ranges, prioritizing key sightlines to secure picks on “squishy” enemies using headshots from your charged primary attack.
- Use grapple to maintain desired mid to long range and positioning. Avoid close range combat.
- Use Venom Mine to protect your flanks and compensate for your reduced peripheral vision.
- Use your ultimate ability to gain vision on all enemies and make it easier to pre-aim targets through terrain. If it happens to help teammates also, great, but usually this isn’t a priority.
What we can hopefully see from this is that Sojourn clearly borrows key elements from Soldier’s and Widow’s “stories”, filling many of the mid-range role responsibilities Solider does while still bringing Widowmaker’s long range lethality into the mid and close ranges. This similarity raises the question: “Why should I play Soldier/Widow over Sojourn?”. At the time or writing, the answer was in Sojourn’s favor, since she filled a lot of the needs filled by Soldier and Widow while providing an unrivaled utility through her Disruptor Shot. I would personally say that the problem isn’t a matter of balance among the three characters, but their similarity. A simple rebalancing would likely decrease Sojourn’s relative viability without providing a substantially different gameplay experience, reducing her playtime and ultimately, her value to Players.
Problem #2: High Skill Floor
The reason why I believe Sojourn requires a high degree of aim skill is that in order to deliver her lethal levels of damage through her secondary fire, the Player must first charge using their primary fire. This requires the Player to be proficient in both tracking and click-timing, and the average Player usually has a low proficiency in both. Additionally, her ultimate ability still requires a high degree of accuracy from the Player to actually get value from it.
These things can make it harder for the average Player to find enjoyment with the character. Typically for the average Player, a character’s ultimate ability is usually a redeeming quality for characters with high skill floors.
For example, Genji requires a high degree of accuracy and familiarity with his mechanics, yet each game, he’ll have at least one chance for a Player to feel powerful when using his Dragon Blade ultimate ability which has a lower accuracy requirement and deals high damage. Hanzo, a character requiring high aim proficiency also can turn the tide of a fight even after overall suboptimal play with his Dragon Arrow ultimate ability which is large and both visually and mechanically impactful.
Other characters attract average Players through their uniqueness of play. Genji definitely falls into this category, jumping and dashing around very dynamically. Tracer fills a similar role as Genji, as a flanker requiring accuracy and mechanical proficiency, but she also plays radically differently from other characters by being able to teleport. Sombra, another close range flanker has a firing action that is very similar to Tracer’s but still has a very unique and distinct gameplay story, being able to hack other Players and move about the battle while invisible.
Finally, some characters attract average Players simply because they are consistent when punishing low-level mistakes. Torbjorn punishes enemies for not dealing with his turret proactively without placing a significant burden of execution on the Player. Junkrat punishes enemies lacking the aim to deal with him from outside his effective range while providing high damage output at close range. Bastion provides Players with additional forgiveness compared to Sojourn or Soldier 76 for punishing bad positioning with tracking aim.
For these reasons, I feel like Sojourn isn’t particularly accessible or appealing to the average Player, and therefore does not deliver them a lot of value.
That being said, I think it becomes relevant to consider where Sojourn is fitting into the meta at top levels of play.
What we can see in the figure to the right is that over the past three months (at the time of writing), Sojourn had a clear presence in the Overwatch League, with a 57.3 % pick rate.
3 Month Overwatch League Participation Rate
These are the percentages for participation for each character over the last 3 months (at the time of writing). Sojourn is in red on the left.
Over the past two months, her presence has grown to 66.7% in the Overwatch League.
2 Month Overwatch League Participation Rate
These are the percentages for participation for each character over the last 2 months (at the time of writing). Sojourn is in red on the left.
And finally, over the past month, her play rate has risen to an astonishing 99.7% pick rate.
I think that these visuals emphasize what a poor state of balance the top levels of play are in. In my opinion, this is a sign that Sojourn is not delivering value to top levels of play, since her relative imbalance is taking away the meaning and richness of decision-making when it comes to picking team compositions.
Not every character needs to be viable for top-level play, but it’s probably not ideal for so few characters to be viable.
1 Month Overwatch League Participation Rate
These are the percentages for participation for each character over the past month (at the time of writing). Sojourn is in red on the left. As you can see the distribution has grown very exaggerated.
Solution: Reevaluate Intended Player Experience and Redesign
I believe that a limited redesign of the character based on her existing gameplay mechanics can address these issues. I believe her abilities can help her tell a more unique gameplay story of staying at mid-close range to get an enemy low before using her movement and AOE to cover a dive on the targeted enemy and subsequently attempting to escape.
While I think rebalancing the character is more realistic, and more likely to happen, I think that it fails to address the intrinsic conflict that lies between Sojourn, Soldier 76, Widowmaker, and possibly Ashe. Nerfing Sojourn until she’s no longer dominant again could potentially lead to her becoming an unpopular or unplayed character which would lead to a redesign anyways.
Currently, I feel like Genji, Sombra, Tracer, and Reaper are the DPS characters that actively dive enemy formations to secure picks. For the most part, they do so because outside of immediate close range, they are fairly ineffective. Each of them has their own unique means for escaping after securing the pick, so although they functionally have similar objectives, they achieve this through significantly different means. I believe these characters have a lot of capacity to deliver value to the average Player, since their gameplay is very unique and involved.
As a result, I think it would be better to move Sojourn’s design more in this direction to help her become more distinctive in a hero pool already filled with numerous hitscan characters. The first specific change I’d suggest is changing her secondary fire into either a projectile weapon or a close-range shotgun style weapon. The intent behind this change is to reduce the effective range of her single-shot lethal capabilities to the mid to close range. I genuinely believe some experimentation and playtesting would be necessary to determine which design would produce the desired outcome.
The second specific change I’d suggest is changing her Power Slide ability to be able to hold charges similar to Tracer’s dashing ability or to have it refill on kill, similar to Genji’s dashing ability. Together with the first change, Sojourn’s new ideal Player experience would be to linger at the mid-close range using her primary fire to get a healer or DPS to low health, use her Power Slide to leap over the enemy tank/formation, throw her AOE slow in midair to prevent enemies from repositioning to stop her from diving, finishing off the DPS or healer with her now charged secondary fire, and finally leaping out using her Power Slide.
Her ultimate ability could remain unchanged, enabling her to fire hitscan, high-damage piercing projectiles, better helping to sell the sensation of her ultimate representing Sojourn “bringing out the big guns” since using her ultimate ability would afford her a kill range she wouldn’t have under normal circumstances.
I believe this would definitely require some playtesting and tweaking to get right, but I feel like a “diving” Sojourn matches her confident and capable personality and this particular design shouldn’t require any additional art assets. That being said, I think it’s more likely that her gameplay properties will be adjusted to reduce her viability in which case her projectile speed and her damage profile on her primary fire may be good places to start.
What Actually Happened
These were analyses and predictions I made in late 2022, and they somewhat closely matched what actually happened. Sojourn proceeded to get nerfed several times to bring her performance back to more reasonable levels. This has put her in a place where she does not see a lot of play time, which I believe is due to the still unresolved issue that her gameplay is not meaningfully different from other characters. My further prediction is that she will eventually see some kind of re-work to address this intrinsic issue with her design.
Grandmaster DPS Pick Rate Over Last 3 Months
This is a slightly different statistic tracking the rate at which the character was picked by Grandmasters in the ranked game mode. Grandmaster is the highest ranking of players. Sojourn is now 7th and below both Widowmaker and Soldier 76.
General DPS Pick Rate Over Last 3 Months
This is pick rate for all players in the ranked game mode. Sojourn is now 6th and gains a spot over Widowmaker likely due to the fact that she is less aim intensive for the average player.
Game Design Rating System Analysis
As I’ve mentioned in previous posts, I’m working through Game Balance by Brenda Romero and Ian Schreiber. The following is an exercise that called for an analysis of a rating system and the expected player behavior.
The Problem
Consider the following rating system for a 1 v 1 game:
- Let D be the difference between two player’s ratings. When D < 20, the winner of the game earns 10 points, and the loser loses 10 points.
- When 20 <= D < 40, the winner of the game earns 9 points, and the loser of the game loses 9 points.
- This decrease in the change in points continues for each 20 point increase of D such that the change in points is max(1, 10-floor(D/20)).
The Solution
In order to determine whether this rating system accomplishes the stated goal, we’ll consider three cases from the perspective of a single player and the expected outcomes in terms of points won/lost in these scenarios. The first is the case where the opponent has an exactly equal rating, the next is where the opponent has a lower rating, and the final is where the opponent has a higher rating.
Case # 1: Expected Point Change When D=0
The expected change in points when D = 0. This is the case when both players have equal ratings.
Case # 2: Expected Point Change when 0 < D < 20
The expected change of points when the value of D is within the range (0,20). This is the case where the player in questions has a higher rating than their opponent.
Case # 3: Expected Point Change when -20 < D < 0
The expected change of points when the value of D is within the range (-20,0). This is the case where the player in questions has a higher rating than their opponent.
In the relationships above, P(D) is some hypothetical function that can be used to compute the probability of a player winning against another player based on their D. While we do not explicitly know P(D), I made 2 assumptions. The first is that rating is at least approximately accurate in determining player skill. The second is that the more skillful player is more likely to win this game. If these assumptions are true, then the expected point changes will follow. In case 1, where D=0, then the either player is equally likely to win, so P(0) = .5. This would mean that the expected point change would be 0, so challenging a player of equal rating isn’t actually valuable and on average will produce no net change in points. In case 2, D= (0,20) so the player in question is rated higher than their opponent. Thus we expect the player to win more often than they lose, so P(D) will be greater than .5 and the expected point change in this case is greater than 0. In case 3, D = (-20,0), so the player in question is rated lower than their opponent. We expect in this case for the player to lose more often than they win, so the expected point change is actually negative.
Furthermore, if we consider that the P(D) is monotonic, then as D grows increasingly negative, then P(D) does also and vice versa. Since E(D) decreases as D grows in magnitude, then this means that the optimal play strategy would be to find the ideal value of positive D that gives you the highest probability of winning a reasonably positive number of points. At the extreme top end of the ratings, it could be optimal to only play against significantly worse players to minimize any possible point loss and nearly guarantee at least 1 point of point gain.
Under this system, playing against a player of equal rating is not an effective use of time and basically amounts to a coinflip. Playing against a player of better rating is explicitly unfavorable.
At first glance, this system appears to encourage playing closely rated players to have a better chance at winning points, and the average player may actually follow this strategy. However, top level players will analyze this system and arrive at the conclusion that in the long run, playing worse players is always desirable, and that potentially, playing significantly worse players may be optimal at particularly high ratings to prevent the loss of points. A pool of players under this system would then consist of average players that don’t look at the system too closely and a set of competitors that actively seek to abuse the average to low rated players, making the overall system fairly unfriendly to new and mediocre players. These lesser players would likely become discouraged and quit, and top-level players would move on to challenge slightly better players, and the cycle would repeat. Over time, the pool of players would shrink to only include top rated players, who would then likely drop the game as most players seek to be top-rated and widely recognized as being so. If the player base for the game is tiny because of this failing rating system, then there would be very little incentive for any player to engage with this rating system. So beyond failing miserably at its objective of encouraging players to play against opponents of equal rating, this system is actually highly likely to create an experience that is likely to be unpleasant for new and average to below-average players who generally compose the majority of player bases.
Unreal Project Update