Creating a bot for Marvel Heroes Omega (PS4)

This post documents how I created a bot for Marvel Heroes Omega using scripting in PS4 Macro.

With the release of PS4 Macro v0.3, I decided to find a game to test the scripting system which is Marvel Heroes Omega, a free to play MMORPG game that was recently released on PS4 and Xbox One.

See the full bot demo in this video:

The plan was to make a typical RPG farming bot that includes navigation, combat (fighting, healing, collect loots), and selling loots. However, there will be no hacking involved since this will be purely a pixel reading bot.

Speaking of pixel reading, I need to use an image processing library. I picked AForge.NET because I have multiple successes in my past projects.

Healing

For my first task, I decided to start with healing since it can become useful right away as some kind of “helper bot” when playing manually.

To be able to heal at the right time, the bot has know how much health the player has left. The first problem that I encountered, was that the health bar was not filled with a solid color which makes it difficult to detect, not to mention the noises caused from streaming the game.

Health BEFORE Posterization

After playing around with the image in Paint.NET, I found an effect called “Posterize”. What this effect does is reduce the colors of an image which will remove the gradients in the image as a result. Luckily, this effect was available in AForge.Imaging package as an image filter.

To use the filter, there is a parameter called PosterizationInterval which defines the amount of colors in the result as intervals. For example, 2^4=64 which will result in an image with maximum of 4 colors.

1
2
3
4
5
6
7
Bitmap bmp = Image.FromFile("test_image.png") as Bitmap;
SimplePosterization filter = new SimplePosterization()
{
PosterizationInterval = 150
};
filter.ApplyInPlace(bmp);

Health AFTER Posterization

Now that the health color is solid it can easily be detected by scanning the pixels from right to left. I also did the same for the energy spirit bar but just with a different color. Then I created a UI to display the the values in progress bars and also a number input box to set the percentage that will trigger healing when the health drops below the number. To make the bot heal I simply check for the condition above and make it press L1.

Combat System (Prototype)

At this point I wanted the bot to fight the enemies along with its healing abilities. It doesn’t have to be good but at least it has to cause some damage. To achieve this first phase of combat system without making it too overwhelming I decided to only detect the enemy health area and blindly attack without caring if the enemy is really close enough or not.

Enemy's health BEFORE Posterization

I used the same technique as detecting the player’s health and spirit energy but with a couple of differences. First, it is in vertical instead of horizontal, and second, I don’t have to detect a value out of it but only detect if it’s there or not.

Enemy's health AFTER Posterization

I also noticed that there are multiple colors of this green bar later in the game, so I also include those colors when checking.

Green Bar

Finally, I have to make the bot attack when the enemy is detected by pressing one of the attack buttons. Later on I made the bot dash to the enemy and press R1 to lock the target which will keep on locking nearby enemies until all of them are out of range.

What I’ve done previously are what I considered as the easy part. The hard part is where we have to actually detect the environment and react to it accordingly. What a working navigation system need to know is: where we are currently, and our destination.

To know where we are, we need to know both the position and rotation.

Rotation

I decided to detect the player’s rotation based on the direction of the arrow of the player indicator since the arrow in the minimap is too small.

Player Rotation BEFORE Posterization

This is possible because the player will almost always be at the center of the screen which means that I can scan for the rotation within one area.

To scan for a rotation I have to recall my middle school trigonometry (which I end up using Google because I never paid attention in class). To find the point on a circle we can use the parametric equation.

1
2
x = r cos(t)
y = r sin(t)

Where r is the radius and t is the parameter (the angle in degrees).

However, if we look carefully we can see that it is not a circle but an ellipse due to the camera angle. That means that the equation we really have to use is

1
2
x = a cos(t)
y = b cos(t)

Where a is the radius in the x axis and b is the radius in the y axis.

I used posterization filter again to reduce the colors then scan for the arrow starting from 0 to 359 degrees.

Player Rotation AFTER Posterization

However, the result is not so good on this part. This is because sometimes the arrow are blocked by the player’s character. I guess I have to also include the arrow of the minimap to improve the rotation detection.

Position

Moving on to position detection. There isn’t an easy way to detect the player’s true location within the game (exactly which map and where). This requires some kind of database to store every single map in the game which will be a huge task. So I will only be detecting the player’s position in relation to only what the player can see in the minimap.

Minimap

To filter the boundaries in the minimap I used an edge detection filter provided by AForge.NET. They offer multiple algorithms for edge detection so I tested all of them for comparison.

Minimap with Sobel-Edge Filter

Surprisingly, only Sobel Edge Detection was giving a usable result which is good enough to create a navigation radar for the bot.

To keep things simple I will only detect and move in 4 directions (up, down, left, right) which will eventually be improved into a full 360 degrees movement. Every loop, the bot should check for the boundaries of each direction from the center of the minimap and calculate its distance. This will be used to rank the best next direction for the bot to move.

I came up with a simple algorithm to allow a basic movement to be used together with the combat system implemented earlier.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public const int TOP = 0;
public const int RIGHT = 90;
public const int BOTTOM = 180;
public const int LEFT = 270;
// Black color for walkable area
public static int FilteredWalkableColor = 0x000000;
// Offset to start searching
public static int PlayerOriginOffset = 6;
// Origin point of player in minimap
public static Point P_PlayerOrigin = new Point()
{
X = 46,
Y = 33
};
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
private int ScanAreaFromOrigin(Bitmap filteredBmp, int direction)
{
int distance = 0;
Color targetWalkableColor = FilteredWalkableColor.ToColorOpaque();
try
{
while (distance <= 10000)
{
Color foundColor = Color.White;
if (direction == TOP)
{
foundColor = filteredBmp.GetPixel(P_PlayerOrigin.X, P_PlayerOrigin.Y - distance - PlayerOriginOffset);
}
else if (direction == RIGHT)
{
foundColor = filteredBmp.GetPixel(P_PlayerOrigin.X + distance + PlayerOriginOffset, P_PlayerOrigin.Y);
}
else if (direction == BOTTOM)
{
foundColor = filteredBmp.GetPixel(P_PlayerOrigin.X, P_PlayerOrigin.Y + distance + PlayerOriginOffset);
}
else if (direction == LEFT)
{
foundColor = filteredBmp.GetPixel(P_PlayerOrigin.X - distance - PlayerOriginOffset, P_PlayerOrigin.Y);
}
else
{
break;
}
if (foundColor != targetWalkableColor)
{
return distance;
}
distance++;
}
}
catch (ArgumentOutOfRangeException) { }
return -1;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
public KeyValuePair<int, int> FindWalkDirection(Script script)
{
// Crop
var bmp = script.CropFrame(R_Minimap);
// Apply filter
Bitmap filteredBmp = Helper.SobelEdgeFilter(bmp);
// Scan distances in each direction
var distances = new Dictionary<int, int>();
distances.Add(TOP, ScanAreaFromOrigin(filteredBmp, TOP));
distances.Add(RIGHT, ScanAreaFromOrigin(filteredBmp, RIGHT));
distances.Add(BOTTOM, ScanAreaFromOrigin(filteredBmp, BOTTOM));
distances.Add(LEFT, ScanAreaFromOrigin(filteredBmp, LEFT));
// Sort by longest distance
var sorted = from pair in distances orderby pair.Value descending select pair;
// Find candidate
var candidate = sorted.First();
// Replace candidate if we just came from there
if (GetOppositeDirection(candidate.Key) == m_LastWalkDirection)
candidate = sorted.ElementAt(1);
// Return candidate
m_LastWalkDirection = candidate.Key;
m_LastWalkDistance = candidate.Value;
return candidate;
}

This algorithm will sort the distances for the longest distance and use the the first one if we haven’t came from that direction earlier, otherwise it will use the direction with the second longest distance.

This allows the bot to patrol around in a small contained area waiting for enemies to respawn.

Conclusion

In conclusion, we now have a bot (created entirely on PS4 Macro scripting API) for Marvel Heroes Omega that can perform simple navigation, combat, and play custom macros. These features are grouped into a larger feature called “Objectives” that allows the user to program them in any order in the UI.

I also added “Attack Sequence” that allows the user to customize what combinations the bot will press during combat. Finally, I implement a simple config save/load feature so the bot settings can be used in the future.

You can check out the source code or test the compiled DLL here.

Screenshot

Resources

Share Comments