Minecraft is one of the most moddable games to date, from resourcepacks to datapacks, all the way up to mods. With the right modifications to the game, Minecraft can change into a completely new genre of sandbox survival. One of the prime ways to modify the game is through a modpack, combining a large list of mods with all sorts of modifications to the game into one tailored package.
For many people, modpacks are downloaded from platforms such as Curseforge or Technic. These provide a premade and predefined experience of playing the game with mods. However, some people may want to create their own experience with their own collection of mods, datapacks, or scripts. Some people may also want to share this custom experience with their friends, and we make that possible by allowing full server customization.
One problem that may arise with modpacks is that bugs and crashes can happen, and when it comes to Minecraft mods, it can be hard to troubleshoot the cause. There could be a mysterious crash with no known source, a mod conflict, or a laggy mod. In cases where we don’t know the cause of an issue, we bring in an elimination technique known as Binary Testing.
What is Binary Testing?
Let’s start off with what this means. If the term “binary” refers to a pair, then a Binary Test would have something to do with a couple or pair. In a way, it is creating a pair by splitting a list of mods in two, then testing if the modpack works. The idea is that if there’s an elusive and unknown issue with a modpack, splitting the list of mods in half over a series of tests can narrow down the list of mods to a more manageable size, making it easier to identify the problematic mods.
Binary Testing in action. The list is reduced in size by half each time until we get down to the source, troubleshooting errors faster than testing one-by-one. If we find the error in one half, we test that half by splitting it again. This process is repeated as many times as it takes to find the problem.
Say we have a modpack with about 400 mods, but it’s failing to load up on a server. Normally, the mod loader will point out what mods have failed to load and why, such as client-side code running on the server or a missing dependency. However, a server may simply fail to load without an indication as to why it failed. With Binary Testing, we split the modpack in half, creating two modpacks with about 200 mods. If one half works but the other doesn’t, then that tells us that the problem exists in one half. This is then split into two modpacks with 100 mods, and this goes down to 50, then 25, and so on until there’s only a couple mods left that could be the problem, or the issue isolates itself.
This is considered the best method of isolation because there’s no guesswork in finding the problematic mod or several mods in some cases. Unlike taking a random stab at what mods might actually be the problem, Binary Testing turns a list of mods into bite-sized pieces, speeding up the process drastically than if we were to guess what the cause of the error is.
How does it work?
First, we need to identify why we are isolating mods. There are a few different approaches to Binary Testing, depending on what the problem is.
Let’s say we are installing a modpack onto a server but it crashes for an unknown reason. If we don’t know why it is crashing, then we need to split the modpack in half, making sure to include any dependencies that will be required of each mod. Once the list of mods starts to narrow down, then the cause of the issue may start to surface by itself.
Now, if we have a source such as an erroring mod but with no clear reason why, we can use that as a clue. Why we might want to test for a problem with a mod instead of removing it is to identify if there is a bug with the mod or if there is a conflict with another mod. This can be used to check both crashes and performance issues with a modpack.
Once the source of the problem has been found, we can slowly put together the list of mods with the conflicting mod or mods removed, all without having to guess where to start.
Binary Testing in Action
Let’s have a look at a modpack we are looking to install on a server. We upload all the mod files, configurations, and any additional files included in the modpack. It loads in our game, but it seems to stop on the server. We can take a look at the Console or the latest.log file to see what the error is.
The error we are diagnosing. We can see that this error is caused by a client-side code, but not what is causing it. To narrow down why, we will use Binary Testing.
Unfortunately, this error message does not tell us what caused this crash, only that the server stopped when attempting to load mods. Because we don’t know what the cause is, we start Binary Testing by removing half the modpack from the server. In this case, our modpack has about 470 mods, so we will want to split up the mod into 235 mods.
What we will notice is that because I split this modpack in twain, there’s mods that are missing dependencies, so these will also need to be included for each test. You might be thinking that maybe it’s a library mod that is the cause of these loading issues and not an actual mod, and while this could be true, this is very rare. If a library mod is at fault and is required on the server, then it can indicate that the library and the dependent is the cause, or just the dependent attempting to run client-side code with the library.
Highlighted here are all the mods that we need installed in the new modpack list. Installing these will be required in order to check if this pack loads.
As highlighted, we have quite a number of dependencies required by this smaller modpack, so we will include these mods, too. We should also make a note of what mods require which dependency in case the dependency or a dependent is the source of the error.
Once we apply all the dependencies that are missing for this half of the modpack, we can start loading up the server again to see if we get new errors or a running server. In this case, we have a new set of errors, so we will need to diagnose and fix these.
There are three types of loading error that help us know if what we have are client-side mods.
As we can now see, there are three different types of errors presented by these mods. These errors can indicate what is called a client-side mod, which are mods that run only on your copy of the game, not on a server. If we remove these, the server can still be joined with the same modpack.
Starting with the obvious ones that mention an “invalid dist.” When this error appears, it means certain code that is designed for the client is attempting to run on the server. In short, this means that the mod does not belong on the server as it is running the wrong kind of code.
Another error is “NoClassDefFoundError.” This can also indicate there is client code attempting to run on the server, or it could be a missing dependency that a mod did not disclose. In this case, a “Toast” and “Narrator” is attempting to be used on the server, but both of these are not server code, so in our case, these mods are client-side.
Finally, there are two mods that simply say “null.” For these, the cause can be unknown, but to be on the safe side, we should remove them. While this could mean we are removing content mods from the server modpack, we could also be removing an elusive client-side mod. Researching the functions of this mod is also a good idea as it can tell you if a mod is specifically for the client. In our case, these two mods are client-side mods.
Framework is missing from the server, but we needed to remove it in the previous step as it attempted to load client code.
Now, this is where things get interesting. In the previous step, we had to remove Framework as it was attempting to use client-side code. However, once we remove that, we can see there are two mods that depend on it. What’s going on?
This is a case where a library mod is being used for both sides of the game. While this library is required for a mod needed on the server, it can also be used for mods that are client-side. Here, we have two possible causes: Goblin Traders and Controllable. If we don’t know which of these two mods are stopping the server from starting, we can remove one and see if the server starts. Since we have two mods, we can also look up their function instead. The first appears to add new entities to the game, while the second adds controller and gamepad functionality. From this, we can conclude Controllable is a client-side mod because the server does not need to know about controller functions.
If we see a message say the server is done loading, that means the test no longer has the error.
With Controllable removed and Framework added back in, we are now successfully loading up the server. Simply put, this means the test is a success. We can put the modpack together again and run it, right? Well, not yet. We still need to make sure the other half of the modpack does work, otherwise we would have to repeat this process from the start. Take out the working half of the mods and put it into a new folder on your computer for safekeeping.
Let’s start up the other half of the modpack. Make sure to add in all the missing dependencies and remove any client-side mods, just like what we did above.
We have an error indicating there is client code running on the server, except that these two mods are not considered client-side.
Hold on, we have these mods here that are saying that they are loading client-side code. This would mean they are client-side mods, but Create and The Undergarden are not client-side, they’re very much mods that need to be on the server. Now what seems to be happening?
What we have is the failing half of the modpack, one that has the problem we are trying to isolate. We need to split this half of the modpack in half again. This process will need to be repeated a few times - add in missing dependencies and remove client-side mods that are throwing errors. However, since we know Create and Undergarden are affected by this error, we want to include either or both of them in every test from here on. This will make sure that we know the pack is able to load with these two mods present.
How do we know we found the mod at fault? We want to keep track of what mods were taken out and whether the server was able to start or if it continues to cause these mods to load client-side code. For every test, we will want to make a new folder, and for each test that still has an error, make another folder. Once the mod list starts getting down to a very small size, we can start looking at each mod, then remove them based on whether they add content or if they deal with visuals, control inputs, or UI elements.
At this point, this is everything that is left in the modpack that still causes Create to error. With the modpack being this small now, we can narrow down what can be removed from the server.
Once we get the modpack down to a manageable size, what we can do now is either remove a small section of mods at a time, or search up what each mod does and if their function is client-side. As there are fewer mods, we can take the time to go through what was in the modpack. After some testing, we find that if the mod Shoulder Surfing is removed, the pack works. From there, we can now slowly reintroduce mods to see if the server runs.
If during the reintroduction of mods we get another error with an unknown cause, we go back and restart the Binary Testing method with the modpack list on the server again. We want to scrub this modpack clean of any mod that is stopping the server, and slow reintroduction means a better chance to find a second, third, or more error mods.
With our Binary Testing efforts complete, we can not only start the server, but join it with all those problem mods removed from the server-side.
Finally, after completing our test, the server is running and we are able to successfully join it. Without Binary Testing, we may have been fixing the modpack for much longer. Now that those troublesome mods have been removed from the server, we can start inviting friends and get to playing.
Conclusion
Binary Testing is an efficient and faster method of diagnosing the most elusive of errors. It is also widely applicable as it can apply to more than just mods. A server could be running plugins, datapacks, scripts, or it might not even be a server for Minecraft. So long as a server contains removable parts like mods, the Binary Testing method can be used to scope out what the problem source is without knowing the cause in any game.
If you are putting together a modpack, make sure to keep this trick in mind the next time an error needs troubleshooting. Need a hand? Come join us over on our Discord, or get in touch with Support by clicking on the live chat bubble. We make sure that server setup is as simple as possible, and all you need to get started is by purchasing a plan.