Cybercriminal gangs from Eastern Europe have always followed a rule: Don't steal from Russians or their former Soviet allies. Groups like REvil or DarkSide put kill switches inside their malicious code, checking if the language on the machine it lands on is Russian, Ukrainian, Georgian, Armenian, or Romanian. If it is, the malware simply fails to install.Such tactics create malware variability\u2014the same piece of code can do different things on different computers, depending on the version of the OS, the libraries installed, or the language settings. "If you try to run the same malware on three or four different machines, you'll potentially get three or four different behaviors," says Erin Avllazagaj, graduate research assistant at the University of Maryland, College Park.While security researchers have long been aware of malware variability, few studies estimated just how extensive it is in the wild. Avllazagaj wanted to know more, so he looked at 7.6 million malware execution traces recorded in 5.4 million real hosts in 113 countries. He noticed many behavior changes across time and across machines, with troubling implications. "You cannot say this malware is a Trojan because it behaves like a Trojan, because maybe in the next execution it will behave like ransomware," he says.Sometimes, these variations are unintentional because the malware fails to work correctly. However, even more interesting are those intentional variations, when the malware is designed to perform certain activities only on the right computer or under the right circumstances, trying to appear benign when these conditions are not met.A piece of malware like this will look benign on most machines, making it difficult to spot. "The information from automated systems or the endpoint detection can be used for positive identification of malware's presence but may not be very useful for confirming its absence," says Peter Kosinar, technical fellow at ESET.Typically, nation-state actors are known to use sophisticated tactics that leverage malware variability, but Kosinar noticed some mass-distributed malware that also employs these tricks. Avllazagaj's research could shed some light on the variability of malicious code, which could help the security community better understand the issue."We empirically evaluated how much malware changes, what part of its behavior changes, and what we can do to account for that," Avllazagaj says. "And we showed that antivirus vendors are in a pretty good position to do something about it."Running a malware sample on seven computersAvllazagaj and his colleagues looked at the behavior of several malware strains, including Ramnit, a worm that affects Windows machines intending to steal information, and DarkComet RAT, which can also steal passwords and take screenshots.By analyzing those 7.6 million samples captured in 2018, the researchers noticed that much of the variability had to do with file creation and naming. That was not all. The DarkComet RAT created Registry keys during some executions, while Ramnit built a lot of Mutexes at times, possibly because it kept running the exploit until it succeeded, the researcher says."For at least 50% of the malware, 30% of the actions observed in an execution will not appear in other machines," the researchers wrote in a paper. Also, half the malware samples only appeared to have 8% of parameters in common across all executions.Avllazagaj says that this proves once again that using sandboxes to analyze malware does not offer the whole picture. When security experts want to see if a particular sample belongs to a specific family, just one execution in one sandbox is not enough.The researchers concluded that security experts who analyze malware need several PCs to catch different behaviors. They can use "three to seven machines to collect parameter tokens that appear in more than 90% of the executions," the paper reads.Avllazagaj also says that variability across machines is higher than across time, so "analysts should re-execute the malware samples three weeks after first receiving them to update their behavior models."Why does malware change behavior?Sophisticated threat actors, such as state-sponsored groups, have enough resources to create custom tools that appear benign on most machines and only detonate if certain conditions are met. Malware authors do it mainly for stealthiness and persistence."Stealthiness as a reason has been seen mostly with very targeted pieces of malware," Kosinar says. The attacker starts with a small tool that performs a few initial checks on the infected computer to see if it's the right one. If it is, it extracts the malicious payload. "If this [initial] tool gets spotted or detected, it is much easier to replace than the full main malicious program," Kosinar says. "Thus, for most computers, the initial tool is pretty much benign\u2014it just self-destructs\u2014and the main one never reaches them at all."APTs are not alone in this. Ransomware groups employ such tactics, too, says Calvin Gan, senior manager of F-Secure's Tactical Defense Unit. "Ransomware nowadays is often deployed as a second-stage payload instead," he explains."There is also ransomware that checks its environment upon being executed and refuses to perform any changes if found to be in a test environment or virtual machine. Leveraging on this, there is also ransomware, Ragnar Locker, that deploys a virtual image to execute their payload to evade detection," according to Gan.Analyzing malwareIt's possible that we\u2019ll see more malware strains with variable behavior in the future, and the effects of this might reverberate across industries. Both state-sponsored groups and cybercriminals are polishing their skills.ESET's Kosinar says antivirus companies are closely studying all these groups and are employing a wide range of techniques when looking at the most unusual samples. The goal is to understand them to the tiny details. "This is usually done by a combination of static analysis in disassemblers and running it in some sandboxing environment, a virtual machine, or under a debugger," he says. "This kind of analysis is unlikely to be thwarted by the malware behaving differently in different runs -- the person doing the reverse engineering can see the actual program without needing for it to be executed so they can explore all the possible paths in the code."While this approach is time-consuming, it can help answer difficult questions such as: "How does [the malware] generate the filenames or registry entries?" or "How does it generate the URLs it connects to?" Kosinar adds.Understanding these questions and the broad range of malware behaviors seen in the wild might become vital in the years to come, as ransomware gangs and state-sponsored groups up their game. This knowledge will allow organizations to be more alert, protecting themselves against a false sense of security.