Thursday, November 15, 2007

In plain English

The post "A storm is brewing" was technical in nature, and was not particularly intended for the audiences it actually received, and as such, a lot of readers did not understand the items at issue.

I'll attempt to make clear and concise statements to help clear things up, and point to the real issues.

  1. Warden is a piece of software that Blizzard Entertainment uses to help protect World of Warcraft (WoW) from a world of cheaters and other perceived enemies, since its inception in a patch of the game on July 12, 2005.
  2. I am regarded as one of the most knowledgeable individuals outside of Blizzard Entertainment on the topic of Warden, and have first-hand knowledge of Warden through reverse engineering nearly every minute detail of the software since its inception.
  3. Warden as a whole is composed of three basic pieces: a piece on servers run by Blizzard, a piece in the World of Warcraft client that remains there until patched with the rest of the game, and a piece sent during the WoW login process that can also be replaced any time afterward
  4. The piece sent during the WoW login process is the piece generally spoken of as simply Warden (and this is the piece I will refer to as Warden hereafter)
  5. Warden is polymorphic. What this means is that they generally create one set of functionality, and create hundreds of non-identical copies (which I will refer to as permutations) of it that produce the same end result. The reason for being polymorphic is to make Warden marginally harder to circumvent, and harder to detect when Warden has been updated with new functionality.
  6. There is typically about 318 permutations of Warden in distribution at any given time, according to our tracking information. This may be different as of the last few days, as at present time, Blizzard is only rotating a single permutation into the wild every few hours. Bear in mind that can change at any time, and may go back to 318, or could literally be any other number bound only by Blizzard's computational power to produce them (without implying any such intent, WoW provides them with a lot of money, if they wanted to this could be a much bigger number than 318).
  7. Warden currently has roughly a dozen scans available to it. Each scan searches for one type of thing, typically being informed of a specific thing of that type to scan for upon request by the server. For example, one scan that was previously used is a scan that could find a window open on your computer, and that scan would be told to run and look for a window titled "My cheat program" (not really that specifically, but for an easy to understand example).
  8. Scan responses typically involve simply a YES or NO answer, for example a NO that it did not find a window titled "My cheat program". Other scan responses do involve bits of memory directly retrieved from the World of Warcraft process, usually not encrypted.
  9. Warden performs a set of scans at random every 15 seconds during World of Warcraft play, per instructions from the game server. The scans are run, and the results sent back to Blizzard.
  10. Warden is effectively useless the vast majority of the time. The process generally works by making the assumption that for some period of time after a Warden update (meaning one specific set of functionality consisting of any number of permutations, not an individual permutation), the scanning capabilities of Warden is unknown to the cheater, and furthermore that the time of the update is unknown to the cheater. During that period, any cheater unwise to the update is vulnerable. However, once it becomes known that Warden has been updated, and how to defeat it, cheaters are no longer vulnerable. Subsequently, during that period, Blizzard is the only entity that "knows" there is no concern for privacy, and customers are required to trust that.
  11. Warden updates have been tracked without Blizzard's assistance since early 2006. As such, any who cared to listen were notified of the update at the time of the update.
  12. On Tuesday, November 13, 2007, Warden was updated to include a new cryptographic (crypto for short) layer, presumably used to prevent man-in-the-middle attacks over network (something done by those who emulate the WoW network traffic in order to automate game play without running the World of Warcraft client software). The cryptographic layer works for that purpose solely because the algorithm is generated, presumably at random, per permutation, and is embedded into Warden. Warden itself is not encrypted as part of this layer.
  13. Prior to the new crypto layer's implementation, all permutations of Warden could be vetted by security researchers in one fell swoop, effectively verifying that all permutations of Warden did, in fact, contain the same functionality.
  14. Ironically, the world of cheaters are the ones tasked with making sure Warden is lawful, and notifying the rest of the World of Warcraft community when something isn't quite right. Consequently, the World of Warcraft community generally responds in favor of Blizzard, regardless of potential infringements of their rights, because they believe that Warden is becoming more effective by whatever is added to it.
  15. Before item #16 is read, I will reiterate that Blizzard has not, in my opinion and to the extent of my knowledge, broken laws with Warden's use in World of Warcraft. Nor do I believe they would knowingly and willingly do so.
  16. The new crypto layer's implementation creates a sort of vulnerability in the system, affecting users of the system, but of no concern to the creators of the system. Specifically, as this algorithm is produced at random per permutation with only the requirement that the server also be aware of the algorithm, it must be assumed that every permutation has a different implementation of the algorithm, and it doesn't make a bit of difference what the algorithm is. In the few copies I have reviewed, it is in fact a cryptographic hash algorithm, and the result is then used to re-key the encryption after sending a hashed copy of the key for verification by the server (the algorithm accepts random data from the server, and produces data that can only be predicted and verified by the server, without manually reverse engineering the permutation of Warden). The real problem is that this implementation can be exploited by Blizzard or an employee of Blizzard, at their sole discretion, with surgical precision if they so choose, to bypass any protective measures taken on behalf of the customer, and retrieve anything they may not be entitled to, even installing malware. There is essentially nothing stopping Blizzard from producing 100,000 permutations of Warden, slipping something unlawful into a single permutation, and slipping right through any network of researchers watching for just that.
  17. Typically this sort of thing is not an issue, as programs consumers purposefully come in contact with are not polymorphic, and it can be generally assumed that every copy of Windows Media Player 10, for example, is identical to the others. Security professionals can take their time in tearing it apart and letting people know if there is something to be afraid of. Warden, however, typically comes in hundreds of flavors, and the software routines are downloaded and executed in real time, and customers must not observe the behavior of those routines, as required by the game's End User License Agreement. This means that the customer is prohibited from viewing what Warden is doing, even if they have the knowledge to do so.
  18. While, again, I do not believe that Blizzard will knowingly and willingly break any laws, I do believe that the customer has the right to reverse engineer the software, if for no other reason than to verify that it does not violate privacy, install malware, and so on. Blind trust is a very good way to get taken advantage of, and you never know until it's too late.
  19. I regret that Blizzard is taking fire in a direct fashion for this, as I do not wish to make this specifically about Blizzard (although yes, I did call on Blizzard to promote transparency in their detection methodology, the issue as a whole goes well beyond Blizzard). I am not attempting to "fearmonger", nor do I see it as a positive thing that the original article was misinterpreted. I am also not raising this issue due to any implied difficulty in continuing to provide software that can hide anything from Warden (if you must know, my solution is waiting until I have solved this vulnerability for those that my software protects, and that solution will be available soon, but cannot address the greater issue).
  20. The issue that happens to affect Blizzard today, is likely to affect more corporations in the future, unless it can be legally curbed. It's a slippery slope, and although they may not be doing something wrong today in the opinions of many, Blizzard or similar corporations may continue dangerously down that slope and eventually the many may change their minds and become interested. With an End User License Agreement and Terms of Use that expressly prohibit research into their tactics, polymorphic code to help hide them, and now random functionality that makes it much more difficult to white list all of Warden (if you ask me what scans Warden has now, I can't tell you for certain), one must wonder exactly how far companies like this will go. Such tactics are usually reserved for malware to hide from anti-virus software! How much of our rights to know what information our own computers are sending out into the world do we have to give up, just to use software? What is stopping other companies from doing the same thing? Why would we trust other companies in the same situation Blizzard is in? In a world where corruption issues routinely make front page news, people need to realize that there are reasons new laws get made. We need to protect our rights as consumers, not blindly accept whatever agreement is thrown at us. Just because the EULA says something is prohibited does not mean they have the right to prohibit it.
  21. Besides, Warden effectively does little to solve the problem it intends to solve! Granted, Blizzard is able to ban some accounts from time to time, but they are showing that they do not need to have this software in order to ban accounts. Server-side detection mechanisms are more effective, as they cannot be subverted by client-side mechanisms, and cannot be discovered by reverse engineering the game client. Instead, server-side mechanisms would attempt to enforce humans playing the game, rather than bots, or attempt to enforce game mechanics where the player may modify them (e.g. by making himself move exceptionally fast). In this fashion, there is no inherent danger to anything on your PC, and quite frankly, if a cheater does not appear to be cheating to the other players, then clearly no harm is done to the game. But again, quite frankly, Warden does not greatly reduce the number of cheaters or botters, trust me they're still here, and in far greater numbers than when Warden was first implemented in World of Warcraft in July 2005. It does not reduce the real-money trade for in-game valuables, that's still here too, and likely in far greater numbers as well (though I don't have the data to back that up, I believe it is a growing industry) -- Server-side mechanisms are, however, at least somewhat effective there. If Warden is good at anything, it's simply delaying cheaters and botters by making them wait for protection, or it's good at putting money in Blizzard's (and Vivendi Games') pocket, because the majority of the accounts they ban end up coming right back and buying a brand new copy of the game, just to continue the cat and mouse game.
  22. Very nearly (if not exactly) the same level of effectiveness that Blizzard is sincerely offered by Warden can be gained without producing polymorphic code at all. Furthermore, removal of the polymorphic code would allow security researchers to ensure that customer data is safe, without blindly trusting Blizzard, to a much higher degree.
  23. I wish to reiterate that it's not my own data I'm concerned about. If this were about any implied difficulty in protecting myself from the system, I wouldn't even bother to blog about it. The problem is that I can no longer ensure the safety of other World of Warcraft players, including my own family, and I believe it is important for someone to do that. And again, not just for World of Warcraft, but for any software that seeks to use cryptic or secret methods to do their bidding.
I hope I have made it clear that I do not have an inherent mistrust of Blizzard as a whole, but while we can share the belief that Blizzard means well, Warden is not stopping people from cheating or botting, and there are precedents to be set here. We can't lay down and give up our rights, or our expectation and even verification of privacy, to companies just to use their software.

I apologize for not having links to back up various statements, such as #21, but with any amount of research, you can verify that the cheating and botting communities have not left the game, and with some 9 million subscribers, I don't think anyone will find otherwise.

Update: Just a few excerpts from F-Secure's Malware Code Glossary that show potential relationships between this type of software (Warden), and what companies like F-Secure aim to protect you from. This is not an indication that I believe Warden itself is clearly any of these things, but definitely is very close to the line if not, and again, this is not just about Blizzard and Warden, but about all current and future companies doing similar things.
  • Polymorphic Virus
A Polymorphic Virus is a virus which changes itself (mutates) as it passes through host files, making disinfection a serious challenge.
  • Rootkit
Rootkits are a technique that allows malware to hide from computer operating systems and from computer users. Rootkit techniques create stealth programs that run at a "lower" level than the user can see with normal software utilities. Malware attempts to use this method to avoid detection by security software.
  • Spyware
Spyware is software that performs actions such as creating unsolicited pop-ups, hijacks home/search pages, or redirects browsing results.

The term Spyware has been used in two ways: In its narrow sense, Spyware is a term for Tracking Software deployed without adequate notice, consent, or control for the user. In its broader sense, Spyware is used as a synonym for Spyware (narrow) and Other Potentially Unwanted Technologies.
  • Trojan
A Trojan or Trojan [Horse] is a software application with hidden destructive functionality. It is a program that appears to do one thing but actually does another.

Wednesday, November 14, 2007

A storm is brewing

Important note: A lot of people are misinterpreting this post because the details are largely technical. Please see the follow-up post "In plain English", as I believe I have covered most if not all of the points people are attempting to make after reading this article.

Coinciding with the most recent World of Warcraft patch (Tuesday, November 13, 2007), Blizzard has begun a more aggressive campaign with Warden. The changes to Warden effectively remove our ability as a community to police Blizzard's activities, and may lead to undetected violations of personal privacy, among other possibilities. I have until now publicly defended Blizzard's actions, which were already under public scrutiny, partly because of Greg Hoglund and his crusades (which I have never agreed with). I do not believe that Blizzard would ever intentionally break privacy laws (or any laws for that matter), at least in any manner that can be traced. However, as we all realize, there are gray areas, which Blizzard is no stranger to (I would consider Warden itself to be in that gray area, which does not seem to be illegal, but that many people would feel is a violation of their rights, and could potentially be deemed illegal in the future), and I do believe that Blizzard would enter those areas until legally bound to leave them (i.e. when the area is no longer gray, and consequences would follow).

I cannot condone or agree with the changes to Warden, and I fear they may be overstepping their bounds. The problem is that Warden has long been a polymorphic program, typically a concept used for viruses, spyware, and other sorts of things that an attacker may wish to hide (see the linked page from the words "polymorphic program", and take note of the described usages). In Blizzard's case, they intend to hide functionality of Warden from what they perceive as attackers, for the obvious reason of catching said attacker without him being tipped off as to how. Clearly, if said attacker knows how, he would attempt to avoid being caught. In itself, this polymorphism is not entirely destructive.

Historically, the polymorphic code produced essentially the same predictable results in the end, and Blizzard's Warden-related activity was kept in check by software like ISXWarden, and to some extent by Glider's Tripwire (at least in the ability to track how often and in what numbers a new Warden was produced, I'm unaware of any additional capabilities Tripwire may have). Unfortunately, Warden now includes a different random cryptographic hash function in every copy, apparently used for cryptographic key exchange, at least in the copies I have reviewed. However, it is nearly impossible to enforce that. The hash function could be replaced with a function that retrieves information from your computer at random (or even precisely defined information, including credit card numbers, or literally anything else) and sends it back to Blizzard, and to electronic enforcement systems, this would be nearly impossible to predict or report.

I formed my opinions of Blizzard's activities and stood on their side of the line on privacy violation arguments, solely because I have been able to automatically keep track of exactly what Warden was doing, how it was doing it, and what information was sent back to Blizzard, regardless of the number of permutations of their polymorphic software. This effectively resulted in checks and balances, much in the way government bodies separate their powers which I believe, in the end, are supposed to preserve the rights of the people in cases of corruption and such. Now, information suggests that Blizzard has begun continually producing replacement copies of Warden -- previously, roughly 318 permutations of Warden existed per patch (according to information from ISXWarden users, as can currently be viewed on the WardenNet stats page), and would be used on a rotating basis. To reiterate what I implied above, all 318 of those permutations could be vetted by software (including ISXWarden), and the behavior of each one could be verified to be identical. Therefore, anything that Blizzard would try to slip into their software was kept in check, and they would not have been able to introduce any significant privacy violations without alerting their customer base. That's actually a very good thing to have on their side.

However, this change to Warden is not a very good thing to have on their side. Given the fact that the randomly generated hash algorithm can be replaced at Blizzard's sole discretion with any other algorithm, including ones that retrieve and use personal, private and/or otherwise confidential information, with only their server to be required to know about the changes, this should be considered a very scary thing for the rest of us. Blizzard, I agree with you wanting to protect your game, I agree with most of the functionality you have placed in Warden, but you're losing a supporter who has conflicts of interest with your policies and still agreed with them, and that would have made a strong argument for your side.

Blizzard, I strongly urge you to promote transparency in your policing efforts. The public cannot be expected to trust a corporation that is hiding information from its own customers. You are governing several million people across the globe, and even though you do not like some of them, you should not attempt to hide your software or the functionality of your software on your customers' personal computers. There is absolutely no excuse for doing so, and I do believe that this is now, without a doubt in my mind, an ethical issue.

Digg it

I wish to clarify a few things, as this post has been read, mis-read, partially ignored, and so on.
There is no issue with Blizzard using a hashing algorithm, or encrypting data. There is no issue with Blizzard attempting to detect its perceived attackers. There is no issue with a key exchange in the detection software. It's not even about any implied difficulty by said attackers to sidestep the new functionality, which at face value, is not a difficult task. The issue is that the hash algorithm can be replaced with any algorithm. The issue is that the hash algorithm is different in every copy of Warden, so there's no simple method of ensuring that every copy of Warden is simply using a hash algorithm, and furthermore that it is one-way. The issue is that the detection software may be exploited, by Blizzard or an employee of Blizzard, with or without the corporation's knowledge, in order to do anything they please on your PC. A resourceful Blizzard employee could, for example, install a virus or other malware on your PC, and have a pretty high chance of that going undetected by the customer. This example may seem extreme, but bear in mind that all customers are required by Blizzard to blindly accept whatever Warden is doing on your PC. By discouraging independent analysis of their tools, Blizzard seems to have something to hide. While I will reiterate (from the first paragraph of the post) that I don't believe that Blizzard would knowingly and willingly break any law, I do strongly believe that Blizzard has a responsibility to show its millions of customers that it is taking these actions in good faith.

Finally, I believe this is an issue that affects not just Blizzard and their customers, but all present and future corporations and customers who may be attempting to hide this sort of process or information from their customers. There is a limit to what they can do, and we can't blindly expect Blizzard or any such company to follow those limitations if they are not being independently verified.