snortattack

This weekend I noticed a ComputerWorld article titled "Massive Android malware op may have infected 5 million users". After reading, it seemed to be exactly the sort of thing many people have been suggesting - an increasingly large-scale outbreak of malicious activity in the Android market, as malware authors saw larger numbers of potential targets. After forwarding the article to my VRT colleagues, we quickly got copies of the few apps that Google hadn't already pulled - specifically, the files com.redmicapps.puzzles.ladies2_v1.02.apk, com.redmicapps.puzzles.ladies3_v1.02.apk, and com.christmasgame.deal_v1.0.1.apk. Eager to see how bad the damage was, we sat down to analyze them yesterday morning.

Some initial static analysis by Alain Zidouemba seemed to confirm what Symantec was saying in its writeup. Not only did the URLs mentioned there appear in the code, the testGetUserID() function pulled the exact information listed in the Symantec writeup:



Dynamic analysis inside the Android emulator was equally promising at first. Within seconds of installation, an HTTP POST to http://www.apperhand.com/ProtocolGW/protocol/commands appeared, and a slew of data was sent off in plaintext:

○ {"initiationType":"first time","needSpecificParameters":true,"applicationDetails":{"abTests":null,"applicationId":"212546654","build":{"brand":"generic","device":"generic","manufacturer":"unknown","model":"sdk","os":"Android","versionRelease":"2.3.3","versionSDKInt":10},"developerId":"987550925","deviceId":"wCxwXphYj3JMoEasWcr+zmVQHjY=","displayMetrics":{"density":1.5,"densityDpi":240,"heightPixels":800,"scaledDensity":1.5,"widthPixels":480,"xdpi":240.0,"ydpi":240.0},"locale":"en_US","protocolVersion":"1.0.6","sourceIp":null,"userAgent":"Mozilla/5.0 (Linux; U; Android 2.3.3; en-us; sdk Build/GRI34) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"},"parameters":{}}

The response that came back made it clear that a unique installation was being tracked:


{"commands":[{"id":"c2d71967-b6a1-451f-9d01-aa91adbfc0d1","parameters":null,"command":"ACTIVATION"}],"commandsInterval":15,"parameters":{},"abTest":"6a13d5ca-f5c7-4805-a12b-c70a4953bb6e","validResponse":true}

Subsequent requests to this URL showed a successful activation, and later returned a "SEARCH URL" of "http://www.searchmobileonline.com/{$CATEGORY$}?sourceid=7&app=4Ek2WZkCbw1Yw9VS%2F6q9D8zE67pPruhMY4SiC6pvyUzqgGNpf%2FjIrlCBA7Bp03eF9wSiv%2FHkJK%2FvkoMTkeCPaA%3D%3D&q={$QUERY$}", which again backed up the Symantec threat report. Figuring we were on to something, we let the malware run, interacting with it as a normal user would (i.e., by playing the games).

The problem was, nothing all that interesting ended up happening. Across the two distinct applications (the two "ladies" puzzles behaved essentially identically), nothing more than a series of advertising-related requests occurred in the background:

• A POST to http://data.flurry.com/aap.do with some device information, which a quick Google search showed was related to mobile ads


• Several POSTs to http://www.jigsaur.com/index.wsgi?method=CheckForReward that contained snippets of data that appeared to be related to the game in progress; visiting the Jigsaur.com home page redirected us to a jigsaw puzzle game on the Android market


• A POST to http://www.umeng.com/app_logs that contained data about the Android version, country and timezone of the user, etc.; this appears to be related to a Chinese mobile analytics tool


• GET requests for URLs on mobclix.com and googleads.g.doubleclick.net, which both returned advertising content

At this point, we started to wonder whether these apps were really malicious, or just using obnoxious ad networks. Going back and re-examining the data from the Apperhand.com requests, we realized that the data was essentially all information an ad provider would find useful: everything from device version to screen size, and an ID that would be useful to track which host was clicking on which ad. From there, we started doing some digging to understand what exactly happens in the world of mobile advertising, and whether this was out of the norm.

We quickly ran across a blog post from Mobclix - one of the advertisers we'd seen in the packet captures - discussing the need for unique IDs in targeted ads. That rendered the most suspicious piece of the data largely moot in our opinion.

Digging a little further, we noticed that Lookout Mobile Security had actually put together a blog post the same day as the ComputerWorld article, refuting Symantec's claim that Counterclank was malware and insisting it was simply obnoxious adware. For instance, they noted that sending off IMEI data - used to uniquely fingerprint modern mobile phones - is common across multiple networks, but that the Apperhand SDK used by Counterclank actually went to the trouble of hashing that value before sending it along to preserve privacy. Looking at the code, this was easy to confirm:



The Lookout report, like Symantec's, noted that Apperhand appears to be based on another SDK known as Plankton, which was much less concerned with user privacy and much more pushy about its ad behavior. This seems to be confirmed by the way these new files are detected by some AV vendors: of the 11 vendors listed in the VirusTotal report that detect the "ladies3" app, four call it some variant on "Plankton", three a variant of "Counterclank", and the remaining four have miscellaneous or generic labels for it. Given that all the reports we've seen thus far describe Apperhand as a kinder, gentler version of Plankton, it seems likely that that SDK may simply be an advertising setup that's not good at respecting user boundaries.

So what's the VRT take on this, you ask? We think it's pretty clear you don't want any of these apps on your phone. Not only do they send out data that may make you uncomfortable to ad networks you have control over, they're actually poorly written anyway - for example, the "Deal & Be Millionare" game didn't even bother to rotate for proper screen orientation:



That said, we think this falls into they gray area of "crappy adware" more than being outright malware. This SDK certainly could use further scrutiny, as do any of the other more pushy advertising setups on mobile phones. Barring further evidence, though, it just doesn't seem to rise to the threshold of other apps that commit SMS fraud, have clear command and control channels, etc.

Since we're not the final authority on the matter, though, we're going to make it easy to decide for yourself how you feel about these apps. We're providing a signature in today's SEU that will detect the POST requests sent to the Apperhand site, and we encourage you to take a look at our packet captures (here and here) to see the raw data yourself. There's also ClamAV coverage under the name Andr.Plangton-12. After all, it's your network, so why shouldn't you be able to defend it, even from crappy adware?

Rep. Mike Rogers (R-MI) and Rep. Dutch Ruppersberger (D-MD) know a secret:  The Federal government is REALLY good at watching people, much better than, say, the private sector.  So they asked themselves (at least they did in my mind), "Why not share some of that information in order to protect American businesses from the ubiquitous cyber-security threat?"

Hey guys…that’s a damn good idea!

Seriously, I thought it was a great idea.  So it was with a good deal of enthusiasm that I printed out H.R. 3523, or to use its more sexy name, the “Cyber Intelligence Sharing and Protection Act of 2011”.[1]  There are only 11 pages, a lot of it standard language stuff, but it essentially lays out that the governement can share with the private sector and vice versa.  Of course, it's never that simple.  For example, the NSA can only share with cleared organizations that can demonstrate they know how to handle classified information.

There is also the small matter of the following statement from the proposed legislation:  "classified cyber threat intelligence may only be … shared consistent with the need to protect the national security of the United States.”  Which, of course, leaves one giant question:  What, exactly, constitutes a threat to national security?

There are, of course, the obvious…terrorists, nuclear proliferation, hostile foreign nations, and the like.  But that isn’t what Rogers and Ruppersberger are thinking here.  They are, according to Mike Rogers, targeting “economic predators, including nation-states, [that] are blatantly stealing business secrets and innovation from private companies.” [2] So we aren’t talking missiles, bombs and airplanes, we’re talking, potentially, about contract negotiations, natural resource surveys and customer lists.

A recent report [3] by the Office of the National Counter Intelligence Executive (ONCIX) states that “Losses of sensitive economic information and technologies to foreign entities represent significant costs to US national security.”  Clearly, this administration, and apparently this congress, are adopting the position that jacking with U.S. companies jacks with the national security.  Given the nature of the world today, I think they're right to do so.

I know...I'm not well known for staunchly backing the ideas of legislators or administrators.  You wouldn't be blamed for thinking I’m a cynical, pessimistic nutter who lived by himself in a wooden hut, eating nothing but pickled ginger and gummy bears while spending his day ranting about the overly generous nature of most computer networks.[4]  But this time -- and I do have trouble saying this -- I think they’re on to something.  The private sector just isn't in a position to match the federal government's ability to generate intelligence.  In fact of all the things the government could provide in the forms of mandates, laws, policies, rules, reporting requirements, CISSP factories, etc... intelligence is really the only thing that makes sense.  It's the only thing that they can provide that industry can't legitimately generate itself.  I think this is a really good piece of legislation.

Of course, there are lots of ways to screw it up, and I'm sure that some of those ways will be found.  But if we get into the habit of having the government share information and letting organizations figure out how to act on the information, we'll be headed down a very good path.

[1] http://www.gpo.gov/fdsys/pkg/BILLS-112hr3523ih/pdf/BILLS-112hr3523ih.pdf
[2] http://dutch.house.gov/2011/11/ruppersberger-rogers-introduce-cybersecurity-bill-to-protect-american-businesses-from-economic-preda.shtml
[3] http://www.ncix.gov/publications/reports/fecie_all/Foreign_Economic_Collection_2011.pdf
[4] And nothing in this blog post would prove you wrong…

Security never sleeps, even if it is the week between Christmas and New Year's, and most of you are on vacation, enjoying time with your family, or just goofing off because the office is empty. Today's reminder of that reality comes from Alexander Klink and Julian Walde, who presented yesterday at the 28th Annual Chaos Communication Congress a method of consuming a web server's entire CPU with a simple, low-bandwidth POST request. In fact, according to the advisory they released after the talk, as little as 30k/sec could be necessary to occupy a single i7 core, depending on the target platform.

While the details of the attack are complex and vary from one target platform to another, the essence of it is that if you can send a large number of key/value pairs where the keys cause collisions in the receiving system's hashing algorithm, each colliding key will consume exponentially more CPU time to parse than the last. This makes for fairly straightforward detection in Snort - exceptionally large numbers of key/value pairs are necessary to trigger the bug, and so it's a matter of counting them up in a given request.

We've released SIDs 20823 and 20824 in an SEU late last night to cover this vulnerability.

For more information on this vulnerability check out the MSRC blog post here. The VRT Snort rules for detecting this vulnerability are discussed in their blog post.

We are working on the additional issues patched in MS11-100, and will provide coverage for those shortly.

The malware sandbox that I've previously discussed on this blog has made for a lot of useful Snort rules - but it's also helped get me some excellent speaking slots around the world this year. This time, I've just wrapped up a presentation titled "Malware Mythbusting" at Ruxcon, Australia's premier technical security conference.

The premise of the talk was simple: there's a lot of hype surrounding malware, and if you're someone tasked with keeping a network secure, there's generally not a lot of good information about the nature of the threat. Can I cut off China and Russia and make all the C&C servers go away? Are spambots really a major threat, or has garden-variety malware moved on? Are the people writing malicious software a bunch of evil geniuses, or can a little bit of diligence and attention locate heaps of nasty behavior on the network?

While I don't claim to have all the answers - no one does - I hope to have done a reasonable job of answering some of these questions during this talk. For those of you who didn't have the chance to make it down here - and for those who did that want to take a closer look at some of the data presented - I've made my slides available here. As I noted in the talk, if you have questions that it left unanswered, or if you're interested in working with us on malware research, drop the VRT a line - we're happy to collaborate with anyone who has good ideas. After all, at the end of the day, we're all on the same team here, and anything that can be done to clean more malicious software from the Internet is a good thing, regardless of the source.

Microsoft recently added a new initiative to its Microsoft Active Protection Program (MAPP), called the Advisory Initiative program, which gives partners up to 96 hours to provide protection for discovered vulnerabilities. Microsoft piloted the program with an advisory release on the Win32K TrueType font parsing engine, related to the Duqu malware (CVE-2011-3402). Sourcefire released its protections for this threat within the first 48 hours, as noted on the MAPP site http://technet.microsoft.com/en-us/security/advisorymapp:

SID: GID 3, SID 20539
http://labs.snort.org/papers/ms/immediate-response.html

Duqu exploits a vulnerability in Windows in the way it parses TrueType fonts and it can create an open tunnel into a user's computer. Then attackers have the freedom to gain full system access and run arbitrary code and modify data, install applications, and, essentially, use the system as the user would. This flaw, for which Microsoft previously issued a workaround, is exploitable across many Windows platforms. Despite this, Microsoft reports that they are currently seeing low customer impact at this time.

More information, as well as other vendors who responded within 48 hours, can be found on the MAPP program web site.

While mobile malware comprises only a tiny fraction of the overall landscape in terms of volume, it is fast becoming essential to address from an enterprise security standpoint. Unfortunately, very few people would even have a clue where to start if charged with analyzing a program on a smart phone. This disconnect provided the rationale for a presentation I recently gave at Hack in the Box Malaysia on how to go from "I've got an Android APK file, now what?" to full static and dynamic analysis.

The slides, available here, contain links to a number of useful tools. The good news for longtime readers of this blog is that the process is even easier now than it was when Alain Zidouemba discussed reversing Android apps last August. Free software is available that can deliver the original Java source for any given Android app. My presentation also provides an overview of the Android permissions system and its relevance to static analysis, as well as some example packet captures from in-the-wild malicious apps.

One useful piece of advice remains the same since Alain's original analysis, however: the vast majority of malicious apps come not from the Google market but from third-party package distribution sources. We're not saying that you shouldn't ever pull an app from outside the market, just that you should do your homework before you do.

This week we are introducing a new rule category into the VRT rule set, named "file-identify.rules". The purpose of this category is to standardize the structure of rules that “set” a flowbit and to enhance detection by looking into file data. The changes will occur in two stages.

Stage 1. The creation of a series of rules that detect the "magic" in files, probably around 70 to start, with more being added as time passes and needs arise. For example:

alert tcp $EXTERNAL_NET $FILE_DATA_PORTS -> $HOME_NET any (msg:"FILE-IDENTIFY PNG file magic detection"; flow:to_client,established; file_data; content:"|89|PNG|0D 0A 1A 0A|"; within:8; fast_pattern; flowbits:set,http.png,fileidentify; flowbits:noalert; classtype:misc-activity; sid:20478; rev:1;)

In this example, the magic at the beginning of the file is detected (the "|89|PNG|0D 0A 1A 0A|”) and the flowbit is set for this particular file type. This will allow a flowbit to be set for file types based on the data in the file and not the file extension in say a URI. For example, if a rule looks for “.jpg” in the URI and sets the “http.jpg” flowbit to track the download for the image requested, but the file is actually a PDF with a .jpg extension, then further detection based on the setting of this flowbit could lead to false positive events at best and false negative events at worst.

Stage 2. Move all URI checks for file extensions over to "file-identify". A lot of work has been done to cleanup these rules. They now have a well defined and consistent structure, with references, flow, message, detection, classtype and pcre options all standardized.

For example:

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"WEB-CLIENT .hta download attempt"; flow:to_server,established; content:".hta"; nocase; http_uri; pcre:"/\.hta(\b|$)/Ui"; flowbits:set,http.hta; flowbits:noalert; classtype:not-suspicious; sid:3551; rev:4;)

Now reads:

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY HTA file download request"; flow:to_server,established; content:".hta"; nocase; http_uri; fast_pattern:only; pcre:"/\x2ehta([\?\x5c\x2f]|$)/smiU"; flowbits:set,http.hta,fileidentify; flowbits:noalert; reference:url,en.wikipedia.org/wiki/HTML_Application; classtype:misc-activity; sid:3551; rev:5;)

And rules like this:

alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"WEB-CLIENT GIF transfer"; flow:from_server,established; content:"image/"; nocase; http_header; pcre:"/^Content-Type\x3a(\s*|\s*\r?\n\s+)image\x2fgif/smiH"; flowbits:set,http.gif; flowbits:noalert; classtype:protocol-command-decode; sid:3535; rev:9;)

Have been changed (or eliminated in this case) and have been split into two:

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY GIF file download request"; flow:to_server,established; content:".gif"; nocase; http_uri; fast_pattern; pcre:"/\x2egif([\?\x5c\x2f]|$)/smiU"; flowbits:set,http.gif; flowbits:noalert; classtype:misc-activity; sid:17394; rev:2;)

alert tcp $EXTERNAL_NET $FILE_DATA_PORTS -> $HOME_NET any (msg:"FILE-IDENTIFY GIF file magic detection"; flow:to_client,established; file_data; content:"GIF8"; within:4; fast_pattern; content:"a"; within:1; distance:1;  flowbits:set,http.gif,fileidentify; flowbits:noalert; classtype:misc-activity; sid:20459; rev:1;)

Over the course of the next week, these changes will be made to the rule set, and a new variable will be introduced in the snort configuration file:

portvar FILE_DATA_PORTS [$HTTP_PORTS,110,143]

Following these two introductions, the structure and formatting of all the flowbit names will be standardized. For example, replacing names like “http.gif” with “file.gif”, will reflect more accurately what is being detected.

Action items for you:

#1. You'll need to add the above variable to your snort.conf, use the snort.conf in the VRT tarball, or download the new snort.conf .
#2. If you are using the Sourcefire product, or PulledPork, the change should be minimal. The Sourcefire product and PulledPork perform flowbit auto-enabling and resolution. If you are using another tool to mange your installation, you will need to pay attention to this rule category.

Upon hearing of the release of THC SSL DoS tool, we decided to download it and look at it in our lab. The idea was intriguing and we were curious to see it in action.

If you are unfamiliar with the method utilized, the THC SSL DoS tool seeks to issue a Denial of Service (DoS) against hosts that offer SSL/TLS encrypted services. Unlike SSL flooding techniques of the past, this attack does not do this with rapid connections. Instead it makes a small number of connections and then rapidly renegotiates the SSL handshake inside those same connections.

The problem is that an attacker no longer needs a large amount of bandwidth or to mount a distributed attack to be able to successfully perform an SSL DoS attack. By utilizing a single SSL connection to a server, thousands of SSL handshake renegotiation requests can be performed very quickly.

To quote THC:
"Traditional DDoS attacks based on flooding are sub optimal: servers are prepared to handle large amount of traffic and clients are constantly sending requests to the server even when not under attack.

The SSL-handshake is only done at the beginning of a secure session and only if security is required. Servers are _not_ prepared to handle large amount of SSL Handshakes." So, what can Snort do for you? We knew that with the default configuration on Snort's SSL preprocessor we were not going to see the renegotiation happening. The reason is that once a successful SSL connection is made, without an SSL decryption appliance (Sourcefire sells them), Snort will ignore the rest of the conversation - the logic being that, since it's now encrypted, we can't do any detection on the traffic anyway. However, all hope is not lost. If you are in a position in which you need to detect this, there is a way. This detection behavior is controlled by the SSL preprocessor option "noinspect_encrypted"; removing that keyword will cause Snort to continue inspection after a session goes encrypted.
So, what next? First, let's look at the SSL/TLS record layer content types:

Hex Dec Type
0x14 20 ChangeCipherSpec
0x15 21 Alert
0x16 22 Handshake
0x17 23 Application

Since the attack utilizes handshake renegotiations, we are interested in the Handshake (0x16). Now we are interested in the two bytes of SSL/TLS version information:

Major Minor Version Type
3 0 SSL 3.0
3 1 TLS 1.0
3 2 TLS 1.1
3 3 TLS 1.2

So, if we take the content type, major version, and minor version within the first three bytes, we can get a pretty decent match. Next we sprinkle in a detection_filter to track the number of renegotiations, and finally we remove the noinspect_encrypted from the SSL preprocessor, and there it is ... SSL negotiation DoS detection.

This isn't a configuration I would recommend unless you've got a good reason because there will be a performance penalty. However, if you need it, you've got it. Run the rules that match your environment, adjust the ports as needed, and tweak the detection_filter to taste. The rules will be released in the next SEU.

This blog post has been brought to you by the letters V, R, T.

alert tcp $EXTERNAL_NET any -> $HOME_NET [443,465,587,995,993] (msg:"DOS multiple SSLv3 Encrypted Handshake messages - THC-SSL tool, potential DoS"; flow:established,to_server; ssl_state:!client_hello; content:"|16 03 00|"; depth:3; detection_filter:track by_src,count 25, seconds 2; reference:url,www.thc.org/thc-ssl-dos/; classtype:attempted-dos; sid:20436;)

alert tcp $EXTERNAL_NET any -> $HOME_NET [443,465,587,995,993] (msg:"DOS multiple TLSv1 Encrypted Handshake messages - THC-SSL tool, potential DoS"; flow:established,to_server; ssl_state:!client_hello; content:"|16 03 01|"; depth:3; detection_filter:track by_src,count 25, seconds 2; reference:url,www.thc.org/thc-ssl-dos/; classtype:attempted-dos; sid:20437;)

alert tcp $EXTERNAL_NET any -> $HOME_NET [443,465,587,995,993] (msg:"DOS multiple TLSv1.1 Encrypted Handshake messages - THC-SSL tool, potential DoS"; flow:established,to_server; ssl_state:!client_hello; content:"|16 03 02|"; depth:3; detection_filter:track by_src,count 25, seconds 2; reference:url,www.thc.org/thc-ssl-dos/; classtype:attempted-dos; sid:20438;)

alert tcp $EXTERNAL_NET any -> $HOME_NET [443,465,587,995,993] (msg:"DOS multiple TLSv1.2 Encrypted Handshake messages - THC-SSL tool, potential DoS"; flow:established,to_server; ssl_state:!client_hello; content:"|16 03 03|"; depth:3; detection_filter:track by_src,count 25, seconds 2; reference:url,www.thc.org/thc-ssl-dos/; classtype:attempted-dos; sid:20439;)

Yesterday we released Razorback 0.3, the result of the Q3 development run.  Q3 focused on building out the scripting nugget, reworking how the Snort-as-a-Collector nugget works and building out a VM image so you can easily tryout the Razorback system.

The scripting nugget is a huge addition to Razorback.  The scripting nugget uses XML across named pipes to pass registration, alerting and logging information back to the system.  This allows the use of any scripting (or even compiled) language that can pass XML out STDOUT with Razorback.  We ship a ruby gem that makes writing detection scripts fairly straightforward as well as a sample ruby nugget.

The scripting nugget calls each script on startup with the --register argument.  This causes the scripts to output their registration information and the script nugget then registers on their behalf.  The scripting nugget then handles retrieving data blocks and calling the nuggets when they are needed for detection.  The scripting nugget then parses the alerting and logging output and uses the standard C API to alert and log on behalf of the scripts.  Finally, the scripting nugget is constantly watching the scripts directory, so adding detection to a running system is as simple as copying a new script into the directory.

There have been a couple of versions of Snort released since we initially built the SAAC and there were some lingering issues we wanted to clean up, so the Amish Hammer sat down and basically rewrote it from the ground up.  The shipping version is now based on Snort 2.9.1.1, has better memory management and is fully integrated with the current API allowing for the data block captured to have the request information attached to it.  Basically this means that for any given captured data block, we have all the information about how it was requested:  hostname, URI, IP addresses, ports etc...  Very useful for forensics work.

Finally, we have built out a FreeBSD based virtual appliance so you can easily bring up and interact with a Razorback installation.  The system comes pre-configured witha ll of the sub-components requried for Razorback to run:  memcached, MySQL and ActiveMQ.  In addition, it provides the following nuggets:  Yara, OfficeCat, ClamAV, Archive Inflate, Scripting, File Inject and a Snort-as-a-Collector nugget.  Provided you have an API key you can also enable the Virus Total nugget and if you have a license, you can activate the PDF Dissector nugget.

Beyond all this are various and sundry bug fixes, performance enhancements and usability improvments.

You can find the source code for 0.3 here:
https://sourceforge.net/projects/razorbacktm/files/Razorback/razorback-0.3.0.tbz/download?source=files

You can find documentation on the VM here:
https://sourceforge.net/apps/trac/razorbacktm/wiki/Manual/Virtual_Machine

You can find the VM itself here:
https://sourceforge.net/projects/razorbacktm/files/VM/

Enjoy!

If you pay attention to the list of new rules in each SEU, you've probably noticed us adding a lot of malware rules lately. While on the surface it may appear that we're just picking random samples out of the millions of different pieces of malware available on the Internet, there's actually a method to our madness that's worth explaining here, to help you make the best possible decisions on which rules you want to enable in your environment.

Outside of cases where we're asked to provide coverage for a specific piece of malware, our primary goal whenever we add a new rule is to cover more than just one sample with any given rule. After all, if there was a 1:1 ratio of rules to malware, we'd end up writing hundreds of thousands of rules and still only touching the tip of the iceberg in terms of total detection - whereas if we can write a rule that catches thousands of different pieces of malware, we can provide much more useful detection in a much more manageable way.

A good example of this principle in action is SID 20232, released in SEU 507:

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BOTNET-CNC Trojan Win32.Cycbot outbound connection"; flow:to_server,established; content:"?v"; http_uri; content:"tq=g"; distance:0; http_uri; content:"User-Agent|3A 20|mozilla/2.0|0D 0A|"; fast_pattern; http_header; pcre:"/(gif|jpg|png)\x3fv\d{1,2}\x3d\d{1,2}\x26tq\x3d/U"; metadata:policy balanced-ips drop, policy security-ips drop, service http; reference:url,www.virustotal.com/file-scan/report.html?id=01fabe4ad1552f4d61b614a319c90b33a6b6b48c5da63965924b687e3f251ca8-1316273623; classtype:trojan-activity; sid:20232; rev:2;)


The analyst who wrote this rule was initially investigating the piece of malware named in the message string specifically, with the string "jpg?v" as a key piece of detection. However, when he began digging through our malware sandbox for samples to test his initial rule with, he realized that a very large number of samples could be detected if he were to broaden his search to look for either "jpg?v", "gif?v" or "png?v" - 3,856 in just the month of September 2011, to be specific. Since relying solely on a five-byte URL match could easily produce false positives, he analyzed several samples by hand, and was able to add the other checks in the rule to keep false positives at bay while still detecting a huge amount of malware. Amazingly enough, that rule will detect 122,630 distinct samples that have run through our sandbox since the start of 2011!

While cases like this are great from a detection perspective, they present a bit of a challenge from a metadata perspective. We can't just have a rule message like "BOTNET-CNC this rule is awesome it finds lots of malware", nor could we possibly list all the different pieces of malware this one rule catches in the message string. The same principle applies to rule references - leaving them out altogether isn't useful, and we can't add references for all the different malware the rule catches. Using data from the targeted piece of malware, or the one that the rule catches most frequently, is a compromise that gives users some idea of what the rule is doing, while still retaining sanity in terms of size.

So the question that users face is, "how do I know when a rule is really useful like this, vs. something more targeted and less broadly applicable?". The answer comes from the default policies that a rule is placed into. If a rule will catch a large amount of malware, and do so without significant false positives or performance problems, we'll place it into the balanced-ips policy. Rules that run more slowly, may generate some false positives, but will still catch more than one piece of malware at a go end up in the security-ips policy. Cases where a rule isn't broadly applicable are not placed into any of the default policies.

Of course, we've received word from multiple customers that they simply enable the entirety of the BOTNET-CNC, BACKDOOR, and BLACKLIST categories with little to no trouble, and plenty of valid detection. Your mileage may vary, of course - but if you're having problems with malware on your network, it may be worth a look. :-)