The contemporary antivirus industry and its problems

The Internet today is a breeding ground for criminal activity. Home users, small and medium businesses, international corporations and governmental bodies all suffer from constant attacks by viruses and Trojans. The reasons why the Internet is in this condition have been widely discussed, and will continue to be discussed. But what do I meant when I say that the Internet is a fertile environment for crime? At bottom, it means that money is being made illegally by creating and distributing malicious programs, which will:

• steal personal and corporate bank account information
• steal credit card numbers
• conduct DDoS attacks, with the instigators then demanding money to stop the attacks – a cyber racket)
• create networks of Trojan proxy servers. These can be used to send spam, and for commercial gain
• create zombie networks, which can be exploited in multiple ways
• create programs which download and install adware to the victim machine
• install Trojan dialers which will repeatedly call pay services
• etc.

It’s difficult to say exactly how widespread criminal activity is. I think that there are dozens, if not hundreds of hacker groups and individual hackers active in the computer underground. The hackers who belong to groups can probably be numbered in the thousands – this is according to the law enforcement agencies of most computerized countries. Over the last few years several dozen hackers and hacker groups have been arrested, and the total number of arrests topped several hundred. However, this doesn’ t seem to have had any real effect on the number of viruses and Trojans.

Another figure which can only be guessed at is the total turnover of the computer underground. Published sources estiamte that between 2004 and 2005 hackers either stole or scammed several hundred million dollars. As the vast majority of cyber criminals have not been arrested or imprisoned, we can assume that the annual turnover is probably billions of dollars. (This figure may well exceed the annual turnover of antivirus companies – for these figures, see below.)

The total damage done to the world economy by the activity of virus writers, hackers and spammers has long since exceeded tens of billions of dollars annually. The amount continues to grow. According to research carried out by Computer Economics, total losses in 2004 were close to $18 billion, with a trend towards a 30 – 40% annual growth rate.

Let’s take a look at the players in the world of cyber threats:

• Virus writers and hackers are creating and distributing viruses and Trojans for their own reasons
• End users’ machines and networks are under constant threat of hacker attacks, and may often fall victim to co-ordinated attacks
• Police and law enforcement bodies throughout the world are only partially successful in investigating and prosecuting cyber crimes
• Antivirus companies create software to counteract cyber threats

There’s been a great deal written about viruses, hackers, and those who hunt them down – there have even been Hollywood films made on the subject. The developers and vendors of antivirus solutions use their web sites to publicize their achievements. However, there isn’t much information about the problems which the antivirus industry faces. This article, therefore, aims to address this topic and, to some extent, rectify the imbalance.

A short overview of the antivirus industry

To start with, let’s take a look at the companies manufacturing standard solutions which protect against computer viruses. (We’ll discuss dedicated solutions and tools a little later in the article.) By standard solutions, I mean software for desktops, file servers, mail servers, and the perimeter of corporate networks.

The total market for such standard solutions was estimated as being $2.7 billion in 2003 and $3.3 billion in 2004, with $3.8 billion being the predicted figure for 2005. (All information in this section is taken from IDC, 2005). All antivirus manufacturers are divided into 3 groups; industry leaders, second tier companies, and others (those which have no significant effect – if any – on the antivirus landscape).

The leaders include Symantec, McAfee (NAI) and Trend Micro – the activity of these companies affects all markets:

These three companies occupy leading positions in all markets, with a few exceptions (for instance, Trend Micro dominates the Japanese market). Symantec and NAI (McAfee) are North American. Trend Micro is originally a Taiwanese company which was floated on the Japanese stock market. It is currently headquartered in the USA.

The second tier includes companies whose turnover is significantly lower than the leading three. However, these companies still have an annual turnover of tens of millions of dollars:

Kaspersky Lab, based in Russia, is also included in this group. However, the company does not disclose financial information.

The majority of second tier companies have a significant presence in their respective domestic markets, but a relatively small presence in foreign markets. For instance, Sophos is most successful in the UK, Panda in Spain, F-Secure in Scandinavian countries etc.

The third group includes several dozen antivirus companies. The best known include:

• Alwil – Awast (the Czech Republic)
• Arcabit – MKS (Poland)
• Doctor Web – DrWeb (Russia)
• ESET – NOD32 (Slovakia)
• Frisk Software – F-Prot (Iceland)
• GriSoft – AVG (the Czech Republic)
• H+BEDV – AntiVir (Germany)
• Hauri – VI Robot (South Korea)
• SoftWin – BitDefender (Romania)
• VirusBuster – VirusBuster (Hungary)

The third group also includes UNA and Stop! (both Ukrainian), Rising and KingSoft (China) and others.

The majority of companies in this group do not disclose any financial information. However, some estimates state that annual turnover is around $10 million.

This information above gives a breakdown of antivirus companies’ market share. However, companies offering products based on licensed technologies aren’t included. Examples are the German company G-Data, whose antivirus solution is based on Kaspersky Lab and SoftWin technologies, and Microsoft, which offers a multi-engine solution developed by Sybari.

There are also some non-standard types of antivirus protection, some of which are relatively specialized. This includes systems which will delete any potential threat from corporate email messages (the end user receives only messages without executable attachments or html scripts), systems which will launch the web browser within a virtual machine etc. There are also some programs which are fairly similar to antivirus solutions: software which protects against DDoS attacks, patch management software etc. However, none of these can be called fully functional antivirus products.

Problems of the antivirus industry

What problems might the antivirus industry be facing, apart from the market headaches which plague any manufactuer of consumer goods. We all know that viruses exist, and so do antivirus solutions. It might seem that antivirus solutions are a standard consumer product – one solution barely differs from the next. Users choose their product according to design, or marketing, or for some other non-technical reason. Given this, an antivirus solution is, in theory, just another consumer product, like washing powder, toothpaste, or cars.

Unfortunately (or perhaps fortunately) this is not the case. Users often chose an antivirus solution for its technical characteristics, and these differ widely between products. Users often focus on whether or not a specific product protects against a specific type of cyber threat, and the overall level of protection offered.

An antivirus solution should be able to protect against ALL types of malicious program. The better the antivirus solution, the happier users and system administrators will be. Anyone who doesn’t understand this in theory will very soon be faced with the practical consequences; without a good antivirus solution, someone can start stealing money from the user’s bank account, or the computer may start dialing phone numbers of its own accord, leaving the user to wonder why outgoing traffic has increased so much. Given this, users should have some idea of what protection is offered by antivirus solutions, so that an informed choice can be made.

Let’s say that antivirus solution X detects, let’s say, 50% of all viruses currently circulating on the Internet; product Y detects 90%, and product Z, 99.9%. N number of attacks will result in either the computer’s integrity being maintained, or the system becoming infected. If the computer is attacked 10 times, then the likelihood of product X failing to detect a malicious program is virtually guaranteed; product Y is more than likely to fail to detect the culprit; and in the case of product Z, the danger is almost infinitesimal.

Unfortunately, there are relatively few products available in shops or on the Internet which offer even close to 100% protection. The majority of products are unable even to guarantee 90% protection. And this is the main problem facing the antivirus industry today.

Problem #1

The number and variety of malicious programs is increasing year on year. The result is that many antivirus companies are simply unable to cope with the onslaught and are losing this ‘virus arms race’. Users who chose products manufactured by such companies will not be protected against all malicious programs. Unfortunately, this may be a large number of users, as a lot of products marketed as ‘antivirus solutions’ shouldn’t really be called this at all.

Incidentally, five or ten years ago, it could honestly be said that an antivirus solution didn’t need to protect systems against every new virus and Trojan. After all, the majority of new malicious programs which were appearing at this time would never penetrate the user’s computer. They were written by adolescent cyber vandals, who either wanted to show off their coding skills, or to satisfy their curiosity. Users only really needed protection against the few In The Wild viruses which managed to actually penetrate victim machines. However, the situation has now changed. More than 75% of malicious programs – i.e. the overwhelming majority – are created by the criminal computer underground, with the aim of infecting a defined number of computers on the Internet. The number of new viruses and Trojans is now increasing every day by a few hundred – the Kaspersky Virus Lab receives between 200 and 300 new samples a day.

These samples come from several sources – honeypots (dedicated machines used to collect malicious files on the Internet); users of infected machines; local network administrators; ISPs; and from other antivirus companies, strange though this may sound. In spite of market segmentation of antivirus companies (which happens with any market, without exception), antivirus companies do work with each other. If a new worm which propagates quickly is detected by one antivirus company, the analysts will inform competitor companies almost immediately, and forward a sample of the worm. And the majority of antivirus companies exchange virus samples at least one a month. They also exchange information at dedicated professional gatherings, which are not open to those outside the industry. It could be seen as professional ethics; antivirus companies do share information with other antivirus companies, except for those companies which may have damaged their standing in the antivirus world through unethical behaviour.

Let’s suppose that a new virus or Trojan is detected in the wild, either on the Internet or on an infected computer. And what does this mean? It means that the likelihood that a certain computer will be infected by a parasite is far from zero, and it’s possible that dozens, hundreds or maybe even thousands of the computers which make up the Internet are already infected. And given how quickly the Internet works, if the latest ‘beastie’ is a network worm, then the number of victims could be in the millions. Consequently, antivirus companies have to able to release rapid updates to antivirus databases, and these updates have to include protection against all the newest viruses and Trojans. This brings us on to the second problem faced by the antivirus industry.

Problem #2

Today, malicious programs propagate so quickly that antivirus companies have to release updates as quickly as possible to minimize the amount of time that users will potentially be at risk. Unfortunately, many antivirus companies are unable to do this – users often receive updates once they are already infected.

Let’s assume that the virus manages to penetrate the victim machine, and the antivirus solution installed on the victim machine doesn’t detect any suspicious activity. (This might be because of the quality of the solution itself, or because the user has been careless, and not downloaded the latest updates to the antivirus databases in good time.) Sooner or later, updates which detected the virus will be released – this means that the virus will be detected, but not necessarily defeated. To get rid of the virus once and for all, the infected files have to be carefully deleted from the victim machine. “Carefully” is the key word here, which brings us to the third problem connected with antivirus programs.

Problem #3

The third problem faced by the antivirus industry is deleting malicious code detected on the victim machine. Very often viruses and Trojans are written in a way which enables them to hide their presence in the system and/ or to penetrate the system so deeply that deleting them is a complex task. Unfortunately, some antivirus programs are unable to delete malicious code and restore the data which has been modified by the virus without causing further problems.

An additional issue is that all software uses system resources, and antivirus programs are no exception. In order to protect the computer, the antivirus program has to perform certain actions – open files, read information in them, open archives to scan them etc. etc. The more thoroughly a file is checked, the more resources are required by the antivirus solution. In this way, an antivirus solution is similar to a security door – the thicker the door is, the more protection it will offer; however, the heavier the door is, the more difficult open and closing it will be. When talking about antivirus solutions, the problem is balancing program speed against the level of protection provided.

Problem #4

Unfortunately, the issue of resource usage is almost insoluble. Experience shows that antivirus solutions which offer rapid scanning are heavily flawed, and will let viruses and Trojans through like water through a sieve. However, the opposite is also not true; antivirus programs which run slowly do not necessarily offer effective protection.

In order to scan files on the fly and provide constant protection for the computer, an antivirus solution has to penetrate relatively deeply into the kernel of the system. It will always penetrate the same levels. Technically speaking, an antivirus program has to install interceptors of system events deep inside the protected system and transmit the results to the antivirus engine in order that intercepted files, network packets and other potentially dangerous objects can be scanned.

However, sometimes it’s simply not possible to install two interceptors in the necessary kernel level of the operating system. The result is incompatibility between the antivirus monitors (which function constantly), as the second antivirus will either be unable to intercept system events, or the attempt to duplicate the interception mechanism can lead to system crash. And this is at the heart of the next problem of the antivirus industry.

Problem #5

Incompatibility between antivirus programs is an issue; in the vast majority of cases, installing two antivirus programs from different vendors on one machine (for increased protection) is technically impossible, as the two programs will disrupt each other’s functioning.

People often think that antivirus companies are acting like toddlers snatching at each other’s toys, that the incompatibility issue is caused by unfair competition, and specially designed in order to squeeze other manufacturers out of the market. However, this is not the case. There is no question of unfair or unethical competition. On the contrary, developers make every effort they can to ensure that their product does not conflict with other popular software (including antivirus solutions.)

Above, I’ve tried to summarize what I think are fundamental issues faceing today’s antivirus industry. So how is the industry going to address this issues? What type of protection will antivirus companies offer in the future?

New technologies vs. traditional solutions

Naturally enough, from time to time antivirus developers want to invent quintessentially new technologies, which will solve the problems listed above at a single stroke, a kind of universal panacea. This proactive protection would make it possible to detect a virus and delete it prior to the virus actually being created and appearing on the Internet – and this could be applied to all emerging virus threats.

Unfortunately, this simply isn’t possible. A ‘universal’ solution is only effective against those threats which act in accordance with constant, well defined rules. As computer viruses aren’t a natural occurrence, but the creation of the intricate workings of hackers’ minds, they are not subject to any fixed rules. Rather, viruses abide by a set of rules which will constantly change in accordance with the goals of the computer underground.

Let’s take the example of the behaviour blocker, which is a competitor to traditional antivirus solutions which are based on virus signatures. These are two completely different approaches scanning for viruses, which are not necessarily mutually exclusive. A signature is a small piece of code which can be compared to files, and the antivirus solution checks to see if the two are identical. A behaviour blocker, on the other hand, tracks application behaviour on launch, and will terminate programs if suspicious or known malicious behaviour is detected. Both methods have their advantages and disadvantages.

One benefit of a signature scanner is that it detects all malicious code that it recognizes. The minus is that it will fail to detect malicious code which it hasn’t encountered before. Another potential minus is the large size of antivirus databases and the resources they consume. Behaviour blockers offer benefits in that they are able to detect even unknown malicious programs. On the minus side is the possibility of false positives; the behaviour of today’s viruses and Trojans is so diverse that devising a single set of rules which encompasses all possible behaviours is simply impossible. This means that the behaviour blocker is certain to fail to detect some malicious programs, and will periodically prevent legitimate applications from functioning.

Behaviour blockers have another inherent disadvantage; they are unable to combat conceptually new malicious programs. Let’s imagine that Company X has developed a behavioural antivirus AVX, which detects 100% of current malicious programs. So what will the hackers do? Of course, they will invent new types of malicious programs. And then of course it will be necessary to update the behavioural rules. And then update them again, because the hackers and virus writers aren’t going to give up that easily. And then update them again and again and again. At the end of the day, we arrive at a signature scanner, except the signatures will be behavioural, and not pieces of code.

This conclusion also applies to the heuristic analyser, another proactive protection method. As soon as hackers perceive that antivirus technologies are preventing them from reaching their victims, they invent new virus technologies which will be used to evade proactive detection. As soon as a product with advanced heuristics and/ or behaviour blocking is widely used, the ‘advanced’ technologies employed will cease working.

This means that ‘reinvented’ proactive technologies are only effective for a relatively short length of time. Where junior hackers need a few weeks or a couple of months to get round proactive protection, professional hackers will need one or two days, or, in the worst case, a few minutes or hours. This means that behaviour blockers or heuristic analyzers, however effective they may be, need constant development and updating. It should also be noted that adding new signatures to antivirus databases is a matter of a few minutes, whereas perfecting and testing proactive protection methods takes much longer. The result is that in many cases signature updates to antivirus databases are far better that the average proactive protection solution. The experience of epidemics caused caused by new email and network worms, new spy programs and other types of malicious code bears this theory out.

Of course this doesn’t mean that proactive protection is useless. It functions well within specific boundaries, and is capable of stopping a certain amount of malware (the programs created by less experienced hackers and virus writers,) For this reason, proactive protection can be an useful addition to signature scanners, but they should not be relied upon to provide total protection.

Comparative testing and its weaknesses

This part of the article looks at the problems users may have when choosing an antivirus solution. It’s assumed that the user will be looking for a product which offers real protection against malicious code. So where can they get information to base their decision on?

The most logical thing is naturally to look at comparative test results from different sources, including professional ones. Do such things exist? Yes, they do, but there aren’t many of them. Most IT publications conduct comparative tests of antivirus solutions on a fairly regular basis. They test the solutions thoroughly, and compare everything from the product price to the quality of technical support provided. However, these tests don’t really prove the quality of the antivirus function. This is understandable, as testers would need a fairly large virus collection, their own tests stands, and automated testing procedures to thoroughly testing the antivirus component. This means a dedicated group which only tests antivirus solutions, and which requires the necessary resources – something which most IT publications don’t have. Comparative tests conducted by IT publications therefore either leave much to be desired, or the publications contact experts who specialize in testing antivirus products.

Currently, the most experienced testers of antivirus products currently are Andreas Marx (Germany
http://www.av-test.org) and Andreas Clementi (Austria
http://www.av-comparatives.org).These tests describe in detail the quality of detection of various types of malicious programs and the speed at which different antivirus companies react to epidemics. The tests are thorough and detailed, and can be used to compare the characteristics of the antivirus solutions themselves. Sadly, these tests only examine the two characteristics described above; they do not address issues of how antivirus solutions perform in real life situations e.g. when curing an infected system, the reaction of the solution to infected web sites, the amount of resources used, and the thoroughness with which archives and installers are checked.

Sadly, tests which provide an in-depth, accurate picture of how products react in typical situations barely exist. The one exception that we know of is the Test Lab at Moscow State University, which conducts tests using a fairly wide range of situations. However, the methodology of these tests still needs working on, and the university’s test lab is not yet known to the public at large.

It’s also worth mentioning the tests conducted by VirusBulletin (an industry publication) – I am sure that if I didn’t include this, readers would ask why the tests and the resulting VB100% award hadn’t been mentioned. Sadly, these tests are far from perfect. The test standards were developed in the mid-1990s and have barely changed since then. Antivirus products are tested using a collection of files infected by ITW viruses. The award is given on the basis of the test results. However, the ITW collection only contains between two to three thousand files – fewer malicious programs than appear in the wild in the space of a single month. Therefore, a VB100% award doesn’t necessarily mean that a product really provides protection against all types of malware. It simply means that the product copes well with VirusBulletin’s ITW collection, nothing more.

Conclusion

I hope that those of you who have read this far now have a better understanding of the issues which the antivirus industry faces, and that it will help you when selecting an antivirus solution for your home computer or network. I think that a computer which is connected to the Internet is rather like sex – it can be safe, or it can be unsafe. In both cases, information is the key to survival, and can protect you from unpleasant consequences.
Happy surfing!