Testing Setup: Methodology

To effectively remove infections, spyware removers must be able to detect and stop many different forms of spyware and adware.  What this means is that a program should not only be able to detect forms of Adware like toolbars, unwanted popups, etc they should also be able to detect keyloggers, trojans, and rogue applications equally as well.  This is a difficult task as each family or variant of spyware and adware requires different methods for detection and removal.

Creators of spyware know this and are constantly updating and changing their programs in an attempt to trick and go unnoticed by anti spyware programs.  It's a constant "see saw" battle and with far more people creating spyware than fighting it makes for a very tough battle.

Actual spyware and adware samples were used to infect test machines (a "lab" of 6 Dell Dimension 1100 desktops).

Testing computer lab used to test and collect spyware and adware infections.

Each of the 6 test machines had clean Windows installations to ensure they were infection free and used VMware which is a program that is able to run "virtual" copies of Windows, this allows for testing in an environment that can easily be "reset" back to pre-infection states. 

Next, each PC was infected with the same sampling of of spyware and adware and then a different anti spyware program was loaded on each machine. This ensured each program was faced with removing the same infections on PC's with the exact same hardware and software setup.

Testing Setup: Samples Used

The following samples were used in my testing and demonstrations. (don't worry, no need to be familiar with these just letting you know what was used in my analysis).

 

This is not a complete list of infections used but represents the mixture of the types of infections used with an emphasis on Keyloggers and Rogue software programs.

	Name	Type	Description
1	SpySheriff	Rogue AntiSpyware	Threat:  Critical fake anti spyware application that tries to trick people into purchasing through fake warnings, popups, and inaccurate scan results
2	ActualSpy	Keylogger	Threat:  High monitors and records all keystrokes, email messages, websites visited and could capture usernames and passwords of sites you visit and then sends this information back to the person who installed the software
3	180 Search Assistant	Adware	Threat:  Medium monitors search queries and then serves up ads usually in the form of popups
4	NewDotNet	Adware	Threat:  High hijacks your search queries in order to show advertising
5	ExactSearchBar	Adware	Threat:  Medium delivers unwanted pop up advertising based on the searches you enter and the websites you visit
6	HotBar	Adware	Threat:  Medium installs a "package" of programs and utilities that are used to serve up different forms of advertising and collect information about your web surfing habits
7	2ndThought	Adware	Threat:  Medium pop up ads are shown on your computer's desktop
8	FunWebProducts	Adware	Threat:  Medium Not a immediate threat to your safety or privacy but some of the software bundled in this download contains adware and can clog up your system with unwanted programs.
9	Active Keylogger	Keylogger	Threat:  High monitors and records all keystrokes, email messages, websites visited and could capture usernames and passwords of sites you visit and then sends this information back to the person who installed the software
10	Virus Burst	Rogue AntiSpyware	Threat:  Critical VirusBurst is a Fake Anti-Spyware product which comes bundled along with a malicious downloader. It is downloaded and installed without the users consent.
11	RegFreeze	Rogue App	Threat:  Critical RegFreeze is a Security Risk that may give exaggerated reports of threats on the computer. The program then prompts the user to purchase a registered version of the software in order to remove the reported threats
12	007 Keylogger	Keylogger	Threat:  High stealth monitoring and surveillance application that can secretly capture information on your PC without the possibility of being detected by the user being monitored! The perfect stealth keylogger
13	All In One Keylogger	Keylogger	Threat:  High program that captures and logs keystrokes on the computer without the user's knowledge and consent. The logged data may be encrypted and is typically sent to a remote attacker. The key logger is usually hidden from the user and may use cloaking (rootkit) technology to hide from other software in order to evade detection by anti-malware applications.
14	Beyond Keylogger	Keylogger	Threat:  High Invisible surveillance tool that registers every activity on your pc to encrypted logs.It allows you to secretly track all activities from all computer users and automatically receive logs to a desire e-mail accounting.

Testing Setup: Scoring

For these tests a weighted average was used based on the type of infection removed (keyloggers were considered a more dangerous threat than say an adware toolbar delivering popups), whether or not a program completely removed an infection, disabled it, or missed it entirely and the ability of each program to detect, block and remove infections in real time with active protection features.

As an extremely simplified example, a program that could remove the majority of keylogger infections in real time without having to run a spyware scan was considered more effective than a program that could remove the same keylogger threats but only after they got by the program's real time protection and running full system scans.