AMTSO & CARO May 2009

During the first full week of May a portion of the AV researcher community gathered in sunny Budapest, Hungary.

The first order of business was attending the third annual CARO workshop. The workshop was attended by more than one hundred researchers from all over the globe.

Appended to the workshop was an AMTSO member meeting. The gathering of people was significantly smaller at about forty people which included Dennis Nazarov and myself from Kaspersky Lab.

We arrived with a plan to vote on four documents:

1. Best Practices for Validation of the Samples,
2. Best Practices for Testing In-the-Cloud Security Products,
3. Analysis of Reviews Process, and
4. Issues around the creation of malware for testing purposes.

After review the membership agreed that Issues Around the Creation of MAlware for Testiong Purposes is not ready for voting. This is by far the most controversial document AMTSO is working on.There’s a clear division between the people who think that it’s OK to modify/create malware for testing purposes and those who feel it’s not.

There had been a complete taboo on creating malware up until a couple of years ago, both for technical and ethical reasons. However there’s been an increase in the amount of people who believe that it’s OK to do as long as proper precaution is met.

Anyway, the other three documents were deemed ready and have been voted on. All of them have been voted in. The validation document focuses on the important issue of making sure that all files in a certain set are valid, i.e. they are not corrupted.

With more and more products gaining ‘cloud abilities’ there comes the issue of how to properly test such products. As the cloud is ever changing it becomes impossible to reproduce testing results. This document gives some great ways around that issue.

The one that will surely have the biggest impact is the Analysis of Reviews Process document. This provides guidleines for published reviews to be officially analyzed by AMTSO.

A special committee will check reviews against the AMTSO fundamental principles of testing and see how they hold up. We’re still streamlining some of the other processes involved but we’re all very excited by this step.

You can find the newly published documents here.

Myself and Costin in sunny Budapest, for lack of a photo of the AMTSO get-together.