The art of finding Cyber-Dinosaur skeletons

Following the release of our report on the Regin nation-state cyber operation, questions were raised about whether anti-malware companies deliberately withheld information — and detections — at the request of governments and customers. A similar question was raised by Bruce Schneier in 2013.

Let’s get the most important issue out of the way immediately. We have never been asked by a customer or a government entity to allowelist or stop detecting any specific malware sample. We’d never comply with such a request, no matter the source.

This simply does not happen. In some cases, investigations into advanced targeted attacks include NDAs (non-disclosure agreements) with customers and we are bound to maintain confidentiality but that never affects adding detection and protecting our entire customer base from threats.

So, why did it take two years for us to release a report on Regin? Without proper context, it may appear that researchers kept something really important in the dark for an inordinately long time. However, security research — not unlike law enforcement investigations — requires meticulous scrutiny and analysis and in many cases, it’s important to watch the crime unfold in real-time to build a proper case.

In our case, without unlimited resources and the fact that we’re tracking multiple APT actors simultaneously (Careto/Mask, EpicTurla, Darkhotel, Miniduke/Cosmicduke, to name a few), this becomes a process that takes months, even years, to gain a full understanding of a cyber-operation.

Sean Sullivan from F-Secure provided a perfect description of APT research, comparing it to the work of paleontologists that find some bones of a dinosaur.  Everyone may have a bone but nobody has the full skeleton.

In the case of Regin, what we first discovered in 2012 was a slightly damaged bone from unknown part of a monster living in a mysterious mountain lake.

(courtesy of southampton.ac.uk)

Someone finding a bone might discard it and keep traveling, but in security research we collect things. The original discovery went to our collection of things stored in the backyard. We have many of these fractured bones from unseen monsters or maybe harmless creatures. Sometimes we hear about other bone fragments being discovered by others and this pushes us to take a closer look but at the early stages, without enough evidence to draw meaningful conclusions, it doesn’t make sense to go public about discoveries until you confirm that the monsters are real, big and dangerous.

We keep working in various channels to collect different artifacts that may or may not match some of the pieces in our collection. Sometimes we join efforts with other “paleontologists” and share our discoveries. Once we collect enough of bones from a monster to understand potential size, danger and possible habitat, we can start the next phase which is a real active investigation that might lead us to the mysterious mountain lake.

A comprehensive APT research project consists of several stages:

1. Adding detection for known modules
2. Collecting samples
3. Reversing the samples
4. Decrypting sophisticated encryption and compression schemes
5. Understanding lateral movement
6. Outlining multiple attack stages in the correct order
7. Mapping C&C infrastructure
8. Setting up sinkholes
9. Analyzing collected traffic and communication protocols
10. Crawling other hosts that understand the same protocol
11. Taking down and acquiring images of C&C servers
12. Identifying victims, sending out notifications to victims and global CERTs
13. Applying forensic analysis and extracting logs, stolen files, other components
14. Collecting and analyzing data from KSN, C&C servers, individual victims who are willing to work with us, sinkholes, crawlers, etc.
15. Writing a comprehensive report

If we are lucky we can find a monster in-the-wild that is the best source for scientific study. In most cases, including Regin, we observe and learn from the behavior of a live monster. We record every single step and intention.

At the same time, we can take it down and study it in a lab like zoologists. However, in many research investigations, we can see only a skeleton of a monster. We have to put everything together and reconstruct how the monster moved, what was it habits, what species it attacked, and how these attacks were coordinated.  Simply put, this requires time and patience.

In addition, when we analyze the characteristics of a certain creature, we understand that the evolution goes on and there are other species like the subject, live and kicking somewhere in a manner that’s not visible at all.

While some of the Regin samples got on our radar early and we continued to find additional samples and artifacts during the research, we are convinced there are others that are currently unknown and undiscovered. Little was known about their life and existence in the past but we know they were out there by finding some tiny fragments over time. And we have to refer to paleontology again, because in the overall picture we have discovered just a small part of the entire beast, but have enough to to release a public alert.

Like Regin, sometimes we find that we had been detecting pieces of malware for several years before realizing that it was a part of global cyber-espionage campaign. One good example is the story of RedOctober. We had been detecting components of RedOctober long before we figured out that it was being used in targeted attacks against diplomatic, governmental and scientific research organizations.

At Kaspersky Lab, we are processing hundreds of thousands of samples every day. The art of figuring out which ones are significant and further yet which ones belong together as part of a big APT attack is akin to finding needles in a huge haystack and then figuring out which ones belong to the same knitting set. We are grateful for every needle we discover, because this makes the world a little safer.