Malware: Gotta catch 'em all!

Today Aptoide is one of the top Android stores worldwide as a result of a 100% year-to-year increase in the last 3 years. This happens because Aptoide offers a highly valuable set of technology and content to serve the needs of Android Manufacturers, Integrators, and Telecoms. With this growth, comes the need for scaling Aptoide's infrastructure to face the increasing demand for accessing, browsing and downloading applications through all Aptoide stores.

Aptoide motto is "Own your Android Store", which means that any user can have a store and upload applications. With more than 100 million users and over 760.000 applications in around 198.000 stores, Aptoide faces the problem of maintaining a malware-free marketplace. In the month of January alone, Aptoide had 873610 uploaded applications, averaging 28180 uploads per day. This makes it clearly impossible to analyse each application manually in a timely manner. In order to provide a real-time uploading system to its users, Aptoide implemented Aptoide Anti-Malware Platform - a system to automatically detect malware, PUAs (Potentially Unwanted Applications) and untrusted developer signatures - with two requirements: efficiency and speed.

Aptoide Anti-Malware Platform is composed by three main components:

1. an automatic malware/PUA/adware detection system codenamed Aptoide Sentinel
2. a developer signature tracker and verification system
3. manual testing

This post is focused on the automatic detection of malware and PUAs. Therefore, it will be about the performance of Aptoide Sentinel, which is the first line of defense implemented in Aptoide to protect its users against unwanted applications.

As stated above, Aptoide saw its users uploading over 873.000 applications in the month of January. Through Aptoide Sentinel, 742.638 applications were detected as not being safe and were taken from the stores. Here we consider different versions of one application as different applications because they are different uploads, meaning that both the number of uploads and the number of infected applications refer to new applications and different versions of the same application. What we see is that 85% of the applications uploaded to Aptoide during January were considered to be some kind of malware, PUA or contained aggressive adware libraries. This amount of unwanted uploads illustrates one of the aspects why some still consider third-party Android marketplaces as unreliable ways to download applications. While some marketplaces do not employ comprehensive means to counteract the high injection flow of unwanted applications, Aptoide has made the protection of its users one of its key concerns. That's why we have developers continuously developing and upgrading Aptoide Anti-Malware System and, for the context of this post, Aptoide Sentinel.

Regarding the way Aptoide Sentinel works, each uploaded application goes through a verification process composed by 6 antiviruses. We are continuously upgrading Sentinel and in the near future, we will have more antivirus integrated into Sentinel while also upgrading the developer signature verification system, providing further protection for our users and partners.

Since we keep integrating more antivirus to Aptoide Sentinel, as well as the antivirus themselves keep getting updated, we keep rescanning the applications in Aptoide stores. Rescanning applications is a way to add extra layers of protection and makes the process of an untrusted developer getting his/her applications to stay in Aptoide and getting downloaded harder. The rescanning scheme we employ ensures that an application is scanned at least 3 times within 48 hours of its upload, at least 4 times within a week and at least 5 times within a month. In addition, we also keep rescanning all the applications in Aptoide over and over, even if they are already in the stores for months or even years, meaning that an application is rescanned several times to ensure its safe to use by Aptoide users and to soften the work of manually testing applications. If almost every infected application is caught in the first layer of Aptoide defenses, the subsequent layers have fewer applications to analyse and can be more thorough in their validation processes, once again ensuring the protection of Aptoide users in the best way possible.

The graph above shows the number of uploads since January of 2014. In blue there is the number of uploads classified as clean by Sentinel and in red the number of uploads categorized as infected. As one can see, the number of uploads more than doubled between September and October of 2015. This was an expected event considering the work being developed by Aptoide with its partners. However, in December of 2015 and January of 2016, the number of uploads was very high as Aptoide was the target of a stream of unwanted uploads. Nonetheless, Sentinel was able to handle both the increased number of uploads and the number of unwanted applications being submitted. Below is a graph showing the evolution of the percentage of infected uploads since January of 2014.

This serves as a first indicative of the improvement of Sentinel since the end of 2015. In addition, we performed a comparative analysis of the detection rate of Sentinel. We gathered a set of 2500 malware samples from different Android malware families and passed them through Sentinel as it was before and as it is now. The results are shown in the following graphs.

The graph above shows the evolution in the live detection rate of Sentinel, i.e. the detection rate of malware in new uploads. The old version of Sentinel could only detect less than 37% of infected applications being submitted. The current version detects above 95% and the rescanning scheme helps Sentinel detect a significant amount of the remaining infected applications. The graph below shows the evolution in the mean of the duration of each scan and there was a significant decrease of 75%.

This increase in the performance of Sentinel, shown by its capacity to scan new uploads virtually in real-time and rescan them without delaying the scan of new uploads, and the increase in the efficiency of detections, is due to 2 main changes:

1. Of the current 6 antivirus in Sentinel, 4 were integrated since February
2. The architecture of Sentinel was changed by the end of 2015

On one hand, change 1 meant that we could introduce more detection power in Sentinel. Each antivirus has its own database of fingerprints and heuristics to identify unwanted behaviours in applications, which makes detection efficiency to vary between antivirus, as well as which kind of malware is detected by a specific antivirus. Integrating more antivirus with better detection rates into Sentinel means that more detections are made.

On the other hand, change 2 was critical to make Sentinel's architecture scalable. The post started with a statement saying that Aptoide has been having a 100% year-to-year increase in the last 3 years. This kind of growth demands that each part of Aptoide infrastructure to be scalable and Sentinel is no exception. This change is what enabled us to perform a lot more application scans per day, allowing for real-time analysis of uploaded applications as well as implementing a thorough rescanning scheme. Moreover, since it's a scalable architecture, we can increase the number of scans per day if needed (and predictions based on our growth say it will be).

As stated, Aptoide keeps developing and upgrading its systems and functionalities. Aptoide Sentinel will keep getting improved, both with the integration of more antivirus and with some other detection tools.

Once again, it's not an overstatement saying that ensuring the protection, and thus elevating the user-experience of Aptoide, is one of our top priorities.