Book cover of Android Malware Detection and Adversarial Methods

Weina Niu, Xiaosong Zhang, Ran Yan and Jiacheng Gong

Android Malware Detection and Adversarial Methods

The Springer logo.

Weina Niu

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Xiaosong Zhang

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Ran Yan

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Jiacheng Gong

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

ISBN 978-981-97-1458-2e-ISBN 978-981-97-1459-9

https://doi.org/10.1007/978-981-97-1459-9

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.

The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

With the development and progress of the Internet, mobile terminals play an increasingly important role in people’s daily lives. Mobile devices such as smartphones and tablets have become indispensable tools for people to communicate, work, entertain, and obtain information. Data show that the number of mobile devices has continued to grow in recent years. The number of smartphone users worldwide currently exceeds billions and is expected to continue to increase in the coming years.

Among mobile device operating systems, the Android operating system occupies the vast majority of the market share and has become the mainstream mobile operating system. According to surveys, 85% of mobile phones around the world use Android as the operating system. In addition, Android OS and its multiple subsystems have also become the mainstream choice of operating systems in smart homes, vehicle systems, and underlying IoT devices. This is mainly due to the open-source nature, flexibility, and extensive hardware support of the Android operating system. However, precisely because of the widespread use and open-source nature of the Android operating system, the number of Android malware has also grown rapidly.

The emergence of Android malware has posed a huge threat to users’ information security and privacy. Malware can cause serious damage to users through various means, such as deceiving users, stealing personal information, controlling devices, etc. In order to solve this problem, both academia and industry are continuously researching and developing detection and countermeasures for Android malware.

The book Android Malware Detection and Adversarial Methods is a systematic academic monograph written under the above background. The book explains in detail the background related to Android malware, including four parts: The Overview of Android Malware Detection, The General Android Malware Detection Method, The Adversarial Method for Android Malware Detection, and The Future Trends of Android Malware Detection. In part, the book explains their related issues, principles, and well-known research. This book is created based on many years of exploration and practice of Android malware detection by the Cyberspace Security Technology Laboratory of the University of Electronic Science and Technology of China. We hope to provide readers with inspiration and guidance.

Weina Niu

September 2023

Preface

Dear readers:

It is my honor to introduce to you this book Android Malware Detection and Adversarial Methods. This book aims to provide readers with a comprehensive and in-depth monograph that discusses the latest research results and methods in the field of Android malware detection.

This book is divided into four parts, covering all aspects of Android malware detection. The first part is an overview of Android malware. Through an overview of Android malware detection, it provides readers with an overall understanding of this field.

The second part is the general method of Android malware detection, including signature-based malware detection method, behavior-based malware detection method, and artificial intelligence-based malware detection method. These methods are currently widely researched and applied technologies. By introducing their principles and applications in detail, readers can have an in-depth understanding and apply them to actual malware detection work.

The third part is Android malware detection and adversarial methods, including static adversarial methods in Android, dynamic adversarial methods in Android, and adversarial technology in machine learning. These methods are designed to help readers understand and respond to the adversarial behavior of malware and improve detection accuracy and effectiveness.

Finally, the fourth part is about the future trends of Android malware. By discussing future development trends, it shows readers the cutting-edge research and development directions in this field. Through this part, we want to show that as technology continues to progress and evolve, Android malware detection techniques need to be constantly updated and improved to adapt to changing security threats.

This book is mainly from an academic perspective. By introducing the commonly used methods and corresponding literature in each chapter, it aims to provide inspiration and guidance to readers with different backgrounds and professions. Whether you are a scholar engaged in security technology research, an engineer engaged in security product development, or a reader interested in Android malware detection, this book will provide you with valuable knowledge and information.

This book was organized and drafted by Associate Professor Weina Niu and Professor Xiaosong Zhang of the University of Electronic Science and Technology of China. Xingyu Liu, Ke Wang, Botao Li, Tianqing He, Jiacheng Gong, Zhiqin Duan, Ran Yan, Anran Hou, and Mingyang Li participated writing of relevant chapters. I would like to thank all those who provided support and help with this book, including the authors, editors, reviewers, and publisher staff.

Due to the author’s level, there will inevitably be omissions or shortcomings in the book. Readers are welcome to criticize and correct them.

Weina Niu

Chengdu, China

September 2023

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Grant No. 62372086), Sichuan Natural Science Foundation Project (Grant No. 24ZNSFSC0038), and the Chongqing Natural Science Foundation Innovation and Development Joint Fund (Grant No. CSTB2023NSCQ-LZX0003). We also would like to express our sincere gratitude to the editor for their valuable comments, which have greatly improved this book.

Contents

Part I The Overview of Android Malware Detection1

1 Introduction of Android Malware Detection3

1.​1 Android Malware Family4

1.​1.​1 Trojan Horse4

1.​1.​2 Viruses5

1.​1.​3 The Back Door5

1.​1.​4 Zombies5

1.​1.​5 Espionage6

1.​1.​6 Intimidation6

1.​1.​7 Extortion6

1.​1.​8 Advertising7

1.​1.​9 Tracking7

1.​2 History of Android Malware Detection8

1.​3 Android Malware Detection Overview9

1.​4 Challenges and Apps of Android Malware Detection9

1.​5 Domestic and International Android Malware Detection11

1.​5.​1 Android Malware Detection Method Based on Static Analysis11

1.​5.​2 Android Malware Detection Method Based on Dynamic Analysis13

1.​5.​3 Android Malware Detection Method Based on Hybrid Analysis15

1.​6 Chapter Summary17

References17

Part II The General Android Malware Detection Method21

2 Feature Code Based Android Malware Detection Method23

2.​1 Detection Based on Traditional Feature Codes23

2.​1.​1 Introduction23

2.​1.​2 DroidAnalyzer:​ A Case Study in Android Malware Analysis24

2.​2 Detection Based on Semantic Feature Codes28

2.​2.​1 Introduction28

2.​2.​2 DroidNative:​ A Case Study in Android Malware Analysis29

2.​3 Chapter Summary40

References41

3 Behavior-Based Detection Method for Android Malware43

3.​1 Privacy Disclosure44

3.​2 Permission Escalation45

3.​2.​1 Permission Escalation Method46

3.​2.​2 Authorization Based on Configuration Files47

3.​2.​3 Code Analysis48

3.​2.​4 Taint Analysis50

3.​3 Machine Learning Technology and Malicious Behavior of Android Software53

3.​4 Chapter Summary54

References54

4 AI-Based Android Malware Detection Methods55

4.​1 Detection Based on Permissions, APIs, and Components55

4.​1.​1 Permissions in Android System60

4.​1.​2 Detection Based on API62

4.​1.​3 Component-Based Detection63

4.​1.​4 Specific Case:​ Drebin67

4.​2 Detection Anchored in Dynamic Runtime Features72

4.​2.​1 Dynamic Analysis and Runtime Features72

4.​2.​2 Overview of Detection Methods Based on Dynamic Runtime Features73

4.​2.​3 Specific Case:​ EnDroid73

4.​3 Detection Through Semantic Code Analysis77

4.​3.​1 Dalvik Bytecode77

4.​3.​2 Overview of Code Semantic-Based Detection Methods78

4.​3.​3 Specific Case:​ MviiDroid81

4.​4 Detection via Image Analysis83

4.​4.​1 Overview of Image-Based Detection Methods84

4.​4.​2 Specific Case:​ R2-D285

4.​5 Detection Through Graph Analysis86

4.​5.​1 Overview of Homogeneous Graph-Based Detection Methods86

4.​5.​2 Overview of Heterogeneous Graph-Based Detection Methods87

4.​5.​3 Case Study:​ HAWK88

4.​6 Chapter Summary91

References92

Part III The Adversarial Method for Android Malware Detection95

5 Static Adversarial Method97

5.​1 Static Obfuscation97

5.​1.​1 Code Obfuscation98

5.​1.​2 Resource Obfuscation98

5.​1.​3 Manifest File Obfuscation99

5.​1.​4 Control Flow Obfuscation99

5.​2 Common APK Static Obfuscation Tools100

5.​2.​1 Obfuscapk100

5.​2.​2 ProGuard100

5.​2.​3 DexGuard101

5.​2.​4 Allatori101

5.​2.​5 DashO101

5.​2.​6 Bangcle102

5.​2.​7 Arxan102

5.​2.​8 Comparative Analysis102

5.​3 Research on Static Obfuscation103

5.​3.​1 Detection Methods Based on New Features103

5.​3.​2 Detection Method Based on Binding Method120

5.​4 Chapter Summary127

References128

6 Dynamic Adversarial Method in Android Malware129

6.​1 Automatic Dynamic Analysis Evasion130

6.​1.​1 Detection Dependent130

6.​1.​2 Detection Independent133

6.​2 Manual Dynamic Analysis Evasion136

6.​2.​1 Direct Detection136

6.​2.​2 Deductive Detection137

6.​2.​3 Debugger Evasion138

6.​3 Related Research About Dynamic Analysis Evasion141

6.​3.​1 Research About Improving Sandbox141

6.​3.​2 Research About Detecting Dynamic Evasion145

6.​4 Chapter Summary149

References150

7 AI-Based Adversarial Method in Android151

7.​1 Introduction to Adversarial Examples151

7.​2 Classification of Adversarial Example Generation Methods152

7.​2.​1 Gradient-Based Attacks153

7.​2.​2 Optimization-Based Attacks153

7.​2.​3 GAN-Based Attacks153

7.​2.​4 Domain-Specific Attacks (Audio, Images, Text, etc.​)154

7.​3 Black-Box Attacks154

7.​3.​1 Introduction to Black-Box Attacks155

7.​3.​2 Common Black-Box Attack Methods155

7.​3.​3 Transfer Learning-Based Black-Box Attacks156

7.​3.​4 Meta-Model Based Black-Box Attacks157

7.​3.​5 Query-Based Attacks158

7.​3.​6 Optimization-Based Attacks158

7.​4 White-Box Attacks159

7.​4.​1 Optimization-Based Attacks159

7.​4.​2 Gradient-Based Attacks161

7.​4.​3 App of Adversarial Attacks in Malware Detection163

7.​5 Chapter Summary165

References165

Part IV The Future Trends of Android Malware Detection167

8 Future Trends in Android Malware Detection169

8.​1 Machine Learning And Deep Learning Techniques169

8.​1.​1 Overview of Machine Learning and Deep Learning for Android Malware Detection170

8.​1.​2 Challenges Faced171

8.​2 Integrated Solutions173

8.​2.​1 Challenges Faced174

8.​3 Blockchain Technology178

8.​3.​1 Introduction to Blockchain Technology178

8.​3.​2 Examples of Blockchain Technology in the Field of Android Malware Detection180

8.​4 Hardware Technology181

8.​4.​1 Advantages of Hardware Technology182

8.​4.​2 Challenges to Hardware Technology182

8.​4.​3 Examples of Hardware Technologies Applied in the Field of Android Malware Detection183

8.​5 BPF Technology185

8.​5.​1 Development of BPF Technology185

8.​5.​2 eBPF Technology Overview186

8.​5.​3 Examples of BPF Techniques in the Field of Android Malware Detection188

8.​6 Chapter Summary189

References189

Part IThe Overview of Android Malware Detection

Android is currently the most popular mobile operating system with over 80% market share. Android’s openness and flexibility provide users and developers with a wealth of features and choices, but it also poses many security risks, generating a large amount of malware that threatens users’ information security.

This part focuses on the introduction of Android malware, the main categories of Android malware and its harm to users from the Android operating system, and secondly, in order to cope with the spreading and propagation of malware, detection techniques to combat malware have emerged, so this part also describes in more detail the development process of Android malware detection, the current state of the research, and the future challenges.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_1

1. Introduction of Android Malware Detection

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

Android is currently the most popular mobile operating system with over 80% market share. Android’s openness and flexibility provide users and developers with a wealth of features and choices, but it also poses many security risks, generating a large amount of malware that threatens users’ information security. Android malware (malware) refers to the use of Android system vulnerabilities or user negligence to carry out illegal or malicious behaviors, such as stealing user information, consuming resources, spreading viruses, and controlling devices. According to statistics, more than 150 million Android devices were infected with malware in 2023, causing huge economic losses and privacy leaks.

In order to protect the security of Android users and devices, researchers want to detect whether there is any security problem in this app before it is used by the users, so there is important research on Android malware detection. Android malware detection is the process of analyzing the behavior, code, permissions, and other characteristics of an Android app to determine whether it contains malicious behavior or code. According to whether the app is actually running or not, it can be categorized into two methods: static analysis (static analysis) and dynamic analysis (dynamic analysis). Static analysis refers to structured or semantic analysis, such as decompilation, anti-obfuscation, data-flow analysis, etc., without running the app. Dynamic analysis refers to behavior monitoring, execution tracking, network traffic analysis, etc., in the case of running apps.

Static and dynamic analysis have their own advantages and disadvantages and generally need to be used in combination to improve detection efficiency and accuracy. In general, a lot of research has been conducted in the industry on these detection methods, and the following chapter will provide an overview of Android malware detection in terms of the Android malware family, the current state of research at home and abroad, and the challenges and apps of the research.

1.1 Android Malware Family

Android malware family refers to a group of malware types developed for the Android operating system, including Trojans, viruses, backdoors, bots, spies, threats, ransomware, advertisements, and tracking. Android malware family uses various technical means including masquerading as legitimate apps, exploiting system vulnerabilities to gain access, self-replicating, and spreading for malicious purposes, such as stealing user’s personal information and bank accounts, sending malicious SMS, controlling user’s devices, encrypting files and extorting them. Based on the intent of malware, it can be categorized as follows:

1.1.1 Trojan Horse

This type of malware usually disguises itself as normal, seemingly useful software but actually performs malicious operations without the user’s knowledge, such as keylogging Trojans that record sensitive information about the user; steal login credentials, bank accounts, and other information; and send it to the attacker. Below is a list of a few Trojans that have been seen in Android devices:

1.

Zone: It automatically sends SMS messages and subscribes to paid content, thus deducting the user’s phone bill.

2.

FakeNetflix: This malware steals account information from Netflix users and sends it back to the server.

3.

FakePlayer: A Trojan that can gain access to SMS sending privileges to form malicious chargebacks; this Trojan disguises itself as a normal media player and spreads via cell phone SMS. Once the user clicks on the linked website Trojan-SMS.AndroidOS.FakePlayer.a, the user will be prompted to install a 13KB APK app file and will be asked to allow the user to get permission to send SMS. Once the user’s cell phone installs the Trojan, it will send a paid SMS to the designated number, thus causing damage to the user’s phone bill account. Once the Trojan is installed on the user’s cell phone, it will send paid SMS to the specified number, thus causing damage to the user’s phone bill account.

4.

Spitmo: It will be injected into the banking app to steal the user’s bank account and password.

5.

Zitmo: This is a Trojan horse that targets banks and is used to steal verification codes sent by banks.

6.

Android.Trojan.Asacub: This is a banking Trojan that steals users’ banking login details and other sensitive information by disguising itself as a legitimate app.

1.1.2 Viruses

This type of malware spreads itself by infecting other files or programs, thus achieving the purpose of large-scale invasion or even destruction of computer systems, for example:

1.

Obad: It is a worm with the ability to replicate and spread itself. It takes advantage of the power-lifting vulnerability of the Android system to gain higher privileges and prevent uninstallation; at the same time, it sends SMS to value-added service numbers to gain profit.

1.1.3 The Back Door

This type of malware is a type of malware with hidden features that are often used to bypass normal security controls in order to gain unauthorized access or control of a target system. The purpose of a backdoor type of malware is to perform potential attacks or surveillance activities on an infected system without the user’s knowledge. Here are some examples:

1.

RATC: A backdoor exploit that enables Root privileges by exploiting the adb setuid exhaustion attack.

2.

Bracebridge: It terminates secure apps and sends profitable SMS to subscribe to value-added services.

3.

Kmin: It steals user information and sends it back to the server.

1.1.4 Zombies

Zombie-type malware is a type of malware that turns an infected computer into a “zombie” or “botnet.” This software can be installed on a computer without the user’s knowledge and can be used to manipulate the infected computer through remote control. These infected computers can be used by hackers to perform a variety of malicious activities, such as sending spam, launching distributed denial-of-service attacks (DDoS attacks), or phishing. Botnets can consist of thousands or even millions of infected computers, which are called “bot hosts.” Hackers can send commands to these zombie hosts through a control server, causing them to perform certain malicious activities at the same time. This ability to centralize control allows hackers to use botnets to carry out large-scale attacks that are difficult to track and stop. Examples:

1.

Geinimi: This malware receives remote commands for controlling mobile devices and collects information about the device to send back to the server side.

2.

AnserverBot: This malware receives remote commands.

1.1.5 Espionage

Spyware malware is a type of malware that has the ability to spy and steal users’ personal information. Its purpose is to collect sensitive information, such as login credentials, bank account information, personal identification information, etc., without the user’s knowledge and send this information to hackers or third parties. These malware usually exists in a hidden form in the computer system and it is difficult for users to detect their presence. For example:

1.

GPSSpy: It disguises itself as another app and uploads the user’s GPS information.

2.

Nickyspy: It records incoming and outgoing phone calls of the victim, tracks the user’s GPS information, and sends SMS to other numbers.

3.

Android.Spyware.Pegasus: It is a spyware that remotely monitors the infected device’s communication, location, passwords, and other information and has the ability to install other malicious features.

1.1.6 Intimidation

Intimidation-type malware is designed to gain profit by threatening, intimidating, and extorting users. This type of malware typically compromises a user’s computer system or mobile device and uses various methods to intimidate the user into paying a ransom or divulging sensitive personal information. The purpose of this intimidating malware is to deceive users and gain financial benefits. They may cause serious damage to the user’s computer system, resulting in data loss, system crashes, or personal information leakage. Example:

1.

Koler: Originally discovered in 2014, once a device is infected with Koler, it masks itself in the form of an image browser or video player and then launches an extortion campaign. Koler usually takes the form of a “warning message” claiming that the user has broken certain laws or violated copyrights and demands that the user pay a fine in order to unlock the device or avoid further legal action. The goal of this software is often to panic the user and obtain money through threats and coercion.

1.1.7 Extortion

Ransomware-type malware is a type of malware that extorts ransom by encrypting the user’s files or restricting access to them. Once a user’s files are encrypted, the ransomware displays a message asking the user to pay a ransom to decrypt the files. This type of software usually exploits the user’s fear by threatening to delete files or disclose the user’s sensitive information. Examples:

1.

Fakedefender.B: It is displayed in the form of a security alert message that induces the user to purchase fictitious services to remove non-existent malware.

2.

Android.Ransomware.Locky: It is ransomware that locks the user’s device and demands a ransom to unlock it. Also, it encrypts the files on the device and makes them inaccessible until the user pays the ransom.

1.1.8 Advertising

The purpose of ad-based malware is to display advertisements on a user’s device for profit. This type of malware usually delivers advertisements by infiltrating the user’s computer, cell phone, or other device, often without the user’s knowledge. Examples:

1.

Uapush.APP: It is an adware that also steals device information.

2.

Android.Adware.HummingWhale: It is a type of adware that generates large amounts of ad revenue without the user’s knowledge through hidden features in malicious apps.

1.1.9 Tracking

Tracking malware is designed to silently monitor a user’s online activity and steal personal information. This type of malware usually hides and runs on the user’s device without the user often realizing its presence. Example:

1.

SMSTracker: an app used to track SMS activity. It helps users to monitor and record SMS messages sent and received by their cell phones and provides information about the content of the messages, sender/receiver, timestamps, etc.

The existence of this malware poses a threat to the security of Android users. In order to protect users from Android malware, the industry wants to intercept the apps in time before they hit the shelves or before they are installed, so how to automate and perform Android malware detection with high accuracy has become a popular research area in recent years. Android malware detection refers to the use of specialized security software or mechanisms to detect and identify malware on Android devices. These detection tools use a variety of technical tools and algorithms, including feature recognition, behavioral analysis, machine learning, etc., to detect and identify malware.

1.2 History of Android Malware Detection

Since 2013, the Android operating system has dominated the mobile operating system market, and the open source and openness of this platform has made the Android platform a popular target for malicious app attacks. More and more malicious apps are being released to users without going through normal and reliable security audits, seriously threatening users’ personal security, and even national security.

The development of Android malware detection has gone through the following stages:

1.

Static feature detection: Early Android malware detection mainly relies on analyzing the static features of the app, such as the app’s permission request, code behavior, etc. Through static feature analysis of suspicious apps, it can quickly identify apps that may carry malicious code. By analyzing the static features of suspicious apps, we can quickly identify the apps that may carry malicious codes. Before 2011, the detection methods based on feature values and rules are absolutely dominant, after 2011, the limitations of the detection methods based on features and rules are becoming more and more prominent, which makes it impossible for this primitive detection method to detect unknown malicious apps.

2.

Dynamic Behavior Analysis: With the popularity of the Android system and the increase of malware threats, the effectiveness of static feature detection gradually diminishes. In order to better cope with malware variants and new types of attacks, dynamic behavior analysis is beginning to be used. This method identifies potential malicious behavior by running the app and monitoring its behavior, network communication, etc.

3.

Artificial Intelligence techniques: With the rapid development of machine learning and deep learning techniques, these techniques are beginning to be used to improve the accuracy of Android malware detection. With the help of AI algorithms, it is possible to categorize a large number of known malicious and benign apps and apply this categorization knowledge to the detection of unknown apps.

4.

Cloud-based detection: Due to the constant changes and evolution of malware, standalone detection methods gradually appear to be out of reach. Therefore, the concept of cloud-based detection emerges. Cloud detection centralizes large-scale samples on cloud servers for analysis and provides updated malware libraries and detection rules to end devices in a timely manner.

5.

Correlation analysis: Traditional malware detection tends to focus only on the behavior of individual apps while ignoring the interactions between apps. In order to better capture malicious behavior, the analysis of behavioral patterns between apps has been gradually introduced to enable more accurate identification of malware.

Inconclusion, with the continuous evolution of malware and the development of attack techniques, Android malware detection methods are constantly updated and improved. The future development direction may include the combination of more powerful artificial intelligence technology, more fine-grained behavioral pattern analysis, dynamic detection technology based on eBPF, cross Native layer, or JavaScript static analysis combined with review.

1.3 Android Malware Detection Overview

Android malware detection is a method of analyzing the behavior, code, permissions, and other characteristics of an Android app to determine the presence of malicious behavior or code. Detection methods can be categorized into static and dynamic analysis based on whether the Android program actually executes or not. Static analysis refers to the structured or semantic analysis of an app without running it, such as decompilation, anti-obfuscation, data-flow analysis, etc. Dynamic analysis, on the other hand, monitors the behavior of an app while it is running, traces its execution, and analyzes network traffic. Static and dynamic analyses have their own advantages and disadvantages, so there are some scholars who use hybrid analysis to combine the features of both static and dynamic detection to improve detection efficiency and accuracy.

However, Android malware detection also faces some challenges, such as static analysis cannot fight against obfuscated and shelled malware, the dynamic analysis will be affected by the program execution environment and the integrity of code execution, and deep learning detection methods have unclear features and the results are not interpretable. To address these challenges, scholars in the industry have conducted many explorations and studies. These include methods that utilize artificial intelligence and machine learning technologies to improve detection efficiency and accuracy; multi-source data fusion and knowledge graph technologies to improve detection coverage and depth; and collaborative computing and blockchain technologies to improve detection credibility and transparency. Through these approaches, the ability to detect Android malware can be enhanced and provide a more secure mobile device environment.

1.4 Challenges and Apps of Android Malware Detection

Although there have been many studies and tools dedicated to Android malware detection, there are still many challenges. Here are some of the major challenges:

1.

Diversity and complexity of malware. Android malware has many types and forms, such as Trojan, spyware, ransomware, adware, botnet, etc. Different types of malware may have different goals, means, and characteristics, making it difficult to detect them with uniform standards or models. In addition, Android malware is also highly complex and hidden, such as using encryption, obfuscation, anti-debugging, self-modification, and other techniques to evade detection or interfere with analysis.

2.

Rapid change and evolution of malware. Android malware is not static, with the development of technology and market demand, new variants or families constantly appear. According to statistics, in 2023 an average of 12,000 new Android malware samples will be discovered every day. These new samples may have new functions, features, or attack methods that render existing detection methods or tools ineffective or less effective.

3.

Limitations of malware detection methods and tools. Methods and tools for Android malware detection also have some limitations, such as the lack of effective feature extraction, classification, clustering, and other techniques, which makes it difficult to deal with large-scale, high-dimensional, and unstructured data; the lack of effective evaluation standards, datasets, benchmarks, and other resources, which makes it difficult to compare and validate the performance of different methods and tools; and the lack of effective collaboration and sharing mechanisms, which makes it difficult to take advantage of different data sources, knowledge bases, experts, and other resources to improve detection capabilities.

In response to the above challenges, Android malware detection also has a number of apps and development directions, such as:

1.

Utilize artificial intelligence techniques to improve detection efficiency and accuracy. Artificial intelligence technology can automatically learn and extract effective features, rules, patterns, etc., from large amounts of data, as well as perform tasks such as classification, clustering, and prediction. For example, deep learning (deep learning) technology can be used to represent learning (representation learning) of the code, behavior, permissions, etc. of Android apps, as well as the use of neural network (neural network) models to identify and classify malware.

2.

Use multi-source data fusion and knowledge graph techniques to improve detection coverage and depth. Multi-source data fusion techniques can collect and integrate relevant information from different data sources, such as app stores, social media, security reports, etc., to increase the amount and quality of data for detection. Knowledge graph technology can transform different data into structured entities, attributes, relationships, etc., as well as construct semantic links between different entities to increase the data dimension and depth of detection.

3.

Collaborative computing and blockchain technologies are utilized to improve detection credibility and transparency. Collaborative computing technology can utilize different computing resources and participants to jointly complete the detection task, such as using cloud computing, edge computing, crowd sensing (crowd sensing), and other technologies to perform static or dynamic analysis in a distributed manner. Blockchain technology can utilize distributed ledger, consensus mechanism, smart contract, and other technologies to record and verify all operations and results in the detection process to increase the traceability and auditability of the detection.

In conclusion, Android malware detection is an important and complex issue that requires continuous research and innovation to address the challenges of malware diversity, complexity, and variability, as well as the use of technologies such as artificial intelligence, multi-source data fusion, collaborative computing, and blockchain, in order to improve detection efficiency, accuracy, coverage, depth, credibility, transparency, and other metrics.

1.5 Domestic and International Android Malware Detection

Android malware detection efforts can be categorized into different categories based on different perspectives. The most common classification is based on whether or not the app to be tested needs to be run during the process of detecting Android malware, and the detection methods can be roughly divided into two categories, which are based on dynamic analysis and static analysis detection methods. The following is a detailed introduction to the two methods and summarizes the current research status at home and abroad.

1.5.1 Android Malware Detection Method Based on Static Analysis

The most notable feature of a detection effort based on static analysis is that it does not require running the app to be detected. Therefore, regardless of the technique used, as long as it does not involve dynamically running the app program, we can refer to this detection method as static analysis. Obviously, the research targets of static analysis are very diverse. Specifically, the apk file of Android or the smali file of class.dex generated after decompilation can be analyzed. The former can parse the AndroidManifest.xml file in the unpacked package, from which we can obtain the permissions applied by the app, various component information, and inter-component communication, etc. The latter contains the program code level. The latter contains program code-level information. Smali is a disassembled form of the language executed by the Android Dalvik virtual machine, which supports basic Java features such as line counting, annotations, and debugging. Therefore, obtaining smali code is equivalent to obtaining the source code of an app, which allows for detailed class-level or method-level static analysis. Aung et al. [1] carried out machine learning based static analysis for malware detection and extracted the permissions of the samples as features. This method is simple and efficient for large-scale automatic screening but has a high rate of underreporting. Hein et al. [2] used an information gain based approach for feature selection to reduce the feature dimensions required while training the model and to improve the detection efficiency of the model. Tian et al. [3] proposed a code heterogeneous analysis based Android malware detection technique using sensitive APIs as a carrier. They address the analysis of repackaged malware by chunking the code into classes and methods and constructing dependencies at function points. The approach takes full account of different levels of granularity, reflecting both coarse-grained program structure features and focusing on fine-grained features at the method level. DroidMat [4] creates a feature vector for each app and applies several machine learning algorithms to distinguish between benign and malicious apps. It uses API calls as features to determine which operations the app tries to perform. The approach considers not only the API calls themselves but also the type of component in which they occur, since the same API used in different components may reflect different intentions of the developer. Kang et al. [5] created signatures for different levels of features and constructed a library of samples that can be used to match malware directly based on the signature information. The detection efficiency of this approach is very fast, but the disadvantage is that a large sample library needs to be maintained and samples that do not exist in the library may be poorly detected. In addition, malware obfuscation techniques may affect the matching accuracy.

In malware family detection, a common approach is to perform multi-classification by indirectly constructing a feature model that transforms easily extractable features into the format required for machine learning or deep learning. The research direction focuses on code-level based features, and common approaches include bytecode analysis and indirect analysis of abstract smali code. Niu et al. [6] proposed an Android malware classification method based on deep learning and opcode-level FCG, which used LSTM model with 97% accuracy. Feng et al. [7]targeted a specified number of malicious families for typical feature extraction for multi-classification. The specific strategy is to extract the maximum suspicious public subgraph of each malicious family sample, and the object of study is the API-level calls and communication relationships between components. MaMaDroid [8], on the other hand, relies on the behavior of the app, constructs a Markov chained behavioral model by extracting the abstract API call sequences executed by the app, and uses it to characterize features and perform classification to distinguish between different malicious families. The advantage lies in the controllable feature set size, which avoids the problem of excessive feature dimensionality. Enck et al. [9] proposed a lightweight detection method based on permission usage rules, which constructs nine rules for security permissions and judges an app as malware if the permissions it applies to violate any of the above rules. This method can identify malware to a certain extent, but there are high false positives. For example, one of the security rules is that you cannot apply for WRITE_SMS and SEND_SMS permissions at the same time, which can identify certain SMS-related malware. However, it is also easy to recognize benign software that needs to send SMS as malware. This can certainly identify some SMS-related malware, but it is also easy to recognize benign software that requires the ability to send SMS as malware. Additionally, early feature codes did not contain program semantics and were therefore easily modified by existing obfuscation techniques to bypass detection, so subsequent approaches began to attempt to combine the program’s semantic information with the feature code. Gao et al. [10] mapped the app’s Android APIs into a large heterogeneous graph, converted the original problem into a node classification problem based on invocation relationships and API usage patterns, respectively. Edges are constructed, and then the heterogeneous graph is fed into a GCN model to iteratively generate node embeddings that combine topology and node features. Ultimately, unlabeled apps are classified based on their final embeddings. This method is able to effectively detect 98.99% of Android malware with a false alarm rate of less than 1%, but the method mainly focuses on classifying in-sample nodes and requires additional computational overhead in classifying new out-of-sample nodes.

In summary, malware detection based on static analysis has the advantage of being fast and batch-processable. However, there may be cases where specific malicious behaviors cannot be triggered, leading to the possibility of low detection rates. Current approaches for dealing with malware families focus on drilling down to the fine-grained code level and performing multi-classification tasks by modeling the app’s code.

1.5.2 Android Malware Detection Method Based on Dynamic Analysis

Although static-based malware detection can achieve a high accuracy rate, in the face of malware programs with code obfuscation or reflection mechanisms, it is very easy to miss the report, this is because many of the malicious features in the program are hidden by the developers using some means, we can not get useful features directly in the static code analysis, so there is a dynamic execution based on the situation to determine whether its whether it is malware or not. Dynamic analysis requires active execution of the Android app. TaintDroid [11] employs the idea of taint tracing to locate details of malicious code by tracing “dirty data” back to the malicious code, in order to discover the leakage of private data. The technique modifies the Native stack to obtain a customized runtime environment. Similarly, TaintART [12] modifies the Android runtime compiler for taint tracing purposes. However, the method of modification at the system level may lead to abnormal system crashes, which may affect the program execution, which in turn may lead to unstable classification accuracy. Logcat [13] implements a dynamic auditing framework for Android malware that monitors kernel information, which consists of a log collector and a log analyzer at the Linux layer. It mainly performs statistical analysis of system resources such as memory consumption, CPU usage, etc. Arora et al. [14] were the first to propose an approach for malware detection based on network traffic analysis, which has good detection results in specific malware families, which are mainly characterized by launching a background to communicate with a remote server over the network. For example, the detection results for the Anserverbot and Basebridge families are remarkable. AntiMalDroid [15] monitors the program in several dimensions, including battery drain, individual components, phone and location information. Theoretically, the more the entry points, the more the malicious behaviors may be captured. DroidScope [16] combines the strengths of related techniques by reconstructing OS-level instructions and Java code to generate tools with a semantic research model capable of representing a wide range of behaviors of the operating system, hardware devices, and the Dalvik virtual machine. This kind of work has a rich set of entry points and achieves better detection results. In addition, since the data acquired by dynamic analysis is after the behavior has occurred, this approach can compensate to some extent for the disadvantages that cannot be handled well by static analysis, such as code obfuscation, reflection, dynamic loading, etc. Andromaly [17] implements a dynamic detection framework that monitors parameters such as CPU usage, the number of WIFI packets, the number of processes, and the amount of power. Afonso [18] proposed a system that extracts functionality from Android API calls and system call traces to dynamically identify whether an Android app is malicious or not. They considered various machine learning algorithms such as SVM, RF, J48, KNN, K2 Bayesian networks, and TAN Bayesian networks and finally chose RF as the decision model but still suffered from lack of adaptability to different samples. They also mentioned the use of network traffic analysis to enhance system performance as a future direction of work. However, they used API Monitor and strace to implement monitoring, resulting in a less scalable system. In addition, their automated testing process is executed for only 5 minutes for all samples, a testing strategy that lacks targeting.

Zhou [19] pointed out that dynamic malware detection has some drawbacks. The first one is the high resource consumption, in order to provide a realistic virtual environment, a high configuration of the simulator is usually required, especially when detecting malicious samples on a large scale. Second, dynamic detection requires the malware to run for a period of time. Automated detection solutions are usually preferred, relying on automated testing procedures to maximize the triggering of malicious behaviors, and code coverage is commonly used to evaluate the operation of the program during dynamic execution. Calculating code coverage requires operations such as source code staking, and this time overhead can increase the overall time consumption of dynamic detection. Meanwhile, manual staking may affect the scalability of the Android malware detection system. In addition, Tam [20] pointed out that malware may detect the current runtime environment, and if it is a non-real machine environment, it may hide its own malicious behaviors and may even implement misleading detection behaviors, such as logic bombs [21]. If malware determines whether the current execution environment is real or not by detecting the CPU architecture, many dynamic detection frameworks that rely on virtual platforms may fail, which will inevitably lead to potential overheads in using real machines for detection. However, there are related researches to detect whether an Android app has dynamic escape behavior, for example, Lumus [22] is a method to detect sandbox escape dynamically through multiple environments, the author detects whether a sample has emulator escape behavior by comparing the different behaviors of the sample in the real machine emulator environment, and the main idea is to use DroidDot to randomly execute the samples for a period of time in the real machine and simulator environments (to ensure the consistency of the triggering events in the two execution environments), and save the log information, then abstract and extract the log information to obtain regular behavioral features and form the call sequence, and then compare the behavioral call sequences generated by the two execution environments to determine whether the simulator escape detection is the same or not.

From a comprehensive point of view, dynamic analysis can assist static analysis and improve the detection rate. Therefore, many studies have been devoted to hybrid detection methods based on dynamic and static analysis.

1.5.3 Android Malware Detection Method Based on Hybrid Analysis

A common hybrid analysis detection process usually includes both static and dynamic analysis, with the results of each part of the analysis being combined to determine whether a sample is malicious. Such a process helps to improve the overall detection rate while balancing between overhead and detection effectiveness.

Surendran et al. [23] utilized a tree-extended plain Bayesian classifier for malware detection by recording dynamic runtime system calls and static permissions. StormDroid [24] enhanced the reliability of the detection by extracting the permissions and API calls of real samples to construct a feature vector. Also, the study used five classifiers for detection. In addition, they also raised the need for feature engineering, pointing out that many “popular” samples play a weak role in the classification process, which may increase the overhead of detection. However, their features are relatively homogeneous and not comprehensive enough.

Xia et al. [25] proposed a hybrid analysis scheme that enables a dynamic testing process customized for different apps. Specifically, they implemented AppAudit to record API calls in the sample that may have suspected data leakage in the static analysis and trace and troubleshoot these potentially problematic APIs in the dynamic analysis. For code execution paths that do not lead to data leakage, AppAudit performs source code pruning operations. At the same time, this customized dynamic testing solution can implement different detection strategies for different apps. Such a scheme not only realizes lightweight detection but also reduces false positives and improves the detection rate.

Zhang et al. [26] proposed CoDroid, a hybrid analysis method based on sequences, a mixture of static decompilation extraction to remove the opcode of operands and the use of Strace to record the system’s dynamic features system calls, and the use of Monkey for the interactive operation, and taking into account the sequences have a certain degree of temporal order, and so the natural language processing and fed into the CNN-BiLSTM-Attention classification model.

Dhalaria et al. [27] proposed a new combination of dynamic and static detection, firstly based on the static analysis extracted to the API calls, command line strings, permissions, and intent information, and then dynamically run each APK for 3 minutes to extract the encryption operations, dynamic permissions, information leakage, and system call information using CuckooDroid dynamically, altogether collected in static 352 static features and 323 dynamic features, and used the information gain sorting algorithm to filter the relevant features, in which the static features are filtered to 110 in binary classification and 35 in multiple classification, and the dynamic features are filtered to 99 features in binary classification and 35 in multiple classification. Moreover, in this paper, the dataset was made public on github and kaggle, samples from VirusShare, ApkMirror and ApkPure were used as the dataset, a collection of 4400, in which the malware was from VirusShare and benign software was from the other two, and 3547 were left over by using md5 de-weighting. Avira Antivirus validation got 1747 malware and 1800 benign software classified 13 families, and finally classified and identified by SVM, DT, RF, NB, KNN, PART, and MLP. The experimental classification comparing the use of only static and dynamic features and the combined use of all the features shows that the Random Forest algorithm has the best results.

Ding et al. [28] also proposed a new hybrid detection method, which compared to other hybrid methods; its static and dynamic work are not processed in parallel but in serial, and the dynamic analysis is triggered only when the static classification probability is not certain enough. It extracts permissions and intent in the static case and uses three feature selection methods, chi-square, ANOVA, and mutual information, to generate a subset of features, compares them using algorithms such as KNN and Random Forest, and finds that Random Forest is better and chi-square’s feature selection method performs the best. The dynamic analysis focuses on traffic detection, but compared to other work that focuses only on unidirectional traffic, HTTP protocols, and transport layer protocols, it focuses on the session and all protocol layers and then classifies them using the Res7LSTM model, which not only works with fewer static features and ensures sufficient accuracy, but also when traffic is cut across all layers of the session and protocols, the accuracy increases from 71.48% to 99%. The malicious samples it uses are from CICAndMal2017.

In KronoDroid [29] the authors argue that Android samples are a dynamic and evolving process and their temporal characteristics should be taken into account, so it discloses a set of hybrid feature datasets with conceptual offsets of Android samples and systematically analyzes a large number of publicly available Android sample datasets, but it does not perform the detection of malicious samples. It analyzes popular datasets of public Android malware detection in recent years and finds that the vast majority of datasets do not take into account temporal effects and are limited to smaller time periods, ignoring the importance of the change and evolution of Android malware over time. What is more serious is that malware detection in recent years is still using these old data. This paper produces a sample library containing time-variant (2008–2020) by merging datasets and collecting both static and dynamic features and is the first to focus on different dynamic data sources (real phones, emulators).

Lu et al. [30] extracted static and dynamic features, respectively, and used Inspeckage to obtain dynamic features including encryption/decryption, file read/write, network data transfer, and other features; the authors concluded that static features are usually discrete so they are processed by deep belief network (DBN), while dynamic features are time-dependent, so the authors used the gate recurrent unit (GRU) for processing.

In the EC2 [31] method, the authors combined the classification and clustering models to classify Android malware families accurately. They used DroidBox to get some dynamic features and static features were only obtained from the Manifest file. The authors also concluded that pure classification can not classify the families with fewer samples well. In contrast, pure clustering can have better results in families with small samples, but the classification accuracy could be better. However, the classification accuracy is poor, so the authors designed a method to combine the clustering algorithm and classification algorithm by thresholding for family classification and discovering some new unknown families.

Scholars have conducted much research on Android malware detection and achieved some results. Static analysis can obtain information such as instructions and permissions by using decompilation technology, which has the advantages of fast and high coverage. However, it cannot handle encrypted data and reflection mechanisms. Dynamic analysis monitors the device data and own data generated by the sample during operation and has a low false alarm rate. However, it is more environmentally demanding, has lower code coverage, and samples may hide their malicious behavior. Hybrid analysis combines the advantages of the two methods to improve detection effectiveness and efficiency further.

1.6 Chapter Summary

This chapter focuses on what Android malware is, the main categories of Android malware, and its harm to users from the Android operating system. Secondly, detection techniques to combat malware have emerged to cope with the spreading and propagation of malware, so this chapter also describes in more detail the development process of Android malware detection, the current state of the research, and future challenges. The following chapters of this book will provide a detailed description of malware detection, the malware adversarial method, and its new techniques for malware detection.

References

1.

Zarni Aung, Win Zaw et al. Permission-based android malware detection[J]// International Journal of Scientific & Technology Research. 2013, 2(4): 228–234.

2.

Hein, Chit La Pyae Myo and Myo, Khin Mar et al. Permission-based feature selection for android malware detection and analysis[J]// International Journal of Computer apps. 2018,181(19): 29–39.

3.

Tian, Ke and Yao, Danfeng and Ryder, Barbara G and Tan, Gang and Peng, Guojun et al. Detection of repackaged android malware with code-heterogeneity features[J]// IEEE Transactions on Dependable and Secure Computing. 2017, 17(1): 64–77.

4.

Wu, Dong-Jie and Mao, Ching-Hao and Wei, Te-En and Lee, Hahn-Ming and Wu, Kuo-Ping et al. Droidmat: Android malware detection through manifest and API calls tracing[C]// 2012 Seventh Asia joint conference on information security. IEEE, 2012: 62–69

5.

Kang H, Jang J, Mohaisen A, et al. Detecting and classifying android malware using static analysis along with creator information[J]. International Journal of Distributed Sensor Networks, 2015, 11(6): 479174.Crossref

6.

Niu W, Cao R, Zhang X, et al. OpCode-level function call graph based android malware classification using deep learning[J]. Sensors, 2020, 20(13): 3645.Crossref

7.

Feng Y, Bastani O, Martins R, et al. Automated synthesis of semantic malware signatures using maximum satisfiability[J]. arXiv preprint arXiv:1608.06254, 2016.

8.

Mariconti E, Onwuzurike L, Andriotis P, et al. Mamadroid: Detecting android malware by building Markov chains of behavioral models[J]. arXiv preprint arXiv:1612.04433, 2016.

9.

Enck W, Ongtang M, McDaniel P. On lightweight mobile phone app certification[C]//Proceedings of the 16th ACM conference on Computer and communications security. 2009: 235–245.

10.

Gao H, Cheng S, Zhang W. GDroid: Android malware detection and classification with graph convolutional network[J]. Computers & Security, 2021, 106: 102264.Crossref

11.

Enck W, Gilbert P, Han S, et al. Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones[J]. ACM Transactions on Computer Systems (TOCS), 2014, 32(2): 1–29.Crossref

12.

Costamagna V, Zheng C. Artdroid: A virtual-method hooking framework on android art runtime[C]//IMPS@ ESSoS. 2016: 20–28.

13.

Isohara T, Takemori K, Kubota A. Kernel-based behavior analysis for android malware detection[C]//2011 seventh international conference on computational intelligence and security. IEEE, 2011: 1011–1015.

14.

Arora A, Garg S, Peddoju S K. Malware detection using network traffic analysis in android based mobile devices[C]//2014 Eighth International Conference on Next Generation Mobile Apps, Services and Technologies. IEEE, 2014: 66–71.

15.

Zhao M, Ge F, Zhang T, et al. AntiMalDroid: An efficient SVM-based malware detection framework for android[C]//Information Computing and apps: Second International Conference, ICICA 2011, Qinhuangdao, China, October 28–31, 2011. Proceedings, Part I 2. Springer Berlin Heidelberg, 2011: 158–166.

16.

Yan L K, Yin H. DroidScope: Seamlessly reconstructing the OS and Dalvik semantic views for dynamic android malware analysis[C]//21st USENIX security symposium (USENIX security 12). 2012: 569–584.

17.

Shabtai A, Kanonov U, Elovici Y, et al. “Andromaly”: a behavioral malware detection framework for android devices[J]. Journal of Intelligent Information Systems, 2012, 38(1): 161–190.Crossref

18.

Afonso V M, de Amorim M F, Grégio A R A, et al. Identifying Android malware using dynamically obtained features[J]. Journal of Computer Virology and Hacking Techniques, 2015, 11: 9–17.Crossref

19.

Zhou Y, Jiang X. Dissecting android malware: Characterization and evolution[C]//2012 IEEE symposium on security and privacy. IEEE, 2012: 95–109.

20.

Tam K, Feizollah A, Anuar N B, et al. The evolution of android malware and android analysis techniques[J]. ACM Computing Surveys (CSUR), 2017, 49(4): 1–41.Crossref

21.

Fratantonio Y, Bianchi A, Robertson W, et al. Triggerscope: Towards detecting logic bombs in android apps[C]//2016 IEEE symposium on security and privacy (SP). IEEE, 2016: 377–396.

22.

Afonso V, Kalysch A, Müller T, et al. Lumus: Dynamically uncovering evasive Android apps[C]//Information Security: 21st International Conference, ISC 2018, Guildford, UK, September 9–12, 2018, Proceedings 21. Springer International Publishing, 2018: 47–66.

23.

Surendran R, Thomas T, Emmanuel S. A TAN based hybrid model for android malware detection[J]. Journal of Information Security and apps, 2020, 54: 102483.Crossref

24.

Chen S, Xue M, Tang Z, et al. StormDroid: A streaminglized machine learning-based system for detecting android malware[C]//Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. 2016: 377–388.

25.

Xia M, Gong L, Lyu Y, et al. Effective real-time android app auditing[C]//2015 IEEE Symposium on Security and Privacy. IEEE, 2015: 899–914.

26.

Zhang N, Xue J, Ma Y, et al. Hybrid sequence’ based Android malware detection using natural language processing[J]. International Journal of Intelligent Systems, 2021, 36(10): 5770–5784.Crossref

27.

Dhalaria M, Gandotra E. A hybrid approach for android malware detection and family classification[J]. 2020.

28.

Ding C, Luktarhan N, Lu B, et al. A hybrid analysis-based approach to android malware family classification[J]. Entropy, 2021, 23(8): 1009.Crossref

29.

Guerra-Manzanares A, Bahsi H, Nõmm S. Kronodroid: Time-based hybrid-featured dataset for effective android malware detection and characterization[J]. Computers & Security, 2021, 110: 102399.Crossref

30.

Lu T, Du Y, Ouyang L, et al. Android malware detection based on a hybrid deep learning model[J]. Security and Communication Networks, 2020, 2020: 1–11.

31.

Chakraborty T, Pierazzi F, Subrahmanian V S. Ec2: Ensemble clustering and classification for predicting android malware families[J]. IEEE Transactions on Dependable and Secure Computing, 2017, 17(2): 262–277.Crossref

Part IIThe General Android Malware Detection Method

In the past ten years, the Android system has rapidly become the most popular mobile operating system due to its open source, rich hardware options, and millions of application markets. At the same time, the open source nature of the Android system also makes it the main target of malicious software. The rapid growth of malware has brought great harm to mobile smartphone users, including tariff consumption, privacy theft, and remote control. Therefore, in-depth research on the security issues of mobile applications is of great significance to the sound development of the smartphone ecosystem.

This part will introduce the existing Android malware detection methods. We mainly divide the literature collected into three categories: behavior-based Android malware detection methods, signature-based Android malware detection methods, and artificial intelligence-based Android malware detection methods. We will introduce these three contents in Chapters 3, 4, and 5, respectively, so that readers can fully understand the whole picture of Android malware detection methods.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_2

2. Feature Code Based Android Malware Detection Method

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

The fundamental principle of the feature code based Android malware detection method is to leverage unique feature codes associated with each app and compare them with the information stored in an existing database of malware feature codes. By doing so, the method determines whether the feature code of a given app matches any known malware feature codes. If a match is found, it indicates that the tested app is a piece of malicious software. Therefore, the construction of a comprehensive database of existing malware feature codes is particularly crucial for these types of methods. Based on whether they contain program semantic information, we can classify the existing malware feature codes into two categories: traditional feature codes (Enck et al. On lightweight mobile phone app certification. Proceedings of the 16th ACM conference on computer and communications security 235–245, 2009; Zhou et al. NDSS 25(4): 50–52, 2012; Seo et al. J Netw Comput Apps 38: 43–53, 2014; Zheng et al. Droid analytics: a signature-based analytic system to collect, extract, analyze, and associate android malware. 2013 12th IEEE international conference on trust, security, and privacy in computing and communications. IEEE, Piscataway 163–171, 2013; Feng et al. Apposcopy: semantics-based detection of android malware through static analysis. Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering 576–587, 2014) and semantic feature codes (Feng et al. Apposcopy: Semantics-based detection of android malware through static analysis. Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering 576–587, 2014; Feng et al. Automated synthesis of semantic malware signatures using maximum satisfiability. Preprint. arXiv:1608.06254, 2016).

2.1 Detection Based on Traditional Feature Codes

2.1.1 Introduction

Early feature codes for Android apps were essentially the digital signatures of each app. According to the requirements of the Android system, every app software must possess a valid digital signature to be uploaded to the app market. Each digital signature uniquely identifies the app, and thus, the most primitive malware detection method involved matching the digital signature of a tested app with those in a database of known digital signatures of malware. However, this detection method was susceptible to bypassing through repackaging techniques, wherein the program could be modified and re-signed, rendering the detection ineffective.

To address these limitations, Enck et al. [1] in 2009 proposed a lightweight detection method based on permission usage rules Kirin. Kirin constructed nine ideal security permission rules, and if the permissions requested by a tested software violated any of these rules (e.g., requesting both SEND$$\_$$SMS and WRITE$$\_$$SMS permissions simultaneously), the software would be identified as malicious and its installation would be denied. Although this method could to some extent identify malicious software, it suffered from high false positives, often misidentifying benign apps that legitimately required access to features like sending SMS. Additionally, by modifying the permissions requested in the AndroidManifest.xml configuration file, it could be easily circumvented. Zhou et al. [2] introduced a permission script-based malware detection method, which is called DroidRanger. This method augmented the permission usage rules with additional information, such as API calls and critical strings, while also considering the loading of remote code and the dynamic execution information of the software. This combined approach aimed to enhance detection effectiveness through a dynamic-static combination.

Apart from using permission usage rules as feature codes, other information like API calls can also be abstracted into feature codes. DroidAnalyzer, proposed by Seo et al. [3], analyzes commonly used API functions in malicious software (e.g., getLine1Number() can get phone numbers) and elevated permission-related commands (e.g., reboot command to restart the device) as feature codes for matching and detecting Android apps under test. Another method, DroidAnalytics, introduced by Zheng et al. [4] employs three layers of feature codes with different granularities. The first layer represents the hash values of API call sequences in each method body, the second layer comprises the corresponding hashes for each class based on the first layer feature codes, and the third layer includes the corresponding hashes for each software app based on the second layer feature codes. This approach of using different granularities of feature codes enables effective detection of different types of malicious code, such as method-level, package-level, and APK-level threats.

2.1.2 DroidAnalyzer: A Case Study in Android Malware Analysis

DroidAnalyzer serves as a quintessential example of an Android malware analysis tool based on traditional feature codes, aimed at uncovering potential vulnerabilities within Android apps. This section elucidates the research methodology and content of Android malware detection using traditional feature codes, focusing on the implementation of DroidAnalyzer.

To enhance the detection accuracy, researchers initially researched commonly used risky APIs and strings within known mobile malware. Additionally, they examined the primary functionalities, commands, and keywords prevalent in root-exploiting malware and root apps to encompass the latest trends in malicious software. Consequently, a definitive list of risky APIs [3] and suspicious [3] root-related keywords was formulated. Leveraging these lists, the core detection algorithm [3] of DroidAnalyzer was designed to identify potential vulnerabilities by detecting suspicious root functionalities. These vulnerabilities encompassed risks such as information leakage, monetary losses due to SMS sending or invocation of advanced services, and potential involvement in mobile botnets.

The primary detection algorithm employed in DroidAnalyzer relies on keyword searching techniques, particularly valuable for detecting root functionalities, as root tools often follow well-known patterns for executing commands and copying files.

2.1.2.1 Suspicious Android APIs and Keywords [3]

In the pursuit of enhancing Android malware detection accuracy, this section focuses on identifying suspicious keywords employed in malicious functionalities, such as permission escalation or involvement in botnets. Specifically, these suspicious keywords represent critical commands or strings associated with potentially malicious behavior. By detecting the presence of these keywords in collected apps, researchers can identify apps exhibiting abnormal behavior. To achieve this, a meticulous analysis of the Android Malware Genome Project dataset, comprising existing mobile malware, was undertaken to identify commonly used suspicious APIs and primary keywords in malicious software. Tables 2.1 and 2.2 show a few risky APIs and keywords that were identified.

Table 2.1

Examples of suspicious APIs [3]

No.

API

Description

1

getNeighboringCellInfo()

Base station leakage

2

TelephonyManager.CALL$$\_$$STATE$$\_$$RINGING

Monitor or intercept telephone calls

3

getLastKnownLocation()

Location leakage

4

getInstalledPackages()

Installation list leakage

5

getAccountsByType()

Account operation

6

getDisplayMessageBody()

SMS Location leakage

7

getSubscriberId()

Device IMSI leakage

8

sendTextMessage()

Sending SMS

9

getLine1Number()

Phone number leakage

Table 2.2

Examples of commands used in privilege escalation [3]

No.

Command

Description

1

su root

Change the current user to root

2

reboot

Reboot the device

3

zergRush, m7, fre3vo

Root exploit keywords

4

insmod

Install a loadable kernel module

5

sh

Execute a shell command

6

mount -o remount

Make a read-only file system writeable

For instance, for the monetization malware HippoSMS, the researchers found that the malware uses sendsms() to send SMS messages and uses localContentResolver2.delete() to delete SMS messages from the cellular service provider.

In the case of Android spyware JackeeyWallpaper, it leaks IMSI using getSubscriberld() and IMEI using getDeviceID(). Furthermore, recent spyware like Zitmo employs getDeviceld(), getSubscriberld(), getActivationCode(), and getLine1Number() to transmit IMEI, IMSI, activation codes, phone numbers, and other information to the Command$$\&amp;$$Control (C$$\&amp;$$C) server. It also utilizes getMessageBody(), getOriginatingAddress(), and getDateTimeinstance() to collect relevant information from SMS messages.

The mobile botnet malware Android-SpywareMinServ contains commands such as “note,” “push,” “soft,” “window,” “xbox,” and “mark,” as well as APIs for sending SMS messages and leaking user information. It uses the “note” command to send SMS messages to high-rate numbers and the “push” command to receive instructions to transfer user information to the C$$\&amp;$$C server. Additionally, the “soft” command installs apps from the C$$\&amp;$$C server, the “window” command navigates to specific websites via the mobile browser, and the “mark” command adds bookmarks for “g3g3.cn,” “android.paojiao.cn,” and “ct2.paojiao.cn.”

Foncy, being a type of malware seeking to gain root privileges, embeds a root exploit program into the assets folder of repackaged apps. The root exploit file is granted permissions of “777,” allowing all users to read, write, and execute it. Then execute the root vulnerability to exploit the program. Upon successful acquisition of root privileges, Foncy-infected phones will download and install additional malicious software from the C$$\&amp;$$C server. Research shows that common commands and keywords used in Root exploits include su, sudo, mount(), and chmod(). These commands are used by Root exploit malware on both Linux OS and Android OS.

2.1.2.2 Main Algorithm of DroidAnalyzer

This section provides an in-depth exposition of the primary algorithm employed by DroidAnalyzer. Given an input app, the algorithm deduces its level of suspicion. Researchers set four possible levels of suspicion as follows:

R: Definitely a rooting app or root exploit malware

RS: Suspected to be a rooting app or apps for rooted phones

S: Suspected to be a malware without function of rooting

C: A normal app without containing any suspicious API calls or keywords

and subsequently identified Android APIs with risky functionalities prevalent in the existing collection of mobile malware. These risky APIs were then utilized to construct an initial filtering mechanism targeting suspicious apps.

Initially, the app is assigned a default suspicion level C, which is the lowest suspicion level. The app’s MD5 hash is calculated and cross-referenced in the given signature database. If a match is found, the suspicion level for this app is recorded. If this suspicion level is R, which is the highest suspicion level, the algorithm terminates and indicates R as the suspicion level for this app. Otherwise, this input app is extracted and the files in this app package are examined. For each file in the app package, its MD5 hash is checked for signatures. If a signature’s suspicion level is higher than the app’s current suspicion level, the app’s suspicion level is updated to the higher of the two values. It indicates this suspicion level and exits, if the algorithm determines that the app’s suspicion level is R.

Subsequently, the algorithm proceeds to the keyword scanning stage, wherein it searches for risky keywords within the disassembled app source code. The list of risky keywords and APIs is defined in Tables 2.1 [3] and 2.2 [3]. The DEX files are disassembled using the Dedexer tool, converting them into assembly code (DDX files). These DDX files, along with other files within the “assets/” folder or resource folder, are scanned to find matches with the list of keywords. Every keyword or keyword cluster receives an assigned suspicion level. Upon each keyword match, the app’s suspicion level undergoes an update determined by the higher value between the app’s current suspicion level and the suspicion level linked to the matched keyword. Finally, after scanning all files to search for risky keywords, the algorithm generates the app’s conclusive suspicion level.

It is crucial to note that the decompressed files within the app may contain archived files that harbor potential malicious code. Alongside decompressing the app, the algorithm recursively extracts the archived files contained within this app package. For example, Fancy is an Android malware. It contains archived files (apps) hidden in form of .png images. These files contain the actual code of the malicious software that sends messages to high-rate numbers. Additionally, they encompass hidden executable files intended to acquire Root access. Consequently, it becomes crucial to delve deeper into the contents of these archived files by employing the previously mentioned keyword-centric search algorithm.

Algorithm 1: Suspiciousness Level and Risky APIs or Keywords [3]

Input: The app to be examined

Output:The suspiciousness level of inApp and risky APIs or

keywords used in inApp

md5=computeMD5(inApp);

suspLevel=C;

suspLevel=search (suspLevel, md5, sigDB);

If{suspLevel == R}{

    report rooting app;

    exit;

}

Else{

    uncompress(inApp);

    isMalicious = false;

    While{true}{

        For{all archive file ‘archFile' inside the working

        directory}{md5 = computeMD5(archFile);

            suspLevel = search(suspLevel, md5, sigDB);

            If{suspLevel == R}{

                report rooting app;

                break;

            }

            Else { decompress(archFile); }

        }

        If{all files are decompressed OR suspLevel == R}  break;

    }

    If{suspLevel != R}{

        For{all .dex file ‘dexFile' in the working directory}{

            run dedexer on dexFile;

            For{all resulting .ddx file 'ddxFile'}{

                search for keywords in keywordsDB in the text

                file ddxFile; update suspLevel accordingly;

            }

        }

        For{all binary file ‘binFile' in the working directory}{

            search for keywords in keywordsDB in the binary file

            binFile; update suspLevel accordingly;

        }

    }

    report suspLevel;

}

2.2 Detection Based on Semantic Feature Codes

2.2.1 Introduction

Early feature codes lacked program semantics, making them susceptible to evasion through existing obfuscation techniques. Subsequent research endeavors sought to incorporate program semantic information into feature codes. For instance, in 2014, Feng et al. [5] introduced Apposcopy, a malware detection approach grounded in semantic feature codes. Unlike traditional feature-based malware detection methods, Apposcopy’s feature codes comprise three types of information: data flow, component type, and control flow. Furthermore, this method utilizes a high-level description language to delineate the feature codes associated with each malware family [26]. However, Apposcopy necessitates security experts to meticulously analyze samples within each family and extract the behavioral feature codes, entailing significant human and material resources. Furthermore, the family feature codes constructed by this method are only effective for known families, making it incapable of detecting unknown family samples.

In 2017, Feng et al. [6] further improved the Apposcopy method and introduced Astroid. Astroid first constructs an Inter-Component Call Graph (ICCG) by analyzing the data flow and control flow information in the software. It then formalizes the ICCG and represents the feature codes as a conjunction normal form. For samples from the same family, Astroid analyzes their corresponding conjunction normal forms and mines the maximum common subgraph, representing the common behavior of the family. When dealing with a suspicious malware sample, the method employs subgraph matching to determine if the sample contains the subgraph feature code of a particular family. Although this method can automatically construct subgraph feature codes from family samples, it still requires some manual involvement in selecting representative samples for feature code construction.

The aforementioned methods do not analyze the native code of programs, leading to potential false negatives when detecting malware that hides malicious behavior within native code. According to studies [7, 8], Native code is present in 37$$\%$$ of market apps and 86$$\%$$ of widely used apps. To address the limitations of native code in malware detection, Alam et al. [9] introduced DroidNative, a technique designed for identifying malicious native code that facilitates cross-platform analysis (both x86 and ARM architectures).DroidNative first parses native code and represents it using an intermediate language (MAILIR0l), constructing relevant behavioral patterns. Based on this intermediate language, DroidNative builds an annotated control flow graph (ACFG) and describes each statement in the control flow graph (CFG). Subsequently, using a sliding window approach on the ACFG, DroidNative splits it and constructs feature codes for malware detection.

2.2.2 DroidNative: A Case Study in Android Malware Analysis

Symantec and F-Secure threat reports [25] indicated that in 2013 and 2014, almost all mobile malware development (approximately 99$$\%$$) was concentrated on the Android platform. Creators of malware employ covert mutations, specifically obfuscation, to generate variations of malicious software. This strategy enables them to elude detection by detectors relying on signature-based methods. Furthermore, a considerable number of advanced detectors utilize static analysis techniques for identifying such variants. However, these detectors operate solely at the bytecode level, rendering them unable to detect malware embedded within native code. In recent investigations, it has been unveiled that native code is present in 86$$\%$$ of the most favored Android apps, thus establishing native code [8] malware as a potential threat vector.

This section presents DroidNative, an Android malware detector that employs distinct control flow patterns to mitigate the effects of obfuscation and offers automation. Remarkably, DroidNative stands out as the pioneering system to formulate semantic-based cross-platform (x86 and ARM) signatures at the Android native code level. This innovation allows for the identification of malware embedded within bytecode or native code. In testing with a dataset of 5,490 samples, DroidNative demonstrated a false positive rate of 2.7$$\%$$ and a detection rate (DR) of 93.57$$\%$$ [9]. Additionally, when confronted with traditional malware variants, it outperformed academic and commercial tools, achieving a DR of 99.48$$\%$$—a notable improvement over the DR range of existing tools, which spans from 8.33$$\%$$ to 93.22$$\%$$ [9].

2.2.2.1 Static Analysis in DroidNative

DroidNative makes an in-depth analysis of the binary code of Android app. In order to achieve this goal, DroidNative adopts Malware Analysis Intermediate Language (MAIL) [10]. This section will delve into MAIL, two techniques for generating malware detection signatures using MAIL, and its extension to suit Android malware detection.

1.

Malware Analysis Intermediate Language (MAIL)

MAIL (Malware Analysis Intermediate Language) [10] is a specialized intermediate language designed for the analysis of malicious software. Serving as a bridge between the raw binary code and higher-level analytical layers, it aims to provide a standardized and extensible representation. By abstracting away the intricacies of low-level binary code, MAIL offers analysts a higher level of abstraction, facilitating a more efficient understanding and interpretation of malicious software behavior.

Distinguished by its flexibility and generality, this intermediate language allows security experts to apply a variety of techniques and algorithms during the analysis process. The design of MAIL is geared toward supporting both dynamic and static analysis of malicious software samples, empowering researchers to comprehensively examine the functionality and interactions of malicious code.

Through the utilization of MAIL, researchers can more easily detect and comprehend potential threats within Android apps. The use of this intermediate language not only streamlines the analysis process but also fosters information sharing and collaboration, contributing to the construction of robust defense mechanisms aimed at safeguarding users from potential security risks.

There are 21 patterns in Table 2.3, and each MAIL statement is marked with one of them. Patterns represent the types of MAIL statements, which makes the comparison and matching of MAIL easier. For example, constant assignment statement and non-constant assignment statement are two different models. Applying these patterns to CFG (Control Flow Diagram) of annotator helps pattern matching, thus detecting malware.

Table 2.3

Patterns used in MAIL. sp is the stack pointer, cf and zf are the zero and carry flags, respectively, and r0, r1, r5 are the general purpose registers [9]

Pattern

Description

ASSIGN

An assignment statement, e.g., $${r_0} = {r_0} + {r_1}$$;

ASSIGN$$\_$$CONSTANT

An assignment statement including a constant, e.g., $$r_0 = r_0 + 0\times 01$$;

CONTROL

A control statement where the target of the jump is unknown, e.g., if $$(zf == 1)$$ JMP [$${r_0} + {r_1} + 0\times 10$$];

CONTROL$$\_$$CONSTANT

A control statement where the target of the jump is known, e.g., $$if (zf == 1) JMP 0x400567$$;

CALL

A call statement where the target of the call is unknown, e.g., CALL $$r_5$$;

CALL$$\_$$CONSTANT

A call statement where the target of the call is known, e.g., CALL 0x603248;

FLAG

A statement including a flag, e.g., $$cf == 1$$;

FLAG$$\_$$STACK

A statement including flag register with stack, e.g., $$eflags = [sp = sp - 0x1]$$;

HALT

A halt statement, e.g., HALT;

JUMP

A jump statement where the target of the jump is unknown, e.g., JMP [$${r_0} + {r_5} + 0\times 10$$];

JUMP$$\_$$CONSTANT

A jump statement where the target of the jump is known, e.g., JMP 0x680,376

JUMP$$\_$$STACK

A return jump, e.g., JMP $$[sp = sp - 0x8]$$

LIBCALL

A library call, e.g., compare($$r_0, r_5$$);

LIBCALL$$\_$$CONSTANT

A library call including a constant, e.g., compare($$r_0, 0\times 10$$);

LOCK

A lock statement, e.g., lock;

STACK

A stack statement, e.g., $$r_0 = [sp = sp - 0x1]$$;

STACK$$\_$$CONSTANT

A stack statement including a constant, e.g., [$$sp = sp + 0x1$$] = 0x432,516;

TEST

A test statement, e.g., $$r_0$$ and $$r_5$$;

TEST$$\_$$CONSTANT

A test statement including a constant, e.g., $$r_0$$ and 0x10;

UNKNOWN

Any unknown assembly instruction that cannot be translated.

NOTDEFINED

The default pattern, e.g., all the new statements when created are assigned this default value.

2.

Signature generation techniques

Malware uses obfuscation techniques [11–13] to avoid detection by general signature-based malware detectors. Confusion technology makes it difficult for general detectors to understand, analyze, and detect malware. DroidNative uses two signature technologies aimed at countering obfuscation technology to detect Android malware using obfuscation technology. Among them, the signature is constructed by MAIL mode.

(a)

Annotated Control Flow Graph

ACFG [14] captures the program’s control flow semantics through the analysis of the control flow diagram generated from the app. It annotates this diagram with patterns outlined in Table 2.3. Before delving into the details of this technology, let us formally define and elucidate how DroidNative constructs ACFG:

Definition 1

A basic block constitutes a segment of code wherein the execution flow can only enter through the first statement and continues uninterrupted until the last statement within the block. In the context of a CFG, basic blocks are typically represented as nodes.

Definition 2

A control flow edge denotes the directional progression of program execution, signifying the transition from one basic block to another. Each control flow edge connects two basic blocks, elucidating the program’s transition from one basic block to another. In a CFG, these edges are conventionally represented as directed edges.

Definition 3

The control flow graph manifests as a directed graph, wherein the nodes signify basic blocks, and the directed edges represent control flow transitions. The inherent directionality of the edges delineates the sequential flow of program execution. ACFG is a kind of control flow graph, which assigns a MAIL pattern to each statement of CFG.

The incorporation of MAIL patterns in annotating the CFG facilitates the generation of considerably reduced CFGs, amenable to reliable analysis for effective malware detection. This stands in contrast to the conventional usage of larger CFGs in CFG-based methodologies. The inherent advantage lies in the ability to disassemble an app into a compilation of smaller ACFGs as opposed to a singular extensive CFG. Consequently, this assortment can undergo expedited analysis and pattern matching with malware, presenting a marked improvement in efficiency compared to the use of larger CFGs.

As an illustration, during the analysis of an app, it is disassembled into a series of compact ACFGs, constituting the unique signature for that particular app. During the detection phase, the ACFGs within this signature undergo comparison with those of established malware. If a substantial proportion of the ACFGs in the app align with those of recognized malware, the app is subsequently identified and flagged as potentially malicious. The efficiency of this matching process is notably rapid, owing to the inherently modest size of the individual ACFGs being compared.

(b)

Sliding window of difference

Differential Sliding Window (SWOD) [15], as an opcode-based methodology for malware detection, stands out prominently in contemporary research. The utilization of opcodes in malware detection confers a significant advantage, particularly in the realm of real-time detection capabilities. Nevertheless, contemporary approaches [16–19] utilizing opcode-based malware detection are confronted with multiple challenges. First and foremost, the dynamic nature of opcode patterns is influenced by factors such as varying compilers, distinct optimization levels within the same compiler, and the compilation of code for diverse platforms. Secondly, the incorporation of obfuscation in different malware iterations has the potential to modify the distribution of opcodes. Thirdly, the runtime of these techniques is contingent upon the selection of features for mining within the program. Striking a balance is critical, as an excess of features may yield high detection rates at the expense of increased execution time, while an insufficient selection leads to a counterproductive outcome.

SWOD is a novel framework that captures the nuanced distribution variances of MAIL patterns within a window, offering a high-level abstraction of program opcodes. This unique approach ensures analysis independence across diverse compilers, compiler optimization levels, instruction set architectures, and operating systems. Significantly, the size of SWOD can be dynamically tailored to align with the specific distribution characteristics of MAIL patterns within a given dataset, facilitating the computation of weights associated with MAIL patterns in said dataset. The differential between the malware distribution of MAIL patterns and benign samples is quantified by the number of SWODs capable of seamlessly traversing this differential window. This numerical representation of SWODs becomes instrumental in determining the weight attributed to MAIL patterns.

To counteract the obfuscation effects introduced by diverse malware variants, SWOD incorporates a heuristic control flow weight (CFWeight) scheme for the extraction and examination of control flow semantics, specifically variations in control flow within the program. The CFWeight scheme adheres to the following heuristics: (a) assigning a weight of 2 to each CONTROL statement, (b) backward jumps (possibly indicating loops) are assigned a weight equal to twice the length of the jump, (c) assigning a weight of 1 to the last statement of a block, as well as each JUMP and CALL statement, (d) attributing a weight equivalent to the length of the jump to every control flow variation (JUMP, CONTROL, or CALL), and (e) assigning a weight to jumps targeting external functions equivalent to their distance from the last block of the function, measured in the number of blocks, plus 1.

The ultimate weight of a MAIL statement is determined by the amalgamation of its CFWeight and the weight associated with its specific pattern. These conclusive weights attributed to the MAIL program statements are then consolidated in a weight vector, effectively constituting the program’s signature. Following this consolidation, the signature undergoes an ascending order arrangement, facilitating a streamlined and efficient comparison methodology through the utilization of index-based arrays.

3.

Extensions

MAIL systems currently in existence exhibit constraints that potentially impact their efficacy in detecting Android malware. Consequently, DroidNative endeavors to augment the capabilities of MAIL through the following enhancements:

(a)

Initially conceptualized for Windows malware, MAIL was tailored exclusively for the support of x86 binary files. In a substantial engineering undertaking, DroidNative broadens the scope of MAIL by incorporating support for ARM binary files. This extension necessitates a meticulous process wherein each ARM instruction is systematically translated into the corresponding MAIL representation.

(b)

During the evaluation of DroidNative, it was observed that SWOD had a fast processing speed but a relatively lower detection rate when applied to Android malware. On the other hand, ACFG had a slower speed but a higher overall detection rate. To tackle this challenge, a hybrid methodology was formulated to amalgamate the strengths of both techniques. The initial step involves the app of DroidNative-SWOD as a pre-filter across all samples. Subsequently, samples identified as benign by DroidNative-SWOD undergo further scrutiny through the app of DroidNative-ACFG. The outcomes derived from each stage are seamlessly integrated to yield the ultimate result. This synergistic approach allows DroidNative to uphold the precision of ACFG while harnessing the efficiency attributed to SWOD.

(c)

The investigation brought to light a noteworthy inefficiency in the training phase of MAIL when confronted with a substantial volume of Android apps. In the training process, MAIL preserves signatures for every malware sample within the training set. In the subsequent testing phase, each app is meticulously compared against each signature within the set of malicious software. The inefficiency becomes pronounced with the expansion of the training set, resulting in a cumbersome and inefficiently large signature database. In response to this challenge, a refinement was implemented in the training stage of DroidNative. Notably, DroidNative now retains solely the unique and distinct signatures of the malware samples. In instances where existing samples exhibit a high degree of similarity to new samples, as measured by a predetermined similarity threshold, only one representative sample is preserved. This strategic pruning of similar signatures results in a significantly diminished signature database. While this adjustment entails a marginally increased training time, it concurrently contributes to noteworthy reductions in both testing time and memory usage.

2.2.2.2 System Design and Implementation

This section delineates the intricacies of the system design implemented in DroidNative, presenting a comprehensive and automated methodology for the detection of malware. The core focus revolves around the analysis of native code within Android apps. The systematic analysis involves scrutinizing the control flow patterns inherent in the code, subsequently comparing these patterns against those discerned in recognized instances of malicious software.

As elucidated previously, the system functions through an in-depth analysis of native code within Android apps. Upon the submission of an app for analysis, it undergoes a segmentation process into bytecode and native code components. Subsequently, any bytecode component is routed to the ART (Android Runtime) compiler, facilitating the generation of native binary files. Simultaneously, the native code component proceeds to the subsequent stage. At this juncture, all elements of the app are present in the form of native code.

Following this, the binary code undergoes a meticulous disassembling process, subsequently undergoing translation into Malware Analysis Intermediate Language (MAIL) code. Following the generation of MAIL, DroidNative progresses through two distinct stages: training and testing. In the training phase, the classifier undergoes a rigorous training process on well-established instances of malicious software. This results in the creation of malware templates, specifically delineating behavioral features for both Annotated Control Flow Graph (ACFG) and Sliding Window of Differences (SWOD). In the subsequent testing phase, the similarity detector adeptly employs these meticulously crafted malware templates to identify and ascertain the presence of malicious software within the app under scrutiny. DroidNative employs a simple decision tree in implementing these two stages. If the analyzed app matches the malware template within a given threshold calculated based on experience, the app is flagged as malicious.

1.

Disassembler The first step in the DroidNative process is to disassemble it when providing native code for an app. The inherent challenge lies in the meticulous endeavor to ascertain the identification and disassembly of every segment of code. Addressing this challenge involves the app of two established and widely recognized techniques [20] for disassembly:

(a)

The LinearSweep methodology initiates the disassembly process from the initial byte of the code, proceeding methodically one instruction at a time until reaching the conclusion of the code segment. This approach operates under the assumption that instructions are stored sequentially and does not discern between embedded data and instructions. It is important to note that the LinearSweep technique may exhibit inaccuracies in scenarios where compilers or malware authors intermingle data with code. Despite this limitation, its notable advantage resides in its capacity to furnish comprehensive coverage of the entire code.

(b)

The Recursive Traversal technique engages in the intricate process of decoding instructions by adhering to the established control flow of the program. In scenarios where one instruction is referenced by another, this technique selectively disassembles a single instruction. A distinctive strength of this approach lies in its discernment between code and data. It is essential to acknowledge that challenges may arise in instances where the target address of a branch cannot be statically determined, potentially leading to the failure to identify and disassemble valid instructions.

To surmount the constraints inherent in linear scanning and recursive traversal, scholars amalgamate these two techniques within the disassembly process. For instance, in scenarios involving ambiguous branches, researchers adeptly annotate such addresses with MAIL and seamlessly transition to the subsequent instruction through the app of linear scanning. This integrative approach harnesses the strengths of both techniques, offering a more nuanced disassembly strategy.

A correlated challenge lies in the prevalent use of stripped binary files in Android, indicating the absence of debugging or symbol information. This absence poses a considerable hurdle in the static construction of a call (API) graph for these binary files and in the identification of function boundaries. Notably, IDA Pro stands out as the preeminent disassembler, leveraging pattern matching techniques to discern these functions. Nevertheless, the dynamic nature of these patterns, fluctuating across diverse compilers and even within different versions of the same compiler, renders this approach arduous to maintain and restricts its widespread applicability.

Since the primary focus of DroidNative is to detect malware variants, researchers may only need to locate points of control flow (i.e., behavior rather than function boundaries) and then compare this behavior with known malware samples used for detection. As a result, DroidNative addresses these challenges by constructing control flow patterns and using them for malware detection.

2.

Optimizer A pivotal challenge within the realm of static analysis for native code resides in the vast array of instructions supported by Intel x86 and ARM microprocessors—both prominent architectures underpinning Android. These architectures encompass an extensive repertoire of hundreds of distinct instructions within their respective instruction sets. In the pursuit of expediting static analysis, a critical imperative is to curtail the number of instructions necessitating manual processing. Many of these instructions prove extraneous to the domain of malware analysis, exemplified by instances like the PREFETCH instruction in Intel x86 and the PRFM instruction in ARM, both designed to shuttle data from memory to the cache. This optimization process involves the systematic elimination of such instructions and their analogs, thereby streamlining the analysis workflow. For a comprehensive enumeration of these instructions, refer to Appendix A.

In the realm of static analysis, a persistent challenge revolves around the intricate task of meticulously assembling accurate Control Flow Graphs (CFGs) [21] during the disassembly of Android binary programs. Simultaneously, there is a pressing need to curtail the runtime matching time for these CFGs. Android apps, with their compositional structure divided into distinct components, each serving unique purposes and activated by various events, introduce an additional layer of complexity. Moreover, the inherent inclusion of exceptions permeates the control flow architecture of Android apps. The dynamic localization of programmer-specified code for handling exceptions further complicates matters, resulting in the emergence of numerous unknown branch addresses within the disassembled Android binary program.

Compounding the challenge is the prevalence of multiple entry points, rendering the construction of an accurate CFG for the program a formidable task within the purview of static analysis. In response to this intricate challenge, DroidNative adopts a nuanced strategy. Instead of adhering to the conventional approach of constructing a singular large CFG, it pioneers the construction of multiple smaller interleaved CFGs tailored for a given program. These condensed CFGs, strategically designed to address the intricacies introduced by exceptions and the presence of multiple entry points, play a pivotal role in facilitating efficient matching and expediting the process of malware detection.

3.

MAIL generation The MAIL Generator is responsible for converting assembly programs into MAIL programs. This component undertakes pivotal tasks integral to the overall process, encompassing the translation of each assembly instruction into its corresponding MAIL statement. Certain instructions, such as PUSHA in the x86 architecture and STMDB in ARM architecture, warrant translation into multiple MAIL statements to ensure precision. Additionally, each MAIL statement is meticulously assigned a distinct pattern as part of the process. Furthermore, a concerted effort is made to curtail the diversity of instructions by amalgamating functionally equivalent assembly instructions. This strategic consolidation serves to streamline the subsequent actions of the MAIL generator, enabling it to utilize the predefined MAIL patterns articulated for the synthesis of a singular, comprehensive MAIL statement.

4.

Malware detection This segment delineates the intricate interplay among the Data Miner, Signature Generator, and Similarity Detector components in harnessing both ACFG and SWOD techniques for the identification of malicious software. The Data Miner meticulously scrutinizes the MAIL program for control and structural insights, playing a pivotal role in the collaborative efforts of the Signature Generator. Together, they embark on the construction of behavioral signatures for the MAIL program, available in two distinctive forms: ACFG or SWOD. Subsequently, the Similarity Detector takes center stage by juxtaposing the program’s signature against those derived from malicious software templates extracted during the training phase. This comparative analysis, guided by an empirically determined threshold, forms the bedrock of malicious software detection, ensuring a nuanced and effective approach.

In the pursuit of enhancing runtime efficiency, a pragmatic approach has been integrated into the architecture of the Similarity Detector—specifically, a streamlined binary decision tree classifier. This innovative design has demonstrated its efficacy in the discernment of variant malicious software. Presently, the classifier within the Similarity Detector leverages features common to malicious software samples. In prospective endeavors, researchers intend to undertake an extensive exploration, identifying shared features not only among benign samples but also in the comparative analysis between malicious and benign samples. These discerned features will serve as the bedrock for training classifiers, contributing substantively to the continual refinement of DroidNative and its capacity to curtail the False Positive Rate (FPR).

(a)

DroidNative-ACFG

In the quest for identifying Android malware variants, the protocol involves the meticulous construction of an ACFG (Annotated Control Flow Graph) for every function discerned within the annotated MAIL program. These ACFGs, essentially constituting the distinctive signatures of Android programs, undergo meticulous comparison with established malware templates. The objective is clear: to ascertain whether the program harbors any traces of malicious software. Should a substantial count of ACFGs within the program be identified as potentially malicious? The overarching classification of the app itself is consequently tagged as malicious.

Subsequently, the malware detection approach in this study is formally defined based on subgraph isomorphism [22].

Let $$G=(V_G,E_G)$$ and $$H=(V_H,E_H)$$ be any two graphs, where $$V_c$$, $$V_q$$, $$E_c$$Ec, and Er represent the vertex sets and edge sets of the graphs, respectively.

Definition 4

If for every edge $$e\in E_G f_V$$ maps the endpoints of e to the endpoints of edge $$f_E(e)$$, an edge bijection denoted as $$f_E = E_G \rightarrow E_H$$ and a vertex bijection (one-to-one mapping) denoted as $$f_V=V_G \rightarrow V_H$$ are consistent.

Definition 5

If there exists a vertex bijection $$f_V$$ and an edge bijection $$f_E$$ that are consistent, G and H are isomorphic graphs. This relationship is denoted as $$G\cong H$$.

Founded upon the outlined definitions, we articulate our approach to ACFG matching in the ensuing manner:

Let $$M=(V_M, E_M)$$ denote an ACFG of the malware and $$P=(V_P ,E_P )$$ denote an ACFG of the program, where $$E_P$$, $$E_M$$ and $$V_P$$, $$V_M$$ are the sets of edges and vertices of the graphs, respectively. Let $$P_sg =(V_sg ,E_sg )$$ where $$V_sg \subseteq V_P$$ and $$E_sg \subseteq E_P$$ (i.e., $$P_sg$$ is a subgraph of P). If $$P_sg \cong M$$, then P and M are considered as matching graphs.

Should a segment of a program’s control flow encompass malware? We categorize the program as such. Specifically, the program is designated as malware if a proportion (relative to an empirically determined threshold) of ACFGs within the signature of a malware program align with those in the signature of a given program.

Upon discovering alignment between the ACFG of the malevolent program and that of the subject program, scholars proceed to employ patterns for matching each statement within the corresponding nodes of both ACFGs. Successful matching necessitates uniform patterns across all statements in the matched nodes, notwithstanding potential divergences within the associated blocks. This methodology enables researchers to identify malware with more compact ACFGs and concurrently diminish the ACFGs’ size (quantified in terms of block count) to optimize detection speed. DroidNative executes this optimization by consolidating blocks within the ACFGs. The consolidation of blocks occurs exclusively if their amalgamation preserves the program’s control flow. When presented with two blocks, denoted as a and b, in the ACFG, consolidation transpires if all paths leading to node b traverse through block a, and all child nodes of a can be accessed through b.

(b)

DroidNative-SWOD

During the transition from assembly language to MAIL, each MAIL statement is ascribed a distinctive pattern. These MAIL patterns are subsequently endowed with weights, informed by SWOD, symbolizing disparities in the MAIL pattern distribution between samples of malicious and benign nature, as elucidated in Sect. 2.2.2.1(2-(b)). The CFWeight (Control Flow Weight) of MAIL statements is computationally derived by aggregating the weights allocated to the fundamental statements implicated, utilizing the heuristic methodology expounded in Sect. 2.2.2.1(2-(b)).

Subsequent to the formulation of the signature for a novel program, as explicated in the preceding section, a comparative analysis ensues with the signatures of all previously established training samples of malicious software. In the event of a correspondence with any existing signature, the newfound program is designated as malicious.

5.

Training The templates of malicious software employed for comparative analysis undergo their genesis in the training phase. This germination process entails an exhaustive examination of recognized malicious software, culminating in the extraction of their distinctive signatures, a procedure expounded upon in Sects. 2.2.2.1(b) and 2.2.2.2(4). During this juncture, researchers embark on the determination of critical metrics. Firstly, the ACFG matching threshold is ascertained, representing the optimal similarity score pivotal for achieving the zenith of Detection Rate (DR) on the designated dataset. In tandem, the SWOD value, denoting the dissimilarities between MAIL patterns of malicious software and benign samples, is computed, elucidated in Sect. 2.2.2.1(2). A case in point is the computed ACFG matching threshold, standing at 80$$\%$$ based on a 25$$\%$$ subsample of the dataset.

Subsequent to the computation of the control flow weight for each MAIL pattern, an intricate synergy with SWOD ensues to determine the aggregate weight for every individual MAIL pattern. Then, the calculated threshold and weight of each MAIL pattern are used to construct the malware template (malware signature).

6.

Implementation DroidNative, constructed in C/C$$++$$, comprises an extensive codebase exceeding 60,000 lines, which includes more than 32,000 lines of open source third-party libraries. At the core of DroidNative’s implementation is the Ahead-of-Time (AOT) compiler. This compiler plays a crucial role in converting Android apps from bytecode to native code. Notably, this conversion takes place on a dedicated testing server through cross-compilation, bypassing the confines of a mobile device. The AOT compiler, sourced from the Android Open Source Project, lacks detailed exposition on its algorithms and techniques within this chapter. However, researchers can obtain scripts used to properly configure and execute the compiler from Riley [23]. To facilitate future research and comparison with their systems against DroidNative, the source code of DroidNative [24] is publicly available.

2.3 Chapter Summary

In summary, traditional methods based on permission or API feature codes offer fast analysis speed, but they are susceptible to attacks by adversaries who use obfuscation techniques to alter the software’s syntactic structure, resulting in changes to the feature codes. Moreover, such methods, although capable of achieving malicious detection through precise matches with existing feature codes, fail to handle unknown malicious software. Feature codes that include semantic information can provide some resistance against obfuscation techniques, but constructing such feature codes often requires a certain level of manual analysis. Therefore, the future viable research direction lies in the automatic construction of semantic feature codes. This approach can enhance the ability to detect and combat malicious software effectively in the face of evolving obfuscation techniques.

x86 and ARM Instruction Lists in Alphabetical Order (Not Translated into MAIL Statements)

x86:

3DNOW, AAA, AAD, AAM, AAS, AESDEC, AESDECLAST, AESENC, AESENCLAST, AESIMC, AESKEYGENASSIST, ARPL BOUND, DAA, DAS, EMMS, ENTER, GETSEC, CLFLUSH, CLTS, CMC, CPUID, CRC32 FCLEX, FDECSTP, FEDISI, FEEMS, FENI, FFREE, FINCSTP, FINIT, FLDCW, FLDENV, FNCLEX, FNINIT FNSAVE, FNSTCW, FNSTENV, FNSTSW, FRSTOR, FSAVE, FSETPM, FSTCW, FSTENV, FSTSW, FXRSTOR FXRSTOR64, FXSAVE, FXSAVE64, FXTRACT INT 3, INVD, INVEPT, INVLPG, INVLPGA, INVPCID, INVVPID, LEAVE, LFENCE, LZCNT MFENCE, MONITOR, MPSADBW, MWAIT, PAUSE, PREFETCH, PSAD, PSHUF, PSIGN RCL, RCR, RDRAND, RDTSC, RDTSCP, ROL, ROR, RSM SFENCE, SHUFPD, SHUFPS, SKINIT, SMSW, SYSCALL, SYSENTER, SYSEXIT, SYSRET VAESDEC, VAESDECLAST, VAESENC, VAESENCLAST, VAESIMC, VAESKEYGENASSIST, VERR, VERW, VMCALL VMCLEAR, VMFUNC, VMLAUNCH, VMLOAD, VMMCALL, VMPSADBW, VMREAD, VMRESUME, VMRUN, VMSAVE VMWRITE, VMXOFF, VSHUFPD, VSHUFPS, VZEROALL, VZEROUPPER WAIT, WBINVD, XRSTOR, XSAVE, XSAVEOPT.

ARM:

BKPT, CDP, CDP2, CLREX, CLZ, CPS, CPSID, CPSIE, CRC32, CRC32C, DBG, DCPS1, DCPS2, DCPS3, DMB, DSB HVC, ISB, LDC, LDC2, MCR, MCR2, MCRR, MCRR2, MRC, MRC2, MRRC, MRRC2, PLD, PLDW, PLI, PRFM SETEND, SEV, SHA1C, SHA1H, SHA1M, SHA1P, SHA1SU0, SHA1SU1, SHA256H, SHA256H2, SHA256SU0 SHA256SU1, SMC, SSAT, SSAT16, STC, STC2, SVC USAT, USAT16, VCVT, VCVTA, VCVTB, VCVTM, VCVTN, VCVTP, VCVTR, VCVTT, VEXT, VLD1, VLD2, VLD3, VLD4 VLD1, VLD2, VLD3, VLD4, VQRDMULH, VQRSHL, VQRSHRN, VQRSHRN, VQRSHRUN, VQRSHRUN, VQSHL, VQSHLU VQSHL, VQSHRN, VQSHRN, VQSHRUN, VQSHRUN, VRINTA, VRINTM, VRINTN, VRINTP, VRINTR, VRINTX, VRINTZ VRSHL, VRSHR, VRSHRN, VRSRA, VRSUBHN, VST1, VST2, VST3, VST4, VTBL, VTBX, VTRN, VUZP, VZIP WFE, WFI, YIELD.

References

1.

Enck W, Ongtang M, McDaniel P. On lightweight mobile phone app certification[C] Proceedings of the 16th ACM conference on Computer and communications security. 2009: 235–245.

2.

Zhou Y, Wang Z, Zhou W, et al. Hey, you, get off of my market: detecting malicious apps in official and alternative android markets[C] NDSS. 2012, 25(4): 50–52.

3.

Seo S H, Gupta A, Sallam A M, et al. Detecting mobile malware threats to homeland security through static analysis[J]. Journal of Network and Computer apps, 2014, 38: 43–53.Crossref

4.

Zheng M, Sun M, Lui J C S. Droid analytics: a signature based analytic system to collect, extract, analyze and associate android malware[C] 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications. IEEE, 2013: 163–171.

5.

Feng Y, Anand S, Dillig I, et al. Apposcopy: Semantics-based detection of android malware through static analysis[C]//Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering. 2014: 576–587.

6.

Feng Y, Bastani O, Martins R, et al. Automated synthesis of semantic malware signatures using maximum satisfiability[J]. arXiv preprint arXiv:1608.06254, 2016.

7.

Afonso V, Bianchi A, Fratantonio Y, et al. Going native: Using a large-scale analysis of android apps to create a practical native-code sandboxing policy[C] The Network and Distributed System Security Symposium 2016. 2016: 1–15.

8.

Sun M, Tan G. Nativeguard: Protecting android apps from third-party native libraries[C] Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networks. 2014: 165–176.

9.

Alam S, Qu Z, Riley R, et al. DroidNative: Automating and optimizing detection of Android native code malware variants[J]. computers $$\&amp;$$ security, 2017, 65: 230–246.

10.

Alam S, Horspool R N, Traore I. MAIL: Malware Analysis Intermediate Language: a step towards automating and optimizing malware detection[C] Proceedings of the 6th International Conference on Security of Information and Networks. 2013: 233–240.

11.

Faruki P, Bharmal A, Laxmi V, et al. Evaluation of android anti-malware techniques against dalvik bytecode obfuscation[C] 2014 IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications. IEEE, 2014: 414–421.

12.

Rastogi V, Chen Y, Jiang X. Droidchameleon: evaluating android anti-malware against transformation attacks[C] Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security. 2013: 329–334.

13.

Zheng M, Lee P P C, Lui J C S. Adam: an automatic and extensible platform to stress test android anti-virus systems[C] International conference on detection of intrusions and malware, and vulnerability assessment. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012: 82–101.

14.

Alam S, Traore I, Sogukpinar I. Annotated control flow graph for metamorphic malware detection[J]. The Computer Journal, 2015, 58(10): 2608–2621.Crossref

15.

Alam S, Sogukpinar I, Traore I, et al. Sliding window and control flow weight for metamorphic malware detection[J]. Journal of Computer Virology and Hacking Techniques, 2015, 11: 75–88.Crossref

16.

Austin T H, Filiol E, Josse S, et al. Exploring hidden Markov models for virus analysis: A semantic approach[C] 2013 46th Hawaii International Conference on System Sciences. IEEE, 2013: 5039–5048.

17.

Rad B B, Masrom M, Ibrahim S. Opcodes histogram for classifying metamorphic portable executables malware[C] 2012 International Conference on e-Learning and e-Technologies in Education (ICEEE). IEEE, 2012: 209–213.

18.

Runwal N, Low R M, Stamp M. Opcode graph similarity and metamorphic detection[J]. Journal in computer virology, 2012, 8: 37–52.Crossref

19.

Shanmugam G, Low R M, Stamp M. Simple substitution distance and metamorphic detection[J]. Journal of Computer Virology and Hacking Techniques, 2013, 9: 159–170.Crossref

20.

Schwarz B, Debray S, Andrews G. Disassembly of executable code revisited[C] Ninth Working Conference on Reverse Engineering, 2002. Proceedings. IEEE, 2002: 45–54.

21.

Aho A V, Lam M S, Sethi R, et al. Compilers: principles, techniques and tools[M]. 2020.

22.

Gross J L, Yellen J. Graph theory and its apps[M]. CRC press, 2005.

23.

https://​gist.​github.​com/​rriley/​ce38ab20532c1d7a​2667

24.

https://​bitbucket.​org/​shahid_​alam/​droidnative

25.

https://​docs.​broadcom.​com/​doc/​istr-16-april-volume-21-en

26.

Wei W, Wang J, Yan Z, et al. EPMDroid: Efficient and privacy-preserving malware detection based on SGX through data fusion[J]. Information Fusion, 2022, 82: 43–57.Crossref

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_3

3. Behavior-Based Detection Method for Android Malware

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

The behavior-based Android malware detection method is a method that detects the presence of malicious behavior by observing and analyzing the behavior of an app during operation. Behavior detection mainly refers to the detection method for specific malicious behavior. Typical malicious behaviors include privacy breaches, privilege escalation, and descriptions that are inconsistent with actual behavior, as described below.

Privacy leakage: The reasons for privacy leakage can be mainly divided into two categories: one is caused by vulnerabilities in the Android system, and the other is caused by vulnerabilities in Android apps.

Android system vulnerabilities are caused by defects in the framework of the Android system itself, with a wide range of impacts and can cause harm to all smartphones installed with the corresponding system. For example, the WebView vulnerability in the Android system is mainly caused by the system allowing JS scripts to make dangerous calls to APIs in Java and native code. Malicious software embeds scripts into WebView components, and when a user clicks to trigger the WebView component, the script triggers a vulnerability and calls related APIs to steal user privacy information. Privacy breaches caused by Android app vulnerabilities are more common than system vulnerabilities, which are caused by developers not paying attention to security when coding, specifically manifested as the lack of legitimacy verification of exposed components in the app. The attacker obtains the interface for stealing user privacy information by analyzing the Android app and hijacks user privacy information through disguise.

Permission elevation: Permission elevation refers to software that does not have certain permission obtaining certain permission, mainly due to the existence of call relationships between Android apps. The Android system provides Android apps with the ability to call other apps, such as ordering takeout on takeout software and then calling payment software for payment, but this kind of behavior may bring certain security risks. If the software applies for permissions it does not need and provides an interface to the outside world, malicious apps can obtain sensitive resources by legally calling the public interface, so as to achieve permission enhancement. In the process of Android software development process, following the principle of least privilege (the software should only state the permissions used by the software, not the permissions irrelevant to the software functions) can ensure the data security of the Android system.

Inconsistent behavior: Inconsistent behavior refers to the actual execution behavior of software that is inconsistent with its declared behavior. For example, if an app has applied for reading CONTACTS permission but there are no statements related to reading contact information in its description, it is said that the app has inconsistent behavior. There are two main reasons for this problem: firstly, a program is often developed by multiple people together, and the personnel who wrote the app description may not have participated in the complete program development process, so their understanding of the program’s functions is not clear enough. Secondly, most Android software embeds third-party packages during development, and these third-party packages do not contain source code, so developers cannot describe their specific behavior.

Behavior-based Android malware detection methods usually use static analysis and dynamic analysis and other program technologies to find malicious behaviors in Android software. At the same time, machine learning technology can be used to enhance the analysis capability of Program analysis technology [1].

3.1 Privacy Disclosure

The discovery of privacy leakage is mainly through program analysis to analyze internal system vulnerabilities, app vulnerabilities, etc.

Program analysis is a technology that automatically analyzes the properties of computer programs such as correctness, robustness, security, and activity. Program analysis focuses on two main areas: program optimization and program correctness. The former focuses on improving program performance while reducing resource usage, while the latter focuses on ensuring that the program proceeds as expected. Program analysis can be carried out without executing the program, that is, static (program) analysis; it can also be performed during runtime, i.e., dynamic (program) analysis; two techniques can also be used simultaneously, namely hybrid (program) analysis.

The Android malware detection method mainly uses techniques in the field of program correctness verification. Program correctness refers to the fact that the characteristics of a program meet specified constraints. In this field, it is usually reflected in the program meeting constraints such as compliance with high-risk permissions for use, ensuring that private data is not transmitted externally, and compliance with the use of mobile device hardware. If the program is incorrect, it represents Android malware. Program constraint conditions are set manually, which usually means that “program features meet certain conditions.” Program analysis extracts features in the program through analysis technology and verifies whether these features meet the constraint conditions.

When analyzing Android software, static analysis technology is usually used to extract features such as permission usage, app Intent information, hardware usage, app metadata, specific API calls, program call graph/data flow graph, stain propagation, bytecode, etc. Dynamic analysis techniques are commonly used to extract system calls, stain propagation, function calls, hardware usage, permission usage, external information interaction, code hijacking, and other functions, such as runtime UI and dynamic code loading. Hybrid analysis technology can combine the features extracted by the above two techniques to set more complex constraints.

3.2 Permission Escalation

Permission escalation is an important aspect of behavior-based analysis. Attackers exploit vulnerabilities or other technical means to elevate their privileges to the level of system administrators or other high-privileged users, in order to gain deeper control over the system. Permission escalation attack is a common attack method that attackers can use to perform malicious operations, such as modifying system configurations, stealing sensitive information, and implanting backdoors. In computer security, preventing privilege escalation attacks is a very important work, which is usually achieved through static analysis.

Static analysis refers to analyzing the structure and code of a program without running it and extracting features. The static analysis uses mathematical tools to estimate the program code. Mathematical tools cannot guarantee the reliability (soundness, no error solution returned) and completeness (completeness, no lack of correct solution) of the results at the same time. This problem is equivalent to the Halting problem. When using static analysis technology, it is necessary to carefully design algorithms to balance the reliability and completeness of the operation results according to the requirements. For example, in the stain analysis task, completeness should be emphasized, and in the program optimization task, reliability should be emphasized.

When conducting static analysis on Android software, the usual process is as follows:

1.

Obtaining an APK file (which may include steps such as extracting APK metadata from the APK repository, including APK name, developer, rating, preview, and download volume).

2.

Unzip the APK file, extract the Android Manifest.xml file, the Java bytecode (. jar) file, and the Native binary (. so) file.

3.

Extract features such as permission requirements, Intent declarations, and hardware usage from the Android Manifest.xml file.

4.

Decompile the Java bytecode file into Java source code.

5.

Use Data-flow analysis and other technologies to extract features such as API calls, system events, graph structures, and stain propagation.

3.2.1 Permission Escalation Method

The basic theoretical knowledge commonly used for privilege enhancement is as follows:

Permission

Permission is a term defined by the Android system for software. Android software must have certain permission to use the corresponding function. For example, read and write permissions allow the software to access external storage space, network permissions allow the software to access the Internet, and so on. Users can restrict software behavior by granting software permissions. For example, when users don’t grant software’s SMS permission, the software will not be able to send SMS information externally. Android Manifest.xml is a file used by software to declare permission requirements, which can be used to extract software permission usage information.

Intent

Intent is the information carrier for communication between components in Android systems (such as initiating components, transmitting broadcasts, etc.). Since Intent can carry additional data while transferring information provided by the system, malware can use Intent to transfer data between components. Static analysis systems that usually use Data-flow analysis and other technologies need to pay attention to Intent information. The data and destinations carried in the Intent can usually be extracted from the code, and the information related to the IntentFilter used for implicit Intent propagation can be extracted from the Android Manifest.xml file.

Hardware Usage

Devices equipped with Android systems typically have some hardware modules, including GPS, microphone, and gyroscope, through which malicious software can collect user personal information. Usually, the use of hardware involves declaring permissions in Android Manifest.xml and calling specific APIs in code, so this feature can be extracted from Android Manifest.xml and code.

Metadata Information

Software metadata information extracted from software repositories such as the software market can be used to classify software, filter some constraints for accelerated analysis, or serve as input for subsequent artificial intelligence technologies.

API Call

This feature can be extracted by querying in the code or using control flow analysis techniques.

Diagram Structure Information

Basic components in static analysis such as Control-flow graphs and data-flow graphs can be extracted through corresponding analysis technology, and this diagram structure information can be used for subsequent stain analysis or as input to subsequent AI technology.

Stain Propagation

The characteristics of stain propagation can be extracted through stain analysis technology. The stain analysis technology is a special Data-flow analysis technology. Some specific APIs are manually marked as source points and sink points. Source points represent the source of sensitive information (stains), and sink points represent the leakage points of sensitive information (stains). In the process of Data-flow analysis, the source point is marked as a point, and all path points of the source point information transmission are marked as a point (propagation). If the sink point is marked as a point, the path from the source point to the sink point is called as a stain propagation path. Using this technology can capture the propagation process of sensitive information in Android software.

Bytecode Features

Large bytecode files can be mapped to small bytecode feature information using a class hash algorithm. Some artificial intelligence technologies compare the similarity between bytecode feature information as a standard for distinguishing malicious software.

3.2.2 Authorization Based on Configuration Files

Android software places information such as permission usage and component structure in Android Manifest.xml. Android static analysis software typically provides the ability to extract information from Android Manifest.xml. This information cannot only be used as features for constraint conditions but also as basic information input to the code analysis module, enhancing the accuracy of code analysis.

Enck et al. [2] proposed in 2009 the ability to determine whether software has malicious behavior based on a permission app list and developed a prototype system Kirin based on this theory. But Felt et al. [3] pointed out that in actual development, developers usually do not follow the principle of least privilege, and the set of app permissions is usually larger than the set of actual use permissions, leading to inconsistent behavior. Such software may be used by malicious software to implement collusive attacks.

Permission usage information is usually only used as features in coarse-grained analysis (such as using artificial intelligence technology), combined with information such as signatures and fingerprints for classification or clustering, or used as labels in information extraction for privacy clauses.

In fine-grained static analysis software, this step is usually the first step of analysis. For example, Android [4] is a commonly used Android static analysis tool that can extract permission usage information from Android Manifest.xml and obtain specific permission usage locations in the code in subsequent code analysis. The above content can be achieved through the following code.

The Android Manifest.xml also contains component information of the Android software, including whether it is exposed to the outside world (exported field), and what kind of intent can be used to wake up the component (intent filter field). Through this information, code analysis tools (such as Amadroid [5]) can build cross-component data-flow propagation relationships. This information can also be used as a feature of coarse particle size analysis.

3.2.3 Code Analysis

Code analysis is the core link of static analysis. Static (code) analysis technology roughly has three directions: Data-flow analysis, constraint solving, and abstract interpretation. Among them, Data-flow analysis is the basis of constraint solving and abstract interpretation. In the behavior-based detection system, stain analysis technology in Data-flow analysis is also commonly used [4–6]. Therefore, this section focuses on the Data-flow analysis technology based on an equation system. As this book does not involve content such as code optimization, it does not introduce static analysis that involves code optimization.

In Data-flow analysis, the program is usually regarded as a graph. The nodes in the graph represent the Basic block, and the edges represent how the control logic makes one Basic block flow to another Basic block. The basic task of Data-flow analysis is to find the flow path of data in the graph and finally form a Data-Flow Graph. This task can be decomposed into two parts, the first part is the construction of the graph:

For graph nodes, code can be decomposed into Basic block through Parsing rules. For example, the syntax rule: if (condition basic block), then then basic block, else else basic block can be used to decompose the if else statement into three Basic block. Of course, this analysis rule should follow the principle of recursive descent, so that the Basic block can be decomposed into several lines of code.

For edge nodes, the decomposed Basic block can be connected through semantic analysis rules. For example, for the above if else statement, according to the semantics, the program may move from condition basic block to then basic block or else basic block during running. Therefore, the condition basic block should be connected to the two Basic block, namely, then basic block and else basic block.

The graph without Data-flow analysis after construction is also called the Control flow graph. It is worth mentioning that for function call nodes, the connection between the calling Basic block and the called Basic block can be built through the Call Graph. This task is also called interprocedural analysis.

The second part is Data-flow analysis. Data-flow analysis often uses the Reaching Definition Analysis algorithm in this field. This analysis attempts to solve the problem that “for each point of the program, when the program executes along a certain path to the point, which assignments may have been executed, but not covered by other assignments.” This analysis result can help Data-flow analysis establish a data-flow relationship between the Basic block that generates values and the Basic block that uses values.

The Data-flow analysis algorithm usually runs in a monotone frame constructed by the mathematical concept of Lattice and uses the work list algorithm to traverse the entire monotone frame from top to bottom (or from bottom to top) for many times. Because the monotone frame has the mathematical property of tending to be unchanged after many times of traversal (hereinafter referred to as convergence, and the convergence point is also referred to as fixed point), the monotone frame can be converged by designing different work list algorithms. The fixed point after convergence is the answer to the analysis algorithm (such as reaching a fixed value analysis).

There are some complex tasks in the field of Data-flow analysis, such as the inter-process analysis mentioned above. The data flow involving function calls is more complex and requires careful algorithm design to ensure the reliability of the analysis results. In the analysis of Android software, since each component of Android software runs independently and communicates with each other through Intent, similar to inter-process analysis, there is a concept of inter component analysis in Android static analysis, and Data-flow analysis between components needs to design corresponding data-flow propagation rules for Intent communication mechanism to establish data-flow propagation relationship [5].

After completing the Data-flow analysis of software code, you can perform stain analysis on the data-flow graph. In this field, stain analysis technology is the main method for extracting behavioral features. For example, for privacy leakage behavior, the API that obtains privacy data can be marked as a Source point, and the API that transmits data externally, such as println(), can be marked as a Sink point. Running a stain analysis algorithm can obtain the propagation path of privacy information.

Algorithm 1

from androguard.core.bytecodes.axml import AXMLPrinter

with open("AndroidManifest.xml","rb") as

/fp:a=AXMLPrinter(fp.read())

xml = a.get_xml_obj()

xml.findall("uses-permission")

from androguard.misc import AnalyzeAPK

a, d, dx = AnalyzeAPK(\somefile.apk")

for meth in dx.get_permission_usage(‘android.

/permission.SEND_SMS',

a.get_effective_target_sdk_version()):

    print(\Using API method {}".format(meth))

    print(\used in:")

    for _, m, _ in meth.get_xref_from():

        print(m.full_name)

3.2.4 Taint Analysis

Due to the widespread use of stain analysis in Android malware behavior detection, this section provides a brief introduction to stain analysis algorithms and demonstrates how the classic Android static analysis framework FlowDroid [6] can propagate stains on an instance code.

Stain analysis can usually be attached to Data-flow analysis, so it is usually only necessary to complete the analysis of Control flow graph and function call graph before stain analysis. Run Data-flow analysis from program entry point after building inter-process Control flow graph. In the process of traversal, if a point is marked as Source, it will be marked as Taint. If a variable reads a value from a variable marked as Taint, it will be marked as Taint until the traversal is completed. At this time, Data-flow analysis has been completed; query all Sink points, and if the point is marked as a Taint, use a path query algorithm (such as Dijkstra algorithm) to check if there is a path from the Source point to the Sink point on the data-flow graph.

Since FlowDroid is analyzed by using the three address code Intermediate representation, its analysis results can be accurate to the three address instruction level, which is usually less than or equal to one line of actual code. For ease of explanation, this example uses pseudocode similar to C language, but function parameters are passed by reference like Java, and a line of pseudocode is equivalent to a line of three address codes. In this example, source() is the function that obtains private data, and sink() is the function that outputs outward. The stain analysis process is as follows:

1.

When Data-flow analysis reaches the second line of bar, the Source point is found, so the point is marked as Taint.

2.

Continuing to the third line of the bar, we found the behavior of reading the Taint point, so we changed the child. The data list label referred to by ref. value, father. child. value, is marked as Taint.

3.

Following the control flow back to the fourth line of foo, it is found that the data list referred to by daughter.value, father. child. value, is Taint, and line17 is marked as Taint.

4.

After Data-flow analysis, query whether Sink points are marked as Taint, and find the path from the third line of bar to the fourth line of foo.

Algorithm 2

struct Child {

    int value;

};

struct Parent {

    Child child;

};

int main() {

    foo();

}

void foo() {

    father = new Parent();

    daughter = father.child;

    bar(father);

    sink(daughter.value);

}

void bar(Parent parent) {

    child_ref = parent.child;

    taint = source();

child_ref.value = taint;

}

Inconsistent Behavior

Inconsistent behavior refers to the significant difference or inconsistency between the behavior of an entity in a system or network and its usual or other normal behavior patterns. This inconsistent behavior may indicate that the entity is engaged in malicious activities or has been attacked, and determining inconsistent behavior is usually done through dynamic analysis.

Dynamic Analysis

Dynamic analysis refers to the analysis of program properties by running a program and observing its state and results at runtime. Dynamic analysis mainly includes sandbox analysis and real machine analysis, both of which require special operating environments to observe and capture information about the app during operation. Dynamic analysis may adopt techniques such as program instrumentation, program slicing, or sandbox to capture runtime program information.

Compared to static analysis, dynamic analysis usually ensures the reliability of detection results, but a single run of software cannot run to all conditional branches, and in order to cover all branches, it may require a high time cost. Multiple technologies can alleviate this problem:

Mixed Analysis

On the issue of control flow/Data-flow analysis, static analysis has great efficiency advantages over dynamic analysis. Therefore, many tools choose to conduct static analysis on the software first and prune the dynamic analysis search space according to the CFG, DFG, and other information obtained from static analysis. This scheme is also called mixed analysis.

Symbolic Execution

Symbolic execution is a program analysis technology based on constraint solving, which can be used to calculate the program input triggering a certain condition. This technique converts variables in a program into symbols, and during program execution, the symbols are not evaluated. Instead, it runs continuously along all conditional branches until the program stops and collects symbol information for all termination points. If a certain constraint is applied to the symbol of the starting state (mathematical relationship, for example, for parameter arg, make arg>1), a constraint solver can be used to solve the constraint condition of the symbol information of the ending state (for example, for the return value ret, based on arg>1, deduce ret<2).

Enforcement

This technique stops the program when it runs under branch conditions, allowing the program to jump to all branches and continue running by modifying the condition variable.

1. The common dynamic analysis characteristics of inconsistent behavior are as follows:

System Call

By monitoring the Android system kernel while running software, it is possible to detect system functions called during dynamic software runtime and analyze the kernel activity logs, which may provide clearer system call characteristics than static analysis.

Information Flow

Information flow refers to the data that devices transmit and obtain in the network. On the one hand, Sink points related to network transmission obtained through monitoring stain analysis can capture stain information with dynamic details. On the other hand, the size, type, and frequency of network packets can be collected as features to distinguish malicious and benign information.

Function Call

By monitoring software function call behavior, the function call path of the software during dynamic operation can be obtained.

Figure Structure

By monitoring the software function call behavior, you can obtain the Control flow graph of the software during dynamic operation. This method is typically used to analyze known malware and label it as a feature of malware.

Hardware Usage

This includes battery status, CPU and GPU utilization, memory usage, and usage of other hardware components.

External Interaction

This includes software use of SMS, phone, URL, WiFi, GPS, Bluetooth, and other situations involving external interaction.

Permission Usage

By monitoring the behavior of software function calls, the permission usage of the software during dynamic operation can be obtained.

Dynamic Code Loading

When the software is running, it can load and run the code that does not belong to its static initial Codebase through the dynamic code loading mechanism. Usually, these codes are stored in the remote server in jar or apk format, and the corresponding characteristics in the dynamic loading module are collected through the above monitoring mechanism.

When using dynamic analysis for inconsistent behavior, the following techniques are commonly used:

Program Instrumentation

Program instrumentation refers to inserting code fragments into a program without affecting the running logic of the analysis target, and using these code fragments to collect program features. Program instrumentation can be divided into binary instrumentation and source code instrumentation, which are analyzed separately for executable files and source code. The choice of technology depends on whether the source code of the analyzed object can be obtained. In this field, binary (bytecode) instrumentation technology is usually chosen.

Soot [6] provides rich instrumentation functions for APK format files. Based on the decompilation and rewriting functions of Dalvik bytecode, Soot can repackage a given APK file into an APK file with instrumentation code. With Soot, monitoring code can be inserted in the context of calling specific functions.

2. Program slicing

Program slicing is a program decomposition technique aimed at extracting code fragments that meet requirements from a program. The use of this technique mainly relies on the analyst’s understanding of the analysis target (disassembly) code. In dynamic analysis, program slicing is mainly used to replace/mask a certain code fragment at runtime.

Xposited [7] is an open source framework designed based on this idea, which can modify the behavior of the system and app programs without changing the source files. It mainly achieves the attachment and shielding of program fragments by modifying the Zygote process (used to create SystemServer and incubate app processes) in the Android system.

3. Sandbox

The purpose of using sandboxes in dynamic analysis of Android software is to collect information about hardware [8], Linux kernel [9], and Android framework [9]. At the same time, since running software in sandboxes will not modify the software, most anti-debugging and anti-instrumentation technologies can be avoided.

3.3 Machine Learning Technology and Malicious Behavior of Android Software

In program analysis technology, features extracted from static analysis or dynamic analysis are usually input into constraint solver for solution, but these features can also be used as features in machine learning technology to cooperate with machine learning technology to classify malware and normal software. These methods usually abandon features that involve heavy computational processes (such as stain analysis) and classify them using features that can be obtained through lightweight computing, avoiding the computational resource requirements for large-scale classification tasks. At the same time, it indicates that machine learning methods are helpful in detecting zero-day vulnerabilities, and the next chapter will provide a specific introduction to machine learning methods.

3.4 Chapter Summary

As the most widely used smart terminal device, smartphones store a large amount of important private data, such as communication information and payment information, so it has also become the target of a new generation of malware attacks. Due to the open source and open nature of the Android platform, it faces complex and diverse security threats, including threats with significant behavioral characteristics such as privacy theft and remote control.

The behavior-based Android malware detection method determines whether there is a malicious attack in the software by determining whether there is a specific behavior in the software. Through targeted summary and induction, the characteristics of malicious behavior are constructed, and the malicious code detection technology of traditional platforms is adapted and improved in a targeted manner, and a variety of malicious behavior detection methods of Android platform are realized. This chapter describes in detail the malicious behavior of Android software and its characteristics, and the theoretical and practical app of static analysis and dynamic analysis techniques in behavior detection.

References

1.

Enck W, Ongtang M, McDaniel P. On lightweight mobile phone app certification[C]//Proceedings of the 16th ACM conference on Computer and communications security. 2009: 235–245.

2.

Felt A P, Chin E, Hanna S, et al. Android permissions demystified[C]//Proceedings of the 18th ACM conference on Computer and communications security. 2011: 627–638.

3.

Androgurad, https://​github.​com/​androguard/​androguard

4.

Argus-SAF, https://​github.​com/​arguslab/​Argus-SAF

5.

Soot, https://​github.​com/​soot-oss/​soot

6.

Xposed, https://​github.​com/​rovo89/​Xposed

7.

Bayer U, Kruegel C, Kirda E. TTAnalyze: A tool for analyzing malware[M]. na, 2006.

8.

Lindorfer M, Neugschwandtner M, Weichselbaum L, et al. Andrubis–1,000,000 apps later: A view on current Android malware behaviors[C]//2014 third international workshop on building analysis datasets and gathering experience returns for security (BADGERS). IEEE, 2014: 3–17.

9.

Enck W, Gilbert P, Han S, et al. Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones[J]. ACM Transactions on Computer Systems (TOCS), 2014, 32(2): 1–29.Crossref

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_4

4. AI-Based Android Malware Detection Methods

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

With the support of artificial intelligence technology, the integration of program analysis techniques and machine learning algorithms has gradually emerged as a research hotspot in intelligent malware detection methods. Specifically, this chapter will elaborate on various detection methodologies, categorized by their distinct feature extraction methods, analysis approaches, and vectorization techniques post feature extraction. These include detection methods based on permissions, APIs, and components; dynamic runtime feature-based detection methods; code semantics-based detection methods; image-based detection methods; and graph analysis-based detection methods.

4.1 Detection Based on Permissions, APIs, and Components

In the pursuit of expediting the process of Android malware detection, certain scholars have directed their focus toward harnessing the elemental traits inherent in Android applications. These rudimentary attributes encompass pivotal facets, notably permissions, APIs, and components. Among these, insights into permissions and components are conveniently gleaned from the AndroidManifest.xml file embedded within the heart of an Android application. Conversely, the compendium of API invocations necessitates a discerning analysis of the dex files.

To elaborate, the AndroidManifest.xml file constitutes an obligatory component integral to each application project, firmly ensconced within the project’s root directory and mandatorily bearing this designated appellation. Such stringent adherence to nomenclature ensures the unswerving acquisition of crucial insights into permissions and components, thereby averting any conundrum pertaining to information retrieval. This manifest file serves as an oracle, transmitting pivotal particulars concerning the application’s foundation to the ambit of Android’s construction tools, the operating system itself, and the expanse of Google Play, thereby furnishing an elemental snapshot of the application’s essential attributes.

The manifest file shall state the following contents:

The application’s package name typically aligns with the code’s namespace. During project building, the Android build tool utilizes this package name to pinpoint the code entity’s location. Upon packaging the application, the build tool substitutes this value with the application ID in the Gradle build file. This ID then serves as the distinctive application identifier both within the system and on Google Play.

The components of an application encompass various elements, including Activities, Services, Broadcast Receivers, and Content Providers. Each component is required to define fundamental properties, such as the name of its Kotlin or Java class. Additionally, the manifest file can declare certain functionalities like the device configurations it can manage and the Intent filter that outlines how the component initiates.

The application specifies permissions needed to access protected parts of the system or other applications. If external applications intend to access this application’s content, the manifest file will also outline the necessary permissions they must possess.

The hardware and software functions that an application requires will influence the compatibility of the application with certain devices on which it can be installed from Google Play.

From the structure of the manifest file, it will reflect some of the most important application characteristics through the following contents.

1.

Package name and application ID

The root element of the manifest file must contain the attribute specifying the application’s package name, typically corresponding to the project directory structure or the Java namespace.

For instance, the code snippet below illustrates the <manifest> root element containing the package name “com.tencent.qqgame.hallinstaller.five”:

A snippet of code denotes the package name within the manifest root element. It also denotes the x m l version, encoding, and standalone values at the top.

When an application is built into its final application package (APK), the Android build tool employs the “package” attribute to achieve two main objectives:

It utilizes this name as the namespace for the generated R.java class within the application. This R.java class is essential for accessing the application’s resources.

Example: In the above listing, an R class can be created at com.tencent. qqgame. hallinstaller.five.R.

This name is utilized to resolve any associated class names declared in the manifest file.

Example: In the list above, the Activity declared as <activity android:name= “.MainActivity”> will be interpreted as com.tencent.qqgame.hallinstaller.five. MainActivity.

Hence, the name specified in the package attribute of the manifest should consistently correspond to the base package name in the project, where the Activity and other application code are stored. While additional sub-packages can be incorporated into the project, these files must be imported into the R.java class using the namespace defined by the package attribute.

However, it is important to note that post APK compilation, the package attribute can also denote the universally unique application ID. During the described tasks executed by the build tools based on the package name, they replace the package value with the value assigned to the application attribute in the project’s build.gradle file (for Android Studio projects). This resulting value of the package attribute must be universally unique, as it is the key factor ensuring the application’s distinct identification within the system and on Google Play.

2.

Application components

For every application component created within an application, the developer needs to declare the corresponding XML element in the manifest file:

Activity is utilized for each subclass of Activity.

Service is utilized for each subclass of Service.

Receiver is utilized for each subclass of BroadcastReceiver.

Provider is utilized for each subclass of ContentProvider.

If a subclass of these components is created and not declared in the manifest file, the system will be unable to initiate that subclass.

It is imperative that the “name” attribute be utilized to specify the designation of the subclass, and further, the said subclass must employ the complete software package nomenclature. By way of illustration, a declaration for a subclass of type “Activity” may be formulated as follows:

A snippet of code denotes the activity name within the application tag inside the manifest root element.

However, if the “name” value begins with a period, the software package nomenclature (derived from the “package” attribute of the <manifest> element) will be used as a prefix for this designation. For example, the resulting “Activity” name will resolve as “com.tencent.qqgame.hallinstaller.five.MainActivity”:

A snippet of code denotes the manifest package name at the top. The application tag inside the manifest root element denotes the activity name.

If the application includes app components located within sub-packages (e.g., com.tencent.qqgame.hallinstaller.five.purchases), it is crucial for the “name” value to include the missing sub-package designation (e.g., “.purchases.PayActivity”) or use the fully qualified software package nomenclature.

3.

Intent filter

The application’s Activities, Services, and Broadcast Receivers are initiated through Intents. An Intent, defined by an Intent object, is a message used to convey an action to be executed. It encapsulates the action’s data, the component’s category responsible for the action, and other relevant specifications.

When the application dispatches an Intent to the system, the system searches for application components that can handle this Intent based on the Intent filters specified in each application’s manifest file. The system then starts instances of matching components and forward the Intent object to the designated component. In cases where multiple applications can process the given Intent, the user is presented with the option to choose which application to use.

Application components can include any number of Intent filters, defined using the “<intent-filter>” element, with each filter specifying distinct functionalities of the component.

4.

Icons and labels

Numerous elements within the manifest possess the “icon” and “label” attributes, serving to, respectively, present users with small graphical icons and textual labels for corresponding application components.

In all circumstances, the icons and labels set within parent elements become default values for all subordinate elements. For instance, icons and labels set within the “<application>” element serve as default icons and labels for each application component (such as all Activities).

Whenever components are presented to users in the form of Intent options, the system displays the icons and labels established within the “<intent-filter>” of said components. By default, this icon inherits from any icon declared within the parent component (“<activity>” or “<application>” element). However, if the Intent filter provides a unique action and developers seek to provide clearer indications within the selector dialog, it might be necessary to modify this icon.

5.

Permission

To access sensitive user data such as contacts and messages, or specific system functionalities like the camera and internet access, Android applications need to request relevant permissions. Each permission is identified by a distinct label. For example, if an application requires the ability to access network state, the following line of code must be included in the manifest:

A snippet of code exhibits the permission to access network state within the manifest root element.

Starting from Android 6.0 (API level 23), users have the ability to grant or deny specific application permissions at runtime. However, irrespective of the Android version an application supports, it is obligatory to employ the “<uses-permission>” element within the manifest to declare all permission requests. Upon being granted these permissions, the application gains access to secured functionalities; otherwise, attempts by the application to access these functionalities will result in failure.

Moreover, applications can employ permissions to safeguard their own components. This can involve the use of permissions specified by Android (as listed in “android.Manifest.permission”), or permissions declared by other applications. Applications are also empowered to define their proprietary permissions. The declaration of novel permissions can be achieved through the “<permission>” element.

6.

Equipment compatibility

The manifest file can also declare the necessary hardware or software feature types required by an application, as well as the types of devices the application is compatible with. The Google Play Store restricts the installation of applications on devices that do not have the required functionalities or system versions specified by the application’s requirements.

4.1.1 Permissions in Android System

4.1.1.1 Permissions in Android System

Application permissions contribute to safeguarding access to and execution of the following data and operations, thereby providing support for user privacy protection:

Restricted data, such as system status and user contact information

Restricted operations, such as connecting to paired devices and recording audio

During the development of Android applications, it is imperative to declare the permissions being requested. Furthermore, when an application’s functionality necessitates access to restricted data or the execution of restricted operations, it is crucial to ascertain whether scenarios exist where pertinent information can be obtained or actions can be executed without requiring permission declaration. Developers can incorporate various functionalities in their applications, such as taking photos, pausing media playback, and displaying relevant advertisements, without requiring any explicit permission declarations.

When an application needs to access restricted data or perform restricted operations to fulfill a specific use case, it should declare the corresponding permissions. Certain permissions are automatically granted to the application upon installation, known as install-time permissions. However, there are runtime permissions that the application needs to request during its execution to access specific features.

Android classifies permissions into different types, including install-time permissions, runtime permissions, and special permissions. Each type of permission defines the range of restricted data the application can access and the limited operations it can execute once granted by the system. The level of protection associated with each permission depends on its type, and you can find detailed information in the permission API reference documentation pages.

1.

Permission during installation

Install-time permissions provide applications with restricted access to specific data or limited operations, minimizing impact on the system or other applications. When install-time permissions are declared in an application, the app store displays a notice to users on the app’s details page. The system automatically grants these permissions to the application during installation.

Android provides several subtypes of install-time permissions, including normal permissions and signature permissions.

Normal permissions: These permissions allow access to data beyond the application’s sandbox and the execution of operations outside the application’s sandbox, all while minimizing risks to user privacy and other running applications. General permissions are designated a normal protection level by the system.

Signature permissions: Signature permissions are only granted to an application if it shares the same certificate signature with the defining application or the operating system. Applications that implement privileged services like autofill or VPN services also make use of signature permissions. These applications necessitate service binding signature permissions to ensure that only the system can bind to the service. The system designates a signature protection level for signature permissions.

2.

Runtime permissions

Runtime permissions, also known as dangerous permissions, grant applications additional access to restricted data or enable them to perform limited operations with a more significant impact on the system and other applications. Therefore, developers need to request these permissions within the application before gaining access to restricted data or performing restricted operations. Numerous runtime permissions involve accessing users’ private data, which is a specific subset of restricted data and may include relatively sensitive information. For instance, location data and contact information are considered user-private data. The system designates a dangerous protection level for runtime permissions.

3.

Special Authority

Special permissions are tied to specific application operations and can only be defined by the platform and Original Equipment Manufacturers (OEMs). Typically, when the platform and OEMs aim to control the execution of powerful functionalities (like drawing over other apps), they define these as special permissions.

The “Special App Access Permissions” page within system settings comprises a collection of user-toggleable operations, many of which are implemented as special permissions. The system designates an “appop” protection level for special permissions.

4.

Permission Group

Permissions can be organized into permission groups, which are sets of logically related permissions. For example, permissions related to sending and receiving SMS messages might be grouped together since they both involve the application’s interaction with text messaging.

The purpose of permission groups is to help the system reduce the number of system dialogs shown to the user when an application requests closely related multiple permissions. When the system prompts the user to grant permissions to an app, permissions from the same group are presented in a unified interface. However, permissions may change groups without prior notice, so one should not assume that specific permissions will always be grouped together with other permissions.

4.1.1.2 Overview of Permission-Based Detection Methods

Permissions serve as a security mechanism that operates on two levels. First, at the level of an individual application, permissions can restrict the utilization of certain privileged features within the program. Second, in the context of different applications, permissions can restrict the interaction between components of different applications. For instance, an application requiring network functionality must request the permission INTERNET.

Malware detection methods based on permission analysis involve extracting permission information from the Android application’s configuration file, AndroidManifest. These permissions are then utilized as features to train classifiers, aiding in the detection of Android malware. Wang et al. [1] employed three different feature ranking algorithms (mutual information, correlation coefficient, and T-test) to evaluate the risk associated with permissions within Android applications. They utilized forward selection and principal component analysis to identify subsets of permissions considered riskier. These subsets were then used to construct feature vectors for training machine learning models. Additionally, for more efficient experimental outcomes, Li et al. [2] conducted pattern mining on extracted permissions and utilized these patterns, composed of permission groupings, for malware detection.

Zhou et al. [3] introduced an extensible malware detection method known as DroidRanger. For known malware, DroidRanger designed a behavior footprint scheme based on permission information for detection. For unknown malware, DroidRanger combined the loading of remote code with the dynamic execution information of software. It employed a heuristic approach to define suspicious behaviors exhibited by malware and finally employed these predefined behavioral rules to identify malicious behavior within unknown malware families.

4.1.2 Detection Based on API

To facilitate both application development and access to specific routines of the Android system without the need for source code access or comprehension of internal implementation details, Android defines functions and provides external interfaces, collectively known as the Android API. In comparison to permissions, APIs offer a finer granularity of features. Consequently, for achieving more precise malware detection, researchers have extracted various types of APIs from applications to employ them as features for training machine learning models.

Asfer et al. [4] extracted five distinct types of APIs through static analysis: the five categories of APIs essentially pertain to applications, the Android framework, DVM, Linux system calls, and utilities. These APIs were utilized to train models for malware detection.

Zhao et al. [5] recorded system call APIs executed by applications through dynamic analysis and employed them as features to train machine learning models. Additionally, to enhance malware detection effectiveness further, some researchers [6] have combined permissions and APIs. Kouliaridis et al. [7] extracted permissions and Intent information as effective features for Android malware and integrated PCA and t-SNE into the base model. They discovered that both modifications substantially enhanced the performance of each individual base model as well as the assembled ensemble model, ultimately achieving AUC and accuracy rates of 95.1$$\%$$ and 91.7$$\%$$, respectively, on the AndroZoo dataset.

Cai et al. [8] introduced JOWMDroid, an Android malware detection method based on feature weights. They extracted eight feature categories from Android applications and employed a combination of joint optimization of weight mapping and classifier parameters, alongside the Information Gain (IG) algorithm, to select the most critical features. JOWMDroid utilized three base models to determine the weight of each feature and devised five weight mapping models to transform initial weights into the ultimate weights. Ultimately, they utilized a differential evolution algorithm to jointly optimize the weight mapping models and base model parameters.

4.1.3 Component-Based Detection

4.1.3.1 Components of an Application

Application components are essential foundational elements of an Android application. Each component acts as a point of entry through which the system or users can interact with the application. Some components depend on others.

Activity

Service

BroadcastReceiver

ContentProvider

Each type serves specific purposes and follows distinct lifecycles, which determine how the component is created and destroyed. The subsequent section will introduce the four types of application components.

1.

Activity

An Activity functions as a starting point for user interaction, representing an individual screen with a user interface. For instance, in an email application, there might be an Activity to showcase a list of new emails, another Activity for composing emails, and yet another for reading emails. Although these activities work together to establish a seamless user experience within the email application, each Activity operates independently of the others. As a result, other applications can initiate any of these Activities (subject to permission by the email application). For instance, a camera application could initiate the email application’s Activity for composing new emails to allow users to share pictures. Activities facilitate essential interactions between the system and applications, including:

The system tracks the content currently in the user’s focus (the displayed screen content) to ensure the ongoing operation of the process hosting the Activity.

The system recognizes that previously used processes contain content the user might return to (paused Activities) and therefore prioritizes retaining these processes.

Assisting applications in handling scenarios where their processes are terminated, enabling users to return to Activities in their previous states.

Offering a mechanism for applications to implement user flows between each other while enabling the system to coordinate these flows.

Each Activity contributes to the seamless interaction between users and the system and helps shape a coherent user experience within an application.

2.

Service

A Service acts as a primary mechanism to keep an application active in the background. It operates as a background component, handling long-running operations or tasks for remote processes without displaying a user interface. For example, a Service could play music or fetch data over the network while the user engages with other apps. Various components like Activities can start or bind to a Service, enabling interaction with it. In fact, there are two distinct semantic modes in which a Service can instruct the system to manage an application: Started Services prompt the system to maintain their functions until they finish their assigned tasks. These tasks may include background data synchronization or keeping music playback active even after the user exits the application. Background data synchronization and music playback represent two distinct categories of started services, and how the system handles their processing can vary accordingly.

Music playback is a perceptible service to the user. Hence, the app sends notifications to the user indicating its desire to be in the foreground, signaling this to the system. In this scenario, the system recognizes the importance of keeping the service’s process active to prevent potential user dissatisfaction that could arise from terminating it.

Regular background services, on the other hand, are often inconspicuous to the user, allowing the system more flexibility in managing its processes. If the system requires RAM for more pressing user-focused tasks, it might allow the service to be terminated (and later restarted) as needed.

Binding to a service takes place when another application (or the system) expresses an intention to use that service. A bound service provides an API to another process, establishing a dependency between the processes. When Process A binds to a service in Process B, the system acknowledges the need to keep Process B and its service running to support Process A. Moreover, if Process A is of user interest, the system comprehends that Process B should also be treated as content the user cares about. Due to their flexibility, both advantageous and potentially challenging, services have emerged as a highly valuable building block, capable of implementing various advanced system concepts. Dynamic wallpapers, notification listeners, screen savers, input methods, accessibility services, and numerous other essential system functionalities can be built as services implemented by apps and bound by the system at runtime.

3.

BroadcastReceiver

Utilizing Broadcast Receivers, the system can communicate events to applications beyond typical user interactions, allowing apps to respond to system-wide broadcast notifications. Broadcast Receivers serve as distinct entry points for applications, enabling the system to send broadcasts to applications that are currently inactive. For instance, an application can schedule reminders to issue notifications, informing users about impending events. By passing this reminder to the application’s Broadcast Receiver, the application need not remain active until the reminder sounds. Numerous broadcasts are triggered by the system, such as notifications for screen power-off, low battery levels, or a photo being taken. Applications can also initiate broadcasts, such as informing other apps that specific data has been downloaded onto the device and is available for their use. Although Broadcast Receivers do not have interfaces, they can generate status bar notifications, informing users when broadcast events take place. However, the primary role of Broadcast Receivers is to act as conduits to other components, facilitating the execution of minimal tasks. For example, they might employ JobScheduler to schedule a JobService for carrying out specific tasks based on an event.

4.

ContentProvider

A Content Provider oversees a set of shared application data that can be stored in various ways, including the file system, SQLite database, network, or any other persistent storage accessible to the developer’s application. Other applications can access and modify this data through the Content Provider, provided it allows for such interactions. For example, the Android system offers a Content Provider to manage user contact information. Consequently, any application with the appropriate permissions can query the Content Provider (such as ContactsContract.Data) to access and modify relevant information for specific individuals. While it is easy to envision Content Providers as abstractions over databases, their fundamental purposes differ from a system design perspective.

From the system’s perspective, Content Providers act as access points for applications to publish named data items identified by URI schemes. This allows applications to define how their encapsulated data is mapped to the URI namespace and share these URIs with other entities. Conversely, these entities can use the distributed URIs to access the data. In the management of applications, the system can perform the following specialized operations:

URI assignment does not require applications to remain active, allowing URIs to persist even after their corresponding applications have exited. When the system needs to retrieve application data from a specific URI, it simply needs to verify that the owning application is still active and running.

These URIs also provide a crucial fine-grained security model. For example, an application can put URIs for its owned images on the clipboard while securing the Content Provider to prevent casual access by other applications. If a second application tries to access the URI on the clipboard, the system can grant temporary URI authorization, enabling the application to access only the data linked to that URI and not any other content within the second application.

ContentProvider is also suitable for reading and writing private data that applications do not share.

Among the four types of components, three (Activity, Service, and Broadcast Receiver) are initiated through asynchronous messages called Intents. Intents establish runtime connections between various components. Developers can conceive Intents as messengers requesting actions from other components, whether they belong to the developer’s application or another application.

Developers utilize the Intent object to craft an Intent, outlining the message to initiate a particular component (explicit Intent) or a specific category of component (implicit Intent).

For Activity and Service, the Intent defines the action to be carried out (e.g., viewing or sending specific content) and can specify the URI of the data to be operated on, along with any relevant information that the launched component might need to be aware of. For example, an Intent might convey a request to an Activity to display an image or open a web page. In certain scenarios, developers can obtain results by launching an Activity, which then returns an Intent containing the result. For instance, a developer can send an Intent to have the user select a contact and return it to them. The returning Intent contains a URI pointing to the chosen contact.

For Broadcast Receiver, the Intent solely defines the notification for the broadcast. For example, a broadcast indicating a low battery level would contain a string with the action “low battery.”

Unlike Activity, Service, and Broadcast Receiver, Content Providers are not activated by Intents. Instead, they are triggered when they are the target of requests from ContentResolver. The ContentResolver manages all direct interactions with the Content Provider, enabling components executing transactions through the provider to abstain from handling the transactions directly and instead invoke methods on the ContentResolver object. This establishes an abstraction layer between the Content Provider and the requesting component, enhancing security.

Each type of component has a distinct initiation method:

To start an Activity, you can pass an Intent to startActivity() or startActivityForResult() (when you expect a result from the Activity) or schedule it as a new task.

In Android 5.0 (API level 21) and later versions, developers can utilize the JobScheduler class to schedule operations. For prior Android versions, starting a Service (or issuing new commands to a running service) involves passing an Intent to startService(). Additionally, binding to a service can be achieved by passing an Intent to bindService().

Broadcasts can be triggered by passing an Intent to methods such as sendBroadcast(), sendOrderedBroadcast(), or sendStickyBroadcast().

Queries on Content Providers can be executed by calling query() on a ContentResolver.

4.1.3.2 Overview of Component-Based Detection Methods

In addition to permissions and APIs, some studies [9] have found that information related to Intents and components in the AndroidManifest.xml file, as well as certain strings within the application code, can also serve as effective features for detecting malicious software. For example, Arp et al. [10] and Daoudi et al. [11] extract not only permissions, components, and Intent information (e.g., detecting malicious behavior triggered by BOOT$$\_$$COMPLETED after reboot) but also hardware information (e.g., GPS and network modules), key APIs, used permissions, suspicious APIs, and network address information (e.g., IP addresses and URLs) from the configuration file and application code. They utilize these eight distinct types of features to train classifiers.

Liu et al. [12], through a comparative analysis of Android application permissions, discovered that Intent information in Android applications is an effective feature for identifying malicious Android applications. Moreover, Su et al. [13] integrate multiple risk factors, including permissions, API calls, Intent Filter action attributes, and data flow, and employ hierarchical clustering analysis to assign weights to subsets of these risk factors. This approach is used to assess the true security risks of Android applications.

4.1.4 Specific Case: Drebin

Drebin [10] is a lightweight approach for detecting Android malware, designed to directly identify malicious applications on smartphones. Given the significant overhead associated with dynamic analysis, Drebin employs extensive static analysis to conserve resources while collecting a wide range of application features. These features are embedded within a unified vector space, allowing for the automated recognition of common patterns associated with malware. This also aids in interpreting the decisions made by the detection method. In an evaluation involving 123,453 applications and 5,560 malware samples, Drebin demonstrated superior performance compared to various pertinent methods, detecting 94% of malware with very few false positives. The explanations accompanying each detection unveil pertinent attributes of the identified malicious software. When tested on five popular smartphones, this approach maintained an average analysis time of 10 seconds, making it suitable for on-device scrutiny of downloaded applications.

At the core of Drebin is the principle of creating a thorough yet lightweight representation of applications to identify common signs of malicious behavior. To accomplish this, the detection method employs in-depth static analysis, extracting feature sets from various sources and analyzing these sets within an expressive vector space.

1.

Extensive Static Analysis: In the initial step, Drebin conducts a comprehensive static examination of the provided Android application and extracts diverse sets of features from the application’s manifest and dex code.

2.

Embedding in Vector Space: Following this, the extracted feature sets are mapped into a unified vector space, allowing for geometric analysis of patterns and combinations of features within this space.

3.

Learning-Based Detection: The embedding of feature sets empowers the utilization of effective machine learning techniques for identifying malicious software, such as linear support vector machines.

4.

Explanation: In the last step, features that contribute to the detection of malicious applications are identified and presented to the user to clarify the detection process.

4.1.4.1 Static Analysis of Applications

As the initial step, Drebin conducts a lightweight static analysis on the provided Android application. Although static feature extraction may seem straightforward, it needs to be executed in a constrained environment and completed swiftly. Lengthy analysis times may prompt users to interrupt the ongoing process and reject the entire method. Hence, the selection of features that can be efficiently extracted is crucial. In this context, Drebin primarily focuses on the application’s manifest and disassembled dex code, both of which can be acquired through a linear scan of the application’s content. To facilitate a generic and scalable analysis, all extracted features are represented as sets of strings, encompassing permissions, intents, and API calls. Specifically, Drebin extracts the following eight groups of strings.

1.

Hardware Components (S1): The first feature set comprises requested hardware components. When an application requests access to smartphone hardware components like the camera, touchscreen, or GPS module, it must declare these features in the manifest file. Requesting access to specific hardware can introduce security vulnerabilities, as certain combinations of hardware usage often indicate malicious behavior. For instance, an application with access to GPS and network modules can collect location data and transmit it over the network to an attacker.

2.

Requested Permissions (S2): One of the critical security mechanisms in Android is the permission system. Permissions are explicitly granted by users during installation, enabling applications to access security-related resources. Malicious software often requests specific permissions more frequently than harmless applications. For example, a considerable portion of current malware sends premium SMS messages, necessitating the SEND$$\_$$SMS permission. Therefore, Drebin compiles all permissions listed in the manifest into a feature set.

3.

Application Components (S3): Android applications are composed of four distinct types of components, each defining different system interfaces: Activities, Services, Content Providers, and Broadcast Receivers. Each application can declare multiple components of each type in its manifest. The names of these components are also gathered into a feature set, as these names may assist in identifying well-known components of malware. For instance, multiple variants of the DroidKungFu series share the name of a specific service.

4.

Intent Filters (S4): Inter-process and intra-process communication on Android primarily transpires through the utilization of intents – a passive data structure for asynchronous message exchange. Intents facilitate the sharing of event-related information between various components and applications. All intents listed in the manifest are aggregated into another feature set, as malware frequently listens to specific intents. An illustrative example of an intent message related to malware is BOOT COMPLETED, triggering malicious activities immediately upon restarting the smartphone.

5.

Restricted API Calls (S5): The Android permission system imposes restrictions on access to a variety of critical API calls. Drebin scans the disassembled code to identify instances of these calls, providing a deeper insight into the application’s functionality. An especially telling scenario indicating malicious behavior is the utilization of restricted API calls without requesting the required permissions. This may suggest that malware is leveraging root exploits to circumvent the limitations imposed by the Android platform.

6.

Utilized Permissions (S6): The complete set of API calls extracted in S5 serves as the foundation for identifying the subset of requested and actively used permissions. Drebin achieves this by establishing a match between API calls and permissions. Unlike S5, this feature set offers a more generalized perspective of application behavior, considering that multiple API calls may be covered by a single permission (e.g., sendMultipartTextMessage() and sendTextMessage(), both necessitating the SEND$$\_$$SMS permission).

7.

Suspicious API Calls (S7): Certain API calls provide access to sensitive smartphone data or resources and are commonly found in samples of malicious software. Given their association with malicious behavior, these calls are isolated and aggregated into a distinct feature set. Specifically, Drebin compiles the following types of API calls:

API calls used for accessing sensitive data, such as getDeviceId() and getSubscriberId()

API calls related to network communication, such as setWifiEnabled() and execHttpRequest()

API calls for sending and receiving SMS messages, such as sendTextMessage()

API calls for executing external commands, such as Runtime.exec()

API calls frequently used for obfuscation, such as Cipher.getInstance()

8.

Network Addresses (S8): Malicious software often initiates network connections to fetch commands or leak data collected from the device. Consequently, all IP addresses, hostnames, and URLs identified in the disassembled code are incorporated into the final feature set. Some of these addresses may be linked to botnets, making them recurrent in various malware samples and thus enhancing learning patterns for detection.

4.1.4.2 Embedding in Vector Space

Malicious activities tend to manifest in distinct patterns and combinations of the extracted features. For instance, malware engaging in sending premium SMS messages might exhibit the SEND$$\_$$SMS permission from set S2 and the hardware component android.hardware.telephony from set S1. In an ideal scenario, one would aim to formulate Boolean expressions to capture these dependencies between features and yield a true indication when detecting malicious software. However, deriving effective Boolean expressions from real-world data is a challenging problem with limited efficacy.

As a solution, Drebin seeks to capture dependencies between features using concepts from machine learning. Since most learning methods operate on numerical vectors, the initial step involves mapping the extracted feature sets into a vector space.

To accomplish this, Drebin establishes a joint set S, encompassing all detectable strings from the eight feature sets. Unique prefixes are appended to all strings within each feature set to prevent any conflicts between elements from different sets. During evaluation, set S encompasses approximately 545,000 distinct features, encoded using one-hot vectors. Applications with akin features are represented closely within this space, whereas those with primarily distinct features are positioned farther apart. Additionally, directions in this space can delineate feature combinations, ultimately facilitating the learning of interpretable detection models.

4.1.4.3 Learning-Based Detection

In the third step, Drebin utilizes machine learning techniques to autonomously discern between malicious and benign applications. The adoption of machine learning obviates the necessity for manually devising detection rules for the extracted features. While several learning methods can be employed to ascertain the demarcation between the two classes, only a few methods can yield effective and interpretable detection models. Drebin employs the Linear Support Vector Machine (SVM) to fulfill this objective.

4.1.4.4 Explanation

In practice, a detection system should not only identify malicious activities but also furnish explanations for its detection outcomes. A common drawback of learning-based methods is their black-box nature. Drebin addresses this issue and extends the learning-based detection to identify application features that contribute to the detection. Furthermore, interpretable detection aids researchers in examining patterns within malicious software and gaining deeper insights into their functionalities.

With the simplicity of linear SVM detection, Drebin can ascertain the contribution of each individual feature “s” to the detection outcome. During the computation of the detection outcome, only the top “k” weights “ws” need to be retained, effectively positioning the application toward the malicious side of the hyperplane. Since each weight “ws” corresponds to a specific feature “s,” it enables an explanation of why an application is classified as malicious or benign. This approach is efficiently implemented by maintaining the top “k” maximum weights “ws” in a heap during the detection outcome computation. After extracting the top “k” features based on their weights, Drebin automatically generates sentences describing the functionalities associated with these features. To achieve this, Drebin creates sentence templates for each feature set, specifically designed to leverage their respective features. These templates are outlined in Table 2.​1. For frequently observed features in malicious software, such as the SEND$$\_$$SMS permission, Drebin provides dedicated descriptions.

S1: App utilizes hardware feature %s.

S2: App requests permission to access %s.

S3: App contains suspicious component %s.

S4: Action is triggered by %s.

S5: App calls function %s to access %s.

S6: App uses permissions %s to access %s.

S7: App uses suspicious API call %s.

S8: Communication with host %s.

For most feature sets, creating sentences from the templates in Table 2.​1 is a straightforward process. For instance, in the case of hardware features, Drebin leverages its naming scheme to compose meaningful sentences. For example, if an application utilizes the android.hardware.camera feature, Drebin generates the sentence “App uses hardware feature camera.”

Similarly, Drebin furnishes explanations for requested and utilized permissions. Explanations for permissions can be deduced from the pertinent descriptions in the Android documentation, which Drebin adjusts slightly to provide meaningful explanations to users. However, given that application developers can define custom permissions, Drebin also provides a generic sentence to users if an appropriate description is not available. A similar approach is adopted for restricted API calls based on specific permissions used. For all other feature sets, the templates are populated directly with feature names or corresponding placeholders.

4.2 Detection Anchored in Dynamic Runtime Features

4.2.1 Dynamic Analysis and Runtime Features

Dynamic analysis encompasses executing an application in a sandbox environment or on a real device, where the application’s runtime state is monitored. This process involves gathering behavioral information, analyzing various data sources such as logs and network traffic. It often necessitates accessing private data or using restricted API calls to monitor application behavior. Dynamic analysis is effective in identifying malicious behavior that static analysis methods might overlook and is particularly potent against malware employing obfuscation techniques. However, the analysis and detection process is time-consuming and computationally intensive.

Dynamic analysis methods collect runtime behavioral information of Android applications and transform it into features. These methods utilize dynamic features, including system calls, file access information, network traffic, encryption operations, service activations, phone calls, user interactions, and system components, to combat obfuscation. Additionally, many researchers gather information about malicious behavior within a confined runtime of the Android application. Malware may not engage in malicious activities within this timeframe, necessitating a lengthy analysis process.

Based on the techniques employed to track Android application behavior, dynamic analysis methods can be categorized into hook-based and log-based approaches. Log-based dynamic analysis involves running Android applications on real devices or emulators and monitoring their behavior using established logging tools. On the other hand, hook-based dynamic analysis methods involve embedding monitoring points (hooks) within the code, recording the application’s activities during execution. These hooks can monitor the execution, collect information about behavior, trace executed instructions, retrieve event sequences, or monitor stored data flows of Android applications.

A significant challenge in dynamic analysis is the issue of code coverage. Obtaining complete code coverage during dynamic analysis is a challenging task, and hidden code that might be triggered under specific conditions or scenarios can be missed. To address this, stimulus mechanisms and triggers are utilized to provoke malicious behavior in Android malware intentionally. Furthermore, tools generating user interface events can simulate regular application usage, mimicking typical interactions between users and applications. This helps in evading the detection of concealed malicious behavior, especially when running within a dynamic analysis environment using an Android emulator.

4.2.2 Overview of Detection Methods Based on Dynamic Runtime Features

Detection methods for Android malware based on dynamic runtime features focus on executing actual applications, monitoring their runtime state and behavioral information, vectorizing the data, and then analyzing and detecting them using artificial intelligence models. These methods often pay attention to features such as system calls, memory usage, interaction events, and API calls of the applications.

Hou et al. [14] introduced an innovative dynamic analysis approach termed “component traversal,” which autonomously executes code routines within specified Android applications. The method involves extracting Linux kernel system calls from samples, constructing a weighted directed graph, employing a deep learning framework, and utilizing a stack-based autoencoder model with graph-based features to detect unknown Android malware. Demonstrating a detection accuracy of 93.68$$\%$$, the approach was tested on a dataset containing 1,500 malicious samples and 1,500 benign samples.

Martinelli et al. [15] developed a dynamic analysis method utilizing convolutional neural networks to analyze system calls. Applications are executed on real physical devices, generating a multitude of interaction and system events during the application’s execution. Raw system call features are extracted and preprocessed for subsequent analysis. The method achieved an impressive detection accuracy of 90$$\%$$ using a dataset consisting of 3,536 benign samples and 3,564 malicious samples.

Yeh et al. [16] integrated monitoring functions into target applications to capture API call events during the preprocessing stage. They utilized DroidBox to dynamically analyze Android applications, label activities, and extract dynamic features. Event logs were transformed into flat data with two-dimensional features, and a model based on convolutional neural networks was employed for malware detection. Training on 16,000 benign and 16,000 malicious software samples, and testing on 1,000 benign and 1,000 malicious samples, their approach achieved an impressive prediction accuracy of 93.012$$\%$$.

4.2.3 Specific Case: EnDroid

EnDroid [17] is a dynamic analysis framework that attains high-precision malware detection by utilizing a variety of dynamic behavioral features. These features encompass tracking system-level behavior as well as common application-level malicious behaviors, including personal information theft, subscription to premium services, and malicious service communication. Additionally, EnDroid applies feature selection algorithms to eliminate noise or irrelevant features, concentrating on extracting crucial behavioral characteristics. By extracting behavioral features through runtime monitoring, EnDroid is capable of distinguishing between malicious and benign applications using integrated learning algorithms.

EnDroid comprises two phases: the training phase and the detection phase. During the training phase, dynamic behavioral features are extracted by monitoring the operations of both benign and malicious applications, resulting in feature vectors for each processed application. These generated feature vectors from benign and malicious applications serve as input for training various base classifiers. Utilizing the predictive probabilities obtained from these base classifiers for each application, EnDroid employs a meta-classifier to train the final classification model. This classification model is subsequently used to distinguish between benign and malicious applications and is applied in the detection phase. In the detection phase, feature vectors are generated for each unknown application, and the classification model determines whether the unknown application is benign or malicious based on its feature vector.

4.2.3.1 Training Phase

1.

Dynamic Behavior Extraction:

In the initial step of the training phase, EnDroid captures runtime behavioral information from the provided applications. To achieve this, EnDroid leverages the open source dynamic analysis tool, DroidBox. DroidBox offers an application sandbox, an extended version of TaintDroid, and utilizes dynamic taint analysis with system hooks at the framework level. It records diverse application operations, including file and network accesses, information leaks, and password operations.

A control script executes the application executable within DroidBox’s monitoring environment using MonkeyRunner. EnDroid extends DroidBox’s monitoring environment by incorporating the strace tool to record system calls related to process and I/O activities. EnDroid observes nine types of application operations and the system calls invoked by the application. Besides using these nine actions directly as features, EnDroid defines formative behavioral features.

The final format of EnDroid’s features is presented in Table 2.​2.

Encryption Operations:

Android malware frequently utilizes encryption operations to encrypt various components such as root exploits, malicious payloads, critical method identifiers, target premium SMS numbers, and URLs pointing to remote malicious servers. This tactic aims to bypass static detection. In some cases, malicious applications shift a majority of their functions to separate DEX files stored in encrypted form. For such instances, a decryption routine is crafted to decrypt the DEX file, which is then loaded through a dynamic class loader. Conversely, legitimate applications typically use encryption APIs to secure data like passwords and personal information. To distinguish between various encryption behaviors, these features can be defined as <action>_<algorithm>, where <action> represents encryption, decryption, and key generation operations, and <algorithm> represents different encryption algorithms.

Network Operations:

Android malware often exploits network traffic to receive commands from Command and Control (C&C) servers and acquire malicious payloads from nefarious websites. Attackers employ this malicious software to remotely control infected devices, harvest personal privacy information, or execute denial-of-service attacks. DroidBox is capable of monitoring four types of network behaviors, encompassing the opening and closing of network connections, as well as sending and receiving network data. These features can be formed as <action>$$\_$$<ip>$$\_$$<port>, where <action> refers to the four network behaviors.

File Operations:

Malicious Android applications commonly scan the file system to search for various information, including bank accounts, installation and debugging details of target applications, and more. These applications also generate external files to store sensitive information downloaded from remote servers or malicious payloads. These functions can be formed as <action>$$\_$$<filename>, where <action> represents reading or writing different files.

Dex Class Loading:

Modern Android applications are confronted with emerging threats like polymorphic malware and composite malware, which leverage dynamic code loading to decrease the probability of detection. Android malware frequently loads extra .jar, .apk, and .dex files at runtime from application assets, another application (collusion attack), or remote systems. This allows the execution of malicious code that was not originally installed as part of the application. These features can be formed as load_<filename>.

Information Leakage:

The security and privacy of private and confidential data have garnered substantial public attention. Android malware frequently engages in the active acquisition of diverse information from infected devices, encompassing IMEI, device-specific identifiers, SMS contents, contact information, social network account credentials, and banking information. This collected information is secretly sent to remote servers. The gathered information can be used for user tracking, profit, obtaining legitimate accounts, etc. Privacy leakage constitutes a significant portion of malware threats. These features can be formed as <source>_<sink>, where <source> represents the operation of obtaining sensitive information and > represents the operation of leaking sensitive information.

SMS Sending:

Malicious software frequently results in economic losses for infected users. Premium services encompass subscription information, game services, charitable donations, and similar offerings. These services, provided by telecom operators, entail additional charges for users, beyond standard communication fees. Android malware clandestinely subscribes to premium services by sending a few SMS messages without the user’s consent. These features can be formed as sms$$\_$$<number>.

Phone Calls:

Android malware commonly initiates phone calls without requiring any interaction from the user. These features can be formed as call$$\_$$<number>.

Service Launch:

Android malware typically carries out malicious actions within background processes contained within service components. These features can be formed as start$$\_$$<servicename>.

Receiver Behavior:

Android malware frequently leverages system events to initiate malicious actions. Registered broadcast receivers serve as effective mechanisms for monitoring these system events. For example, registering the BOOT_COMPLETED intent in malicious software signifies the initiation of malicious activities immediately after the mobile device boots. These features can be formed as broadcast_<intentname>.

System Calls:

System calls are a means through which an application requests services from the operating system kernel. They offer valuable functionality for applications, including power management, device security, access to hardware resources, and interactions related to processes. Android malware frequently invokes system calls such as ptrace, sigproc masks, and getuid to exert influence over the execution of other applications and ensure the successful execution of their processes.

2.

Feature Vector Generation:

After the dynamic behavior extraction process, EnDroid obtains a set of dynamic behavior log files. Each behavior extracted from these log files is considered a detected feature by EnDroid. Consequently, the dimensionality of detected features corresponds to the size of the attribute database. If we denote the size of the attribute database as S, EnDroid establishes an S-dimensional feature vector space. For every application, a feature vector is constructed by mapping its dynamic behavior log files to this S-dimensional vector space. Subsequently, by setting the respective dimension of each extracted feature to 1 and the remaining dimensions to 0, the feature vector of the application can be effectively constructed.

EnDroid aims to extract an extensive set of dynamic behavioral features from the given application and then proceeds to construct feature vectors by mapping these behavior log files to the vector space. It is important to note that the dimensionality of the feature vector is contingent upon the count of the extracted dynamic behavioral features. In cases where EnDroid extracts an excessive number of these features during the dynamic behavior extraction, it could result in high-dimensional feature vectors during the feature vector generation process. However, some of these features might be interrelated and could potentially contain noisy, irrelevant, or redundant information. This situation could adversely affect the performance and efficiency of machine learning algorithms. To address this, feature selection is employed to choose a subset of features that effectively describe the training data. This process aims to reduce dimensionality, eliminate redundant features, and mitigate overfitting.

EnDroid applies the well-known filtering method chi-square to its feature selection process. The chi-square algorithm plays a pivotal role in feature selection by eliminating irrelevant or redundant features. It assesses the correlation between features and classes, allowing the retention of an original functional subset that is adequate for effectively classifying Android applications. As a result, this method proves effective in significantly reducing the dimensionality of the feature vectors.

3.

Learning:

In the training phase, EnDroid employs two primary classification algorithms to learn dynamic behavior features from both benign and malicious applications. Using input feature vectors generated from both types of applications, EnDroid applies supervised learning algorithms to produce a classification model capable of effectively distinguishing feature vectors originating from benign applications and malicious software. This classification model is then carried forward to the detection phase for subsequent application.

To capitalize on the unique strengths of various machine learning algorithms and to further boost the effectiveness of malicious software detection, an ensemble approach is implemented subsequent to obtaining classification results from these learning algorithms. This ensemble comprises multiple base classifiers, including decision trees, linear support vector machines, extremely randomized trees, random forests, and boosting trees. Additionally, a meta-classifier, Logistic Regression, is utilized to aggregate the outputs of these base classifiers.

4.2.3.2 Detection Phase

During the detection phase, EnDroid extracts dynamic behavioral features from unknown applications, generating their respective feature vectors. Leveraging the classification model, the system then determines whether the application is benign or malicious based on this feature vector.

4.3 Detection Through Semantic Code Analysis

4.3.1 Dalvik Bytecode

Dalvik was a custom-built virtual machine by Google tailored for the Android operating system, finely tuned to meet its specific requirements. While programs on Android are developed using Java, Dalvik and the standard Java Virtual Machine (JVM) are distinct entities. Dalvik VM utilizes a register-based architecture, contrasting with the stack-based architecture of the JVM. Additionally, Dalvik utilizes its proprietary executable format called “dex” (Dalvik Executable), while the JVM executes Java bytecode. Dalvik VM demonstrates faster execution speed and consumes less memory compared to the JVM.

Despite developers using Java to write Android applications, Dalvik does not directly load .class files. Instead, the dx tool is employed to optimize .class files into .dex files, which are then loaded by the Dalvik VM. Consequently, the analysis of APK files cannot be conducted directly through analyzing .class files. Instead, the baksmali.jar tool is employed to decompile .dex files into corresponding smali files. Smali files represent the bytecode equivalent of Dalvik, although they are not entirely identical.

Smali File Structure

Upon decompiling with baksmali.jar, each .smali file corresponds to a class in Java. Each smali file is composed of Dalvik instructions and adheres to a specific structure. Smali encompasses a variety of instructions used to depict the corresponding Java code, with all instructions commencing with a period “.”. Commonly used instructions are presented in Table 4.1.

Table 4.1

Some keywords and descriptions in smali file

Keyword

Description

Keyword

Description

.class

Defines package and class names

return-void

Function returns void

.super

Superclass name

.method...end method

Defines a method

.source

Source file name

.annotation...end annotation

Defines an annotation

.field

Defines a field

.implements

Defines interface directive

invoke-super

Calls parent function

.local

Specifies number of local variables

invoke-direct

Calls a function

.registers

Specifies total registers used in a method

invoke-static

Calls a static function

.prologue

Marks the beginning of method code

new-instance

Creates an instance

.line

Indicates a specific line in Java source file

iput-object

Assigns object

.parameter

Specifies method parameters

iget-object

Calls object

.param

Same as .parameter, but with a different format

Smali Data Types

Similar to Java, Dalvik bytecode has two main data types: primitive types and reference types. There are eight primitive data types, while objects and arrays are reference types. The descriptions of data types in Dalvik bytecode align with those in the JVM.

Primitive Data Types: Z, B, S, C, I, L, F, and D are primitive data types. As shown in the table above, the descriptors for Dalvik bytecode primitive types are generally derived from the initials of Java’s primitive types, with the exception of Boolean, which corresponds to Z.

Object Types: Object types are represented using “L” followed by the fully qualified name of the class or interface, such as Lpackage/objectName. For example, the descriptor for the String type is Ljava/lang/String, and the descriptor for a class “test” in the package “com.biyou” would be Lcom/biyou/test.

Array Types: Arrays of primitive types are represented by appending “[” to the descriptor of the base type. For example, an array of int is represented as [I, a two-dimensional array of int is represented as [[I, and so on.

Arrays of object types are represented by appending “[” to the descriptor of the object type. For instance, an array of String is represented as [Ljava/lang/String.

Java data types and Dalvik bytecode data types have a one-to-one correspondence, as shown in Table 4.2.

Table 4.2

The correspondence between Smali descriptors and Java types

Java Type

Smali Descriptor

Description

Boolean

Z

Boolean type

byte

B

Byte type

short

S

Short integer type (16-bit)

char

C

Character type

int

I

Integer type

long

J

Long integer type (64-bit)

float

F

Floating-point type

double

D

Double precision type (64-bit)

void

V

Void type, used only as return type

L

Object type

[

Array type

4.3.2 Overview of Code Semantic-Based Detection Methods

Android applications are usually coded in Java and then compiled into Dalvik bytecode (DEX), which is stored in a file named “classes.dex.” Decompiling an APK file provides access to the corresponding Dalvik bytecode, stored in smali files. Each smali file corresponds to a class in Java and has its own set of instructions, similar to assembly language. A Dalvik instruction consists of an opcode and operands. Additionally, the Dalvik instructions within a function can be grouped into basic blocks based on their execution order. A basic block is a sequential sequence of statements executed in a program, characterized by a single entry point (the first statement) and a single exit point (the last statement). Basic blocks do not contain branching instructions; execution enters through the entry point and exits through the exit point.

In the realm of bytecode features, a common approach involves applying k-gram techniques [18] to opcode sequences. This involves using a sliding window of size k to partition opcode sequences into features composed of k consecutive opcodes, enhancing feature robustness [19, 20]. However, such methods exhibit exponential growth in the number of features as k increases. Typically, k values range from 1 to 5, and the feature space can vary significantly across different datasets.

An alternative approach, exemplified by Suarez-Tangil et al.’s Dendroid [21], utilizes basic blocks as features. Dendroid represents each sample as a collection of basic blocks and designs a syntax to characterize basic block elements. This approach constructs feature vectors for family recognition.

In summary, code semantic-based detection methods for Android malware often involve analyzing the Dalvik bytecode using techniques such as k-grams for opcode sequences or utilizing basic blocks as features, with the goal of enhancing the detection and recognition of malware families.

4.3.3 Specific Case: MviiDroid

MviiDroid [22] is a framework for detecting malicious Android applications, leveraging multi-view information integration technology to improve the representation capabilities of current classification models. The approach involves extracting various components of an application, transforming them into embedded feature vectors, and utilizing a multi-kernel learning model as the classifier.

In the static analysis phase, foundational analysis is conducted on each app, involving disassembling, splitting, and reverse engineering of the app package. Subsequently, during the feature generation phase, features are extracted from each component of the app package and transformed into multiple vectors. Once all feature vectors are generated, the model training phase is executed to train the classification module, determining whether a given APP is malicious.

4.3.3.1 Static Analysis Phase

In this phase, the focus is on unpacking the Android APK package and conducting preprocessing on its components. The authors of MviiDroid designed an additional tool to extract the APK package, decode the manifest file, and decompile the DEX files. Additionally, the armno-eabi-objdump tool is utilized to decompile native library files extracted from the APK package into ARM instructions. Some fundamental components of Android applications include:

1.

AndroidManifest.xml: This file contains declarations for all necessary permissions, utilized components, and intent filters of the Android application. MviiDroid extracts this manifest file from the APK package and converts its binary encoding to text. All required permissions can be extracted by scanning the USES-PERMISSION tags.

2.

DEX files: Android applications are generally developed using the Java language. Java code is translated, refactored, and compressed into .class files, which are then further converted into DEX files. MviiDroid extracts all Classes.dex files from the APK package and converts them into Dalvik instructions for subsequent statistical analysis.

3.

Native library files: For performance and security considerations, Android applications often employ C/C++ to implement critical code sections. These codes are compiled into Native libraries, extending functionality to the Java layer or other modules. Unfortunately, malicious software frequently exploits native libraries to inject malicious code and conceal nefarious activities. MviiDroid supports the decompilation of native libraries, extracting ARM instruction opcodes to analyze the code characteristics of these libraries.

4.3.3.2 Feature Generation Phase

MviiDroid employs permissions, sensitive APIs, Dalvik opcodes, and native opcodes to describe the behavioral features of Android applications. All these features are computed into feature vectors using either method-based or frequency-based approaches.

1.

Perm-Vector:

The Android system offers approximately 179 permissions, making the permission vector a binary representation of length 179. Each bit in this vector represents a specific permission. When a permission is utilized in an app (Used-method), the corresponding bit in the permission vector is set to 1. Conversely, bits corresponding to permissions not used in that app are set to 0. In addition to the standard Android system permissions, MviiDroid also gathers user-defined permissions found in samples from the MalGenome and Drebin datasets. The 100 most frequently used permissions from these datasets, combined with the 179 predefined permissions, create a binary permission vector of length 279.

2.

SAPI-Vector:

This type of vector is generated using a frequency-based approach. The process involves scanning the bytecode in the decompiled classes.dex file, extracting all API calls, and calculating their occurrence frequency in the Android application. The frequency-based method entails combining the frequency values of APIs within a defined range to form the vector. Following the Pscout reference, MviiDroid selects 500 sensitive APIs and calculates their frequencies based on importance and occurrence, forming a sensitive API vector of length 500.

3.

Dopcode vector:

The frequency distribution of malicious software Dalvik opcodes significantly deviates from benign applications. Calculating 1-opcode and 2-opcode vectors can effectively capture these differences. The process to generate an n-opcode vector is as follows: Let P be the set of opcodes, p represents an opcode, and m represents the number of types of Dalvik opcodes. The count of n-opcode combinations $$(p_1, p_2, \ldots , p_n)$$ can be computed as $$m^n$$. Let f denote the frequency of composing $$(p_1, p_2, \ldots , p_n)$$ within a code segment. To generate an n-opcode feature vector in the form $$&lt;f_1, f_2, \ldots , f_(m^n)&gt;$$, each $$f_i$$ is computed. MviiDroid chooses n to be 2 for calculating the 2-opcode vector. In the Android operating system, m is approximately 250, resulting in a length of $$250^2=62{,}500$$ for the 2-opcode vector, which poses a high-dimensional sparse problem with excessive computation. Considering this factor, MviiDroid selects the most frequent 2-opcode sequences from the MalGenome and Drebin datasets while disregarding other 2-opcode types. Additionally, all 1-opcode types are included to compensate, aiming to significantly enhance computation speed with only a slight loss in expressive capability. MviiDroid selects 250 types of 2-opcode and includes all 250 types of 1-opcode, forming a Dopcode vector of length 500.

4.

Nopcode-Vector:

This type of vector is generated by extracting opcodes from Native library files, which are then reverse-converted into ARM instruction sequences. Similar to Dopcode-vector generation, MviiDroid defines the Native n-opcode vector as $$&lt;f_1, f_2, \ldots , f_(m^n)&gt;$$. Approximately 250 commonly used ARM instructions are present in the Android system. When adopting the 2-opcode approach, the length of the native opcode vector becomes 62,500, leading to a high-dimensional sparse issue. To address this, MviiDroid employs a strategy similar to the Dopcode vector, selecting 250 types of 2-opcode and 250 types of 1-opcode to form a Dopcode vector of length 500.

4.3.3.3 Model Training Phase

Through the mentioned steps, four types of vectors can be obtained, each representing a coordinate point in a Hilbert space under different views for mapping an app. These vectors serve as the basis for determining whether a specified app is malicious and for identifying its family classification. In contrast to approaches utilizing deep generative models and deep Siamese attention models to handle multi-channel/view data, it is crucial to construct a model that enables fast processing and efficient feature training. After extensive research, MviiDroid adopts the Multiple Kernel Learning (MKL) algorithm to tackle this challenge.

For the detection task, training a classifier can effectively differentiate between malicious and benign applications. In multi-class scenarios, the MKL algorithm necessitates training multiple models. There are two primary methods: one-versus-one (OVO) and one-versus-all (OVA). If there are m classes in the dataset, the OVA method entails training m classifiers, whereas the OVO method requires training m(m$$-$$1)/2 classifiers. In MviiDroid, the OVA method is employed.

4.4 Detection via Image Analysis

The core idea of image-based Android malware detection methods is to transform Android applications into visual images and utilize advanced techniques and models from the field of computer vision for constructing malware detection models. These methods often employ program analysis techniques to extract features from Android applications and utilize specific transformation methods to convert program features into grayscale or RGB images. Among the most commonly used program features are bytecode from DEX files, permission features, and API call features. The incorporation of computer vision methodologies enables the construction of effective malware detection models.

4.4.1 Overview of Image-Based Detection Methods

Zegzhda et al. [23] generated control flow graphs from SMALI files, extracted features based on API call sequences, mapped protection levels using required permissions, and converted API call sequences and protection levels into RGB image pixels. They utilized a convolutional neural network model for malware detection and achieved an impressive classification accuracy of 93.64$$\%$$ on a dataset comprising 7,192 benign samples and 24,461 malicious samples.

Ganesh et al. [24] applied static analysis to extract 138 permission features categorized into four groups. They then transformed these permission features into $$12\times 12$$ PNG images and utilized a convolutional neural network for training and detection. Remarkably, they achieved a high detection accuracy of 93$$\%$$ on a dataset consisting of 2,500 Android applications, encompassing 2,000 malicious and 500 benign samples.

Huang et al. [25] introduced a unique approach where they converted Classe.dex bytecode into RGB color codes, effectively transforming them into fixed-size color images. These images were then fed into a convolutional neural network for automatic feature extraction and training. Their experimental dataset was extensive, including around two million Android applications, both benign and malicious, collected from January to August 2017. Impressively, this method achieved a remarkably high detection accuracy of 98.4225$$\%$$.

Shiqi et al. [26] proposed a malware content feature extraction method based on texture fingerprints. This approach utilized information from texture images extracted from sample application code. By mapping these code segments to uncompressed grayscale values according to texture image methods and combining API call features, they employed deep belief networks for Android malware detection. Achieving a detection accuracy of 95.6$$\%$$ on a dataset of 6,956 samples.

Yen and Sun [27] presented an innovative method for malware detection by visualizing the importance of words in APK code as images. They decompiled DEX files to obtain Java source code and calculated the significance of each word using term frequency-inverse document frequency. Utilizing algorithms like SimHash and GJB2, they converted this information into images. On a dataset consisting of 720 benign and 720 malicious samples, their model achieved an impressive detection accuracy of 92.67$$\%$$.

Xu et al. [28] utilized semantic graph representations, encompassing control flow graphs, data-flow graphs, and various combinations, to characterize features of Android applications. These graphs were encoded into matrices and utilized to train a classification model based on convolutional neural networks. By leveraging samples from four datasets – Marvin, Drebin, VirusShare, and ConagioDump – they accomplished an impressive detection accuracy of 99$$\%$$.

Pektaş et al. [29] utilized API call graphs as graphical representations, encompassing all possible execution paths that malware could follow during runtime. They employed graph embedding techniques to transform these graphs into low-dimensional numerical feature vectors, which were then integrated into a deep neural network. After conducting evaluation experiments involving various graph embedding algorithms and network structure parameters, they achieved an impressive classification accuracy of 98.86$$\%$$ on a dataset comprising 33,139 malicious samples and 25,000 benign samples.

4.4.2 Specific Case: R2-D2

R2-D2 [25] is an innovative Android malware detection system that leverages a color-inspired Convolutional Neural Network (CNN). One of its key advantages is its ability to minimize the requirement for manual feature engineering by circumventing the need for pre-selected feature extraction. The system achieves this by converting Classes.dex bytecode, extracted from Android archive files, into RGB color codes and representing them as fixed-size color images. These images are subsequently fed into a convolutional neural network, which autonomously extracts features and facilitates the training process for accurate malware detection.

The algorithmic process of R2-D2 is explained in detail below. In the initial stage, Android applications are classified into distinct types, such as benign and various categories of malicious types like Trojan, RiskTool, HackTool, Adware, Banker, Clicker, DownLoader, Dropper, FakeAV, Monitor, SMS, Spy, Ransom, Vulnerability, and Backdoor. The application is then decompressed, allowing retrieval of Classes.dex, which is represented in bytecode format. Following this, hexadecimal values are mapped to RGB color codes utilizing predefined rules. For instance, values like 646578 are mapped to (R: 100, G: 101, B: 120), 0A3033 is represented as (R: 10, G: 48, B: 51), 3500D1 translates to (R: 53, G: 0, B: 209), 4B1222 corresponds to (R: 75, G: 18, B: 34), 565778 corresponds to (R: 86, G: 87, B: 120), and so on. This mapping procedure culminates in the creation of an Android color image, exemplified which portray four distinct Android applications, showcasing both benign and malicious software. These images are provided to the CNN, which trains a model for detecting Android malware.

In the system architecture of R2-D2, users are not required to upload data, minimizing network traffic consumption and enhancing processing efficiency while reducing computational resource demands. Additionally, this approach addresses the challenge of a CNN training model being excessively large, making it impractical for execution on user devices.

Step 1

User scans applications on the Android device.

Step 2

If all applications are recognized, the scan results are directly provided to the user.

Step 3

If there are unknown apps, the system converts the classes.dex into Android color images.

Step 4

The Android color images are then uploaded to the backend.

Step 5

The Android color images are fed into TensorFlow’s GPU computation pool.

Step 6

The trained Inception-v3 model is employed to identify the applications.

Step 7

The results are sent back to the user’s mobile device.

4.5 Detection Through Graph Analysis

The core idea behind graph-based Android malware detection methods is to employ graph-structured data (Graph) to represent an application’s behavior, interdependencies between features, interactions, and more. Subsequently, artificial intelligence algorithms are utilized to analyze and construct a malware detection model based on this representation. Depending on whether the nodes in the graph structure data are homogenous or heterogeneous, these methods can be categorized into two distinct approaches: graph-based detection methods utilizing homogenous graphs and those utilizing heterogeneous graphs.

4.5.1 Overview of Homogeneous Graph-Based Detection Methods

Homogeneous graph-based Android malware detection methods often represent information such as function calls, control flow, and system calls of an application as graph-structured data. These methods vectorize the graph data into a format suitable for artificial intelligence models and generate the final detection model through extensive training with large datasets.

Zhang et al. [30] introduced a malicious software detection model based on graph convolutional neural networks (GCNs). This model captures different patterns of sensitive API calls and designs corresponding methods to simplify function call graphs, effectively reducing their size. However, the effectiveness of this method heavily relies on the content of the sensitive API list, making it unable to generate effective graph features for malicious software that does not involve sensitive APIs.

Feng et al. [31] presented a lightweight static analysis approach for Android malware detection using graph neural networks (GNN). This method involves analyzing the application’s source code and generating approximate call graphs based on the internal function call relationships, effectively representing the application. It further extracts features within functions using word2vec, including required permissions, security levels, and semantic information of smali instructions, to form node features within the graph structure. GNN is then used to generate vector representations, achieving 98$$\%$$ accuracy. However, this method utilizes approximate call graphs, sacrificing some precision and not fully capturing intricate behaviors like malicious code hidden through reflection, implicit callbacks, and implicit control flows.

Yue et al. [32] construct an API call graph to preliminarily characterize APKs. They introduce the Structural Deep Network Embedding (SDNE) graph embedding algorithm to learn both structural and content features from the API call graph. An attention network is employed to comprehensively fuse neighbor node feature vectors, generating graph embedding vectors for the final detection task.

4.5.2 Overview of Heterogeneous Graph-Based Detection Methods

Heterogeneous graph-based Android malware detection methods combine the intrinsic features of applications with their inter-relationship to construct heterogeneous graphs for comprehensive analysis and detection.

Gao et al. [33] have mapped the Android APIs of applications into a comprehensive heterogeneous graph, effectively converting the original problem into a node classification task. They construct edges based on call relationships and API usage patterns and then feed the heterogeneous graph into a Graph Convolutional Network (GCN) model to generate node embeddings. Unlabeled applications are classified based on their resulting embeddings. This method effectively detects 98.99$$\%$$ of Android malware with a false positive rate below 1$$\%$$. However, it primarily focuses on in-sample node classification, requiring additional computational overhead for out-of-sample node classification. To address the time and computational cost issue when dealing with out-of-sample nodes in heterogeneous graphs.

Hei et al. [34] have introduced a novel malware detection framework named HAWK. This framework models entities and behavioral relationships in Android apps using a Heterogeneous Information Network (HIN), leveraging the rich semantic meta-structure of HIN to specify implicit higher-order relationships and considerably reduce detection time. However, this method still faces challenges related to model aging.

Niu et al. [35] proposed a graph compression algorithm using Graph Compressed Reachability Relationship (GCRR) extraction. They developed an Android malware detection and classification method called GCDroid based on this algorithm. Theoretical analysis demonstrates that GCRR is capable of extracting meaningful reachability relationships between APKs and efficiently compressing large, heterogeneous APK-API relationship graphs into homogeneous APK graphs. Experimental results indicate that GCDroid based on GCRR not only improves detection accuracy but also significantly reduces the required time consumption.

4.5.3 Case Study: HAWK

HAWK, presented by Hei et al. [34], offers a pioneering malware detection framework designed to adapt to the evolution of Android applications. The framework models Android entities and their behavior relationships using a Heterogeneous Information Network (HIN), leveraging the intricate semantic meta-structure to delineate implicit higher-order relationships. Additionally, HAWK integrates an incremental learning model to effectively handle dynamically evolving applications, eliminating the need for complete reconstruction of the entire HIN and subsequent embedding models. This approach allows for rapid determination of the proximity of new applications to existing in-sample applications, aggregating their numerical embeddings across various semantics.

The architecture of HAWK, comprising two main components: the Data Modeler and the Malware Detector.

In the Data Modeler, the process begins with the Relationship Extractor, which initially extracts Android entities based on feature engineering. This involves compiling and analyzing a large volume of Android applications. The extracted entities fall into seven types of nodes, including “App” and six distinct features. Additionally, there are six types of edges representing different relationships.

Next, the HIN Constructor takes the extracted entities and relationships, organizing them into nodes and edges within the Heterogeneous Information Network (HIN).

The App Graph Constructor focuses on generating isomorphic relationship subgraphs from the HIN, specifically containing only App entities. This is achieved by utilizing meta-structures, including meta-paths and metagraphs.

Following this, the Malware Detector utilizes two distinct representation learning models to digitally embed in-sample and out-of-sample nodes. Maximizing the utilization of node affinity within the provided meta-structure and aggregating embeddings of the same node across different meta-structures is of paramount importance. HAWK employs specific strategies to achieve effective embedding learning:

To represent in-sample App nodes, HAWK introduces MSGAT, a meta-structure-aware Graph Attention Network (GAT) solution. MSGAT aggregates embeddings between neighboring nodes belonging to a specific metagraph using an attention aggregation mechanism within the meta-structure. In the subsequent meta-structure cross-stage, the obtained embeddings from different meta-structures are further fused to effectively represent their semantics in the final embedding.

To effectively address out-of-sample node embeddings, HAWK reuses embeddings of selectively chosen in-sample App nodes that are proximate to the target node. These embeddings are aggregated to incrementally generate embeddings for new nodes. This requires the model to determine the similarity between existing in-sample App nodes and the target node. Additionally, embeddings are collected at the neighboring node level within a specific meta-structure before conducting cross-meta-structure aggregation.

The malware classifier trains a classification model using the generated vector embeddings to determine whether a given application is malicious or benign. Subsequently, the effectiveness of the model is validated. Commonly used techniques such as Random Forest (RF), Logistic Regression (LR), and Support Vector Machine (SVM) can be employed as classifiers.

4.5.3.1 Feature Engineering

Android applications need to be packaged in APK format for installation on the Android system. An APK file contains various components, including code files, the configuration file androidManifest.xml, signing and verification information, a lib directory (containing platform-specific compiled code), and other resource files. To conduct thorough analysis of Android applications, reverse engineering tools such as APKTool2 are commonly utilized to decompile APK files, converting .dex source files into .smali files. HAWK employs this approach to extract six essential types of entities that describe key features of an app.

1.

Permissions (P): Permissions specify particular actions that an application is allowed to undertake. For example, an application granted the READ$$\_$$SMS permission is permitted to access a user’s email information.

2.

Permission Type (PT): Permission types categorize specific permissions.

3.

Classes (C): Classes are abstract modules in Android code that can directly access interfaces and variables. HAWK uses class names from .smali code to represent classes.

4.

APIs: Predetermined callable functions in the Android development environment.

5.

Interfaces (I): Interfaces refer to abstract data structures in Java. Interface names can be extracted from .smali files.

6.

.so Files (S): .so files, also known as shared object files, are Android dynamic link libraries that can be extracted from the decompiled lib folder.

Following this approach, HAWK downloaded over 200,000 APKs from open repositories, filtered and extracted 181 APKs after deduplication and decompilation. Then, it selected 63,902 entities. This provides a rich data source for establishing the HIN and mining its inherent semantics.

4.5.3.2 Constructing Heterogeneous Information Network (HIN)

Extracting Entity Relationships into HIN

Metaschema refers to a meta-level template defining the relationships and type constraints of nodes and edges within the Heterogeneous Information Network (HIN). HAWK identifies a meta-pattern capable of encoding crucial relationships among Android entities. Employing domain knowledge, HAWK meticulously explores the following fundamental semantic relationships:

(1)

[R1] App-API: Represents a specific App having a particular interface. Utilizing the relationship between Apps and APIs to mine and depict connections between two Apps is effective.

(2)

[R2] App-Permission: Denotes that a particular app holds specific permissions. Apps with permissions such as READ_SMS, SEND_SMS, and WRITE_SMS demonstrate a significant correlation. For example, if Send_SMS is common between App1 and App2, and Read_SMS is common between App2 and App3, it implies a probable implicit association between App1 and App3.

(3)

[R3] Permission-Permission Type: Describes the classification of permissions into specific permission types. Permissions can typically be classified into different types.

(4)

[R4] App-Class: Represents an App containing specific classes from external SDKs. Malicious software often generates instances using classes from malicious SDKs.

(5)

[R5] App-Interface: Indicates that an application includes specific interfaces from an external SDK.

(6)

[R6] App-.so: Denotes that the App has a particular .so file.

Storing Entity Relationships

HAWK uses relation matrices to store each relationship independently. For example, a matrix A is generated where element $$A_{i,j}$$ indicates whether App i contains API j. In a similar manner, the transpose of the matrix represents reverse relationships, such as API j belonging to App i. As outlined in the following , six matrices ([r1] to [r6]) are employed to represent and store relationships. However, in the presence of intricate semantic links, i.e., higher-level relationships, establishing connectivity between two Apps is essential.

R1 [A]: If App i contains the API j, $$A_{(i,j)}$$ is 1; otherwise 0.

R2 [P]: If App i has the permission j, $$P_{(i,j)}$$ is 1; otherwise 0.

R3 [T]: If the type of permission i is j, $$T_{(i,j)}$$ is 1; otherwise 0.

R4 [C]: If App i owns the Class j, $$C_{(i,j)}$$ is 1; otherwise 0.

R5 [I]: If App i uses the interface j, $$I_{(i,j)}$$ is 1; otherwise 0.

R6 [S]: If App i calls the so file j, $$S_{(i,j)}$$ is 1; otherwise 0.

4.5.3.3 Constructing Application Graph from HIN

To create a homogeneous graph comprising only App nodes, a crucial step involves integrating relationships between App entities and other entities into the overall connectivity between Apps. HAWK identifies hidden higher-order semantics by calculating the adjacency of application nodes within this graph using the provided meta-paths or meta-graphs within the HIN. This calculation results in the node adjacency matrix of this graph. In essence, under a given meta-structure, the HIN can be transformed into an exclusive homogeneous graph, where each node has meta-structure-specific neighboring nodes.

Certainly, meta-paths establish meaningful connections between pairs of nodes. HAWK takes this a step further by enriching meta-structures to include meta-graphs, which are essentially Directed Acyclic Graphs (DAGs). These meta-graphs serve as extended templates capable of capturing various and meaningful combinations of existing relationships between pairs of nodes. In essence, meta-structures provide a filtered view to extract isomorphic node graphs, where all nodes adhere to specific intricate semantics. It can be argued that nodes will be distinctly organized in different graphs based on different meta-structures. Each graph, to some extent, can be viewed as a subgraph of the overall HIN under a certain perspective, with each subgraph adhering to the semantic constraints imposed by the respective meta-structure.

(1)

Meta-Structures: HAWK leverages domain knowledge from the field of system security to thoughtfully select meta-structures that encompass intrinsic relationships. Initially, a comprehensive set of possible meaningful semantic meta-structures is composed. Subsequently, a careful selection is made through extensive experimentation to ensure sufficient precision. Six meta-paths and three meta-graphs are ultimately proposed, effectively summarizing the structural semantics between two Android Apps within the HIN and capturing rich relationships. For instance, A-P-A describes the relationship between two Apps having the same permission (MP5), while A-P-PT-P-A signifies Apps sharing the same type of permission (MP6). The chosen nodes must adhere to all predefined constraints, implying tightened semantic constraints.

(2)

Isomorphic Application Graphs Corresponding to Meta-Structures: A sequence of matrix operations applied to the modeled relation matrices accurately computes the adjacency relationships of nodes within the graph.

4.6 Chapter Summary

This chapter provides a comprehensive discussion of Android malware detection methods based on artificial intelligence. These methods are categorized into five distinct approaches based on variations in feature extraction techniques and post-processing methods. For each category of detection approach, a thorough presentation of its feature extraction and processing techniques is provided. Subsequently, the core principles underlying each method are expounded upon, followed by a presentation of concrete detection cases that illustrate the procedural workflow of each respective approach.

References

1.

Wang W, Wang X, Feng D, et al. Exploring permission-induced risk in android applications for malicious application detection[J]. IEEE Transactions on Information Forensics and Security, 2014, 9(11): 1869–1882.Crossref

2.

Li J, Sun L, Yan Q, et al. Significant permission identification for machine-learning-based android malware detection[J]. IEEE Transactions on Industrial Informatics, 2018, 14(7): 3216–3225.Crossref

3.

Zhou Y, Wang Z, Zhou W, et al. Hey, you, get off of my market: detecting malicious apps in official and alternative android markets[C]//NDSS. 2012, 25(4): 50–52.

4.

Aafer Y, Du W, Yin H. Droidapiminer: Mining API-level features for robust malware detection in android[C]//Security and Privacy in Communication Networks: 9th International ICST Conference, SecureComm 2013, Sydney, NSW, Australia, September 25-28, 2013, Revised Selected Papers 9. Springer International Publishing, 2013: 86–103.

5.

Zhao M, Ge F, Zhang T, et al. AntiMalDroid: An efficient SVM-based malware detection framework for android[C]//Information Computing and Applications: Second International Conference, ICICA 2011, Qinhuangdao, China, October 28-31, 2011. Proceedings, Part I 2. Springer Berlin Heidelberg, 2011: 158–166.

6.

Zhu Z, Dumitraçş T. Featuresmith: Automatically engineering features for malware detection by mining the security literature[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016: 767–778.

7.

Kouliaridis V, Potha N, Kambourakis G. Improving android malware detection through dimensionality reduction techniques[C]//Machine Learning for Networking: Third International Conference, MLN 2020, Paris, France, November 24–26, 2020, Revised Selected Papers 3. Springer International Publishing, 2021: 57–72.

8.

Cai L, Li Y, Xiong Z. JOWMDroid: Android malware detection based on feature weighting with joint optimization of weight-mapping and classifier parameters[J]. Computers & Security, 2021, 100: 102086.Crossref

9.

Miao Xiaochuan, Wang Rui, Xu Lei, et al. Security analysis for Android applications using sensitive path identification[J]. Journal of Software, 2017, 28(9): 2248–2263.

10.

Arp D, Spreitzenbarth M, Hubner M, et al. Drebin: Effective and explainable detection of android malware in your pocket[C]//Ndss. 2014, 14: 23–26.

11.

Daoudi N, Allix K, Bissyandé T F, et al. A deep dive inside drebin: an explorative analysis beyond android malware detection scores[J]. ACM Transactions on Privacy and Security, 2022, 25(2): 1–28.Crossref

12.

Liu Wei. Research on a Method of Security Detection for Android Based on Intent[J]. Computer technology and development, 2019, 29(05):102–106.

13.

Chao F, Yang Z, Du X, et al. Classified risk assessment method of Android application based on multi-factor clustering selection[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 161–173.

14.

Hou S, Saas A, Chen L, et al. Deep4maldroid: A deep learning framework for android malware detection based on linux kernel system call graphs[C]//2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW). IEEE, 2016: 104–111.

15.

Martinelli F, Marulli F, Mercaldo F. Evaluating convolutional neural network for effective mobile malware detection[J]. Procedia computer science, 2017, 112: 2372–2381.Crossref

16.

Yeh C W, Yeh W T, Hung S H, et al. Flattened data in convolutional neural networks: Using malware detection as case study[C]//Proceedings of the International Conference on Research in Adaptive and Convergent Systems. 2016: 130–135.

17.

Feng P, Ma J, Sun C, et al. A novel dynamic Android malware detection system with ensemble learning[J]. IEEE Access, 2018, 6: 30996–31011.Crossref

18.

Tian Z, Zheng Q, Liu T, et al. Software plagiarism detection with birthmarks based on dynamic key instruction sequences[J]. IEEE Transactions on Software Engineering, 2015, 41(12): 1217–1235.Crossref

19.

Canfora G, De Lorenzo A, Medvet E, et al. Effectiveness of opcode ngrams for detection of multi family android malware[C]//2015 10th international conference on availability, reliability and security. IEEE, 2015: 333–340.

20.

Zhang B, Xiao W, Xiao X, et al. Ransomware classification using patch-based CNN and self-attention network on embedded N-grams of opcodes[J]. Future Generation Computer Systems, 2020, 110: 708–720.Crossref

21.

Suarez-Tangil G, Tapiador J E, Peris-Lopez P, et al. Dendroid: A text mining approach to analyzing and classifying code structures in android malware families[J]. Expert Systems with Applications, 2014, 41(4): 1104–1117.Crossref

22.

Wu Q, Li M, Zhu X, et al. Mviidroid: A multiple view information integration approach for android malware detection and family identification[J]. IEEE MultiMedia, 2020, 27(4): 48–57.Crossref

23.

Zegzhda P, Zegzhda D, Pavlenko E, et al. Applying deep learning techniques for Android malware detection[C]//Proceedings of the 11th International Conference on Security of Information and Networks. 2018: 1–8.

24.

Ganesh M, Pednekar P, Prabhuswamy P, et al. CNN-based android malware detection[C]//2017 international conference on software security and assurance (ICSSA). IEEE, 2017: 60–65.

25.

Hsien-De Huang T T, Kao H Y. R2-d2: Color-inspired convolutional neural network (cnn)-based android malware detections[C]//2018 IEEE international conference on big data (big data). IEEE, 2018: 2633–2642.

26.

Shiqi L, Shengwei T, Long Y, et al. Android malicious code Classification using Deep Belief Network[J]. KSII Transactions on Internet & Information Systems, 2018, 12(1).

27.

Yen Y S, Sun H M. An Android mutation malware detection based on deep learning using visualization of importance from codes[J]. Microelectronics Reliability, 2019, 93: 109–114.Crossref

28.

Xu Z, Ren K, Qin S, et al. CDGDroid: Android malware detection based on deep learning using CFG and DFG[C]//Formal Methods and Software Engineering: 20th International Conference on Formal Engineering Methods, ICFEM 2018, Gold Coast, QLD, Australia, November 12-16, 2018, Proceedings 20. Springer International Publishing, 2018: 177–193.

29.

Pektaş A, Acarman T. Deep learning for effective Android malware detection using API call graph embeddings[J]. Soft Computing, 2020, 24: 1027–1043.Crossref

30.

Zhang Xuetao, Wang Jinshuang, Sun Meng. GCN-based Android Malware Detection Model[J]. Software Guide, 2020, 19(7): 187–193.

31.

Feng P, Ma J, Li T, et al. Android Malware Detection via Graph Representation Learning[J]. Mobile Information Systems, 2021, 2021.

32.

Yue Ziwei, Fang Yong, Zhang Lei. Android malware detection based on graph attention networks[J]. Journal of Sichuan University (Natural Science Edition), 2022, 59(05): 88–95. https://​doi.​org/​10.​19907/​j.​0490-6756.​2022.​053002.

33.

Gao H, Cheng S, Zhang W. GDroid: Android malware detection and classification with graph convolutional network[J]. Computers & Security, 2021, 106: 102264.Crossref

34.

Hei Y, Yang R, Peng H, et al. Hawk: Rapid android malware detection through heterogeneous graph attention networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021.

35.

Niu W, Wang Y, Liu X, et al. GCDroid: Android Malware Detection Based on Graph Compression with Reachability Relationship Extraction for IoT Devices[J]. IEEE Internet of Things Journal, 2023.

Part IIIThe Adversarial Method for Android Malware Detection

Since the advent of Android malware, academia and industry have proposed many detection schemes for Android malware, which we have introduced in detail in Section II. But security-related researchers are always in an arms race for malware detection and defense. Aiming at the emergence of malicious software, security researchers propose corresponding detection methods. Against the existing malware detection methods, attackers have proposed many countermeasures to bypass them.

In this part, we will introduce common countermeasures, defense methods, and related research of Android malware. According to the classification of existing countermeasures, this part will be divided into three chapters to introduce these countermeasures: static adversarial method, dynamic adversarial method, and machine learning algorithm adversarial method. I hope that after reading this part, readers can understand the basic principles and detailed classification of these countermeasures and have a deeper understanding of the defense methods and related research of these methods.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_5

5. Static Adversarial Method

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

As Android systems become targets of malicious software attacks, researchers and developers are exploring methods to prevent malware infiltration into Android devices. To counter researchers’ security measures and further increase the spread and penetration of malware, malicious authors use static obfuscation techniques to evade detection. This chapter first introduces three static obfuscation techniques: code obfuscation, resource obfuscation, and manifest file obfuscation. Code obfuscation increases code complexity and ambiguity by modifying identifiers and removing unused code. Resource obfuscation increases the difficulty of decompilation by randomizing resource file names and directories. Manifest file obfuscation enhances app security by replacing component names, permission names, and metadata. Static obfuscation techniques effectively prevent attacks, such as APK decompilation, injection, and tampering, but may impact app performance and user experience and increase app complexity and maintenance costs. Then, common APK static obfuscation tools are presented, including Obfuscapk, ProGuard, DexGuard, Allatori, DashO, Bangcle, and Arxan. These tools were originally designed to protect app security by obfuscating the code to prevent reverse engineering, tampering, and piracy. However, they are also used by malicious software developers to achieve static evasion. To combat malware that uses static obfuscation techniques, researchers have begun using new detection strategies. They explore more difficult-to-obfuscate static features, such as user interface perceptual hashes and grayscale images, to identify malicious software. They also combine dynamic analysis or machine learning to counter the impact of static obfuscation on analysis. Finally, the chapter summarizes some research work related to static obfuscation.

5.1 Static Obfuscation

Static obfuscation is a technique used to enhance the security of apps. It involves obfuscating the code, resources, and manifest files, making the source code and structure of the app difficult to understand and analyze. Static obfuscation techniques include code obfuscation, resource obfuscation, manifest file obfuscation, and control flow obfuscation. These techniques effectively prevent apps from being decompiled and analyzed, thus protecting the app’s security. However, static obfuscation also poses challenges such as performance impact and maintenance costs. Striking a balance between security and performance is crucial when using static obfuscation techniques.

5.1.1 Code Obfuscation

Code obfuscation is a technique used to protect the source code of Android apps, making the decompiled code difficult to read and understand. It involves modifying identifiers such as class names, method names, variable names, and string constants to make them meaningless or inconsistent, thereby increasing code complexity and ambiguity. Additionally, code obfuscation may remove unnecessary code, such as comments, whitespace, and debug information, reducing the size of the APK file. The primary purpose of code obfuscation is to prevent malicious third parties from decompiling the APK file, stealing or tampering with the source code, and safeguarding the app’s intellectual property and security. However, it is also utilized by malicious software developers to evade malware detection.

5.1.2 Resource Obfuscation

APK resource obfuscation is an optimization technique for Android apps. It involves randomizing the names and directories of resource files, which increases the difficulty of reverse engineering, reduces the APK file size, and enhances the app’s security. The principle of APK resource obfuscation lies in modifying resource IDs and paths using obfuscation tools during the APK building process. This process transforms meaningful resource names into meaningless strings, as shown in Table 5.1.

Table 5.1

Example of static resource obfuscation

No.

Origin name

Obfuscated name

1

res/drawable/ic$$\_$$launcher.png

r/a/a.png

2

res/layout/activity$$\_$$main.xml

r/b/a.xml

3

res/values/strings.xml

r/c/a.xml

After applying the described resource obfuscation, even if someone manages to reverse engineer the APK file and access its binary resource files, it will be challenging for them to identify the corresponding resource names and directories. This level of obfuscation helps protect the app’s sensitive resource files and enhances the overall security of the app. It adds an extra layer of complexity for potential attackers, making it more difficult for them to understand and misuse the resources within the APK.

5.1.3 Manifest File Obfuscation

By using Manifest Obfuscator or other tools, an Android app’s manifest file can be obfuscated to improve app security and protection. Information such as component names, permission names, and metadata in the manifest file can be replaced with random strings or encrypted content, and useless information is deleted, making it difficult to analyze and tamper with the manifest file. In this way, malicious third parties are prevented from decompiling, modifying, or repackaging the app.

5.1.4 Control Flow Obfuscation

APK control flow obfuscation is a technique to protect Android apps from reverse engineering and code analysis. The principle of control flow obfuscation is to modify the execution logic of an app to make it difficult for humans or machines to understand. Control flow obfuscation can be achieved by inserting useless instructions, changing jump conditions, adding exception handling, rearranging code blocks, etc. Control flow obfuscation can improve app security, but it can also increase app size and runtime.

APK static obfuscation technology can effectively prevent APK from being decompiled, injected, tampered with and other attacks, and protect the app’s business logic and data security. However, APK static obfuscation technology also has some disadvantages, such as:

1.

The obfuscated APK may increase the startup time and memory usage of the app, affecting the performance and user experience of the app.

2.

The obfuscated APK may cause some functional failures or abnormalities, such as reflection, annotation, multi-channel packaging, etc., which require full testing and debugging by developers.

3.

The obfuscated APK may increase the complexity and maintenance cost of the app, such as error log parsing, version update compatibility, etc., requiring developers to carry out standardized management and documentation.

5.2 Common APK Static Obfuscation Tools

Currently common APK static obfuscation tools include: Obfuscapk, ProGuard, DexGuard, Allatori, DashO, Bangcle, Arxan, etc. These tools can obfuscate Android apps to protect the security of the app.

5.2.1 Obfuscapk

The Obfuscapk tool is a modular Python tool for black-box obfuscation of Android apps. It does not require the source code of the app, only the original apk file. It uses apktool to decompile and repack the apk file and applies various obfuscation techniques on the decompiled smali code, resource files, and Manifest files. The obfuscated app maintains the original functions, but the internal structure and source code are quite different from the original app, which can effectively prevent decompilation and analysis. Obfuscapk tools are highly extensible, and new obfuscation techniques can be added or existing ones modified through a plug-in system. It also supports the Android App Bundle format and can obfuscate aab files. The Obfuscapk tool is an open source project, and its source code and instructions can be found on GitHub.

5.2.2 ProGuard

The ProGuard tool is an open source command-line tool that minifies, optimizes, and obfuscates Java code. It optimizes bytecode and detects and removes unused instructions. The ProGuard tool is free software, licensed under the GNU General Public License version 2. The ProGuard tool is available for Java Micro Edition and Android platforms. Features of the ProGuard tool include:

1.

Bytecode-level optimization: Bytecode is optimized through techniques such as control flow analysis, data-flow analysis, partial evaluation, static single assignment, global value numbering, and liveness analysis. The ProGuard tool can remove many types of unused and duplicated code, perform more than 200 optimizations, reduce variable allocations, inline constants and short methods, simplify tail-recursive calls, remove logging code, and more.

2.

Obfuscation: By renaming classes, fields, and methods with meaningless names, it makes the final app harder to reverse engineer.

5.2.3 DexGuard

DexGuard tool is an advanced protection and optimization tool specially designed for Android apps. It is based on ProGuard but offers more features and security. DexGuard can obfuscate and encrypt all components of the app, including bytecode, manifest files, resource files, native libraries, etc. It also detects and blocks reverse engineering at runtime through Runtime app Self-Protection (RASP) technology.

5.2.4 Allatori

The Allatori tool is a Java code obfuscator that protects your intellectual property and prevents code from being reverse engineered. The Allatori tool not only obfuscates the code but also reduces the size of the app and improves its running speed. The Allatori tool also supports watermarking technology, which can add the appropriate license to your software. The method of using the Allatori tool is to add corresponding plug-ins and configuration files to the Maven project and then package them through the Maven command to generate an obfuscated jar or war file. The Allatori tool is a second-generation Java obfuscator, which is more powerful and intelligent than the first-generation obfuscator in protecting code.

5.2.5 DashO

DashO tool is a professional Java code obfuscation and optimization software, which can effectively protect Java apps from the threat of reverse engineering and tampering. DashO tools have the following features:

1.

Support Java 17 and Android 12, as well as the latest frameworks and libraries.

2.

Provide powerful obfuscation techniques, including string encryption, control flow transformation, renaming, removal of useless code, etc.

3.

Provide flexible configuration options, which can customize obfuscation rules and strategies according to different needs.

4.

Provide a friendly graphical interface and command-line interface, which can be easily integrated into the development and construction process.

5.

Provide detailed reporting and analysis functions, which can help users evaluate obfuscation effects and performance impacts.

5.2.6 Bangcle

Bangcle tool is a professional Android app reinforcement and protection software, which can effectively prevent apps from being cracked, decompiled, tampered, and repackaged. Bangcle tool adopts multi-layer encryption and obfuscation technology to comprehensively protect app code, resources, data, and signatures. It also provides various security detection functions such as anti-debugging, anti-injection, and anti-tampering, as well as the function of custom hardening rules and policies, allowing developers to customize hardening solutions according to their needs. Bangcle tool supports multiple Android versions and architectures and is compatible with mainstream third-party frameworks and plug-ins, without affecting the normal operation and performance of the app.

5.2.7 Arxan

Arxan Tools is a professional software protection solution that helps developers prevent code from being reverse engineered, tampered with, injected, or stolen. The Arxan tool can integrate multiple protection mechanisms into the code during the compilation phase, such as obfuscation, encryption, anti-monitoring, anti-tampering, etc. Arxan tools support multiple platforms and languages, such as X86, ARM, Windows, Linux, Mac OS, iOS, Android, Java, .NET, etc. Arxan tools can also provide detection and reaction capabilities, allowing code to self-heal, notify, or terminate execution if it is attacked. Arxan tool is a security solution suitable for enterprise-level commercial software, which has been widely used in financial services, health care, high-tech, Internet of Things, digital media and gaming and other fields.

5.2.8 Comparative Analysis

The security of Android apps is critical to defend against malware attacks. To protect apps from the threats of reverse engineering and analysis, developers can use various APK static obfuscation tools. We will conduct a comparative analysis of the Android APK static obfuscation tools involved in this chapter, discuss their characteristics, advantages and disadvantages, and organize the results as shown in Table 5.2.

Table 5.2

Static obfuscation tool evaluation

Tool

Advantage

Disadvantage

Obfuscapk

Does not require source code supports the Android App Bundle format, open source project.

The obfuscation effect may be limited and the scalability is low.

ProGuard

Open source command-line tool that supports Java Micro Edition and Android platforms, bytecode-level optimization and obfuscation.

The obfuscation effect may not be as good as other advanced tools.

DexGuard

Advanced protection and optimization tool designed for Android apps, supports comprehensive obfuscation and encryption, provides runtime app self-protection technology

Commercial software, may have a higher cost

Allatori

Protects intellectual property, improves running speed, supports watermarking technology second generation of Java obfuscator.

Not as powerful as other tools

DashO

A professional Java code obfuscation and optimization software, supports the latest Java and Android versions, provides powerful obfuscation techniques and flexible configuration options Commercial software.

Higher cost

Bangcle

A professional Android app hardening and protection software, adopts multi-layer encryption and obfuscation technology, supports custom hardening rules and policies.

Have poor compatibility and affect app performance

Arxan

Professional software protection solution, supports multiple platforms and languages, provides detection and reaction functions, is suitable for enterprise-level commercial software

May have a higher cost

According to comparative analysis, different tools have their own characteristics and applicable scenarios. Developers can choose the appropriate obfuscation tool based on the app’s needs, budget, and performance requirements. For open source projects and limited budgets, Obfuscapk and ProGuard are ideal choices. For commercial apps and projects with high security requirements, DexGuard, DashO, and Arxan provide more comprehensive and advanced protection functions. The performance, effect, and cost of a comprehensive evaluation tool, as well as the compatibility with the development environment, are the key factors for choosing a suitable obfuscation tool.

5.3 Research on Static Obfuscation

5.3.1 Detection Methods Based on New Features

5.3.1.1 Static Detection Based on Perceptual Hashing

As researchers and developers strive to prevent malicious software attacks on Android devices, the ease of reverse engineering Android apps has become a serious problem in the market. To address this issue and improve security, researchers have proposed a novel repackaging detection system based on perceptual hashing of verified Android apps and their related dynamic user interface (UI) behavior. The results show that the average hashing method achieved 88$$\%$$ accuracy in a sample set of 4,878 Android apps, including 2,151 repackaged apps, indicating low false positive and false negative rates. This approach is the first to use image-based hashing technology in a dynamic method, performing comparably to other known dynamic methods and with potential for practical implementation when new apps enter the Android market.

The key technology used in this study is perceptual hashing, a fingerprinting technique for multimedia that extracts various features from its content. Unlike cryptographic hash functions, which heavily rely on slight variations in media to produce completely different hashes, perceptual hashing generates “close” hashes based on the quantity of changes. Therefore, cryptographic hash functions are used to measure whether two images match completely, while perceptual hashing generates a fingerprint that can be measured for image similarity using Hamming distance.

There are three widely used perceptual hashing methods: average hashing, perceptual hashing, and difference hashing.

Average hashing is the simplest form of perceptual hashing. The method takes an image and converts it to grayscale, then reduces the image to an 8x8 pixel size and generates the average pixel value for each pixel. Finally, each pixel value is compared to the average value to generate a 64-bit hash.

Difference hashing is similar to average hashing but relies on gradient changes instead of average pixel grayscale to generate the hash. The algorithm first converts the image to grayscale and then reduces the image to an $$8\times 8$$ square of grayscale values. It calculates row hashes for each row from left to right. If the next grayscale value is greater than or equal to the previous one, the output is 1; otherwise, it is 0. Finally, these bits are concatenated to generate the final hash.

Perceptual hashing relies on Discrete Cosine Transform (DCT) instead of average grayscale. The algorithm operates as follows:

1.

Downsize the image to $$32\times 32$$ pixels.

2.

Reduce the color: Convert the image to grayscale to simplify calculations.

3.

Compute DCT: DCT separates the image into a set of frequencies and scalars.

4.

Calculate the mean: Compute the mean of DCT.

5.

Further reduce DCT: Based on whether each of the 64 DCTs has a mean higher or lower than the average, set each bit of the 64-bit hash to 0 or 1.

6.

Construct the hash: Concatenate the bits to form the 64-bit hash.

In this research, perceptual hashing is applied to dynamic screenshots of Android apps. The experiment aims to validate the effectiveness of the perceptual hash (pHash) algorithm in identifying falsely repackaged apps with the premise that original apps have been recorded. During the experiment, each trusted (benign) app is run through a sandbox environment for analysis to obtain one or more screenshots. The captured screenshots are processed using perceptual hashing functions and stored in a JSON database along with the app’s APK, version information, and certificate. These UI component screenshots serve as the dataset for repackaging and cloning detection.

The implementation architecture used in the study involves five different steps to generate the experimental data:

1.

DroidBox is prepackaged for Android 4.1.1. Therefore, to insert new functionalities into the script, a custom Python script is developed to patch DroidBox for providing an experimental version of DroidBox for this research.

2.

The second step involves creating Docker containers for executing DroidBox. This includes creating a DockerFile and specifying the build instructions for the container, including: (1) Ubuntu 14.04 environment, (2) commands for tinydb, pillow, and imagehash libraries, (3) Android SDK and Android 4.1.2 emulator, (4) SSH to log and forward traffic to the emulator system, and (5) VNC library for allowing control of the emulator screen.

3.

In the third step, the DroidBox Docker container is run, and a large-scale dynamic analysis of APKs is conducted automatically.

4.

Generating perceptual hashing data. During sandbox runtime, the adb tool and monkey tool are used to obtain updates of the Android emulator’s status. For image collection, the monkey tool instantiates the APK and uses adb commands to take periodic screenshots. Images are stored on the host at a rate of one image every five seconds. The ImageHash library is used to generate average hashing, perceptual hashing, and difference hashing. The hash values are then inserted into the centralized tinydb database in JSON format.

5.

The fifth step, prior to analyzing the image hashing data, involves preprocessing the data for reliability. If an app crashes, the hash values of the main Android display screen and error screen are obtained. Then, all three databases are scanned to find images matching the two preprocessed hashes. This ensures that the dataset does not contain any erroneous samples and removes repackaged apps that were not processed in the benign dataset to ensure accurate predictions for effective training.

Nguyen et al. [1] implemented this detection method and applied it to the scalable dynamic analysis technique of sandboxed Android apps. The sandbox environment’s main goal is to support real-time screenshot generation and extraction of Android apps, outputting results as perceptual hashes and storing them in a database. For this, they successfully implemented a scalable Docker container. The final dataset contains 2,151 pairs of benign apps, 2,151 pairs of repackaged apps, and 576 benign apps. Their research results show that average hashing produced the best accuracy compared to perceptual and difference hashing. The results show that at a Hamming distance of 10, it was able to match repackaged apps with their benign pairs with an accuracy of 88.16$$\%$$. The experimental framework and results demonstrate the practical potential of implementing the perceptual hashing method using available tools, software, and simulation environments.

5.3.1.2 Static Detection Based on Semantic Feature Set

Researchers have proposed an extensible method for detecting similar Android apps based on semantic information, which they implemented in a tool called AnDarwin. They further evaluated this method on a dataset of 265,359 apps from 17 app markets, including Google Play and various third-party markets. AnDarwin offers four key advantages compared to earlier approaches: it significantly improves scalability by avoiding pairwise app comparisons, it enhances reliability by analyzing only the app code without relying on other information like market, signature, or description, it can detect both full and partial app similarities, and it automatically detects and excludes library code from the similarity analysis. Within a time frame of 10 hours, AnDarwin detected at least 4,295 cloned victim apps and 36,106 renamed apps. Additionally, it identified instances of injected similar code present in multiple apps, which could indicate the spread of malicious software.

A Program Dependency Graph (PDG) is used to represent a method in a program, where each node represents a statement, and each edge represents the dependency between statements. There are two types of dependencies: data and control.

1.

Data Dependency: If the value of a variable in statement s2 depends on the value assigned to that variable in statement s1, there exists a data dependency edge between statements s1 and s2. For example, if s1 is an assignment statement that assigns a value to a variable, and s2 references the same variable with the assigned value, then s2 has a data dependency on s1.

2.

Control Dependency: If the execution of the second statement depends on the truth value of the first statement, there exists a control dependency relationship between the two statements. In other words, the second statement is executed based on the outcome of the first statement.

In summary, the PDG captures the interdependence between statements in a method, representing both data dependencies and control dependencies.

Over the years, researchers have developed various methods to detect code clones. Code clones refer to two or more code segments that have the same semantics but come from different sources. Finding and eliminating code clones offer several software engineering advantages, such as increased maintainability and improved security, as vulnerabilities in clones only need to be identified and fixed once.

Plagiarism detection and code clone detection share a common goal: identifying reused code. However, code clone detection mainly focuses on code reuse within the same app, while plagiarism detection emphasizes code reuse across different apps, where the apps have different codebases and are identified to have different authors.

Tools for detecting code clones typically fall into one of four categories: string-based, token-based, tree-based, and semantic-based. Among these, semantic-based detection is the most powerful but also the most time-consuming. In the early methods, code was treated as a collection of strings, often based on lines, and reported clones based on identical lines.

DECKARD and its successors used abstract syntax trees (ASTs) of the codebase to create vectors and then clustered these vectors to find similar subtrees, representing code clones.

Semantic-based clone detection methods provide a deeper understanding of the code’s meaning and logic, making them more effective at detecting clones that may have undergone modifications. However, due to the increased complexity of analyzing semantics, these methods are generally more computationally intensive compared to other approaches.

AnDarwin consists of four stages. Firstly, it represents each app as a set of vectors computed on the program dependency graph of the app. Next, it performs clustering on all the vectors of the apps to find similar code segments. After that, it eliminates library code based on cluster frequency. Finally, it detects similar apps while considering both complete and partial app similarity. Additionally, there is a post-processing method to detect apps injected with similar code. The entire workflow of AnDarwin is as follows:

1.

Extracting Semantic Vectors: In the first stage of AnDarwin, each app is represented as a set of semantic vectors. Initially, AnDarwin calculates the undirected PDG (Program Dependence Graph) of each method in the app using only the data dependencies of edges (as control dependency edges may be easier to modify). Then, each PDG is partitioned into connected components, as multiple data-independent computations may occur within the same method. These connected components are referred to as semantic blocks, as each component captures a building block of the method and represents the semantic information stored in the PDG. Finally, AnDarwin computes a semantic vector to represent each semantic block. Each node in the semantic block represents a statement in the method and has an associated type corresponding to that statement. For instance, a node representing addition may have a binary operation type. To capture this information, the semantic vector is computed by calculating the frequency of each type of node in the semantic block. Continuing the above example, a semantic block with only an addition of “x” will have a “1” in the dimension corresponding to the binary operation. AnDarwin utilizes the T. J. Watson Analysis Library (WALA) to construct the PDG. WALA’s PDG comprises a total of 20 node types, resulting in AnDarwin’s semantic vector having 20 dimensions. To enhance the precision of the semantic vector, more information stored by WALA for each node, such as specific binary operations performed, can be used. Doubling or tripling the dimensions would not significantly increase AnDarwin’s overall time complexity. AnDarwin discards semantic blocks with fewer than 10 nodes, as small semantic blocks typically represent trivial and atypical code. Researchers manually analyzed apps to determine this threshold and found that once the code was translated into Static Single Assignment (SSA) form, even a few lines of code would produce relatively large semantic blocks, fulfilling the requirement for building the PDG.

2.

Identifying Similar Code: When two semantic blocks are code clones, they share most of their nodes, resulting in similar semantic vectors. Thus, by finding the nearest neighbors of semantic vectors, potential code clones can be identified. However, not all nearest neighbors are code clones, as two completely unrelated semantic blocks may have the same node type frequencies due to the lossy nature of the semantic vectors – many PDGs can generate the same semantic vector. Specifically, the number of PDGs that generate a given semantic vector increases with the size combination of the semantic vector. Fortunately, evaluations have shown that these collisions are not common in practice.

To determine all nearest neighbors, one could attempt to calculate the pairwise similarity between all semantic vectors. However, this approach is quadratic in the number of vectors, which is computationally prohibitive considering there can be millions of vectors. Instead, Local Sensitive Hashing (LSH) can be used, which is an algorithm that efficiently finds approximate nearest neighbors among a large number of vectors. LSH achieves this by hashing the vectors using many hash functions from a specialized family, such that if the vectors are similar, these hash functions are likely to produce collisions. LSH first hashes all vectors using the specialized hash functions and then looks for nearest neighbors within hash collisions. This allows LSH to identify approximate clusters of similar vectors (code clones), which AnDarwin uses to detect similar apps.

Since semantic blocks of vastly different sizes are unlikely to be code clones, further scalability can be achieved by grouping vectors based on their sizes. To ensure that code clones near group boundaries are not missed, the computation of groups is slightly overlapping. Then, LSH can quickly cluster each group as each individual group is much smaller than the entire set of vectors. Additionally, each LSH computation is independent, enabling all groups to be processed in parallel. This also has the additional benefit of adjusting the clustering radius of each group based on the size of vectors within the group – thus detecting more code clones.

From LSH, AnDarwin identifies many clusters of semantic vectors representing semantic blocks, which could be code clones. However, AnDarwin’s goal is not to identify code clones – it aims to detect app clones. Hence, the representation of apps in the feature space must be determined. Fortunately, code clones can be used to represent apps. Specifically, an index can be assigned to each code clone, and then each app can be represented as a binary vector, where the i-th dimension is 1 if the app contains a semantic block whose semantic vector is a member of the i-th code clone, otherwise 0. This binary vector has many dimensions, one for each code clone; however, it will be very sparse as each app only has a relatively small number of semantic blocks. Thus, to save space, a set containing the indices of non-zero dimensions can be used to store this vector for each app. This set is referred to as the feature vector of the app.

3.

Excluding Library Code: Libraries are collections of code used for sharing among many apps. In Android, libraries are embedded in apps, making it challenging to distinguish between app code and library code. This is problematic because app similarity detection tools should not consider library code when analyzing the similarity of apps. Previous methods used whitelists and manual identification of libraries. However, these approaches are inherently non-scalable and prone to omissions.

In contrast, AnDarwin automatically detects libraries by leveraging the results of its similar code clustering. Libraries consist of many semantic blocks, which AnDarwin maps to semantic vectors. When an app contains a library, it inherits all the semantic vectors derived from the library code. Thus, when computing the feature vector of an app, features from library code will appear in more feature vectors compared to non-library code. Additionally, features representing boilerplate code and compiler-generated structures will occur more frequently.

To exclude these atypical features, AnDarwin disregards any features that appear in more than a threshold number of apps. By doing so, it effectively filters out the features associated with libraries and ensures that the representation of an app is based on its unique code and not the shared library code. This automated approach makes the process scalable and eliminates the need for manual intervention to identify libraries, enhancing the accuracy and efficiency of app similarity detection.

4.

Detecting Similar apps: AnDarwin employs two methods based on these features to determine app similarity: one for complete app similarity and the other for partial app similarity.

For complete app similarity detection, AnDarwin represents each app as a set of features. In the simplest case, two highly similar apps will have a large overlap or complete set of shared features. Conversely, the feature sets of different apps should have minimal overlap. This is reflected in the Jaccard Similarity between their two feature sets FA and FB, reducing the problem of finding similar apps to finding similar sets. The above method successfully identifies apps sharing most of their code but lacks robustness in finding clones that share only part of the code. For instance, consider an app and its copy that adds many methods and removes many original methods to maintain a similar size. Although the app feature sets of these two apps are consistent in many features, their Jaccard Similarity could be low.

To detect partial similarity, for each feature not excluded in the previous section, AnDarwin calculates the set of apps containing that feature. If two features have similar app sets, determined by Jaccard Similarity, then these two features are shared by the same group of apps. If a sufficient number of features are shared by the same group of apps, AnDarwin discovers significant non-library code sharing. By creating clusters of feature sets based on app sets, AnDarwin can detect partial app similarity by finding similar sets. Both complete and partial app similarity detection require finding similar sets. We could attempt to compute pairwise similarity between all sets, but this would be computationally prohibitive given the large number of sets. Fortunately, this can be efficiently approximated using MinHash. MinHash is another Locality Sensitive Hashing (LSH) algorithm initially developed at Alta Vista for detecting similar websites represented as sets of features. MinHash algorithm first computes signatures for each set based on hash and random permutations of the feature space. It then identifies nearest neighbors in sets with hash collisions. By using MinHash, AnDarwin can effectively detect both complete and partial app similarity. The output of MinHash is a list of pairs of similar sets, and we combine them to create clusters of similar sets.

For this purpose, a union-find data structure is initialized, enabling rapid cluster merging and element lookups, with each set initially placed in its own cluster. Then, each pair (X, Y) is processed, and the two clusters containing X and Y are merged if they are not already in the same cluster. By merging clusters in this way, the average similarity within clusters decreases with each processed pair. For example, A may be similar to B, B to C, and C to D, but this does not necessarily mean that A must be similar to D.

5.

Detecting Common Injection Code: Once AnDarwin has clusters of apps, automatically detecting whether similar code has been injected into many apps would be beneficial. For instance, malicious actors might download numerous harmless apps, inject malicious code, and then upload these apps. We introduce two concepts: cores and differences.

Each app cluster computed by AnDarwin has a core, representing the common code shared among all apps in the cluster. AnDarwin calculates the core by computing the intersection of the feature sets of apps in the cluster. Using this core as a reference point, AnDarwin calculates the differences for each app in the cluster by subtracting the cluster’s core from the app’s feature set. This new set represents a superset of the modified code in the app. AnDarwin computes these cluster cores and corresponding app differences for each app cluster, enabling complete and partial similarity detection. Subsequently, it uses MinHash to cluster the differences and identify apps with similar injected code.

Compared to earlier methods, AnDarwin significantly improves its scalability by avoiding pairwise comparisons between apps. AnDarwin achieves this through two clustering stages: LSH groups semantic vectors into features, and MinHash detects apps with similar feature sets (complete apps) and frequently occurring features. To test AnDarwin, Crussell et al. [2]evaluated it using 265,359 apps obtained from 17 markets.

AnDarwin identified at least 4,295 cloned apps and an additional 36,106 renamed apps. From the clusters discovered by AnDarwin, 88 new variants of malware and 169 new malware instances were found. Crussell et al. also proposed a post-processing method for clustering to identify apps with injected similar code. AnDarwin’s false positive rate is very low, with a mere 3.72$$\%$$ for complete app similarity detection.

5.3.1.3 Static Detection Based on Static Data Streams

The Android operating system dominates the smartphone market, leading to a rapid increase in the number of Android apps to meet market demands. These apps handle large amounts of sensitive data, which can lead to various issues, including data leaks and privacy violations. For instance, apps may misuse sensitive data stored on Android devices, compromising user privacy. Therefore, protecting user privacy and preventing sensitive data leaks are crucial. Static data-flow analysis methods are employed to analyze Android apps and discover security and privacy issues. However, due to specific challenges posed by Android apps, such as Inter-Component Communication (ICC), reflection, and implicit flows, these methods often generate false positives.

Researchers have proposed the DroidRista [6] approach for static data-flow analysis of Android apps to detect sensitive data leaks. DroidRista analyzes ICC, reflection, and implicit flows in Android apps. To evaluate its performance, DroidRista was tested on three datasets, demonstrating better performance in detecting data leaks compared to existing static data-flow analysis methods.

Static data-flow analysis is one of the primary mechanisms used for analyzing Android apps. One key feature of this analysis is its ability to detect vulnerabilities, errors, and defects in the app before it becomes available to users. It achieves this by collecting information about the data flow at each app point in the analyzed program, revealing valuable insights about the app without executing it. The mechanism of static data-flow analysis for Android apps can be summarized as follows: first, it identifies sensitive data, also known as sources, which are methods used to retrieve data, such as methods that return GPS coordinates. Then, it traces these sensitive data through static code analysis. If the sensitive data flows outside the app (e.g., sent via SMS), the data-flow analysis identifies this as a sensitive data leak and reports the path from the source to the sink as a data leak path.

Static data-flow analysis methods are classified based on their sensitivity to the following attributes:

1.

Context-sensitivity: Context-sensitive analysis considers the calling context when analyzing method call targets and calculates independent information for different calls to the same procedure.

2.

Object-sensitivity: Object-sensitive analysis is a form of context-sensitive analysis that considers the calling objects, identifying method calls performed on different objects.

3.

Field-sensitivity: Field-sensitive analysis independently handles all fields of the underlying objects, while field-insensitive analysis aggregates all fields with their corresponding objects.

4.

Flow-sensitivity: Flow-sensitive analysis considers the order of statements. For example, for a sequence of statements a = 2; b = a+1; a = 3, flow-sensitive analysis would yield b = 2. However, flow-insensitive analysis would give b = 3 or b = 4.

5.

Path-sensitivity is a characteristic of static data-flow analysis that considers the execution paths in the program, particularly with respect to conditional statements, and distinguishes information from different paths. For example, if a branch contains the condition “a > 0,” the analysis will take into account both “a > 0” on the true branch and “a <= 0” on the false branch.

Android apps are composed of components, which are the basic building blocks of the app and provide its functionality. There are four types of components: (1) Activity, which presents an interactive window to allow users to perform specific tasks. (2) Service, which performs long-running tasks in the background. (3) Broadcast Receiver, which responds to system notifications. (4) Content Provider, which controls access to a set of data.

Android components can communicate with each other, which is known as inter-component communication (ICC). Certain Android methods are used to trigger ICC, and these methods are referred to as ICC methods. They use a special parameter called the Intent object, which holds information about the sender component’s interaction with the receiver component. All ICC methods require at least one Intent as a parameter.

An Intent can be defined as an asynchronous message that connects app components at runtime. The Intent object carries information that Android uses to determine which component to start and also includes additional information for the target component to perform its operation. Basic information included in an Intent consists of: the component name (the class name of the target component), data (the data to be processed), action (the action to be performed), category (the type of the target component), extras (additional information required for the operation), and flags (metadata of the Intent).

Intents can be explicit or implicit. In explicit intents, the target component is explicitly defined. In implicit intents, the target component is implicitly specified by identifying specific metadata, such as action, data fields, or categories.

To receive implicit intents, a component must declare an intent filter in the AndroidManifest.xml file. The intent filter defines the types of intents that the component can receive. Specifically, the intent filter includes the types of actions the component can perform or the types of data the component can handle.

In some modern programming languages, reflection is a feature that allows code to inspect itself or other code at runtime and make modifications based on the inspection results. In Java, reflection is used to examine and modify classes and methods at runtime. Android inherits the Java reflection API, which is available within the Android SDK for developers to use. Reflection can be utilized in Android apps to achieve the following functionalities:

1.

Maintain backward compatibility: Android apps use reflection to ensure that new improvements in the app can still function on older versions of Android.

2.

Provide generic support: Developers can use reflection to perform certain generic tasks.

3.

Enhance app security: Developers can separate the essential functionality of the app into independent libraries. When specific conditions are met at runtime, reflection can be used to dynamically load these libraries.

4.

Access inaccessible APIs: Some APIs in Android, such as hidden and internal APIs, are not accessible to third-party Android apps and cannot be called through regular Java method invocations in the developer’s SDK. However, using reflection, developers can leverage these hidden and internal APIs, allowing third-party apps to access these APIs and function properly on Android devices.

In Android apps, data flow can be explicit or implicit. Explicit data flow arises due to data dependencies, while implicit data flow arises due to control dependencies. Put simply, a program with the pattern a1 = a2 exhibits explicit data flow, indicating a data leak. Here, sensitive data is stored in variable y and directly passed to variable a1, whose value can be inferred from external sources. Typically, tracking implicit data flow is more challenging than explicit data flow. A typical example of implicit data flow is: if (y == 0) a1 = 0; else if (y > 0) a1 = 1; else a1 = 1. Even though there is no direct data flow between variables a1 and a2, monitoring a1 can still reveal some information about a2.

DroidRista utilizes the original APK file as input and employs the DroidRA tool to instrument the APK, generating an enhanced version of the APK. Then, DroidRista extracts ICC links from the Android app using the IC3-Modified tool (a modified version of the IC3 tool). Next, DroidRista uses a sub-module of Soot called Dexpler to convert the enhanced APK’s Dalvik bytecode into Soot’s internal representation, Jimple. Based on the extracted ICC links, DroidRista modifies the Jimple code to immediately connect the components for data-flow analysis. Finally, the modified Jimple code is passed to a highly precise component-wise analysis tool called FlowDroid to generate the complete control flow graph of the entire app. This enables extremely accurate data-flow analysis, listing all potential data-flow paths.

Sensitive data leakage poses a significant security threat to Android apps. Researchers have discussed using static data-flow analysis to detect data leakage issues in Android apps. Many apps require necessary permissions to access sensitive data stored on Android devices, such as phone numbers and contact lists. Therefore, users must ensure that apps only use this sensitive data when needed and do not leak the data to unauthorized parties. Researchers propose DroidRista, an efficient and accurate tool for analyzing Android apps to detect sensitive data leakage. Unlike previous methods, DroidRista addresses three challenges faced in static data-flow analysis in Android apps: ICC (Inter-Component Communication), reflection, and implicit flows. Experimental results show that DroidRista effectively addresses these specific security issues and performs excellently among static data-flow analysis tools for Android apps.

5.3.1.4 Static Detection Based on Grayscale Images

Researchers have proposed a framework [7] based on visualization and machine learning for classifying Android malware. In the first stage of the experiment, Android malware apps from the DREBIN dataset are converted into grayscale images. This detection method initially divides Android malware into fifteen different image segments and uses handcrafted features related to Android malware images to identify malicious files. Handcrafted features are extracted from image segments using algorithms based on Gray-Level Co-occurrence Matrix (GLCM), Global Image Descriptor (GIST), and Local Binary Patterns (LBP). Further, machine learning algorithms such as K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forest are used to classify the extracted features. In the second stage of the experiment, handcrafted features are combined with Convolutional Neural Network (CNN) features, forming a feature fusion strategy. The results obtained from the feature fusion strategy are then compared with the results from the handcrafted features. Experimental results demonstrate that the Feature Fusion-SVM model is most suitable for the identification and classification of malware images using certificates and Android manifests (CR + AM), achieving an accuracy rate of 93.24$$\%$$.

Researchers conducted experimental evaluations on the DREBIN dataset, which has been widely used by many researchers for studying Android malware. Malware authors can generate many variants, and any type of malware can be viewed as an image that captures even subtle changes. Images have the ability to preserve the original semantics of the code. All files and folders included in the APK package are bound together to develop the app. The literature survey [11] shows that there are four main resources available for identifying malicious behavior in malware. These main resources include the classes.dex, resource, manifest, and certificate files. Malware developers are likely to utilize these files to store malicious behavior. Below is an explanation of the files in the APK.

(a)

Meta-Inf/: Contains signature files such as CERT.SF and CERT.RSA. It also includes the manifest file MANIFEST.MF.

(b)

Assets/: The app uses the AssetManager object to retrieve app resources in the assets folder.

(c)

Res/: This folder contains descriptions of resources. These resources are not compiled into the resources.arsc file.

(d)

Lib/: The software layer of the processor is associated with specific types of compiled code stored in this folder.

(e)

Resources.arsc: The compiled APK resources are included in this file. It contains the paths to strings, styles, and image/layout files. The data is processed only in XML format.

(f)

Classes.dex: Java code is compiled into class files. These class files are merged into a single dex file using some standard dex tools. Classes.dex contains Dalvik bytecode, which is executed by the Dalvik virtual machine. Any changes to the dex file will affect the APK.

(g)

AndroidManifest.xml: It contains the permissions required by the app, hardware or software components, and links to API libraries. It also shows the SDK version.

The main four files, such as classes.dex, resources, manifest, and certificate files, make up a stable APK structure. The binary bits of the malware are paired into 8-bit vectors and converted to grayscale images in this way. Several key steps are involved in converting any malware binary sample to a grayscale image. The entire malware substring can be seen as a combination of several substrings. Each substring in the binary code has a length of 8 bits, called a pixel. The 8-bit length bitstream can be further converted into a decimal number representing the range from 0 to 255. After calculating the unsigned decimal number, the malware code matrix needs to be generated. All executable substrings of the malware are further divided into one-dimensional vectors of decimal numbers. The one-dimensional vector space can be seen as a linear vector space. It is further processed into a two-dimensional matrix of a specific width. In the study, the DREBIN dataset is used for malware classification. Twenty malware families’ malicious executable files are converted into grayscale images following the steps mentioned above. These grayscale images involve different regions of the APK. Researchers used fifteen different combinations of Android file structures to create these images. The mentioned malware families’ variants have dissimilar textures. These images have different grayscale image textures when generated using different file structure combinations. The texture varies with the content of the malware APK. The size of the generated malware images is different. The height of the image is adjusted based on the file size of the malware sample.

Texture in images is a function of spatial variations in pixel intensities, revealing how pixel values change within a region. It ultimately defines the visual interpretation of an image. In the era of digital image analysis, image texture has been used for various purposes, such as image segmentation, image classification, texture synthesis, and shape recognition using textures. Texture involves the spatial distribution of grayscale levels. It has multiple apps across interdisciplinary fields, including the food processing industry, biometric analysis (fingerprint, iris, or retina matching), medical image analysis, remote sensing data analysis (geographical information systems), and network security. Texture features are computed using statistical methods, including GIST, Gray-Level Co-occurrence Matrix (GLCM) features, and Local Binary Pattern (LBP) features, among others. The Gray-Level Co-occurrence Matrix (GLCM) is one of the most commonly used texture features and finds extensive app in content-based image retrieval, medical image classification, and object recognition. In this method, texture information is extracted based on the spatial relationship between pixels. These spatial relationships between image pixels are defined by distance and direction.

Firstly, the GLCM matrix is computed, which estimates the probability density function of grayscale pairs with specific spatial relationships in the image. The most common distance selection is a distance of 1 in four directions (0 degrees, 45 degrees, 90 degrees, and 135 degrees). Subsequently, multiple statistical measures are computed from this matrix to describe the image’s texture. In this work, only 19 statistical measures are used to represent the texture of malicious software images. These features include contrast, correlation, energy, entropy, uniformity, square sum, total mean value, total variance, total entropy, difference variance, difference entropy, and correlation information measures. These features are measured for each distance and direction combination, resulting in a total of 76 features. LBP texture descriptors are computed on grayscale images of malicious software. In a small block/matrix of the image, neighboring pixels surround the center pixel. If the value of a neighboring pixel is greater than the center pixel’s value, it is replaced with 1; otherwise, it is replaced with 0. For example, considering a $$3\times 3$$ matrix, there are 8 neighbors around the center pixel, resulting in an 8-bit sequence. For each 8-bit sequence, there are 8 such rotations, and each rotation has an associated integer representation. LBP is typically defined as the integer value of the minimum among these rotations. The parameter value of the radius is usually set to 8. The GIST algorithm is known for its good accuracy in computer vision tasks. The GIST algorithm uses 8 orientation parameters at each scale and operates on 4 different blocks. It convolves the image with 32 Gabor filters, with 8 filters at each scale and orientation, to generate 32 feature maps of the same size as the input image. Each feature map is divided into 16 regions ($$4\times 4$$ grid), and the average value of the features within each region is computed. Then, the 16 averaged values of all 32 feature maps are concatenated to obtain a $$16\times 32=512$$-dimensional GIST descriptor.

Feature fusion has been widely adopted in computer vision research related to detection and classification tasks. Researchers used default GIST descriptor values, generating a total of 512 features. For GLCM and LBP, 76 and 58 features were used, respectively. The CNN architecture generated a 4096-dimensional feature vector. In this work, researchers employed feature fusion by concatenating the features column-wise.

In a series of experiments, various handcrafted features such as GLCM, GIST, and LBP were used for the analysis and classification of Android malicious software images. The results revealed that LBP features did not provide rich information for malware classification, whereas GIST features combined with SVM, KNN, and RF classifiers exhibited good classification accuracy. The normalization of the dataset using min-max normalization had a significant impact on the proposed method. Among the handcrafted features, the GLCM-SVM model achieved the highest classification accuracy of 92.20$$\%$$ on AM malicious software images, followed closely by the GLCM-RF and GIST-SVM models, which reached accuracies of 91.81$$\%$$ and 91.35$$\%$$ on CR + AM malicious software images, respectively. Additionally, a feature fusion strategy was adopted to combine CNN and handcrafted features for the classification of Android malicious software images. The classification results of feature fusion outperformed those using only handcrafted features. When using the Feature Fusion-SVM classifier for the classification of CR + AM malicious software images, the highest accuracy reached 93.24$$\%$$. The feature fusion method not only improved the classification performance of all classifiers but also reduced the workload of inspecting the entire APK structure.

The proposed feature fusion-based visualization technique enables efficient classification with fewer resources and time.

5.3.1.5 Static Detection Based on Permission Pairs

Due to inherent vulnerabilities in Android smartphones, apps can access internal resources when users grant permissions explicitly or implicitly, making Android smartphones susceptible to the spread of malicious software. As a result, researchers have been striving to identify significant permissions that contribute to malicious software detection. While many of these permissions apply to both malicious and legitimate apps, they exhibit different patterns and impact the attacks differently. Therefore, identifying potentially dangerous permission combinations is crucial. Existing research has not utilized permission pairs for detecting malicious software.

In response, researchers have introduced an innovative detection model called PermPair [8]. The evaluation results demonstrate that the proposed approach achieves 95.44$$\%$$ accuracy in detecting malicious samples, outperforming other similar methods and mainstream mobile anti-malware apps. Furthermore, an efficient edge elimination algorithm has been proposed, removing 7$$\%$$ of unnecessary edges from the malicious software graph and 41$$\%$$ from the benign graph. This reduction not only minimizes space usage but also reduces the detection time by 28$$\%$$.

Analyzing permission combinations involving more than two permissions can be challenging due to the potential explosion of permission patterns. If an app contains N permissions and we want to analyze permission patterns in R permission groups, the total number of permission patterns would be N CR, where C represents the combination. Increasing the value of R would result in a larger number of permission patterns or increased complexity in representation. Therefore, the proposed method focuses on analyzing permission pairs.

To conduct a comprehensive analysis of permission pairs, researchers considered malicious samples from three different sources: Genome, Drebin, and Koodous. Additionally, they downloaded normal Android apps from the Google Play Store. The model consists of three stages. The first stage is the graph construction stage, where permission pairs are extracted from each app to form a graph. Four different graphs are constructed in this stage: Genome graph (GG), Drebin graph (GD), Koodous graph (GK), and normal graph (GN). Typically, for each new dataset used in analysis and detection, a graph needs to be constructed. The next stage is the graph merging stage, which involves combining all malicious graphs into a single malicious graph (GM). As a result, the final model consists of two graphs: a normal graph (GN) and a malicious graph (GM). A detection algorithm is then proposed in the subsequent stage to differentiate between normal and malicious apps. Finally, the model optimizes by eliminating irrelevant edges from GM and GN that do not affect the detection results.

1. Graph Construction Algorithm: The proposed model utilizes graphs to represent the relationships between individual permissions in each app. The algorithm constructs a graph G(V, E), where each unique permission is represented as a vertex V, and the permission pairs are connected by weighted edges E. The weight of each edge increases with the occurrence frequency of the same permission pair in different apps. Only apps with a single or no permissions would not have any impact on the analysis or detection.

The algorithm extracts permissions from each app and generates all possible permission pairs. If an app has N unique permissions, the algorithm creates N vertices, with each permission representing one vertex. The number of possible permission pairs added to the graph among all permissions is N $$\times $$ (N$$-$$1)/2. After processing all apps in the dataset, the weight of each edge is divided by the total number of apps in that dataset to normalize the weights of permission pairs among different graphs. This stage describes how permission pairs extracted from apps are represented.

After generating individual Genome (GG), Drebin (GD), and Koodous (GK) graphs, they are merged into a common malicious graph (GM) to represent common malicious patterns.

2. Graph Merging Algorithm: The graph merging algorithm combines three malicious graphs (GG, GD, GK) into a single malicious graph (GM) by merging the edges. There are two types of edges: common edges and disjoint edges. All disjoint edges and their weights are directly inserted into GM. The challenge lies in determining the weight of common edges. Consider a common edge e, with weights ew1, ew2, and ew3 in GG, GD, and GK, respectively. There are different ways to determine the weight of edge e in the final GM, such as taking their average, minimum, or maximum.

Researchers propose a novel approach to detect malicious apps using permission pairs extracted from the manifest file. They construct graphs to analyze permission pairs in normal and malicious samples and assign edge weights based on the occurrence frequency of permission pairs in the samples. The researchers analyze three malicious software datasets, namely Genome, Drebin, and Koodous, containing newer samples from 2014 to 2018 and older samples from 2010 to 2014. Initially, three different graphs are constructed, one for each dataset, and certain differences in permission pairs between newer and older samples are observed. The researchers use a weighted summation method to merge the different malicious software graphs into a single graph. They further perform edge elimination to remove unnecessary edges. The results demonstrate that the proposed method outperforms eleven popular mobile anti-malware apps.

5.3.1.6 Static Detection Based on Static Sensitive Subgraphs

As the most popular mobile platform, Android has become a primary target for malicious software, making effective defenses against them urgently needed. Recently, machine learning-based techniques have emerged as a promising solution for malware detection, heavily relying on distinguishing features to differentiate between malicious software and benign apps. Although there are hundreds of features available for machine learning-based malware detectors, attackers can also leverage knowledge about these features to develop variant malware that evades detection. Therefore, an important role of the Android security community is to continuously propose new features capable of characterizing malicious behavior.

Ou et al. [9] introduced a novel Android malware detection feature based on static sensitive subgraphs, known as S3Feature. Firstly, to represent Android apps with advanced features, researchers extended the sensitive function call graph (SFCG) by marking sensitive nodes on the function call graph (FCG). Sensitive nodes were identified by evaluating their malicious scores. Secondly, a large number of sensitive subgraphs (SSGs) and their neighboring subgraphs (NSGs) were mined from the SFCG to characterize suspicious behavior in apps. Finally, after removing duplicate or isomorphic subgraphs, the remaining SSGs and NSGs were encoded into feature vectors to represent each app. For malware detection, S3Feature achieved an F1 score of 97.04$$\%$$, outperforming other extensively studied features. Combining S3Feature with other features led to an F1 score of 97.71$$\%$$, indicating that S3Feature is a promising feature for improving the performance of malware detection methods or tools.

The entire feature extraction framework of S3Feature consists of three components: Sensitive Function Call Graph (SFCG) generation, subgraph mining, and feature vector generation. Given an Android app, S3Feature uses SFCG to represent the app’s global behavior using graphs. Since access to the user’s private data and system resources is always implemented through system APIs rather than developer-defined functions, malicious behavior should involve system APIs. Therefore, the SFCG generation component aims to extend the FCG to the corresponding SFCG by explicitly marking sensitive system APIs on the FCG to better represent malicious behavior. Unlike methods that rely on heuristic approaches to identify malicious functions, researchers analyzed a large number of benign apps and malware, and transformed each app into the corresponding SFCG. Subsequently, the subgraph mining component is used to discover a large number of sensitive subgraphs (SSGs) from the corpus of SFCGs, where each SSG represents a local suspicious behavior pattern and may provide valuable insights into API-level malicious behavior. Then, due to the existence of duplicate or isomorphic subgraphs, the subgraph mining component applies graph hashing algorithms to remove them. By doing so, a refined set of SSGs is obtained, representing all suspicious behavior patterns that may occur in an app. The feature vector generation component is responsible for converting the SFCG into a binary feature vector, where each element of the resulting feature vector corresponds to an SSG. The dimension of the feature vector equals the number of discovered SSGs. When an SSG represented by a feature occurs in the app, the value of the feature is set to 1; otherwise, it is set to 0.

In the context of static detection based on static sensitive subgraphs, researchers proposed sensitive subgraphs (SSGs) as features for Android malware detection, representing APKs through feature vectors provided by S3Feature. Through comprehensive experiments, it was demonstrated that the local behaviors captured by SSGs are beneficial for improving the performance of malware detection, highlighting that S3Feature offers a new perspective for feature engineering in the field of malware detection. Additionally, due to its vector-based representation, S3Feature easily integrates with other feature sets.

5.3.1.7 Static Detection Based on Malicious URLs

In recent years, the scale and diversity of malicious software on mobile networks have significantly increased, posing considerable risks to users’ property and personal privacy. As a response, researchers have proposed a malware detection method that utilizes the URLs accessed by apps. They employ a multi-view neural network to create a malware detection model that emphasizes both depth and width. This neural network automatically generates multiple views of the input and assigns soft attention weights to focus on different features of the input. The multiple views retain rich semantic information of the input for classification without the need for complex feature engineering. Comprehensive experiments have been conducted to compare the proposed method with other approaches and validate the effectiveness of the detection model. The experimental results demonstrate that this method achieves robust and timely malware detection. It can effectively detect malware discovered in different months of the same year and even identify potential malicious apps on third-party app markets. Wang et al. [10] also compared the detection results of the proposed method with ten popular antivirus software on real-world apps, and the final results showed that this method ranked second in terms of detection performance.

The researchers presented a malware detection method based on malicious URLs. Each URL is divided into several segments using specific characters, and skip-gram algorithm is used to train embedding vectors for each segment. This vectorization approach addresses the issues of data sparsity and semantic loss caused by conventional encoding methods. The URL vectors are then fed into a multi-view neural network, which automatically creates multiple views and leverages the input data to generate these views. Each view focuses on a specific subset of features from the input data. The multiple views are combined to perform classification on the URL vectors. The network emphasizes both depth and width, allowing it to automatically select features from multiple perspectives.

5.3.2 Detection Method Based on Binding Method

5.3.2.1 Static Detection Combined with Dynamic

One of the challenges faced in static code analysis is the encryption of sensitive strings, such as function names invoked through reflection. App developers can achieve this encryption by writing custom code or using existing obfuscators. However, dynamic code analysis, which involves running the app in an emulator, is not affected by such obfuscation techniques as sensitive data is decrypted by the app’s code at runtime. To leverage this advantage, researchers devised a combination analysis process. The dynamic module extracts decrypted data and passes it to the static module. However, there is a challenge: while static analysis can examine every line of code in the APK, dynamic analysis only perceives the code actually executed. In other words, researchers may have to ensure that the dynamic analysis captures all possible flows that use encrypted data during its execution. Their solution is to make the dynamic module more active by displaying the right behaviors. The static module searches for all calls to decryption code in the app code, which typically appears in the form of static functions (e.g., DashO, KlassMaster). It provides the dynamic module with a list of function calls, including parameter values. The dynamic module executes these function calls and returns the results to the static module, which then uses the decrypted strings to patch the app code. Researchers implemented this concept and tested it on samples obfuscated with DashO. As expected, this approach enabled static analysis to detect new suspicious behaviors in apps that were previously analyzed with limited scope.

The experiments utilized Check Point’s existing app analysis infrastructure, which includes the following core elements:

1.

A sample database containing millions of apps.

2.

Static analysis infrastructure supporting custom analysis algorithms for a large number of samples.

3.

Dynamic analysis infrastructure enabling custom detection for operating systems and analysis apps.

4.

An orchestration infrastructure for collecting, transmitting, and storing data generated by analysis algorithms.

Data Collection

The first step of the research was to collect data. While there were numerous apps, they were not correctly labeled with obfuscation information. To locate the required obfuscated apps for further analysis, the following signatures were used:

1.

Static functions

2.

Return values of type String

3.

Functions that receive 2–3 integers and a String as input parameters

4.

Exception handling inherited from RuntimeException

5.

Functions that only call methods of the String class

DashO Experiment

As DashO’s implementation of string encryption is relatively straightforward, it was chosen for the first experiment. A dataset of 586 samples, each comprising an Android app, whether malicious or benign, containing strings encrypted by DashO, was created. The researchers ran all apps in the dynamic analysis infrastructure, which first utilized the static analysis infrastructure to identify all decryption calls and extract parameter values. After the initial static analysis phase, the dynamic analysis executed all decryption calls. The decryption results were saved in JSON files, which were then passed to the static analysis infrastructure for full static analysis of each sample. The researchers also ran the static analysis infrastructure on the same samples without providing any decrypted string data.

Moses et al. [3] implemented a complete deobfuscation solution and tested it on hundreds of Android apps. The experiment demonstrated that this method could detect new behaviors in apps that were previously analyzed with limited scope. Importantly, the study highlighted the need for a holistic approach to app analysis for modern threats. Relying on a single method is insufficient–effective threat prevention requires interdisciplinary collaboration and a team that adopts a wide range of technologies.

5.3.2.2 Static Detection Combined with Machine Learning

Android is currently one of the most popular and widely used smartphone operating systems. It boasts millions of apps available for download on both official and unofficial app stores. However, some of these apps may be malicious and spread through these app stores, deceiving users into downloading them onto their smartphones. To address this issue, researchers have conducted investigations into Android zombie networks. They employed static analysis techniques to reverse engineer the source code of apps and extract potential features. Subsequently, these features were used to develop machine learning models for detecting such malicious apps. The researchers also proposed a novel set of features related to resource access on target phones. These features were extracted from 1928 Android zombie network apps and 2224 benign apps, and they underwent classification. By employing four common machine learning classifiers for training and testing, the researchers compared the results of different feature sets. They found that features related to URLs played a critical role in detecting Android zombie networks. Overall, the random forest classifier achieved the best detection results based on all feature sets.

Data mining is the process of analyzing data to discover hidden patterns and unexpected relationships. Typically, the dataset used for data mining consists of examples (referred to as instances) and multiple attributes (referred to as features). Supervised machine learning techniques are employed in data mining to predict unseen attribute values (known as class labels) by understanding the relationships and correlations between features and their labels in the training instances, followed by applying the trained model to test data (referred to as classification tasks).

Naive Bayes classifier is a probabilistic classifier based on the Bayes classifier and belongs to the category of probability classifiers. It is simple to use, requires no parameter estimation, and is easy to implement. Naive Bayes classifier is a statistical classifier that assumes that the influence of attribute values on a given class is independent and unrelated to the values of other attributes. On the other hand, MLP (Multi-Layer Perceptron) is a complex modeling technique based on the model of human brain neurons. The basic elements of MLP are perceptrons. Each perceptron receives multiple inputs and passes these input values through a summation function and an activation function to generate the output value of that perceptron. The commonly used activation function in this method is the sigmoid function. MLP is a feed-forward neural network with multiple layers, including an input layer, an output layer, and one or more hidden layers. Perceptrons within the same layer typically have the same activation function. The network learns through the app of learning algorithms. Backpropagation is the most commonly used learning algorithm for this task. Decision tree is generated by an algorithm that segments the dataset into segments. A decision tree consists of nodes and directed edges, forming a tree-like structure similar to a flowchart. The root node is the starting point and has no incoming edges, while internal nodes have one or more outgoing edges. Nodes without outgoing edges are called leaf nodes or decision nodes. Each internal node splits the dataset based on attribute values and assigns each leaf node to a class value. Therefore, each path from the root to a leaf node in the decision tree can be represented as a rule. The C4.5 algorithm is an improvement over the Iterative Dichotomiser 3 (ID3) algorithm. C4.5 has an enhanced tree pruning method to reduce misclassification errors and handles both continuous and categorical attributes. C4.5 calculates the information gain for each attribute using the information gain method, selects the attribute with the highest information gain as the root node of the current subtree, and recursively repeats this process. Random Forest is a powerful ensemble classifier composed of multiple decision trees. Each decision tree is generated using randomly selected attributes and a random training sample set, which gives each decision tree independence and diversity. Random Forest makes the final classification decision through majority voting. It exhibits efficiency in handling large datasets and the ability to avoid overfitting. Moreover, Random Forest can evaluate the generalization capability of the model by using out-of-bag data for validation. Random Forest is a powerful and widely used machine learning algorithm applied in various fields.

To determine the domain of the feature sets that are most relevant to mobile zombie network detection, a special process needs to be defined as the first step in the research methodology. This process helps to understand the problem and identify its components: propagation vectors, C$$\&amp;$$C (Command and Control) channels, topology structure, and techniques used to propagate code to mobile network services (e.g., SMS, MMS), internet access (3/4G, Wi-Fi), Bluetooth, and USB. Additionally, this process is used to identify the control channels and commands, which are the main components of mobile zombie networks. These channels and commands enable the remote propagation of tasks, messages, and payload updates between zombie nodes and the main control, using Bluetooth, SMS, internet, P2P, or any combination thereof.

To create a dataset for training and testing machine learning models, two types of apps were used: Android benign apps and Android zombie network apps. Two main sub-processes were employed to generate forms almost identical to the source files used: the Android app collection sub-process and the APK extraction sub-process. In the Android app collection sub-process, the ISCX Android Zombie Network Dataset and apps downloaded and scanned from the Google Play app store were used. In the APK extraction sub-process, APKtool and a special tool developed as part of this research were selected and used for extraction. To collect data related to Android zombie network apps, the ISCX Android Zombie Network Dataset [12] was used. This dataset contains 1,928 Android zombie network apps of 14 different types. The data collection period spans from 2010, when Android zombie networks first appeared, to 2014. To collect data on Android benign apps, a sub-process was defined to download a large number (over 2,000) of Android apps from the Google Play app store as APK files. These downloaded apps were then scanned using popular malware detection services, such as VirusTotal. To determine the best APK extraction tool, four commonly used tools, including Android SDK Asset Packaging Tool (AAPT), APKtool, Dex2jar with JD-GUI, and Baksmali, were evaluated. APKtool was ultimately chosen to extract the APK files of both Android zombie network and benign apps into source code folders. APKtool was selected because it can be used in conjunction with other scripts and can decode APK resources into close-to-original source code. Additionally, it can correctly decode the AndroidManifest.xml file and disassemble DEX files into Smali extension files, resulting in more organized and readable content. Consequently, a tool was developed using Visual Basic 2017 to call APKtool to extract the target APK files. The concept of static analysis is to study the behavior of the app without executing its code. All the features to be extracted are related to aspects hard-coded in the source code. APKtool is not a static analysis tool; it only allows reverse engineering of apps to obtain their source code. Once the source code is available, static analysis is used to search for specific aspects, patterns, or flaws. Furthermore, a VB.NET tool was developed to scan the AndroidManifest.xml file and Smali source code files in the extracted APK folder and extract 406 features. The features used in the literature were collected, and new features not previously used were added. Therefore, a total of 406 features were used in this study to train the classifiers. These features can be categorized into six groups: requested permission features (138 features), permission protection level features (7 features), software/hardware access features (73 features), software/hardware access category features (21 new features), Uniform Resource Locator (URL) features (8 features), and broadcast receiver features (159 features). Afterward, four well-known machine learning classifiers (Random Forest, Multi-Layer Perceptron, Decision Tree, and Naive Bayes classifier) were used to build detection models using the WEKA open-source Java tool. All models were developed based on seven feature sets, each representing a group of features from the six categories, with one additional set combining all categories.

To assess the performance of the models, three evaluation metrics were used: accuracy, recall, and precision. Accuracy measures the proportion of correctly classified instances among both classes. For the Android zombie network detection problem, the positive class is the zombie network class. Therefore, precision is the ratio of the number of Android apps correctly classified as zombie networks to the total number of Android apps classified as zombie networks. On the other hand, recall is the ratio of the number of Android apps correctly classified as zombie networks to the total number of Android apps that are actually zombie networks. To determine the most influential features that improve the detection results for the Android zombie network detection problem, the importance score of each feature was calculated. The variable importance evaluation function in the CARET package in R was used to estimate the contribution of each feature to the random forest model. For each tree in the model, mean squared error was calculated based on the out-of-bag (OOB) dataset. Then, for each predictive feature, the error was computed again after permutation. The difference in error counts was then averaged over all trees. After calculating the importance score of each feature, the 20 most influential features were listed and visualized using plotting methods.

Finally, to determine the minimum number of features required to achieve the best evaluation results, four models were developed using the RF classifier based on the 5 most influential, 10 most influential, 15 most influential, and 20 most influential features, respectively. Accuracy, precision, and recall were then measured for these models to evaluate their performance.

Alqatawna et al. [4] studied the Android zombie network detection problem based on machine learning and static analysis. Additionally, they developed all necessary and important tools for web scraping the Google Play Store, downloading APK files, and scanning them; building a benign app dataset, and ultimately decompiling APK files. Moreover, a tool was developed to extract 406 features from the extracted APK folder, including all the features used in the literature: requested permission features (138 features), software/hardware access features (73 features), Uniform Resource Locator (URL) features (8 features), and broadcast receiver features (159 features). Furthermore, they proposed two previously unused feature groups: permission protection level features (7 features) and software/hardware access category features (21 features). The predictive ability of six feature groups and the new combination of extracted features was evaluated using random forest, multi-layer perceptron neural network, decision tree, and Naive Bayes classifier. Finally, feature importance analysis was conducted, and the 20 most influential features were listed. The experimental results showed that the random forest classifier achieved the best results across all feature groups.

5.3.2.3 Static Detection Combined with Deep Learning

The widespread adoption of Android platforms in smartphones and other IoT devices has led to a surge in malicious software attacks. Malware poses a serious threat to the security of devices and the services they provide, such as stealing sensitive data stored on mobile devices. Researchers have proposed a stacked ensemble framework called SEDMDroid [5] to identify Android malware. Specifically, to ensure individual diversity, it adopts random feature subspaces and bootstrapping techniques to generate subsets and runs Principal Component Analysis (PCA) on each subset. By preserving all principal components and training each base learner, the Multi-Layer Perceptron (MLP), using the entire dataset, detection accuracy is achieved. Then, Support Vector Machine (SVM) acts as the fusion classifier, learning implicit complementary information from the outputs of ensemble members and generating the final prediction. The researchers demonstrated the effectiveness of SEDMDroid through experimental results on two independent datasets collected using static analysis methods. The first dataset extracted widely used features in Android malware, such as permissions, sensitive APIs, and monitored system events, achieving 89.07$$\%$$ accuracy in multi-layer static features. The second dataset was a large publicly available dataset, extracting sensitive data-flow information as features, with an average accuracy of 94.92$$\%$$. Promising experimental results indicate that the proposed method is an effective approach to identify Android malware.

The system overview of SEDMDroid can be decomposed into three main components:

1.

Feature Extractor: It extracts permissions, permission rates, monitored system events, and sensitive APIs as raw features through static analysis. It should be noted that these features can be extended or replaced by other static analysis features, such as data-flow information.

2.

Stacked Ensemble Deep Learner: The goal of this component is to design and build a stacked ensemble approach that uses the features extracted from labeled apps in the previous component to learn ensemble classifiers and fusion classifiers.

3.

Malware Detection: For each unknown Android app, its raw features are constructed first. Based on these raw features and the learned stacked ensemble deep learning classifier and fusion classifier, it will be determined whether the app is benign or malicious.

Ensemble learning has gained increasing attention in various app domains as it can combine multiple base learners to achieve more robust, stable, and accurate results. When given a set of base learners, stacked ensemble learning combines the predictions from base learners using a fusion model. Over the past few decades, research on stacked ensemble learning has mainly focused on two issues: how to generate a new set of classifiers and how to search for the best fusion of base classifiers. As a result, various ensemble learning methods have been proposed. Zhang et al. [13] proposed an ensemble classifier based on Deep Belief Network (DBN), called MODBNE, for estimating the remaining lifetime of a system. Nanni et al. [14] proposed an ensemble classifier by weighting the features in a random subspace. Sun et al. [15] proposed a stacked ensemble classifier for predicting the annual river ice breakup date, where they used Bayesian Regulated backpropagation artificial neural network (BRANN) as the base classifier and Adaptive Neuro Fuzzy Inference System (ANFIS) as the fusion model.

A key task in stacked ensemble learning is obtaining base learners that individually perform better than random guessing on the original training data and have small correlations in training errors with each other. Then, these base classifiers’ predictions are integrated using a combination strategy to maximize accuracy and diversity. However, the challenge lies in the conflict between accuracy and diversity in ensemble learning. To enhance individual diversity, accuracy may be sacrificed, and vice versa. Furthermore, most traditional fusion methods choose simple combination strategies, such as majority voting, weighted voting, or fuzzy integrals. These methods seldom consider exploring and utilizing complementary information from a set of training base classifiers with different performances. To address the first challenge, SEDMDroid utilizes PCA, random feature subspaces, bootstrapping, and deep learning construction to ensure the diversity and accuracy of its individual members. In the fusion stage, SEDMDroid utilizes advanced machine learning algorithm SVM to learn implicit complementary information from the output of well-trained base classifiers to optimize the stacked ensemble learning method.

Accuracy and diversity are key features for achieving excellent generalization performance in ensemble models. To ensure accuracy, researchers selected the commonly used Multi-layer Perceptron (MLP) as the base learner in the deep learning framework. MLP is a flexible computational framework, making it well-suited as a base learner in ensemble learning. It is a feed-forward neural network composed of multiple layers of neurons, and the number of layers can be adjusted as needed.

Researchers proposed a deep learning-based stacked ensemble method for detecting Android malware. It consists of a two-layer structure, including the integration of base learners (MLP) and the fusion classifier (SVM). Inspired by the success of the Rotation Forest method, SEDMDroid aims to construct a set of base learners with accuracy and diversity. Specifically, to ensure individual diversity, it randomly splits feature subspaces and generates a set of bootstrapped subsets from the training set. Principal Component Analysis (PCA) is applied separately on each subset to obtain the principal component coefficients while preserving individual diversity. The accuracy is verified by retaining all principal components and training each base learner on the entire dataset. However, a challenge lies in the fact that some base learners’ outputs may be erroneous, or certain base learners may be more correct or reliable than others. Support Vector Machine (SVM) is used as a maximum-margin classifier. Particularly, SVM is trained on data with irregular or unknown distributions due to its ability to generate non-linear decision planes and filter out irrelevant features. Therefore, SVM is incorporated into the fusion strategy to optimize the use of base learners and their combinations.

In response to the explosive growth of Android malware, researchers proposed a static malware detection framework called SEDMDroid. The framework adopts a two-layer architecture, integrating the base learners MLP and fusing their outputs through SVM. In the first stage, diversity in the training subsets is ensured through dual perturbations in the feature space and sample space, followed by separate PCA runs on these subsets. For each branch, all principal components obtained through PCA are retained, and the entire training dataset is transformed into a fresh collection, on which MLP is executed to ensure the accuracy of the base learners. In the second stage, also known as the fusion stage, the learning process extracts implicit complementary information from the trained base learners’ outputs to optimize the classification efficiency. Extensive experiments were conducted on two different datasets, validating the effectiveness of the proposed method. The experimental results demonstrate that SEDMDroid is an effective malware detection approach.

5.4 Chapter Summary

This chapter introduces the importance and apps of static obfuscation techniques in Android apps. Static obfuscation is a crucial technology for securing apps, as it modifies code, resources, manifest files, and execution logic to increase complexity and ambiguity, thereby preventing reverse engineering. Common static obfuscation techniques include code obfuscation, resource obfuscation, manifest file obfuscation, and control flow obfuscation.

Furthermore, the chapter presents several popular APK static obfuscation tools, including Obfuscapk, ProGuard, DexGuard, Allatori, DashO, Bangcle, and Arxan. These tools can obfuscate Android apps and enhance their security. Each tool has unique features and characteristics, allowing developers to choose the most suitable one for obfuscating their apps based on their specific requirements. Lastly, the chapter discusses several static detection tools that have achieved outstanding results in relevant research. These tools implement static detection of Android apps through different detection features and methods, categorized into new feature-based detection and combination-based detection. Researchers can select the appropriate static detection method according to their needs to enhance the security of their apps.

References

1.

Nguyen T , Mcdonald J , Glisson W ,et al. Detecting Repackaged Android apps Using Perceptual Hashing[C]//Hawaii International Conference on System Sciences.2020.https://​doi.​org/​10.​24251/​HICSS.​2020.​813.

2.

Crussell J , Gibler C , Chen H .AnDarwin: Scalable Detection of Android app Clones Based on Semantics[J].IEEE Educational Activities Department, 2015.https://​doi.​org/​10.​1109/​TMC.​2014.​2381212.

3.

Moses Y, Mordekhay Y. Android app deobfuscation using static-dynamic cooperation[J]. VB2018, 2018, 625.

4.

Alqatawna J, Ala’M A Z, Hassonah M A, et al. Android botnet detection using machine learning models based on a comprehensive static analysis approach[J]. Journal of Information Security and apps, 2021, 58: 102735.Crossref

5.

Zhu H, Li Y, Li R, et al. SEDMDroid: An enhanced stacking ensemble framework for Android malware detection[J]. IEEE Transactions on Network Science and Engineering, 2020, 8(2): 984–994.Crossref

6.

Alzaidi A, Alshehri S, Buhari S M. DroidRista: a highly precise static data flow analysis framework for android apps[J]. International Journal of Information Security, 2020, 19(5): 523–536.Crossref

7.

Singh J, Thakur D, Gera T, et al. Classification and analysis of android malware images using feature fusion technique[J]. IEEE Access, 2021, 9: 90102-90117.Crossref

8.

Arora A, Peddoju S K, Conti M. Permpair: Android malware detection using permission pairs[J]. IEEE Transactions on Information Forensics and Security, 2019, 15: 1968-1982.Crossref

9.

Ou F, Xu J. S3Feature: A static sensitive subgraph-based feature for android malware detection[J]. Computers & Security, 2022, 112: 102513.Crossref

10.

Wang S, Chen Z, Yan Q, et al. Deep and broad URL feature mining for android malware detection[J]. Information Sciences, 2020, 513: 600–613.Crossref

11.

A. Sadeghi, H. Bagheri, J. Garcia, and S. Malek, “A taxonomy and qualitative comparison of program analysis techniques for security assessment of Android software,” IEEE Trans. Softw. Eng., vol. 43, no. 6, pp. 492–530, Jun. 2017, https://​doi.​org/​10.​1109/​TSE.​2016.​2615307.Crossref

12.

Kadir AFA, Stakhanova N, Ghorbani AA. Android botnets: What URLs are telling us. In: International conference on network and system security. Springer; 2015, p. 78–91.

13.

C. Zhang, P. Lim, A. K. Qin, and K. C. Tan, “Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics,” IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 10, pp. 2306–2318, Oct. 2017.Crossref

14.

L. Nanni and A. Lumini, “Evolved feature weighting for random subspace classifier,” IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 363–366, Feb. 2008.Crossref

15.

S. Wei and B. Trevor, “A stacking ensemble learning framework for annual river ice breakup dates,” J. Hydrol., 2018.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_6

6. Dynamic Adversarial Method in Android Malware

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

With the development and popularization of Android malware detection technology, there are sundry Android malware detection and defense methods in the industry and academia. In order to evade these detections, malware developers have also begun to adopt countermeasures to remain in user terminals to achieve malicious purposes. The previous chapter has introduced the common methods and detection methods of static confrontation technology. However, in reality, deploying such detection methods and confrontation methods requires a lot of cost. The best way to combat static confrontation is dynamic analysis. Because security personnel only need to run the sample to collect the observed behaviors during dynamic analysis and do not need to perform special treatment on the obfuscation, encryption, hardening, dynamic code loading, and other confrontation methods used by malicious samples. Some Android malicious samples that have been discovered so far use dynamic confrontation technology, also known as dynamic evasion technology. For example, Android.Hehe, obad, Pincer family, etc. can evasion dynamic analysis by detecting their own environment. If they detect that they are in a sandbox or dynamic debugging, these malicious samples will not execute malicious behaviors and appear benign. Once these malicious samples use dynamic evasion technology to bypass the defense mechanism we deployed, they can remain in the user terminal stably and obtain income through continuous evil, which will seriously endanger the privacy and property rights of users. Therefore, the research on dynamic evasion technology is very necessary. In this chapter, we will introduce Android dynamic evasion technology and its defense methods. According to the evasion object, Android dynamic evasion technology is mainly divided into two categories: dynamic sandbox evasion and dynamic debugging evasion (Afianian et al., ACM Comput Surv 52(6):1–28, 2019). We will introduce the specific methods of these two categories in Sects. 6.1 and 6.2, respectively. At last, we will introduce the defense methods and related research of dynamic evasion in Sects. 6.3.

6.1 Automatic Dynamic Analysis Evasion

6.1.1 Detection Dependent

The core principle of the detection-dependent method is to judge its own environment by finding the difference between the sandbox and the device [1]. According to malware detection targets, such methods can be divided into sandbox fingerprint-based, artifact value distribution-based, wear-and-tear feature-based, and hardware architecture-based differences. The method based on the sandbox fingerprint detects whether it is in the sandbox environment by detecting the salient features of the observable artifacts of the sandbox.

For example, the mobile phone number of many sandboxes is directly set to: 155552155xx, or its build.HARDWARE property is directly set to goldfish [2]. At present, many detection methods have been proposed. For example, the detection method proposed by Vidas and others uses the differences in the hardware attribute as one of the detection [2]; Yokoyama et al. [3] through the inherent features of collecting sandboxes to form a sandbox fingerprint, thereby training the classifiers who can distinguish the sandbox equipment and the real equipment. The framework used to detect the differences between the sandbox and the real equipment workpiece and automatically generate the heuristic rules. It also uses the principle of detecting real equipment and sandbox fingerprint differences. The method based on the distribution of workpiece values judges its own environment by comparing the distribution of the workpiece value of the real device with that of the sandbox device. For example, the trajectory generated by a person touching the screen is different from the trajectory generated by the simulator; the method proposed by Diao et al [4]. The real device sensor values are not the same in such a way to detect. The detection method based on wear characteristics is more difficult to defend than the first two methods. The detection method based on wear characteristics is to judge whether the environment belongs to a sandbox or a real device by detecting the usage traces generated by the device. Miramirkhani et al. [5] judged whether the target environment is a real device or an emulator based on the use and loss characteristics of the system, disk, network, registry, and browser: such as cache, logs, and number of files. Meanwhile, Kondracki et al. [6] proved the incompleteness of existing sandboxes, that is, some detectable features, especially wear features, can always be used for detection. Finally, there are some methods based on the detection of hardware architecture differences. Since most of the existing sandbox CPUs are based on QEMU simulation, the sandbox environment can be judged by detecting QEMU. Sahin et al. [7] judge the target environment by detecting the difference between QEMU and the real ARM CPU at the instruction level. Petsas et al. [8] also used the difference between QEMU and ARM CPU when dealing with cache inconsistencies to determine the target environment. Wang et al. [9] proposed a static method to analyse the fingerprint and trigger-based evasion malware.

Sandbox evasions that rely on detection is subdivided into three categories: fingerprint-based, reverse Turing test-based, and target-based. The rationale for detection-dependent sandbox evasions is detailed below by category.

6.1.1.1 Fingerprint

Fingerprint recognition is a strategy adopted by malware to detect the existence of the sandbox by looking for an environmental workpiece or logo that may reveal the signs of virtual/simulated machines. The range of these signs can be from the device driver on the disk, the exposed files and registry keys, to the difference between the emulation/virtual processor, or the characteristics or simulation characteristics of the VM dispersed at different levels. We divide these functions into different categories:

Hardware

Malware may look for devices and drivers to identify its environment. In a real machine, the target device can usually be easily detected with some obvious characteristics, such as Reptile malware uses such a technique. In addition, the virtual machine uses a specific driver to interact normally with the host operating system. These drivers are other characteristics of malware analyzing the environment. For example, there are such flags in the path: C:$$\backslash $$Windows$$\backslash $$System32$$\backslash $$Drivers, which may expose VMWare, VirtualBox, etc. (such as Vmmouse, m3dgl.dll, VMToolsHook.dll, etc.).

Execution Environment

When malware is placed in a sandbox, it operates in a slightly different environment than in its natural state. One way this is noticeable is that the memory values in the kernel space differ between the sandbox and the native system. Additionally, artifacts at this level can exist either in memory or during execution, making them detectable by certain types of malware like Agobot and Storm Trojans. In VMWare, a channel called “ComChannel” is established between the host and guest OS to enable inspection and control of memory artifacts. Similarly, Virtual PC hooking is also implemented.

App

When an app is used in an analysis environment, it can be exposed to the analysis software due to installation and execution artifacts that may be detected by malware such as Rbot or Phatbot. Even if an app is not executed, traces of analysis tools can still be found on disks, registry keys, etc. Malware can easily detect these traces, especially if they have obvious filenames or locations, such as SYSTEM$$\backslash $$CurrentControlSet$$\backslash $$Control$$\backslash $$VirtualDeviceDrivers. Malware can easily identify virtual machines from real environments by querying processes such as VMtools.exe, Vmwareuser.exe, or vboxservice.exe.

Behavioral

Imperfect virtualization/emulation or inherent characteristics of these environments can cause leaks that are difficult for anti-malware vendors to hide. The difference in behavior between simulated and physical CPUs is particularly problematic. Apps running in a simulated environment do not perform as well as real systems due to translation or interception of apps in simulation/virtualization. Timing attacks can reveal this discrepancy. Malware can calculate relative performance to compare the performance ratios of two or more operations on the same system. Malware can also observe and compare the performance of the cache in the emulated environment and the real environment to detect if it is in simulation. Red-pill testing is another notable set of techniques where malware can detect sandbox sharing behavior caused by imperfect CPU emulation and resident instruction bugs.

Web malware can also search for signs of a sandbox on the network. These signs can appear in many forms, such as known fixed IP addresses, sandbox restrictions related to blocking internet access or extremely fast internet connections. For example, malware can detect sandboxes based on known IP addresses obtained through decoy malware in previous attacks.

6.1.1.2 Reverse Turing Test

One method for detecting malware involves analyzing the interactions between humans and the system. This is because sandboxes are automated machines that lack human interaction. Therefore, if malware does not detect any human activity, it assumes that it is in a sandbox. This approach is known as the reverse Turing test, where the machine tries to differentiate between human and artificial intelligence. Various techniques can be used for this, such as waiting for a left mouse button click or monitoring user inactivity. Malware may also wait indefinitely for any user input before executing its payload. Simulating human behavior can be detected, and devising new reverse Turing tests is a constant challenge. This tactic is becoming more common, but it is still not as widespread as fingerprinting.

6.1.1.3 Target

The third strategy that depends on testing is object detection. This strategy is slightly different from the previous strategy, because malware fingerprint verification whether the host is a completely expected (target) machine, rather than directly detect or evading sandboxes. In other words, malicious software is looking for goals, not a sandbox. This strategy can take the following different approaches:

Environmental Targeting

Stuxnet is believed to be the first cyberweapon to use targeting tactics as part of its evasion tactics. Stuxnet explicitly looks for the presence of a particular industrial control system and otherwise remains dormant. Depending on the method of infection, APT attacks follow different paths.

Individual Targets

In Stuxnet’s case, the strategy was to keep probing the victim (a worm behavior) until the target was reached. Other types of APTs include attacks such as DarkHotel, where infection is delivered via spear phishing (i.e., directly targeting the target). In other words, attackers ensure that their malicious code is delivered directly to their targets.

Environment-Dependent Encryption

This type of targeted malware has an encrypted payload whose decryption is affected by a key from the victim’s environment. A key could be a hardware serial number, a specific environment setting, etc.

6.1.2 Detection Independent

The main difference between these technologies is that they do not depend on the detection target environment, and their evasion technology has nothing to do with the target system, so that they can get rid of their demand for complex detection technology. Therefore, the efforts to improve transparency have nothing to do with these strategies. In the following, we introduce these strategies and deploy several technologies in detail.

6.1.2.1 Stalling

This evasion strategy uses such a fact, that is, the time for the sandbox to be assigned to each sample is limited. Malicious software must simply delay its malicious activities to the post-analysis stage. To this end, the author of the malicious software proposes delayed ideas, which can be achieved through a series of different technologies, from simple calls to sleep () functions to more complex methods. Here we examine some of the major known stall techniques.

Ordinary Sleep

Sleep is the easiest way to procrastinate, and it is easy to defeat. This idea is to keep it for N minutes without active, so as to check the sandbox time before observing any malicious activity. After falling into the network, malicious software executes malicious and effective loads. The infamous Duqu (DUQU) shows this technology as one of the prerequisites for ignition. It requires the system to be idle for at least 10 minutes. Another example is Kheelios Botnet. The new variant (called 20 NAP) found in the KHELIOS sample in 2013 was based on a 10-minute timeout () API. The delay of this execution exceeded the sandbox analysis timeout and then reached the harmless logo. To fight this technique, the sandbox proposes a simple way to speed up time (called a sleep patch). Although it seems logical, sleep plaques have an unpredictable effect. In a specific malware sample, the interesting side effects of sleep stipulation are observed. In this sample, the acceleration actually causes the predefined period of malicious software instances to wait for human interaction. Although there are side effects, in some cases, the sleep patch has been proven to be valid, and the author of malware has proposed more advanced technology.

Advanced Sleep

Finding new malware (such as Pafish) chose more advanced sleep techniques. They use the RDTSC instructions combined with Sleep () to detect sleep patch to check the speed of execution.

Code Stalling

Delaying the execution during sleep allows malware to stagnate in the code and select execution to execute unrelated and time-consuming benign descriptions to avoid elevating any suspicion in the sandbox. Rombertik is spyware designed to steal confidential data. To confuse the sandbox, it writes approximately 960 million bytes of random data to memory. The obstacles this technology brings to the sandbox are twofold. The first problem is that the sandbox cannot suspect a stall while the sample is running. Second, too many writes can overwhelm tracking tools. In another high-profile code stalling technique, the malware encrypts the payload using a weak encryption key and enforces it during execution.

6.1.2.2 Trigger-Based

The second strategy that does not rely on detection is trigger strategies or more traditional logical bombs. As we have already pointed out, the foundation of evasion of the sandbox is not malicious in the sandbox. Another strategy to maintain a dormant state is to wait for triggers. There are many environmental variables that can act as malicious software triggers, from system date to specialized network commands. For example, at some days, IDoom is triggered and DDOS attacks, or some keyboard recorders only record the hit key on specific websites; in the end, only the DDOS robot is activated when the appropriate command is given. In the literature, this behavior is called trigger behavior. The following are some trigger factors embedded in malware as protective sandboxes.

Keyword Triggering

If the malware noticed, the name of the app or the title of the window, then it would trigger special behavior depending on the purpose and context. For example, malicious software may start recording a pink key.

System Time

In this case, the system date or time will act as the trigger.

Network Input

A part of malware is triggered when it receives a specific input from the network, just like the tribal flood network.

Stealth Triggering

Previous techniques often assume a lack of code inspection to remain undetected (as is often the case with compiled malware). However, progress has been made in the automated detection of these procedural routes, for example, state-of-the-art covert trigger techniques are being devised from disassemblers. In addition, they implemented secret control transmission based on the use of triggers, which makes it difficult for dynamic analysis to find appropriate triggers, and the return value of the system call is other examples of malicious software triggers. The trigger strategy was originally used to dodge the sandbox, which is still relatively visible in the wild. These trigger factors are usually achieved by path exploration methods, such as the purpose of symbolic execution is to find and cross all condition branches in an automated manner. These methods are densely resource -intensive, and the efficiency is still not as reasonable.

6.1.2.3 Fileless Attack

Fileless malware is a unique type of malware that is difficult to analyze and evade security mechanisms. Unlike traditional malware, fileless malware is immune to the analysis environment. This type of malware exploits vulnerabilities in the target system, such as the operating system, browser, and plugins, to inject malicious code directly into memory. A new trend in fileless malware is to use Windows PowerShell to perform its tasks. Defending against this type of attack is complex and requires advanced techniques.

6.2 Manual Dynamic Analysis Evasion

6.2.1 Direct Detection

One strategy used by fileless malware is called “direct detection,” where the malware discovers the presence of a debugger by observing its artifacts. Debuggers were originally designed to debug legitimate software, so they provide no stealth countermeasures. Enabling debugging capabilities requires instrumenting the system with necessary tools, which leads to extensive tracing of the system at different levels. Malware uses this tracing to detect the presence of debuggers. In this section, we detail these system changes and how malware attempts to reveal them. These tactics tend to be easy to use but can be beaten by moving from user-mode debuggers to emulation-based debuggers.

6.2.1.1 Read PEB

The Process Environment Block (PEB) is a structure that contains information about every process in the system. Malware can use different parts of the PEB to detect the presence of a debugger. For example, the “BeingDebugged” field in the PEB can be read directly or through specific APIs like IsDebuggerPresent() and CheckRemoteDebuggerPresent(). These techniques are commonly used by malware, but they can be solved by changing debugged bits or API hooks. Other PEB-based techniques, such as utilizing NtGlobalFlags and processheap, are slightly more complicated but still commonly used in anti-debugging strategies.

6.2.1.2 Breakpoint Query

Debuggers set breakpoints to stop execution, which can be hardware- or software-based. Malware can detect these breakpoints by looking for the special opcode “0xCC” or using GetThreadContext to check the CPU registers. This technique is used by malware like CIH or MyDoom.

6.2.1.3 System Artifacts

Debuggers leave traces at different levels of the operating system, such as file system, registry, and process names. Malware can detect debuggers by looking for these traces using APIs like FindWindow() and FindProcess(). Randomizing the names or changing the results of these queries through API hooks can defeat these techniques.

6.2.1.4 Parent Process Detection

To check for the presence of a debugger, you can use functions like GetCurrentProcessId(), CreateToolhelp32Snapshot(), Process32First(), and Process32Next() to check the parent process name and ID. If the parent process ID does not match explorer.exe, it could indicate the presence of a debugger. However, this method is challenging and seldom used in practice. Malware can also exploit the logic that processes specific instructions, which is done through Structured Exception handling (SEH). For instance, the max$$++$$ malware includes the “int 2dh” instruction in its code. According to SEH, this instruction throws an exception without a debugger, and malware can handle it through a try-catch structure. But if a debugger is attached, the exception is diverted to the debugger instead of the malware, and this is the logic the malware employs to detect the presence of a debugger. Malware may also use other techniques, such as embedding particular instruction prefixes or other instructions like 41h, to set traps. A more subtle way to detect and evade debuggers is through indirect detection, which is the subject of the next section.

6.2.2 Deductive Detection

In this category of strategies, the goal is to infer the presence of a debugger indirectly, rather than observing it directly. A policy is used to calculate and evaluate the probability of a debugger being present, which relies on exploiting the system’s logic, such as the SEH exception handling in Windows. Malware can use various tactics to achieve this, which we will explore below.

6.2.2.1 Trap

The “trap” technique is not to be confused with trap flags. With this strategy, malware provides code that, when stepped by a debugger, reveals information that helps the malware infer the presence of a debugger. Many techniques fall into this category, and we will detail a few of them here.

One technique malware uses is to embed specific instructions in its code that exploit the logic of the system, such as SEH. For example, the “max$$++$$” malware embeds the “int 2dh” instruction in its code. According to SEH, when this instruction is executed, an exception is thrown without a debugger, which the malware can handle through a try-catch structure. However, if a debugger is attached, this exception will be diverted to the debugger, revealing the presence of a debugger to the malware. Malware may use other techniques, such as embedding specific instruction prefixes or other instructions like 41h. These techniques are relatively simple and can be implemented in less than 20 lines of code in assembly. To combat these traps, one can ask the debugger to step over these instructions. According to our research, exploiting traps is a common method used by malware.

6.2.2.2 Time-Based Detection

Detecting the presence of a debugger can be effectively done through time-based detection strategy. This tactic involves analyzing the time taken for a particular function or set of instructions to execute. If the time exceeds a certain threshold, the malware infers the presence of a debugger. This technique can be performed locally using native APIs such as GetTickCount() and QueryPerformanceCounter(). It can also use the CPU rdtsc to query external resources for timing. Local timing is easy to use but difficult to bypass. However, using external sources like NTP or tunneling NTP to counter time-based detection remains a challenge. Malware such as W32/HIV, W32/MyDoom, and W32/Ratos are notorious for exploiting these time differences.

6.2.3 Debugger Evasion

This category focuses on ensuring that malware executes fully or partially, rather than detecting or inferring the presence of a debugger. Malware uses a technique called catch to achieve this goal. Below is an overview of this strategy.

6.2.3.1 Control Flow Manipulation

Control Flow Manipulation is a technique where malware exploits the implicit flow control mechanism of the Windows operating system. Malware authors typically use callbacks, enumeration functions, thread local storage (TLS), and other notable techniques to achieve the goal of “debugger evasion,” each of which deserves a brief introduction.

Thread Hiding

Thread hiding is a simple but effective technique that prevents debug events from reaching the debugger. Microsoft has provided a special API called NtSetInformationThread() to implement this technique. It sets the HideThreadFromDebugger() field of the ETHREAD kernel structure. This technique is powerful and easy to implement but can be solved by hooking related functions. LockScreen is an example of malware that uses this technique.

Hanging Threads

Hanging Threads is a more aggressive approach where malware tries to stop the debugger’s process, allowing it to continue its own execution without much trouble. This technique is only valid for user-mode debuggers and can be implemented by using SuspendThread() or NtSuspendThread() in ntdll. The Kronos banking malware has this technique in its arsenal.

Multithreading

Multithreading is another technique to bypass the debugger and continue execution. One way to achieve this is through the CreateThread() API. Packed malware typically spawns a separate thread within its process to execute decryption routines. However, in some cases, malware executes a portion of malicious code through a different thread than the debugger. McRat and Vertexnet employ this technique. Dealing with this technique is tricky, and one way is to set breakpoints at each entry point.

Self-debugging

Self-debugging is an interesting technique that prevents a debugger from successfully attaching to malware. Each process can only be attached to one debugger by default, and malware exploits this by running a copy of itself and attaching to it as a debugger. This prevents another debugger from owning it. There are several ways to implement this strategy, such as utilizing DbgUiDebugActiveProcess() or NtDebugActiveProcess().

6.2.3.2 Lockout Evasion

The malware can successfully continue its execution through a lockout strategy that obstructs the functioning of a debugger. Several techniques are utilized by malware to achieve this, such as using the BlockInput() function in Satan RaaS, which blocks mouse and keyboard input until certain conditions are met. Other techniques involve exploiting a feature in Windows NT-based platforms that allows multiple desktops to exist. LockScreen, for instance, can choose a different active desktop with the help of CreateDesktop() and SwitchDesktop() and continue working without detection from the debugger.

6.2.3.3 Debugger Identification

To evade specific debuggers, malware exploits vulnerabilities that are exclusive to them. These vulnerabilities are challenging to find but easy to act on. For instance, OllyDBG has a format string bug in its regular version that can cause it to crash by passing incorrect parameters to the OutPutDebugString() function. Another technique that was prevalent was related to the SoftICE debugger, which was vulnerable to multiple DoS attacks due to two vulnerable functions. Malware can exploit these vulnerabilities and cause a “blue screen of death.” Exploiting such vulnerabilities can prove to be an effective anti-debugging technique if a vulnerability is found in a specific debugger.

6.2.3.4 Fileless Malware

Fileless malware, also known as Advanced Volatile Attack (AVT) attacks, is the latest trend in malware evolution. Unlike previous malware, fileless malware does not require files to operate, and they reside purely in memory and utilize existing system tools such as PowerShell. The purpose of this attack is to make forensics more difficult as there is no executable to analyze. In some cases, capturing the attack live as it happens is the only way to get a sample for analysis. While these attacks are not easy to carry out by themselves, they can be obtained with the help of an exploit kit. According to a 2018 McAfee report, fileless malware increased 432 percent in 2017 and is expected to account for 35 percent of attacks in 2018.

To overcome the limitations of static analysis, analysts can use a debugger for manual dynamic analysis. However, new malware trends and the real world, where vendors face thousands of new malware samples every day, require a more agile approach that goes beyond the capabilities of manual dynamic analysis. In the next section, we will discuss the emergence of automated dynamic analysis methods and sandboxes as a response to these challenges and detail malware strategies to thwart them.

6.3 Related Research About Dynamic Analysis Evasion

The defense of dynamic evasion is considered from two perspectives: one is to improve the sandbox, and to continuously repair the loopholes in the sandbox to narrow the difference between the sandbox and the real device to defend against dynamic evasion; the other is to use the characteristics of dynamic evasion for detection, such as multi-environment execution.

6.3.1 Research About Improving Sandbox

6.3.1.1 The Droid is in the Details: Environment-Aware Evasion of Android Sandboxes

In order to evaluate the attacker’s ability to build a sandbox evasion based on the environment, the research [6] implemented an Android app, which can obtain statistical data from the environment through a large number of API lists, and send it back to its own server for analysis, and finally use these data to train a machine learning model to identify the simulator.

Through the collected data, the users and institutions of the crowdsourcing platform are invited to participate in this experiment and let them download and install this app. Through this process, they obtained data from 1,245 user devices and used this data to train a machine learning classification model. Based on this data, they built a classifier that was able to distinguish between the real environment and the sandbox with 98.54% accuracy. Results show that model training is robust to sandbox patches, with defenders able to remove more than 60 of 81 features without affecting classifier performance at all.

Finally, to build manufacturer-specific models based on the settings on devices from mainstream manufacturers, they found that only three sandboxes had declared device models that matched the environmental characteristics under the manufacturer-based classifier. For the other nine sandboxes, their classifiers identified that they were actually lying. Based on their conclusions, sandboxes are discoverable both because they share common features and because sandboxes are not like their real counterparts (manufacturer-based sandbox evasions).

Due to the continuous upgrade of existing malware, some malware can suspend malicious activities when detected in a sandbox environment. Existing sandboxes evade malware analysis by simulating the runtime properties of real machines. This work proposes a new mobile sandbox evasion technology, which detects the simulator by the characteristics of the mobile environment rather than the performance of the specific simulator, from the available APIs such as screen brightness, the number of camera songs, hardware sensors, etc., through their feature distributions are used to distinguish between real and simulated environments. Attackers can use this method to build a classifier to distinguish real devices from sandboxes with 98.54% accuracy. Furthermore, this study focuses on inconsistent properties between sandbox manufacturers and real devices from these manufacturers. To demonstrate the ineffectiveness of patching APIs alone in a sandbox, the findings of this study highlight the difficulty of creating a robust sandbox environment. By evaluating the latest sandbox, it is proved that the latest existing mobile sandbox cannot protect against such attack.

Data Collection

This study released an app to collect data from the user and send the information to a remote server. They uploaded this app to different apps in different markets, and invited users and institutions of the crowdsourcing platform to participate in this experiment, let them download and install the app. The following feature information is mainly collected from the app, mainly including three types of features: wear characteristics, device configuration characteristics, hardware device characteristics. Wear characteristics refer to the characteristics generated by the user’s use; device configuration characteristics are similar to wear characteristics, referring to some settings and configurations of the user on the mobile phone; device hardware characteristics refer to the unique configuration of device components and settings.

The APK developed in this research mainly collects three types of operation permissions and two types of installation permissions: calendar, contacts, files, wifi status, and user dictionary. The data is sent to the remote server. Finally, the final dataset is formed based on the collected data: a total of 1245 real devices, 398 baseline devices, and 210 sandbox devices. Baseline devices are real Android phones that provide cloud-based app testing through Google Firebase Test Lab and Amazon Device Access services. They also label the baseline device as an emulator during training and testing. 200 real devices, 100 baselines, and 100 sandboxes are used as training sets. The remaining 408 sandbox devices and 1045 normal devices are used as the test set.

Experiments and Conclusions

For interpretability and to find the splitting boundaries of features, decision trees are used as classifiers. Using multiple classification algorithms to evaluate, the final gradient boosting algorithm works best. And considering the impact of features on the classification results, it was finally found that among the top 20 key classifiers, 13 belong to wear-and-tear features. Therefore, API patches for certain features of the simulator cannot affect the detection results.

The results on the test set show that the classifier achieves an accuracy rate of 98.54%, a false positive rate of 1.58%, and a false negative rate of 0.57%. Overall, the classifier produced only one case of false negatives, a dataset from the sandbox device. In addition, this study also used 10-fold cross-validation to test on the same test set and finally reached 99.27% effect.

It is found that when only one type of feature is used, the performance of device configuration features and wear features is the best for detection, and only using device hardware attributes can still detect well.

Then they study the results of testing using mobile devices and emulators across different time and price ranges, and the results show that the time and price of the device are not enough to affect the difference between the real device and the mobile sandbox.

Considering that if too many permissions are required or sensitive permissions are involved, it is very likely that various detection methods will be judged as malicious samples. The relationship between the features required for detection and the required permissions. The results show that only some basic permissions are used. This allows the app to detect the emulator.

Manufacture Detection

In order to further prove the correctness of the previous experiments, this research also studies the difference between the manufacturer’s equipment that the sandbox claims and the actual equipment produced by the manufacturer.

Start by creating specific machine learning classifiers for each device manufacturer with a subset of the above features. The features considered are mainly hardware configuration features and bloatware that come with the system.

A specific Android device will always have the same hardware setup. Device models from the same manufacturer may have consistent hardware settings. The number of environmental sensors and the configuration options of the communication module may vary greatly depending on the manufacturer: light, gravity sensor, heart rate, and gravity sensor.

Operating systems often include a bundled portion of bloatware, and by studying the amount of bloatware included by each manufacturer, they were able to determine the expected size of each manufacturer’s system image. From their research, they can conclude that real devices from the same manufacturer have more pre-installed software packages than sandboxes of the same device.

Collect the number of system software packages by re-uploading samples, retrain multiple classifiers (note that the classifier here is a binary classification of multiple manufacturers, and each classifier can distinguish whether the device is a certain manufacturer or not, not a multi-classifiers, because the number of potential device manufacturers is large, and there will be a great negative effect if multi-classifiers are used), and finally found that on real devices, they can use these features to identify device manufacturers with a higher recognition rate, while on the simulator, the accuracy can only be lower.

The author evaluates the conformity between the sandboxes of various manufacturers and their real devices and finally shows that only a small part of the sandboxes match the characteristics of their declared devices.

Finally, the author believes that for malware, once it is discovered by detection means, its signature and hash value will be published on the Internet, and the malware will be considered invalid. So the most important thing for malware is to eliminate the false positives of the classifier, not to reduce the false positives. To achieve this, the authors adopted a two-stage tandem detection method. Among the devices identified as non-sandbox by the first classifier, and the devices that were misjudged as non-sandbox (which were actually sandboxes), the manufacturing quotient detection to reduce false positives. The author uses the experiment as an example, in the first stage of the experiment, 1019 negative samples were obtained, and multiple classifiers were used for detection in the second stage. The results showed that 379 of them belonged to a certain manufacturer, and the other were all negative in these binary classifiers. Among these 379 samples, 378 were true negatives and 1 was false negative when detected by the one-stage classifier. Put them into a two-stage classifier for detection. It was found that the hardware configuration and bloatware of the false negative samples were indeed different from the devices it claimed, while the other 378 had no differences, so it was an emulator or sandbox environment.

Discussion

The experiments in this research show that it is not enough to patch a single API; it is necessary to make the declared device manufacturer model, details, etc. match across the environment. While some sandboxes fully emulate Android hardware, their usage characteristics can still be used to differentiate them. To mitigate this flaw, real devices can be used to build sandboxes, monitor the signatures collected by malware, and dynamically return real values to them. In addition, the author believes that although they have collected a large number of samples, the number of samples is not enough for machine learning. They encountered some problems in the process of using the crowdsourcing model. In the future, other work can use the crowdsourcing model to help collect a large number of real statistics. Data, sandbox builders can use this method to build a more realistic sandbox environment. The authors also suggest that sandboxes can also use adversarial example machines to circumvent such machine learning classifiers.

6.3.1.2 Morpheus: Automatically Generating Heuristics to Detect Android Emulators

Malware can use heuristics to effectively detect whether the environment is in an emulator or a real device. This research [10] developed Morpheus, which leverages their discrepancy detection on real devices and simulators, to automate the generation of such heuristic rules. To this end, it observed sandboxes and simulators as well as real devices and finally revealed 10,000 heuristic methods that can be used to detect simulators. It is mainly divided into four steps:

1.

Sandbox analyzer to identify sources of observable artifacts

2.

Artifact finder, which traverses the source of observable artifacts and outputs all observable results to the pool of the next stage

3.

Heuristic extractor to extract discrepant artifacts

4.

Heuristic screening, sorting candidates, this problem is similar to the feature selection problem in supervised learning

Based on these heuristics discovered, the authors propose three countermeasures against simulator detection of malware:

1.

Basic countermeasures: Although this heuristic detection method cannot be absolutely identified, it can serve as a warning. For example, a dynamic analysis system can monitor access to files and attributes that are rarely used by benign apps. API heuristics are more stealthy because benign apps often use them as well.

2.

Short-term countermeasures return virtual values, such as ASM, so that apps can return virtual values. Make the app run in the virtual device, but get the data of the real device. Strict DAC and MAC are recommended to limit unwanted access.

3.

Long-term countermeasure: Fix all differences in the simulator.

6.3.2 Research About Detecting Dynamic Evasion

6.3.2.1 CamoDroid: An Android App Analysis Environment Resilient Against Sandbox Evasion

This research [11] proposes the CamoDroid dynamic analysis framework, by extracting network features, sensors, user data, and user input features, hooking functions in the native layer and java layer, creating a list of sensitive APIs, and implementing a monitoring in camodroid module, the monitoring module provides a dynamic analysis report on the security-sensitive behavior of the app by recording parameters, return values, and sensitive calls. This dynamic framework is extensible. This research studies the resistance of a centralized dynamic sandbox environment to simulator detection sex, the results showed that theirs was better.

In order to defend against dynamic sandbox evasion, this study proposes a method of spoofing malware by simulating sensor data, user input, network, and other characteristics, which proves that existing commercial and academic methods cannot detect it, and can be found in real-world environments 96% of dynamically evasive malware. Joker, Obad, Pincer, Android.hehe, and DenDroid malware can detect its environment and evade it.

This study has the following 4 contributions:

1.

Propose a dynamic analysis framework that can resist simulator detection.

2.

This dynamic analysis framework can provide detailed sensitive API call behavior.

3.

Thanks to the open source extensible framework, this framework can be extended.

4.

Reveal vulnerabilities of existing sandboxes.

Features

Some malware developers write C or C$$++$$ code and compile it to native layer code, such as Droidkungfu.H Stiniter. This is an example of malware using native layer code, another evasion technique reflection, which allows a program to A method for checking its internal properties at runtime. In a more complicated situation, malware downloads files containing sensitive function names from the Internet and obtains them using hook technology. The features can be divided into:

1.

Static attributes: Static attributes including the values of the android.os.build class, these values are generally used as environment configuration variables, malware can determine whether the environment it is in by identifying these values for the simulator.

2.

Network characteristics: Network characteristics including special APIs such as TelephonyManager.getLine1Number, which may be set to special values in the simulator, such as getDeviceId will return 00000000000, but the real device does not. There will be such a value, so an attacker can use this value to determine whether his environment is in the emulator.

3.

Sensors use randomly generated sensor events to call onSensorEventChanged, but it believes that there should be a distribution in the real world, but the framework they provide allows workers to modify this distribution.

4.

User data can be divided into private and shared. Shared data is usually stored in shared folders by apps and downloaded from the Internet. All apps can access shared storage. Therefore, it is necessary to simulate contacts, phone records, text messages, user input, and interactions with monkeys. The sensitive API calls come from the SDK API calls and corresponding permissions collected by Backes and others, a total of more than 400.

Experiment

This research first put the self-developed tool Camodroid into the well-known emulator detection tools Morphes and RootBeer. Judging from the results, these two tools cannot effectively judge Camodroid as an emulator. The result shows the results of Camodroid on the well-known malware Android.Hehe.A. From the results, it can be seen that the tool can effectively obtain the APIs that the monitored malware tries to call.

In this study, a total of 227 samples were collected from mini dump and malpedia download malware, and they were executed for 30 seconds and the logs were observed. A total of 219 evasion samples were found out of 227 samples. The sample size used in this study is small because it assumes that the same family has a lot of shared information, so increasing the data size will not significantly affect the results.

6.3.2.2 Lumus: Dynamically Uncovering Evasive Android apps

This research [12] proposes a method for screening Android malicious samples that evasion dynamically. They run Android apps on real devices and emulators, analyzing their logs, filtering out common operations such as writing to shared memory addresses and reviews, and analyzing the logs After normalization. Then, they extract the collections that are not available in real machine-specific simulators, analyze the calling sequences of these collections starting from the program entry point, compare them with the complete calling sequence of the program, analyze the reasons for the differences, and identify whether it is a path in a specific environment in the end.

Method

The non-mobile terminal uses distance equations and hierarchical structuring to compare similarities and uses tracking system calls as input to identify Windows malware. Their approach was to identify differences in the actual behavior of Android malware running on bare metal and in a simulated environment, independent of the characteristics between the two environments. First, determine the cause of the different behavior by gathering information readily available on the Android, it includes following steps:

1.

The execution method of the program being analyzed

2.

The method from the framework called in (1)

3.

System call from the app

4.

The interaction of the app leads to the creation of threads or indirectly changes the execution flow

5.

Information from the system about program stop execution

6.

Information about external stimuli

It is possible to trace the call sequence of certain behaviors that only occur on bare metal. By doing this, we can identify the cause of the behavior and any external stimuli that may have caused it.

To determine the cause of any discrepancies between the sequence obtained from bare metal and the behavior observed in the simulator, we compare them. This allows us to identify any events that were not generated, discrepancies in the execution of certain methods, or any other issues that may have affected the system.

We collect logs (1)–(6) and use this information to create a comparison report. When the bare metal device returns to a consistent state, we compare the information in the report with the emulator and the bare metal execution data.

Evaluation and Results

In their study, the researchers used AVD and LG G2Mini devices and a modified version of Android 5.1 for both bare metal and emulators. They executed each sample for a maximum of three minutes in a bare metal environment and up to 10 minutes in a simulated environment. Samples were selected from VirusShare, Malgenome, contagio Mobile, androMalShare, and Drebin. Subsets were chosen through Virustotal, and families were identified by ESET-NOD32. Then, a maximum of five APKs were selected for each family, resulting in 1470 samples.

Out of the 50 samples randomly selected from different families and manually checked for evasion, their technique detected seven evasions. False negatives were not investigated, but samples exhibiting behavior different from the simulated system were considered false positives.

Lumus considers behavior that only occurs in the bare metal system but not in the simulated system to be an evasion, even if this discrepancy is not due to the analysis system’s identification. For instance, if a sample attempts to send an SMS to a contact stored on the phone but only does so in the bare metal environment because there are no contacts in the simulated environment; Lumus treats it as an evasion. This is because it successfully prevents certain actions from being observed in the simulator and can be used as an evasion technique. Disarm and Barecloud, two tools for Windows, can be compared to Lumus.

The analysis of some specific samples is shown below:

1.

Sample 1: If /system/xbin/busybox, /system/bin/busybox, or /bin/busybox exist in the system, it changes its behavior and the difference causes the malware to write shared prefs/config.xml and many files to/SD card/LuckyPatcher/ directory. This is probably not an evasion technique since most user systems do not have these files. However, since it does prevent some malware behavior from being observed in the emulator environment, they consider that behavior TP.

2.

Sample 2: It identifies if the phone number starts with “15555,” if the IMEI starts with “00000000,” or if the IMSI starts with “31026.” Once detected, System.exit(0) will be called.

3.

Sample 3: It copies the icons of the apps installed in the system to the directory /data/data/com.pintudog/files/icons/. Since the list of apps installed in the emulator and the bare metal environment are different, the monitored actions end up being different. However, at a higher level, it is still the same behavior, so they consider it FP.

4.

Sample 4: It verifies that the IMEI contains the string “000000000000000” and then calls System.exit(0). Similar to sample 2, the actions resulting from the divergence are as follows: start the service, create a wakelock, and connect to the dnsproxyd device to issue DNS requests.

5.

Sample 5: Because the graphics library used in the bare metal and the emulator is different. Since this is not actually related to the behavior of the malware, they consider this sample as FP.

6.

Sample 6: During bare metal execution, it verifies which Wifi networks are available. In the simulator, it does not recognize any Wifi networks, so it takes a different execution path. Because of this difference, the only behavior that is performed in bare metal is writing a file in the SD card. Since this behavior is related to malware execution, they consider this sample as TP.

7.

Sample 7: This sample randomly selects a domain name to visit from a list of predefined names. This results in a different domain name being used in the emulator than in bare metal. They behave the same except for the domain name, so they think it is FP.

In this research, 192 instances of evasion software were found among 1470 samples. The results were classified into three categories: static, dynamic, and sensors.

Static

Static analysis involves querying attributes through the android.os.Build class, such as checking if “google_sdk” exists in PRODUCT or MODEL, “generic” exists in BRAND or DEVICE, or “Goldfish” exists in HARDWARE. Specific files, like /system/bin/qemud, can also be queried. Using package names, such as com.google.android.gsf, and checking the total number of installation packages can also help detect emulators.

Dynamic

Dynamic analysis involves detecting false environments using WifiManager and BluetoothManager. Apps may also abuse the simulated network connection itself. Interactions with the Google Play Store or unavailable services and apps can also result in exceptions. However, the introduction of Google’s SafetyNet Attestation API poses a threat to most dynamic analysis systems.

Sensors analysis involves comparing the sensors emulated on the AVD with the actual model. There are still differences that can be exploited, such as user behavior information and differences in the system. The name of the query sensor is displayed as “Goldfish-*” instead of the actual sensor name.

Overall, this research provides useful insights into detecting evasion software but has some limitations. Differences in user behavior and data and system variations may lead to different actions. The SafetyNet Authentication API allows malware to implement easy-to-use server-side runtime environment checks and app tampering detection.

6.4 Chapter Summary

This chapter mainly introduces Android dynamic adversarial technology and its defense methods, makes a comprehensive classification of Android dynamic evasion technology, and discusses each category in detail. In Sects. 6.1 and 6.2, they, respectively, elaborated two classifications of dynamic evasion: the main principles, divisions, and specific methods of dynamic sandbox evasion and dynamic debugging evasion. In Sect. 6.3 we describe dynamic evasion defenses and related research. In subsequent chapters we will discuss machine learning adversarial techniques in Android malware detection.

References

1.

Afianian A, Niksefat S, Sadeghiyan B, et al. Malware dynamic analysis evasion techniques: A survey[J]. ACM Computing Surveys (CSUR), 2019, 52(6): 1–28.Crossref

2.

Vidas T, Christin N. Evading android runtime analysis via sandbox detection[C]//Proceedings of the 9th ACM symposium on Information, computer and communications security. 2014: 447–458.

3.

Yokoyama A, Ishii K, Tanabe R, et al. Sandprint: Fingerprinting malware sandboxes to provide intelligence for sandbox evasion[C]//International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, Cham, 2016: 165–187.

4.

Diao W, Liu X, Li Z, et al. Evading android runtime analysis through detecting programmed interactions[C]//Proceedings of the 9th ACM conference on security & privacy in wireless and mobile networks. 2016: 159–164.

5.

Miramirkhani N, Appini M P, Nikiforakis N, et al. Spotless sandboxes: Evading malware analysis systems using wear-and-tear artifacts[C]//2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017: 1009–1024.

6.

Kondracki B, Azad B A, Miramirkhani N, et al. The Droid is in the Details: Environment-aware Evasion of Android Sandboxes[J].

7.

Sahin O, Coskun A K, Egele M. Proteus: Detecting android emulators from instruction-level profiles[C]//International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, Cham, 2018: 3–24.

8.

Petsas T, Voyatzis G, Athanasopoulos E, et al. Rage against the virtual machine: hindering dynamic analysis of android malware[C]//Proceedings of the seventh European workshop on system security. 2014: 1–6.

9.

Wang X, Zhu S, Zhou D, et al. Droid-AntiRM: Taming control flow anti-analysis to support automated dynamic analysis of android malware[C]//Proceedings of the 33rd annual computer security apps conference. 2017: 350–361.

10.

Jing Y, Zhao Z, Ahn G J, et al. Morpheus: automatically generating heuristics to detect android emulators[C]//Proceedings of the 30th Annual Computer Security apps Conference. 2014: 216–225.

11.

Faghihi F, Zulkernine M, Ding S. CamoDroid: An Android app analysis environment resilient against sandbox evasion[J]. Journal of Systems Architecture, 2022, 125: 102452.Crossref

12.

Afonso V, Kalysch A, Müller T, et al. Lumus: Dynamically uncovering evasive Android apps[C]//Information Security: 21st International Conference, ISC 2018, Guildford, UK, September 9–12, 2018, Proceedings 21. Springer International Publishing, 2018: 47–66.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_7

7. AI-Based Adversarial Method in Android

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

In recent years, the app of machine learning (ML) in Android security has attracted widespread attention, and its research enthusiasm continues to rise. Many scholars focus on exploring the potential and app of machine learning algorithms in Android security. This chapter will focus on the integration of machine learning technologies in the Android security field. In Android security, adversarial examples are an important research direction. Adversarial examples are generated by making slight modifications to the original sample so that it can fool the machine learning model. These modifications are usually within a range that is imperceptible to humans but can have a significant impact on the model. Research on adversarial examples helps to understand the vulnerabilities of machine learning models and propose corresponding defense strategies. Regarding adversarial examples, researchers have proposed different classification methods and attack techniques. Among them, black-box and white-box attacks are two common perspectives. A black-box attack means that the attacker can only infer the behavior of the model through its input and output, without direct access to the specific structure and parameters of the model. A white-box attack means that the attacker can fully access and understand the internal information of the model. In these two attack methods, researchers have proposed various adversarial methods and techniques. In order to conduct in-depth research on black-box and white-box confrontation, many typical works have been proposed. These works include different methods and technologies, such as transfer learning, meta-adversarial training, zero-order optimization, and query-based adversarial attacks. By introducing these works in detail, we can deeply understand the app of machine learning in Android security and related technologies. First, we will introduce adversarial examples and their principles. Second, we will introduce common adversarial methods and typical works from the perspective of black-box and white-box attacks.

7.1 Introduction to Adversarial Examples

Adversarial example attacks are a type of attack that uses vulnerabilities in machine learning models to generate inputs that are imperceptible to humans but cause the model to make classification errors [1]. The essence is that by making slight modifications to the input data, machine learning models are fooled into misclassifying adversarial examples as a different class than the original input. These modifications are usually small enough to be undetectable to humans, but significant enough to confuse the model.

Adversarial attacks pose a serious threat to machine learning models, as they can cause high accuracy models to produce erroneous outputs. For example, adversarial attacks could be used to fool automated driving systems, detection systems, and facial recognition systems, potentially leading to dangerous consequences.

To counter adversarial attacks, researchers have proposed numerous defenses, such as defensive training, adversarial training, randomization defenses, etc. These methods can increase model robustness and reduce the impact of adversarial attacks to some extent. At the same time, robustness testing and analysis of machine learning models need to be strengthened to improve detection and defense capabilities against attacks.

Adversarial examples could be used in Android apps in the following scenarios:

Malicious apps: Attackers could use adversarial examples to generate inputs that appear harmless but fool Android apps, bypassing security screening or deceiving users. For example, attackers could use adversarial example attacks to enable malicious apps to evade malware detection or identification systems.

Android system vulnerabilities: Adversarial examples could also be used to exploit vulnerabilities in the Android system to bypass security protections. For example, attackers could use adversarial examples to trick the recognition systems in Android, bypassing security screening.

Speech recognition systems: Adversarial examples could also be used to attack Android speech recognition systems to achieve dangerous attacks. For example, attackers could use adversarial examples to deceive the Android speech recognition system to recognize dangerous commands as normal commands, achieving attack goals.

To prevent the impact of adversarial example attacks on Android apps, researchers have proposed some defensive measures, such as adversarial training, randomization defenses, defense in feature space, etc. These methods can improve the robustness of Android apps and reduce the impact of adversarial example attacks. At the same time, Android app developers can also adopt some best practices, such as using the latest deep learning models, dataset cleansing, limiting input data, etc., to reduce the risks of adversarial attacks.

7.2 Classification of Adversarial Example Generation Methods

Adversarial example generation methods can be classified into gradient-based, optimization-based, GAN-based, data augmentation-based, and black-box attacks. Each method attempts to balance attack success rate and stealthiness. With the extensive app of deep learning, adversarial attack techniques in different scenarios continue to evolve. Current adversarial example research mainly focuses on improving the transferability, universality, and imperceptibility of attacks, as well as building adversarially robust models [2]. Defense methods have also shifted from detecting adversarial examples to enhancing models’ intrinsic adversarial robustness. It is foreseeable that adversarial examples will be a long-term topic in machine learning security research, requiring continuous advancement in attack and defense techniques to build more trustworthy AI systems. This section will introduce different adversarial example generation methods [3].

7.2.1 Gradient-Based Attacks

Gradient-based attacks (FGSM, BIM, DeepFool, etc.): This type of attack method is the earliest and simplest adversarial example attack method. By calculating the gradient information of the model, slight perturbations are made to the input data to cause errors in the model’s output. Common methods include Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), DeepFool, etc.

7.2.2 Optimization-Based Attacks

Optimization-based attacks (C&W, PGD, etc.): This type of attack method utilizes optimization algorithms to perturb input data, minimizing the distance between the adversarial and original examples while maximizing the error under the premise of maintaining adversarial properties. Common methods include Carlini and Wagner (C&W), Projected Gradient Descent (PGD), etc [4].

7.2.3 GAN-Based Attacks

GAN-based attacks (GAN, CycleGAN, etc.) use Generative Adversarial Networks (GANs) or variants to generate adversarial examples. This approach does not require access to the model’s gradient information but instead uses the generative model to sample from the training data to produce adversarial examples. Common methods include Generative Adversarial Networks (GAN), CycleGAN, etc [5].

7.2.4 Domain-Specific Attacks (Audio, Images, Text, etc.)

These types of attacks target machine learning models in specific domains, such as audio adversarial examples, image adversarial examples, text adversarial examples, etc.

Audio adversarial examples, image adversarial examples, and text adversarial examples are all forms of adversarial example attacks, with similar principles and ideas as general adversarial example attacks, but with differences in specific implementation and technical details [6].

Audio adversarial examples: Audio adversarial examples refer to adversarial example attacks targeting audio signals. The basic idea is to add certain noise or perturbations to the original audio signal so that it gets incorrectly classified or causes other undesirable effects. Audio adversarial examples can be achieved by manipulating parameters of the audio signal like frequency, amplitude, phase, etc. Common methods for generating audio adversarial examples include gradient-based methods, evolutionary algorithm-based methods, GAN-based methods, etc.

Image adversarial examples: Image adversarial examples refer to adversarial attacks on images. The basic idea is to add certain noise or perturbations to the original image so that it gets incorrectly classified or causes other undesirable effects. Common methods for generating image adversarial examples include Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), the DeepFool method, C&&W method, app-specific adversarial example attacks, etc.

Text adversarial examples: Text adversarial examples refer to adversarial attacks on text. The basic idea is to change the content of the text or add certain noise/perturbations so that the original text is misjudged or incorrectly classified. Text adversarial examples can be achieved by attacking different text processing methods like one-hot encoding, word embeddings, etc. Common methods include gradient-based methods, substitution-based methods, language model-based methods, etc.

These adversarial attack methods pose challenges to the security and robustness of artificial intelligence and need to be studied and defended against.

7.3 Black-Box Attacks

Black-box adversarial attacks refer to attacks against machine learning models where the attacker can only infer the behavior of the model through its inputs and outputs, without direct access to the model’s internal structure and parameters. By changing the model’s input data, the attacker causes the model to make incorrect judgments or errors, achieving the attack goal. This section mainly introduces black-box confrontation and common black-box confrontation methods.

7.3.1 Introduction to Black-Box Attacks

Black-box adversarial attacks refer to attacks against machine learning models where the attacker can only infer the behavior of the model through its inputs and outputs, without direct access to the model’s internal structure and parameters. By changing the model’s input data, the attacker causes the model to make incorrect judgments or errors, achieving the attack goal.

Black-box adversarial attacks are an important research direction in the field of machine learning security, with the goal of improving the robustness and security of machine learning models, thus avoiding the harm caused by malicious attacks. In black-box attacks, attackers usually use generative adversarial networks (GANs), evolutionary algorithms, and other methods to generate inputs with deception that can fool the target model. For example, in the field of image recognition, attackers can add noise, occlusion, and other means to images to fool the model. In speech recognition, attackers can use methods like adding noise, changing speed, changing tone, etc. to fool the model. In natural language processing, attackers can use methods like adding grammatical errors, replacing words, etc. to fool the model [7].

Black-box adversarial attacks have very extensive apps in Android systems, such as in the fields of image recognition, speech recognition, natural language processing, etc. By constructing specific input data, for example, by adding noise, disturbance, etc., attackers can fool machine learning models, causing them to make incorrect judgment or errors.

In order to improve the robustness and security of machine learning models and prevent black-box adversarial attacks, researchers have proposed a series of defense methods, such as adding noise, adversarial training, model distillation, etc., to improve the robustness and security of machine learning models. At the same time, the security analysis of machine learning models needs to be strengthened to improve detection and defense capabilities against attacks.

7.3.2 Common Black-Box Attack Methods

Common black-box attack methods include:

Transfer-based attacks: Transfer learning can apply knowledge learned from other models to new models. In black-box attacks, the attacker can learn useful information like class traits, similarity traits, etc. from known models and then apply this information to attack the target model. The advantage of this method is attacks that can be conducted without knowing the target model structure, but related model data needs to be prepared in advance.

Meta-model based attacks: Meta-model based attacks build a substitute model to attack the target model. The attacker can construct a substitute model by sampling inputs and outputs of the target model and then use the substitute model to generate adversarial examples. This method can perform attacks without knowing the target model structure but requires a large amount of input-output data samples and may lead to reduced attack effectiveness.

Query-based attacks: Query-based attacks are a type of attack method that obtains information about the model by querying it multiple times. The attacker queries the model to obtain output and gradient information and uses this information to construct adversarial examples. Since each query has a high cost, this method typically requires a lot of computational resources and time.

Optimization-based attacks: Optimization-based attacks are a type of attack method that searches for adversarial examples using optimization algorithms. The attacker can use different optimization algorithms and loss functions to search for adversarial examples and continuously adjust algorithms and parameters to improve attack effectiveness. This method typically requires a lot of computational resources and time and has issues like large search space and unstable attack effectiveness.

These methods are often combined to improve attack effectiveness and success rate [8].

7.3.3 Transfer Learning-Based Black-Box Attacks

Transfer learning-based adversarial attacks are a common black-box attack method. The basic idea is to learn useful information from known models and then apply this information to attack the target model. This method can perform attacks without knowing the target model structure but requires related model data prepared in advance.

Specifically, transfer learning-based adversarial attacks usually consist of the following steps:

1.

Pre-train a source model: Select a source model that is similar to the target model, then train it on large-scale data.

2.

Extract source model features: Extract useful features from the source model, such as class traits, similarity traits, etc.

3.

Generate adversarial examples: Based on the extracted features, generate adversarial examples capable of fooling the target model.

4.

Verify attack effectiveness on target model: Input the generated adversarial examples into the target model to verify attack effectiveness and success rate.

Transfer learning-based adversarial attacks have been widely applied in many fields, including image recognition, speech recognition, natural language processing, etc.

Related papers include:

“Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples” This paper mainly studies the transferability phenomenon in black-box adversarial attacks based on transfer learning and proposes a decision tree-based attack method [9].

“Practical Black-Box Attacks against Machine Learning” The method proposed in this paper is based on transfer learning and optimization algorithms by minimizing the distance between the adversarial examples and the target model to generate adversarial examples [10].

7.3.4 Meta-Model Based Black-Box Attacks

Meta-model based attacks are an emerging adversarial attack method highly suitable for black-box adversarial attacks. The core idea is to build a meta-model based on the original model to generate more effective adversarial examples. Compared with traditional adversarial attack methods, meta-model based adversarial attacks are more efficient, requiring fewer adversarial examples while achieving a better attack success rate and sample quality.

Meta-model based adversarial attacks can be divided into the following steps:

1.

Train a source model: Choose a source model similar to the target model and train it on large-scale data.

2.

Build meta-model: Generate a meta-model from the source model to attack the target model. The meta-model is typically a fast, low-complexity model that can quickly generate adversarial examples.

3.

Generate adversarial examples: Use the meta-model to attack the target model and generate adversarial examples. In this process, the meta-model typically automatically learns features of the target model in order to generate more effective adversarial examples.

4.

Verify attack effectiveness on target model: Input the generated adversarial examples into the target model to verify attack effectiveness and success rate.

Meta-model based adversarial attacks have been widely applied in computer vision, speech recognition, natural language processing, and other fields, achieving excellent results in some competitions.

Related papers include:

“Meta-Attackers: Defending Against Adversarial Examples Using Meta-Adversarial Training” This paper aims to defend against adversarial examples using meta-adversarial training.

“Meta-Transfer Learning for Few-Shot Learning” This paper proposes a method called Meta-Transfer Learning, aimed at leveraging knowledge from existing large-scale high-dimensional datasets to improve performance on few-shot learning tasks.

7.3.5 Query-Based Attacks

Query-based attacks are a type of attack method against black-box models. They do not require access to the model’s internal parameters and structure but only obtain the model’s prediction results by inputting some queries (input-output pairs), in order to construct adversarial examples.

The basic idea of query-based attacks is that by continuously querying the model’s prediction results, sufficient information can be obtained to construct high-quality adversarial examples. The main advantage of this attack method is that it does not require access to the target model’s internal information, so it can attack any black-box model. However, it requires a large number of queries, making the attack efficiency relatively low.

Some query-based attack methods include:

Boundary Attack: Proposed in the 2018 paper “Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models” by Brendel et al., boundary attack is a query-based adversarial attack method that aims to generate adversarial examples capable of fooling machine learning models with efficient query complexity [11].

ZOO: Proposed in the 2017 paper “ZOO: Zeroth Order Optimization-Based Black-box Attacks to Deep Neural Networks Without Training Substitute Models” by Chen et al., Zeroth Order Optimization (ZOO) introduces a black-box attack method based on zeroth order optimization, for adversarial attacks against deep neural networks without training substitute models [12].

7.3.6 Optimization-Based Attacks

Optimization-based attacks refer to attack methods that use optimization algorithms to construct adversarial examples by optimizing an objective function. These types of attacks typically require access to the model’s gradient information and tweak adversarial examples through optimization algorithms to fool the target model and cause incorrect predictions.

Optimization-based attacks can be divided into single-step attacks and iterative attacks. In recent years, optimization-based attacks have become one of the most popular and widely used attack methods in the field of adversarial attacks and are extensively applied in image classification, object detection, speech recognition, and other domains [13].

7.4 White-Box Attacks

White-box attacks refer to attacks where the attacker has full access to the target model’s internal structure and parameters and can precisely calculate the model’s gradient information to construct high-quality adversarial examples. This type of attack is more challenging compared to black-box attacks because the attacker needs to deeply understand the target model’s internal structure and parameters in order to conduct effective attacks.

White-box attacks can be divided into two categories: optimization-based attacks and gradient-based attacks.

7.4.1 Optimization-Based Attacks

Optimization-based attacks refer to attacks where the attacker searches for adversarial examples that can make the target model output incorrect results within a certain search space. This type of attack usually requires optimization algorithms like L-BFGS, Adam, Newton, etc. Optimization-based attacks can generate high-quality adversarial examples but have high requirements on computational resources and require a thorough understanding of the target model’s structure and parameters.

7.4.1.1 C&W Attack

Proposed by Nicolas Carlini and David Wagner in 2017, C&W (Carlini and Wagner) is an adversarial attack method designed to fool deep neural network models with high effectiveness and broad applicability.

The main goal of C&W attacks is to generate adversarial examples with minimal perturbations that cause incorrect classification by the targeted model. Compared to other adversarial attacks, the C&W method uses a more complex optimization process to improve the attack success rate [14].

The key steps and principles of C&W attacks are:

1.

Choose target model and input data: The attacker selects a target deep neural network model and an input sample to attack.

2.

Define loss functions: C&W uses two important loss functions - the deception loss and the regularization term.

Deception loss: This function aims to cause the adversarial example to be misclassified as a selected target class by the model. Specifically, it minimizes the output probability of the target class.

Regularization term: To control the magnitude of the generated perturbations, C&W introduces a regularization term. The regularization can take various forms, such as the L2 norm or the L0 norm.

3.

Optimization process: C&W uses an iterative optimization process to generate adversarial examples. Specifically, it employs a method called “binary search optimization.”

Initialize perturbation: The attacker initializes a perturbation for the input sample, usually setting it to zero or a small random value.

Iterative optimization: In each iteration, the attacker performs steps like optimizing perturbation, binary search, and repeating iterations until satisfied.

4.

Generate adversarial example: Add the resulting optimized perturbation to the original input sample to obtain the adversarial example.

By using a complex optimization process and introducing regularization terms, C&W is able to generate adversarial examples with smaller perturbations and achieve better attack success rates. However, C&W has a high computational cost and requires multiple iterations of optimization.

7.4.1.2 PGD Attack

Projected Gradient Descent (PGD) is an optimization algorithm used for adversarial attacks and defense. It is an iterative, gradient-based method used to generate adversarial examples or perform adversarial training.

The main goal of PGD is to minimize an objective function subject to given constraint conditions. In adversarial attacks, the objective function is usually a loss function designed to maximize misclassification or minimize confidence of the attack target [15].

The key steps and principles of the PGD algorithm are:

1.

Initialization: Select an initial input sample as the starting point of PGD.

2.

Iterative optimization: In each iteration, compute the gradient, update the input sample, and project the updated sample to satisfy constraints. Repeat until satisfied.

3.

Generate adversarial examples or update model parameters: Depending on the app, PGD can be used to generate adversarial examples or perform adversarial training.

The PGD algorithm minimizes the objective function by iteratively applying gradient descent and uses projection operations to ensure the generated examples satisfy constraints. This iterative optimization process enables PGD to fine-tune the input sample at each iteration, gradually moving closer to the adversarial goal or improving model robustness.

Several factors affect the performance and efficacy of PGD algorithms, including the number of iterations, learning rate, constraint conditions, and choice of objective function. These factors need to be carefully balanced when applying PGD algorithms. In addition, understanding PGD principles and corresponding defenses is crucial for defenders to improve model robustness and security.

7.4.2 Gradient-Based Attacks

Gradient-based attacks are a type of common white-box attack method against deep learning models. Such methods typically leverage the gradient information of neural networks to construct adversarial examples, so that the adversarial examples can fool the neural network into making incorrect classification results.

The main idea of gradient-based attack methods is to incorporate an adversarial loss into the loss function of the target model, making the model sensitive to adversarial examples. In the process of crafting adversarial examples, the attacker needs to use backpropagation to calculate the gradient information of the target model and update the adversarial examples according to the gradient information. Through multiple iterations of this process, the attacker can progressively optimize the adversarial examples so that they can cause misclassification effects on the target model [16].

Some examples of gradient-based attack methods include FGSM, BIM, MI-FGSM, etc. The main differences between these methods lie in the gradient update schemes, number of iterations, choice of loss functions, and so on. Gradient-based attack methods have been applied in multiple domains such as computer vision, natural language processing, etc.

7.4.2.1 FGSM Attack

Proposed by Ian Goodfellow et al. in 2014, the Fast Gradient Sign Method (FGSM) is a method used for adversarial attacks on machine learning models.

The target of FGSM is to add perturbations with a specific sign to the input value, causing the machine learning model to produce incorrect predictions. It utilizes the model’s gradient information to calculate the perturbation, nudging the input data slightly so as to cause misclassification.

The detailed steps of FGSM are:

1.

Select target model and input data: The attacker needs to select a target model, which will be the object of the attack. At the same time, select an input data sample, which is the image the attacker wants to modify.

2.

Calculate gradient: For the given input data sample, the attacker needs to calculate the gradient of the loss function of the target model with respect to the input data. The gradient indicates the sensitivity of the target model to the input data.

3.

Determine perturbation direction: Using the computed gradient information, the attacker decides in which direction to tweak the input data in order to maximally increase the loss. The sign of the gradient (positive or negative) indicates the direction of the maximal increase in loss.

4.

Control perturbation magnitude: The attacker uses a hyperparameter called epsilon to control the magnitude of the perturbation. Epsilon denotes the maximum magnitude of perturbation that is added to the original input data. The attacker can tune the epsilon value as needed to balance the attack effectiveness and perceptibility of the perturbation.

5.

Add perturbation: According to the determined direction and perturbation magnitude, the attacker adds the perturbation to the original input data to obtain the perturbed input data.

6.

Evaluate model output: Using the perturbed input data, the attacker can re-evaluate the target model’s output. Typically, the perturbed data will cause the model to produce incorrect classification results.

The key idea of FGSM is to add a small perturbation with a predefined sign to the input data, so as to alter the model’s prediction. It utilizes the model’s gradient information to guide the generation of the perturbation, in order to maximally affect the model’s output.

Note that FGSM is a gradient-based attack method and may be less effective for some defensive mechanisms such as adversarial training or gradient masking. In addition, adversarial examples generated by FGSM may only be effective for the target model, lacking universality. Therefore, the characteristics of the model and app scenario need to be considered when researching and applying FGSM. In summary, FGSM is a simple yet effective adversarial attack approach that can be used to evaluate and improve model robustness and security [6].

7.4.2.2 BIM Attack

BIM is an adversarial attack method that is an extension of Fast Gradient Sign Method (FGSM). BIM stands for “Basic Iterative Method”, also known as “Projected Gradient Descent.” BIM improves upon FGSM by applying FGSM iteratively to generate stronger adversarial examples.

The detailed steps of BIM are:

1.

Select target model and input data: Same as FGSM, the attacker needs to select a target model and an input data sample to attack.

2.

Initialize perturbation: The attacker initializes a perturbation for the input data sample, usually setting it to zero or a small random value.

3.

Iterate to generate perturbation: For each iteration, the attacker performs steps like calculating gradient, determining direction, controlling magnitude, updating perturbation, and clipping.

4.

Repeat iteration: Repeat the iterative steps until a preset number of iterations or some other stopping criterion is reached.

5.

Generate adversarial sample: By adding the resulting final perturbation to the original input sample, generate the adversarial sample.

Compared to FGSM, BIM generates adversarial samples by iteratively adjusting the perturbation, so it can more finely control the direction and magnitude of the perturbation at each iteration. This allows BIM to generate more powerful adversarial examples that can more effectively fool the target model. However, BIM may require more computational resources and time since it involves multiple iterative computations. Nevertheless, it typically achieves stronger attack effectiveness than FGSM [17].

7.4.3 App of Adversarial Attacks in Malware Detection

Adversarial attacks are widely applied in malware detection. Malware detection refers to using computer security techniques to detect and identify malware in order to protect computer systems and network security. However, malware authors can use adversarial attacks to evade malware detection, preventing detection algorithms from accurately identifying and blocking malware.

Adversarial attacks can be divided into white-box attacks and black-box attacks. In white-box attacks, the attacker has full knowledge of the detection algorithm’s internal structure and parameters and thus can use various adversarial attack techniques to fool the detection algorithm. In black-box attacks, the attacker only knows the input and output of the detection algorithm and therefore needs to use black-box optimization algorithms to generate adversarial examples to deceive the detection algorithm.

Apps of adversarial attacks in malware detection mainly include:

Adversarial example generation: Attackers can use adversarial attacks to generate adversarial examples, enabling malware to evade detection algorithms and run on victim computers.

Assessing model robustness: Researchers can use adversarial attacks to evaluate the robustness of malware detection algorithms and discover and fix vulnerabilities and flaws in the detection algorithms.

Adversarial training: Adversarial training refers to using adversarial examples to train malware detection models to improve model robustness and resilience against attacks.

In summary, adversarial attacks have extensive apps in malware detection and are very important for improving the robustness and security of malware detection algorithms.

Recent related papers include:

“Adversarial Example Generation with Syntactically Controlled Paraphrase Networks”

This paper introduces a method to generate adversarial examples using syntactically controlled paraphrase networks. In traditional adversarial example generation, optimization algorithms are often used to minimize the loss between the input sample and target model to generate adversarial examples. However, this approach often leads to large semantic changes in adversarial examples, limiting their applicability in real apps. This paper proposes a new method that uses syntactically controlled paraphrase networks to generate adversarial examples while maintaining semantic consistency [18].

The method includes:

1.

Build syntactic controller: Use pre-trained syntactic parsers to analyze the syntax of input samples and extract syntactic features.

2.

Train paraphrase network: The network takes original input sample, target label, and syntax features and generates adversarial examples syntactically similar to the original sample.

3.

Generate adversarial examples: Use optimization algorithms and paraphrase network to minimize the loss between adversarial examples and the target model, generating adversarial examples that fool the target model.

By introducing syntactic control during adversarial example generation, this method can generate syntactically correct and semantically similar adversarial examples with higher interpretability and applicability. Experiments prove its effectiveness on multiple tasks and datasets.

“Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN”

This paper introduces a method to generate adversarial malware examples to attack black-box machine learning models using Generative Adversarial Networks (GANs). In traditional black-box attacks, the attacker cannot directly access the model’s internal information. This paper’s method overcomes this limitation by using GANs to generate adversarial malware examples to fool the target model [19].

The method includes:

1.

Construct GAN with generator and discriminator.

2.

Train the GAN model using real malware samples and generate adversarial examples.

3.

Use a trained generator to take noise vectors and generate adversarial malware examples.

4.

Input generated examples into target model for the attack.

By using GANs, the method can generate malware examples similar to real ones but adversarial. These examples can fool the target model into misclassifying them. The advantage is it allows black-box attacks without accessing the target model’s internal information. Experiments show attack effectiveness and practicality.

7.5 Chapter Summary

Android malware detection faces significant challenges due to the use of adversarial examples. Researchers and security experts are striving to enhance the accuracy and resilience of Android malware detection systems. Malware developers are also adapting by leveraging adversarial examples to deceive machine learning models. Common techniques for generating adversarial examples include Fast Gradient Sign Method (FGSM), evolutionary algorithms, and GAN-based methods. These methods subtly alter malware samples to trick machine learning models into misclassifying them as benign. The generation process depends on factors like feature selection, perturbation intensity, and transferability.

Researchers have proposed defense techniques to counter adversarial examples. Adversarial training, which mixes adversarial and original samples for model training, improves model robustness. Detection methods monitor input and output spaces to identify potential adversarial attacks. Developing more robust machine learning models using techniques like ensemble learning, regularization, and adversarial training is crucial. Challenges persist in Android malware detection. Adversarial techniques are evolving, introducing new threats. Enhancing model robustness remains complex. Exploring multimodal adversarial examples, considering attacks on various data types (e.g., images, speech), is essential.

In summary, this chapter outlines adversarial example attacks in Android security. It covers their principles, apps, black-box and white-box methods, and key research papers. Understanding adversarial example generation, influencing factors, and defense technologies illuminates the ongoing competition between malware developers and security experts. Continued research and development are critical for improving Android malware detection’s accuracy and resilience against increasingly sophisticated threats.

References

1.

Alexey K. Adversarial examples in the physical world[J]. arXiv preprint arXiv:1607.02533, 2016.

2.

Sun Q, Liu Y, Chua T S, et al. Meta-transfer learning for few-shot learning[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 403–412.

3.

Pan W W, Wang X Y, Song M L, et al. A review of adversarial sample generation technology[J]. Journal of Software, 2019, 31(1): 67–81.Crossref

4.

Wang Y, Du S, Balakrishnan S, et al. Stochastic zeroth-order optimization in high dimensions[C]//International conference on artificial intelligence and statistics. PMLR, 2018: 1356–1365.

5.

Tai J W, Li YK, Jia X Q, et al. Overview of Speech Recognition System Against Sample Attacks and Defense[J]. Journal of Cyber Security, 2022, 7(5).

6.

Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[J]. arXiv preprint arXiv:1412.6572, 2014.

7.

Zhang J, Li C. Adversarial examples: Opportunities and challenges[J]. IEEE transactions on neural networks and learning systems, 2019, 31(7): 2578–2593.MathSciNet

8.

Yuan X, He P, Zhu Q, et al. Adversarial examples: Attacks and defenses for deep learning[J]. IEEE transactions on neural networks and learning systems, 2019, 30(9): 2805–2824.MathSciNetCrossref

9.

Papernot N, McDaniel P, Goodfellow I. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016.

10.

Papernot N, McDaniel P, Goodfellow I, et al. Practical black-box attacks against machine learning[C]//Proceedings of the 2017 ACM on Asia conference on computer and communications security. 2017: 506–519.

11.

Brendel W, Rauber J, Bethge M. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models[J]. arXiv preprint arXiv:1712.04248, 2017.

12.

Chen P Y, Zhang H, Sharma Y, et al. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models[C]//Proceedings of the 10th ACM workshop on artificial intelligence and security. 2017: 15–2.

13.

Cheng M, Le T, Chen P Y, et al. Query-efficient hard-label black-box attack: An optimization-based approach[J]. arXiv preprint arXiv:1807.04457, 2018.

14.

Carlini N, Wagner D. Towards evaluating the robustness of neural networks[C]//2017 ieee symposium on security and privacy (sp). Ieee, 2017: 39–57.

15.

Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint arXiv:1706.06083, 2017.

16.

Dong Y, Liao F, Pang T, et al. Boosting adversarial attacks with momentum[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 9185–9193.

17.

Kurakin A, Goodfellow I, Bengio S. Adversarial machine learning at scale[J]. arXiv preprint arXiv:1611.01236, 2016.

18.

Iyyer M, Wieting J, Gimpel K, et al. Adversarial example generation with syntactically controlled paraphrase networks[J].arXiv preprint arXiv:1804.06059, 2018.

19.

Hu W, Tan Y. Generating adversarial malware examples for black-box attacks based on GAN[C]//International Conference on Data Mining and Big Data. Singapore: Springer Nature Singapore, 2022: 409–423.

Part IVThe Future Trends of Android Malware Detection

In order to effectively protect the security of users, Android malware detection techniques are constantly developing and evolving. These techniques are able to detect and identify Android malware more accurately and efficiently by employing a variety of different technical means, such as static analysis and dynamic analysis. In the previous chapters, we have introduced these commonly used Android malware detection techniques in detail.

However, Android malware detection is an important and increasingly complex field, and as technology continues to progress and evolve, Android malware detection techniques need to be constantly updated and improved to adapt to changing security threats. As the popularity of Android systems and the number of malware grows, we need to constantly update and improve our detection techniques to better cope with changing security threats. At the same time, we also need to keep an eye on the development of emerging technologies to better safeguard user security and privacy in the future.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

W. Niu et al.Android Malware Detection and Adversarial Methodshttps://doi.org/10.1007/978-981-97-1459-9_8

8. Future Trends in Android Malware Detection

Weina Niu1  , Xiaosong Zhang¹, Ran Yan¹ and Jiacheng Gong¹

(1)

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

To effectively protect users’ security, the field of Android malware detection is constantly evolving. Various technical means, such as static analysis and dynamic analysis, are employed to accurately and efficiently detect and identify Android malware. In the previous chapters, we discussed these commonly used techniques in detail. However, as technology advances, the complexity of Android malware detection increases, requiring constant updates and improvements to adapt to evolving security threats. With the growing popularity of Android systems and the increasing number of malware, it is crucial to continuously enhance our detection techniques to effectively address changing security threats. Additionally, we must also monitor the development of emerging technologies to safeguard user security and privacy in the future. In the upcoming chapters, we will delve into the future trends of Android malware detection technology and explore the implications and challenges posed by emerging technologies in this field. Our aim is to enhance user security and privacy. Some potential future technologies for Android malware detection include:

➀

Machine learning and deep learning techniques

➁

Integrated solutions

➂

Blockchain technology

➃

Involvement of hardware technology

➄

BPF technology

8.1 Machine Learning And Deep Learning Techniques

In this section, we will delve into the fascinating realm of Android malware detection and its connection to machine learning and deep learning. We will begin by outlining the pivotal role and significance of these cutting-edge technologies in this field. Following that, we will explore the various challenges that machine learning and deep learning face when it comes to detecting Android malware. By thoroughly examining these aspects, we will gain a deeper appreciation for the crucial role that machine learning and deep learning techniques play in safeguarding Android devices from malicious software.

8.1.1 Overview of Machine Learning and Deep Learning for Android Malware Detection

Machine learning and deep learning techniques play a crucial role in the detection of Android malware, making them indispensable for the future. These technologies are able to identify and predict malware behavior more accurately by learning and understanding the behavioral patterns and features of malware.

Machine learning techniques can learn the features and behavioral patterns of malware by training models to achieve automatic detection and identification of malware. This method can greatly improve the efficiency and accuracy of malware detection and reduce the need for human intervention. Deep learning technology, on the other hand, can deeply learn and understand the complex behavior and hidden features of malware through deep neural network models. This approach can effectively identify complex and hidden malware and improve the depth and breadth of malware detection.

For example, Machine Learning-aided Android Malware Classification [1] proposes two machine learning-aided methods for static analysis of Android malware. The first approach is based on permissions and the other on source code analysis using bag-of-words representation models. A Machine Learning Approach to Android Malware Detection [2] extracts a number of features and trains a class of support vector machines in an offline (off-device) manner in order to utilize the higher computational power of a server or a cluster of servers.

As of now, artificial intelligence techniques such as machine learning and deep learning have been widely used for Android malicious code detection, and with the continuous development of artificial intelligence and machine learning techniques, the future of Android malware detection will be more dependent on these techniques. By training a large number of data samples from which malware features are extracted, including static features (e.g., file size, permission request, etc.) and dynamic features (e.g., runtime behavior, network communication, etc.), the machine learning model can automatically identify the features and behavioral patterns of the malware, thus improving the accuracy and efficiency of the detection [2]. Although there have been a large number of related research results showing that machine learning and deep learning have good results in the field of Android malware detection, there are still many problems in it.

8.1.2 Challenges Faced

In this section, we will explore the challenges faced by machine learning and deep learning in detecting Android malware. Currently, these challenges include insufficient datasets, adversarial attacks, interpretability issues, backdoor attacks, and more.

By examining these challenges, we can gain a better understanding of the limitations of machine learning and deep learning in Android malware detection and work toward finding solutions to enhance the effectiveness and reliability of detection.

1.

Insufficient data sets

The quality and quantity of the dataset are crucial for machine learning models. Firstly, the dataset’s quality significantly impacts the accuracy of malware detection. If the dataset lacks sufficient or representative malware samples, the detection algorithm may miss detections or produce false alarms. Therefore, it is essential to ensure the dataset’s quality, including its source, labeling, and regular updates. Secondly, the dataset’s size greatly affects the efficiency of malware detection. If the dataset is too small, the detection algorithm may not cover all malware samples, resulting in missed detections. Conversely, if the dataset is too large, the training and testing time of the detection algorithm will be prolonged, potentially impacting real-time detection. Therefore, the size of the dataset needs to be selected according to the specific app scenario. In addition, the diversity of the dataset also has an impact on the effectiveness of malware detection. If the dataset contains only a certain type of malware, the detection algorithm may not be able to detect other types of malware. Therefore, the diversity of the dataset needs to be ensured, including aspects such as different types of malware, different sources of malware, and so on. And unlike domains such as images, many sample generation and enhancement methods applicable to other domains are not applicable on Android app datasets. The following datasets are currently commonly used for Android:

(a)

AndroZoo [3]: AndroZoo is a dataset of more than 1 million Android apps, including both malware and normal software. The dataset is maintained by researchers at McGill University, Canada.

(b)

Drebin [4]: Drebin is a dataset of more than 120,000 Android apps, including malware and normal software. The dataset is maintained by researchers at the Technical University of Braunschweig, Germany.

(c)

CICMalDroid [5]: The CICMalDroid app dataset contains over 4033 benign software (Benign), 1512 adware, 2467 banking malware, 3896 mobile riskware, and 4809 SMS malware. The dataset is maintained by researchers at the Canadian Institute for Cyber Security Research.

(d)

VirusShare [6]: VirusShare is a dataset of over 500,000 malware samples, including Android malware and other malware. The dataset is maintained by researchers and security experts.

(e)

Contagio [7]: Contagio is a dataset containing over 100,000 malware samples, including Android malware and other malware. The dataset is maintained by both researchers and security experts.

2.

Against attacks

The uncertainty inherent in machine learning models makes them vulnerable to adversarial attacks [8]. An adversarial attack is when an attacker deceives a machine learning model by making minor modifications to the input data so that it produces incorrect output results. The impact of adversarial attacks on machine learning models is very serious, mainly in the following aspects:

First, adversarial attacks can reduce the accuracy of machine learning models. An attacker has the ability to manipulate the input data in such a way that it is misclassified into different categories or recognized as normal data when it is actually malicious. This can result in a decrease in the accuracy of the model, which in turn affects its effectiveness in practical apps.

Furthermore, adversarial attacks can raise the false alarm rate of machine learning models. The false alarm rate represents the proportion of normal data that is incorrectly identified as malicious. By making slight alterations to normal data, an attacker can increase the false alarm rate, thereby diminishing the reliability of the model and impacting its effectiveness in real-world scenarios.

In addition, adversarial attacks can undermine the robustness of machine learning models. Robustness refers to the model’s ability to withstand errors in the input data. Through minor modifications to the input data, an attacker can undermine the model’s detection mechanisms, resulting in reduced stability. This reduction in stability can have a detrimental effect on the model’s effectiveness in practical apps.

Lastly, adversarial attacks pose a threat to the security of machine learning models. An attacker can compromise the security of a model by introducing malicious code or other dangerous elements into the input data. This highlights the potential risks associated with adversarial attacks and their impact on the security of machine learning models. This can lead to a decrease in the security of the model, thus affecting the effectiveness of the model’s app [9]. In the field of Android malicious code detection, attackers can often also consciously create adversarial samples as a way to bypass the defense of the Android malicious code detection system.

3.

Interpretability issues

In the field of Android malicious code detection, the interpretability of machine learning models is very important. Due to the black-box nature of machine learning models, it is often difficult to explain why a particular app is classified as malicious. If a machine learning model is unable to explain the reasons and rationale for its output results, it is difficult for users to understand and trust the model’s output results [10]. This can lead to users’ skepticism about the effectiveness of the model’s app, which in turn affects the effectiveness of the model’s app. Secondly, the interpretability of machine learning models is very important for the control of false alarm rate of malware detection. The false alarm rate refers to the percentage of normal data that is mistakenly identified as malicious. When a machine learning model cannot provide an explanation for its output results, it may have a high false alarm rate. This can undermine the reliability of the model, which in turn affects its effectiveness in practical apps. The interpretability of machine learning models is crucial for optimizing malware detection. If a machine learning model cannot explain the reasoning behind its output results, it becomes challenging to optimize the model. Consequently, the model’s performance may suffer, impacting its effectiveness in practical apps. Moreover, the interpretability of machine learning models is essential for ensuring the security of malware detection. If a machine learning model cannot explain the basis for its output results, attackers may exploit its vulnerabilities to carry out malicious attacks. This compromises the security of the model and diminishes its effectiveness in practical apps [11].

4.

Backdoor attack

In the context of machine learning, the backdoor problem refers to the ability of attackers to manipulate a machine learning model by inserting malicious code. This can result in the generation of false outputs under specific conditions [12]. In the case of Android malware detection, the backdoor problem can significantly reduce the effectiveness of the machine learning model, thereby impacting the accuracy and reliability of malware detection. Specifically, if an attacker successfully implants a backdoor, the machine learning model may produce incorrect output results in certain situations. This can lead to an increase in false positives or missed positives during malware detection, thereby reducing the accuracy and reliability of the model. Consequently, the effectiveness of malware detection apps is compromised. Furthermore, the backdoor problem can also compromise the privacy of machine learning models. If an attacker manages to implant a backdoor, they can exploit it to gain unauthorized access to the model’s private information. This jeopardizes the privacy of the model and diminishes its reliability, thereby affecting its practical app.

8.2 Integrated Solutions

Integrated solutions require combining multiple detection methods [13] to achieve comprehensive detection of malware. For example, An Integrated Static Detection and Analysis Framework for Android [14] proposes an integrated static detection framework that consists of a four-layer filtering mechanism, i.e., Message Digest (MD5) values, combinations of malicious permissions, dangerous permissions, and dangerous intentions. An intuitive threat level model is proposed, especially for dangerous permission detection.

The integrated methods include static analysis, dynamic analysis, machine learning, and artificial intelligence based detection, signature-based detection, behavioral based detection, sandbox based detection, cloud-based technology detection, and other approaches. Each method has its advantages and disadvantages, and combining them can improve the accuracy and efficiency of detection.

8.2.1 Challenges Faced

Integrated systems face several challenges in practice, which cover technology convergence, data sharing and collaboration, systematic optimization, adaptive capabilities, privacy protection, scalability, real-time performance, and maintenance and updating. By overcoming these challenges, integrated systems can achieve more efficient, reliable, and secure operations and provide better services and experiences for users.

1.

Technological convergence

Integrated solutions need to effectively combine different malware detection techniques (e.g., static analysis, dynamic analysis, sandboxing techniques, machine learning, cloud-based techniques, etc.). This requires an in-depth study of the various technologies to understand their strengths and weaknesses. There is technical heterogeneity among different malware detection technologies, including differences in technical principles, data formats, and algorithm implementations. This makes technology integration more difficult, and it is necessary to conduct in-depth research on various technologies to understand their strengths and weaknesses, so as to give full play to their respective advantages and make up for the shortcomings in the integration process. Technology integration also requires the establishment of a unified evaluation standard so that different detection technologies can be compared and evaluated in order to select a more suitable detection technology, and therefore the characteristics and advantages of different technologies need to be taken into account in order to establish a fair, objective, and comprehensive evaluation standard.

2.

Data sharing and collaboration

In order to achieve an efficient and integrated solution, data sharing and collaborative work between the various inspection modules is required. This means that an exchange and communication mechanism needs to be designed so that modules can access each other and utilize data generated by other modules. Different malware detection techniques may use different data formats, which makes data sharing and collaboration more difficult. To solve this problem, a generic data format needs to be designed so that modules can access each other and utilize the data generated by other modules. In the process of sharing and collaborating, data security needs to be considered. Malware detection involves sensitive data such as malicious code, traffic, etc., which need to be protected from being stolen or tampered with by attackers. Therefore, some security measures such as data encryption and identity are needed to ensure data security. In the process of data sharing and collaboration, data access rights need to be considered. Different technology modules may need to access different data, so an access control mechanism needs to be designed in order to control the access rights of each module to the data. In the process of data sharing and collaboration, the management of data volume needs to be considered. Malware detection involves a large amount of data, such as malicious code libraries, network traffic data, etc., which need to be effectively managed and stored so that each module can access and utilize the data efficiently [15].

3.

Systematic optimization

Integrated solutions may increase the complexity of the system and thus affect performance. Therefore, performance optimization needs to be considered when designing an integration solution to ensure that the system can maintain efficient operation even when handling a large number of malware detection tasks. The complexity of the system needs to be considered when designing an integration solution so that the system architecture can be simplified as much as possible to reduce the complexity of the system during the integration process. Integration techniques may affect the performance of the system and thus the accuracy and reliability of malware detection. Different technology modules may need to handle a large amount of data, such as malicious code bases, network traffic data, etc., which need to be processed efficiently to ensure that the system can keep running efficiently while handling a large number of malware detection tasks. Therefore, the performance of the system needs to be considered when designing the integration solution so that the integration process will maximize the performance of the system. Integration techniques may also affect the resource management of the system, which may affect the accuracy and reliability of malware detection. Therefore, when designing an integration scheme, the resource management of the system needs to be considered and the system redundant resources need to be designed so that the resource utilization of the system can be optimized as much as possible during the integration process to ensure that the system can utilize the limited resources efficiently [15].

4.

Adaptive

As malware characteristics and behaviors are constantly changing, integrated solutions need to be self-responsive so that they can quickly adapt their detection strategies in the face of new types of malware. This may require the introduction of machine learning and artificial intelligence techniques that enable the system to automatically learn and adapt to new malware characteristics. First, the self-adaptive capability requires the ability to sense and learn. Perception refers to the ability of a system or an individual to perceive changes in the environment and context, and learning refers to the ability of a system or an individual to learn new knowledge and experiences from the environment and context. Perception and learning are the basis for realizing adaptive capability, but perception and learning themselves face some difficulties, such as the accuracy of perception, the efficiency of learning, and generalization ability. Secondly, adaptive ability needs to have the ability for decision-making and execution. Decision-making ability means that the system or individual can formulate corresponding decision-making strategies according to the results of perception and learning, and execution ability means that the system or individual can effectively execute the decision-making strategies. Decision-making and execution are the key to realizing adaptive capability, but decision-making and execution itself also face some difficulties, such as the accuracy of decision-making, and the efficiency and stability of execution. Third, adaptive capability needs to have the ability to collaborate and interaction. Synergy and interaction refer to the ability of a system or an individual to effectively synergize and interact with other systems or individuals in order to effectively improve adaptive capability. Collaboration and interaction is an important means to realize adaptive capability, but collaboration and interaction itself also face some difficulties, such as efficiency, stability, and security of collaboration and interaction. Fourth, adaptive capability needs to have the ability of interpretability and credibility. Interpretability refers to the ability of a system or an individual to explain and illustrate its decisions and behaviors so that users and other systems or individuals can understand and trust their own behaviors and decisions. Credibility refers to the ability of a system or individual to gain the trust and recognition of users and other systems or individuals. Interpretability and trustworthiness are important guarantees for realizing adaptive capability, but interpretability and trustworthiness themselves also face some difficulties, such as the accuracy of explanation and illustration, the establishment of trust and recognition, and so on.

5.

Privacy

User privacy protection needs to be considered in the implementation of an integrated solution. Designing a mechanism for malware detection without disclosing user data ensures that user privacy is adequately protected. Privacy protection needs to consider the security and trustworthiness of the data. The security and trustworthiness of data is the foundation of privacy protection but the security and trustworthiness of data also face some difficulties, such as data leakage, data tampering, and data loss. In order to ensure security and trustworthiness, it is necessary to use some effective technical means, such as data encryption, data backup, data recovery, and so on.

6.

Scalability

As malware detection technologies continue to evolve, integrated solutions need to be scalable in order to introduce new detection techniques and methods in the future. This means that a flexible architecture needs to be designed to allow for expansion without compromising existing functionality. First, scalability requires consideration of the architecture and design of the system. The architecture and design of the system are the basis for realizing extensibility. A well-designed system architecture can enhance the flexibility, scalability, and maintainability of a system. However, there are challenges in achieving an optimal balance between performance and extensibility, as well as managing the complexity of the system. Another consideration for scalability is the impact on system performance and resource utilization when the number of users and data volume increases. To address this, effective technical measures such as load balancing, caching, and distributed computing can be employed. Additionally, scalability involves managing and storing large-scale data, which can be achieved through techniques like distributed storage, data slicing, and data compression. Ensuring the security and reliability of the system is also crucial as the user base and data volume grow. Measures such as data backup, fault-tolerance mechanisms, and security authentication can be implemented to address these concerns.

7.

Topicality

To effectively combat the ever-evolving threat of sophisticated malware, it is crucial for integrated solutions to possess real-time detection capabilities. Achieving this may necessitate harnessing the power of cloud computing resources, which enable efficient operations in large-scale data processing and real-time detection tasks. First and foremost, achieving real-time detection requires careful consideration of the system’s response time. The response time serves as the foundation for achieving real-time functionality. A shorter response time translates to a more adaptable, dependable, and efficient system. However, optimizing the system’s response time poses certain challenges, such as striking the right balance between system performance and response time, as well as managing system complexity. Secondly, real-time functionality must take into account the system’s concurrency. As the number of users and data volume increases, the system’s concurrency faces significant pressure. Overcoming this challenge and achieving real-time concurrency is no easy feat. It requires the implementation of effective technical measures, such as multi-threading, asynchronous programming, and distributed computing. Thirdly, real-time functionality necessitates considering the system’s data processing capability. As data volume grows, the system’s ability to process data also faces substantial strain. Enhancing the system’s data processing capacity in real-time poses a formidable challenge. To address this, effective technical approaches such as data slicing, data compression, and data caching must be employed. Fourthly, real-time functionality must prioritize the security and reliability of the system. As the number of users and data volume escalates, ensuring the system’s security and reliability becomes increasingly challenging. Achieving real-time security and reliability is a complex endeavor. It requires the implementation of effective technical measures, such as data backup, fault-tolerant mechanisms, and security authentication. Cloud computing emerges as a promising solution to address the aforementioned challenges. By leveraging cloud computing, organizations can access efficient computing and storage resources, benefit from high availability and fault tolerance, and utilize distributed computing and data processing services to support real-time apps.

8.

Maintenance and updates

To ensure the integrated solution remains effective in detecting malware, regular maintenance and updates are necessary. This involves tasks such as updating malware signature libraries, optimizing detection algorithms, and addressing known issues. The ever-changing nature of malware presents a significant challenge in maintaining and updating the Android malware detection system. Malware mutations can enable it to evade detection, posing a threat to the system. To address this issue, constant updates and improvements to the detection algorithms are needed to adapt to malware mutations. Furthermore, Android malware detection techniques must maintain a comprehensive malware database to support detection and identification. Maintaining and updating this database is a crucial task that involves continuously collecting and analyzing new malware samples for inclusion. Additionally, the database requires optimization and cleaning to ensure its performance and accuracy. Moreover, Android malware detection technology must be compatible with various Android system versions and devices, necessitating consideration of system compatibility and stability. To ensure the system functions properly in different environments, thorough testing and optimization are essential. All of the aforementioned points require substantial staffing and financial support.

8.3 Blockchain Technology

In this section, we will introduce Blockchain technology and its app in the field of Android malware detection. First, we will briefly introduce the basic principles and characteristics of Blockchain technology. Then, we will explore the app examples of Blockchain technology in the field of Android malware detection. These examples demonstrate the potential apps of Blockchain technology in the field of Android malware detection. However, it is important to note that Blockchain technology still faces some challenges in practical apps, such as performance, scalability, and privacy protection. Therefore, when applying Blockchain technology to Android malware detection, the advantages and limitations of the technology need to be comprehensively considered and rationally designed and implemented.

8.3.1 Introduction to Blockchain Technology

Blockchain technology is a distributed database technology that ensures data security and integrity through decentralization, encryption, and consensus mechanisms [16]. In recent years, Blockchain technology has been widely used in many fields, including finance, supply chain management, and the Internet of Things [17]. In the field of Android malware soft detection, Blockchain technology also has potential app value.

In the field of Android malware detection, data sharing, and collaboration are very important. Due to the variety of malware, each organization may only be able to collect some of the malicious soft samples, so data sharing among multiple organizations is needed to improve the efficiency and accuracy of malware detection. Blockchain technology can help enable the sharing and collaboration of Android malware detection data. By creating a decentralized malware detection database, individual participants can share malware samples, detection methods, and analysis results. It helps to improve the efficiency and accuracy of malware detection while reducing the cost of investment in malware detection for individual organizations.

Decentralized data storage refers to storing data in a distributed network rather than centrally on a single centralized server. In this type of storage, data is stored decentralized on multiple nodes in the network and each node can access and process the data. In Blockchain technology, decentralized data storage is achieved through a distributed ledger. Each node has a complete copy of the ledger and all the copies of the ledger are independent of each other, but its consistency is ensured through a consensus mechanism. When a new transaction is submitted to the network, all nodes verify the legitimacy of this transaction and add it to their ledger. When most of the nodes agree on the transaction, the transaction is confirmed and added to the ledger of the entire network. In decentralized data storage, data is usually divided into multiple blocks and stored on different nodes. Each block has a unique identifier by which the data can be found and accessed. When the data needs to be accessed, the client sends requests to multiple nodes in the network to get copies of the data blocks and combine them into complete data. Since the data is stored on multiple nodes, the client can get the data from the nearest node, thus increasing the speed and efficiency of data access [18]. This storage method has the following advantages:

➀

Data is stored on multiple nodes and there is no single centralized server, so there is no single point of failure and no single organization or individual in control of all the data.

➁

Since the data is stored on multiple nodes, attackers need to attack multiple nodes to tamper with the data, so the data is more secure.

➂

Since the data is stored on multiple nodes, more nodes can be added as needed to expand storage capacity and processing power.

In Blockchain technology, data integrity and traceability is achieved through consensus mechanism and distributed ledger. When a new transaction is submitted to the network, all the nodes verify the legitimacy of this transaction and add it to their ledger. When most of the nodes agree to the transaction, the transaction is confirmed and added to the ledger of the entire network. This process ensures data integrity as no single node can tamper with a transaction that has been confirmed. In addition, distributed ledgers allow for data traceability. Each transaction has a unique identifier that can be used to trace the origin and history of this transaction. Since all nodes have a complete copy of the ledger, the source and history of the data can be verified by comparing the ledgers of different nodes. This traceability can help prevent data from being tampered with or falsified, thus increasing the trustworthiness and security of the data.

In addition, in Android malware detection, a large amount of user data is involved, such as the user’s address book, text messages, location, and so on. These data need to be protected from being stolen or leaked by malware. Blockchain technology can protect data privacy by using smart contracts to control access to data. A smart contract is an automatically executed contract that can be written and executed on the Blockchain [19]. Through smart contracts, access control to data can be realized so that only users who meet specific conditions can access the data. The user’s privacy and data security are greatly ensured. The security of smart contracts is an important difficulty. Smart contracts may have loopholes that lead to tampering with malware detection results or data leakage. To ensure the security of smart contracts, strict security auditing and testing of smart contracts are required. In addition, techniques such as formal verification can be used to ensure the correctness of smart contracts. The execution efficiency of smart contracts is also a key issue. Malware detection needs to be performed in real-time, so the execution speed of smart contracts has a great impact on the detection effect. In order to improve the execution efficiency of smart contracts, techniques such as optimized compilers and parallel execution can be used.

When using Blockchain technology for Android malware detection, the performance problem of Blockchain needs to be considered [20]. The performance issues of Blockchain include transaction speed, storage capacity, network bandwidth, and other aspects. Therefore, when using Blockchain technology for Android malware detection, it is necessary to consider how to optimize the performance of the Blockchain to improve detection efficiency and accuracy. Transaction speed is a key indicator of Blockchain performance. In the scenario of Android malware detection, a large amount of data and transactions need to be processed in real-time. To improve transaction speed, techniques such as sharding and sidechaining can be used to decentralize transactions to multiple subchains for processing. In addition, more efficient consensus algorithms such as Proof of Stake (PoS) or Delegated Proof of Stake (DPoS) can be used to increase transaction confirmation speed. Storage capacity and network bandwidth are also important factors in Blockchain performance. In the scenario of Android malware detection, a large amount of user data and detection results need to be stored. In order to solve the problems of storage capacity and network bandwidth, data compression, data slicing, and other technologies can be used to reduce the storage space and transmission bandwidth requirements of data.

8.3.2 Examples of Blockchain Technology in the Field of Android Malware Detection

The field of applying Blockchain technology to detect Android malware is still emerging, and there have not been many research findings yet. However, a few researchers have proposed methods for Blockchain-based malware detection. For instance, in a paper titled “Collective Intelligence: Decentralized Learning for Android Malware Detection in IoT with Blockchain”[21], a new approach is presented that combines Blockchain and multi-level deep learning models to detect malware activities in real-time environments, particularly on Android IoT devices. Here is how the proposed approach works: ➀ The developer creates a dataset containing malware. ➁ The deep learning models with multiple layers are categorized into different clusters based on the characteristics of the distributed malware. The most suitable deep learning model is then chosen for each cluster. ➂ Decisions are made by analyzing the data that has been previously stored in the distributed ledger of the Blockchain. Additionally, new features related to malware activities are stored in the Blockchain. ➃ Finally, during the process of uploading or downloading an Android app, the Blockchain smart contract sends notifications to the user to validate the app for malware. To enhance security and detect malware on IoT devices in real-time, the authors collected a vast number of Android app features (both malware and benign) stored in the Blockchain database. They designed a multi-layer deep learning model specifically for these features, which are received in large quantities on Android IoT devices. The model supports multi-level clustering of a single data distribution. Through smart contracts, malicious apps are verified when uploaded or downloaded over the network. This system can either approve or reject harmful uploads and downloads of Android apps. The model is highly effective in identifying malware, thereby providing enhanced network security.

The operational steps of the architecture are as follows.

1.

User uploads App to the network.

2.

The app version is saved in the Interplanetary File System (IPFS).

3.

A deep learning model extracts features from the uploaded app to determine its nature.

4.

Updated deep learning models are stored in IPFS to reduce Blockchain costs. Model weights are aggregated to lessen computational burden on Blockchain nodes.

5.

Hash values of the app and decision results from the deep learning model are stored in the distributed ledger.

6.

During the app download process:

(i)

The user sends the hash value to the network.

(ii)

The smart contract compares and validates the hash values of the downloaded app.

(iii)

Finally, the user is notified of the app’s nature (malicious or benign).

8.4 Hardware Technology

This section discusses the advantages, challenges, and app examples of hardware technologies in the field of Android malware detection. By utilizing the advantages of hardware technology and overcoming the challenges faced by hardware technology, the effectiveness and performance of Android malware detection can be further improved. In the future, with the continuous development and innovation of hardware technology, we can expect more innovative hardware acceleration solutions in the field of Android malware detection.

8.4.1 Advantages of Hardware Technology

Android malicious code detection currently mainly relies on software means; in fact, hardware also has great potential to assist in the detection of malicious code [22]. Android malware detection methods based on hardware technology have the following advantages:

1.

Higher detection efficiency

Hardware techniques can utilize dedicated hardware gas pedals to accelerate the malware detection process, thereby improving detection efficiency. Compared with software-based methods, hardware technology can complete the malware detection task in a shorter time.

2.

Stronger anti-reverse analysis capability

Hardware technology can provide a higher level of security protection, making it difficult for attackers to obtain detailed information about malware detection algorithms through reverse analysis. This helps prevent attackers from conducting targeted attacks against the detection method, thus improving the accuracy of detection.

3.

Better isolation

Hardware technologies can isolate the malware detection process from other system resources, reducing the impact of malware on the system. This helps protect user data and system security from malware attacks that exploit system vulnerabilities.

4.

Lower false alarm rate

Malware detection methods that utilize hardware technology can employ hardware fingerprinting techniques and other approaches to enhance detection accuracy. This helps minimize false alarms and prevents legitimate apps from being mistakenly identified as malware.

5.

Greater adaptability

By upgrading hardware security modules and employing other methods, hardware-based malware detection can swiftly adapt to new attack techniques. This enhances the ability of detection methods to counter evolving security threats.

6.

Lower energy consumption

By utilizing dedicated hardware accelerators for malware detection, these methods consume less energy compared to software-based approaches. This not only reduces power consumption in mobile devices but also extends their lifespan.

8.4.2 Challenges to Hardware Technology

However, hardware technology also faces challenges in its development and app. Researchers are actively working to reduce the cost of hardware technologies, improve compatibility and scalability, and enhance security and privacy protection. Through continuous research and innovation, we can overcome these challenges and further advance the use of hardware technologies in Android malware detection.

1.

High cost

The introduction of hardware technology can increase the manufacturing cost of a device. To efficiently detect malware, additional hardware components like hardware gas pedals and security modules may be necessary. Consequently, the price of the device may rise, potentially impacting users’ willingness to purchase it.

2.

Compatibility issues

Compatibility issues may arise due to the diverse range of hardware technologies produced by different vendors. This can lead to inconsistencies in the performance of malware detection methods based on hardware technology across various devices, thereby affecting their effectiveness.

3.

Difficulty in updating

Updating hardware technology is more challenging compared to software-based methods. When new types of malware or attack methods emerge, physically replacing or upgrading the hardware device may be necessary, resulting in additional costs and time consumption.

4.

Scalability limitations

Hardware technologies are designed and manufactured with specific functionality and performance in mind. As a result, the scalability of hardware technology may be limited when there is a need to expand or enhance malware detection capabilities.

5.

Dependence on specific hardware

Malware detection methods based on hardware technologies may depend on specific hardware platforms. Consequently, these methods may not function properly on devices that do not support the particular hardware technology, thereby restricting their applicability.

6.

Privacy and Security Concerns

While hardware technology can offer a higher level of security, it is not without potential privacy and security risks. For example, hardware components may be subject to physical attacks, resulting in threats to the security of malware detection systems.

8.4.3 Examples of Hardware Technologies Applied in the Field of Android Malware Detection

For example, in the article NCScope: Hardware-Assisted Analyzer for Native Code in Android Apps [23], it is mentioned that Android devices use the arm architecture CPU, and the arm processor core provides a hardware module—the Embedded Trace Microcell (ETM). It is used to trace and record processor execution and system bus interactions with very low overhead. ETM can record instruction streams, data accesses, interrupts, exceptions, and processor state information of the processor execution, which can provide detailed debugging and optimization information for software developers to better understand and optimize the performance of the system. ETM consists of several modules, including instruction Trace Module, Data Trace Module, Branch Trace Module, and System Bus Trace Module, as shown in Table 8.1.

Table 8.1

The modules of ETM

ETM registers

Field

Description

TRCACVRO/TRCACVR1 (Address Comparator Value Registers)

ADDRESS

Set the virtual memory range for branch broadcast tracing

TRCBBCTLR (Branch Broadcast Control Register)

RANGE

Set the address range comparator pair for branch broadcast tracing

TRCCIDCCTLR0 (Context ID Comparator Control Register)

COMPO

Enforce Context ID comparison with relevant byte in TRCCIDCVR0

TRCCIDCVR0 (Context ID Comparator Value Register)

VALUE

Set PID to be compared with during Context ID tracing

TRCCONFIGR (Trace Configuration Register)

CID

Validate context ID tracing

BB

Validate broadcast tracing

TRCIDR0 (ID Register)

TRCBB

Examine whether branch broadcast tracing is supported

The instruction tracking module can record the instruction flow and the number of instruction cycles executed by the processor; the data tracking module can record the processor’s access to data, including reads, writes, and the type of access. The branching tracking module can record the branching instructions executed by the processor and the type of branching; the system bus tracking module can record the interaction information on the system bus. These modules can be configured and controlled through the programming interface to meet different tracking needs. These modules can be configured and controlled through programming interfaces to meet different tracing needs. These features can be used for dynamic analysis of Android apps. The advantage of using ETM for dynamic analysis of programs is that it can capture information about the processor instructions executed without affecting the processor’s performance and provide an uninterrupted stream of data. Since ETM can record detailed program execution and save it in real-time, it can be used to analyze program flow and execution time, perform analysis, and locate software errors. In the case of a runaway program counter, code tracing is valuable to find out why and how a “program crash” occurred. The debugger provides a user interface and stored trace data. The debugger is a powerful tool that allows for the utilization of all ETM features and provides a comprehensive display of the captured trace information. It is commonly known that Android malware employs tactics to conceal malicious code, which is often dynamically released during runtime. By employing ETM for command trace analysis, one can ensure that even the most elusive malicious commands are captured with minimal overhead.

In the paper titled “Happer: Unpacking Android Apps via a Hardware-Assisted Approach”[24], the authors introduce a novel method for dehulling Android apps using hardware assistance. They argue that traditional software-based approaches tend to be sluggish and ineffective, while hardware-assisted approaches offer superior performance and accuracy. The proposed method, known as Happer, leverages a hardware-based tracing mechanism to meticulously record the app’s execution and generate a trace file. This trace file is then used to reconstruct the code and data of the app, thus allowing it to be analyzed and modified.

Hardware-assisted methods are those that use a hardware-based tracing mechanism to capture the execution of an Android app and generate a trace file. This trace file can be used to reconstruct the app’s code and data, thus enabling the app to be decloaked and analyzed. Specifically, Happer uses the ARM CoreSight hardware tracing technology to capture app execution. This technology records the execution of an app’s instructions and data access at the hardware level and stores this information in a trace file. Happer then uses this trace file to restore the app’s code and data to enable app unpacking and analysis. Compared to traditional software-assisted approaches, hardware-assisted approaches provide higher performance and accuracy because they capture app execution directly at the hardware level, avoiding performance bottlenecks and errors at the software level. As a result, Happer’s hardware-assisted approach can decloak Android apps faster and more accurately.

8.5 BPF Technology

8.5.1 Development of BPF Technology

BPF (Berkeley Packet Filter) is a powerful module of the Linux kernel. In 1992, BPF was born out of Steven McCanne & Van Jacobson’s 1993 publicly available paper: The BSD Packet Filter: A New Architecture for User-level Packet Capture [25]. The performance of packet filtering techniques is improved by more than 20 times by the introduced BPF technique. Mainly through:

A new virtual machine (VM) design that works efficiently on CPUs with register-based architecture.

The BPF program performs efficient filtering at the kernel level, avoiding the need to copy data to user space for processing.

The VM design is very simple, with 2 32-bit registers A/X, 16 memory bits: M(0-15), and 33 instruction sets.

In early 2014, Alexei Starovoitov implemented extended Berkeley Packet Filter (eBPF). eBPF first appeared in the Linux kernel 3.18. Since then, BPF has been known as classic BPF, abbreviated cBPF, which is now largely deprecated. Today, the Linux kernel runs only eBPF, which transparently converts loaded cBPF bytecode into eBPF before executing it. eBPF was extended to user space in June 2014, marking a turning point for eBPF technology. It evolved into a general-purpose execution engine, with apps no longer limited to network packet filtering. Gradually, it expanded to kernel submodules, becoming a major player in areas such as observation, networking, and security.

8.5.2 eBPF Technology Overview

eBPF is based on the concept of a virtual machine, which allows users to write eBPF programs and run them in the kernel. eBPF programs are written using eBPF bytecode, and then the bytecode is converted to native machine code by the JIT (Just-In-Time) compiler to improve execution efficiency. eBPF programs can interact with the kernel through eBPF system calls to enable kernel behavioral monitoring and modification [26].But at the same time eBPF has the following limitations:

➀

eBPF programs are limited to calling BPF auxiliary functions listed in the kernel module. The function support list is increasing with the evolution of the kernel, and the latest progress allows direct calls to specific kernel functions.

➁

eBPF programs cannot contain unreachable instructions to prevent loading invalid code and delaying program termination.

➂

The number of loops in an eBPF program is limited and must terminate in a finite amount of time. Linux 5.3 introduced support for bounded loops in BPF, which have a verifiable upper limit on runtime.

➃

The eBPF stack size was limited to MAX_BPF_STACK, which was set to 512 as of kernel Linux 5.8. Initially, eBPF bytecode sizes were limited to 4096 instructions, but as of kernel Linux 5.8, this limit has been relaxed to 1 million instructions. However, the limit of 4096 instructions is still retained for unprivileged BPF programs. The new version of eBPF also supports multiple eBPF program cascade calls, which can be combined to achieve more powerful functions.

The app scenarios of eBPF include observation and tracking, network, security, and more. In terms of observation and tracking, eBPF programs can be attached to tracking points, kernel probe points, and user app probe points, providing unprecedented visibility into the runtime behavior of apps and the system itself. Unlike relying on static counters and measurements exposed by the operating system, eBPF enables the collection of custom metrics and in-kernel aggregation, as well as the generation of visibility events from various sources. Regarding networking, eBPF’s programmability and efficiency make it well-suited for packet processing requirements. It allows for the addition of protocol parsers within the Linux kernel’s packet processing context and facilitates easy programming of forwarding logic to adapt to changing needs. The JIT compiler ensures that eBPF’s execution performance is comparable to that of natively compiled kernel code. In terms of security, eBPF offers a revolutionary approach by providing insights into the basis of all system calls and combining it with a packet- and socket-level view of network operations. This integration allows for the creation of secure systems that operate on multiple contexts with a higher level of control. Traditionally, system call filtering, network-level filtering, and process context tracking are handled by separate systems, but eBPF enables the combination of visibility and control aspects for enhanced security.

Event objects supported by eBPF tracking:

➀

kprobes/kretprobes: implements dynamic tracing in the kernel. Tracks function entry or return points, unstable ABI interfaces in the Linux kernel; (5.5 fentry/fexit replacement for better performance and usability).

➁

uprobes/uretprobes: user-level dynamic tracing. Similar to kprobes, except that the traced functions are those in the user program;

➂

tracepoints: static traces in the kernel. tracepoints are maintained by kernel developers and can provide a stable ABI interface, but the number and scenarios may be limited due to maintenance by developers.

➃

perf_events: timed sampling and PMC.

Generally speaking, the workflow of eBPF has the following steps:

➀

Compile: you can use LLVM or GCC tools to compile the eBPF code program into eBPF bytecode.

➁

Load: then use the loader program Loader to load the bytecode into the kernel through the bpf() system call.

➂

Verification: the kernel uses the verifier component to ensure the safety of executing the bytecode to avoid disaster to the kernel, and loads the corresponding kernel module to execute the bytecode after confirming that it is safe; verification process:

Firstly, depth-first search is performed to disable loops; CFG verification.

The instructions generated in the previous step are used as input to traverse all possible execution paths. Specifically, we simulate the execution of each instruction and observe the state changes of registers and stacks.

➃

Trigger Run: The eBPF bytecode program runs when the corresponding event is triggered by the kernel and can interact with the user program through map or perf_event.

The eBPF program usually consists of the following parts: ➀ Backend: this is the eBPF bytecode that is loaded and run in the kernel. It writes data into the kernel map and ring buffer data structures; ➁ Loader: it loads the bytecode backend into the kernel. Normally, the bytecode is automatically unloaded by the kernel when the loader process terminates; ➂ Front-end: it reads data from the data structures (written by the backend) and displays it to the user; ➃ Data structures: these are the means of communication between the back end and the front end. They are map and ring buffers managed by the kernel, accessible via file descriptors, and need to be created before the backend can be loaded, and the data persists until there are no more backends or frontends to read or write to.

eBPF programs are usually written in C or a BPF-specific assembly language. In order to simplify the development of eBPF programs, the community has created projects such as BCC, bpftrace, etc.: these provide easy-to-use frameworks for writing, loading, and running eBPF programs and can be accessed by writing simple Python programs in any language other than “Restricted C.” This can be achieved by writing simple Python, Go, or Lua scripts.

8.5.3 Examples of BPF Techniques in the Field of Android Malware Detection

The Android operating system is also based on the Linux kernel and also has access to all the features of eBPF. In Android 10 (Android Q), Google introduced eBPF support, allowing developers to use eBPF to optimize the Android software and operating system. Using the features of eBPF, Android apps can be monitored and tracked, and the same can be applied to the dynamic analysis of Android apps. In the field of Android malware detection, eBPF has many advantages. First, because eBPF is a virtual machine program based on the Linux kernel, it can directly monitor the sensitive API calls of the kernel without any dead angle and will not interfere with the kernel operation; second, many self-protection measures of the malware, such as shelling, obfuscation, inlining, etc., will also be monitored in the kernel-based monitoring of eBPF. Secondly, many self-protection measures of malware, such as shelling, obfuscation, inlining, etc., will also be broken under eBPF’s kernel-based monitoring. In addition, eBPF’s program overhead is extremely small, and the resource consumption of the Android device is far less than that of the traditional dynamic detection tools. For example, in the article BPFroid: Robust Real-Time Android Malware Detection Framework [27], the authors propose an eBPF-based malware detection framework called BPFroid, whose design and implementation details include the following aspects:

➀

Multi-layer probes: BPFroid uses multiple eBPF event probes to monitor different events, including system calls, process creation, and file accesses. These eBPF probes operate at the kernel level and thus can capture malware behavior without being easily bypassed by malware.

➁

Event Handlers: BPFroid uses event handlers to handle events captured by eBPF probes and send them to the data analysis module for analysis. Event handlers can categorize and filter events based on event type and other attributes to reduce the overhead of data analysis.

➂

Data analysis: BPFroid uses machine learning algorithms to analyze event data to detect malware behavior. The data analysis module can determine whether an event is a malware behavior based on its attributes and contextual information, and generate alerts or block malware execution.

8.6 Chapter Summary

This chapter summarizes the future trends in Android malware detection, emphasizes the importance of machine learning and deep learning techniques, and proposes integrated solutions, blockchain technology, hardware technology, and BPF technology as methods to deal with malware. All of these technologies and approaches have potential but also face some challenges. The effectiveness and efficiency of Android malware detection can be improved by applying these techniques and methods in an integrated manner.

References

1.

Milosevic N, Dehghantanha A, Choo K K R. Machine learning aided Android malware classification[J]. Computers & Electrical Engineering, 2017, 61: 266–274.Crossref

2.

Sahs J, Khan L. A machine learning approach to android malware detection[C]//2012 European intelligence and security informatics conference. IEEE, 2012: 141–147.

3.

Kevin Allix, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. 2016. AndroZoo: collecting millions of Android apps for the research community. In Proceedings of the 13th International Conference on Mining Software Repositories (MSR ’16). Association for Computing Machinery, New York, NY, USA, 468–471. https://​doi.​org/​10.​1145/​2901739.​2903508

4.

Arp D, Spreitzenbarth M, Hubner M, et al. Drebin: Effective and explainable detection of android malware in your pocket[C]//Ndss. 2014, 14: 23–26.

5.

Datasets. (n.d.). Canadian Institute for Cybersecurity. https://​www.​unb.​ca/​cic/​datasets/​index.​html

6.

VirusShare.Com. (n.d.). VirusShare. https://​virusshare.​com/​about

7.

Contagio. (n.d.). Contagio. https://​contagiodump.​blogspot.​com/​

8.

Qiu S, Liu Q, Zhou S, et al. Review of artificial intelligence adversarial attack and defense technologies[J]. Applied Sciences, 2019, 9(5): 909.Crossref

9.

Zhang C, Benz P, Lin C, et al. A survey on universal adversarial attack[J]. arXiv preprint arXiv:2103.01498, 2021.

10.

Zhang Y, Tiňo P, Leonardis A, et al. A survey on neural network interpretability[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2021, 5(5): 726–742.Crossref

11.

Zhang Q, Zhu S C. Visual interpretability for deep learning: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 27–39.Crossref

12.

Bagdasaryan E, Veit A, Hua Y, et al. How to backdoor federated learning[C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2020: 2938–2948.

13.

Foster I, Kesselman C, Nick J M, et al. Grid services for distributed system integration[J]. Computer, 2002, 35(6): 37–46.Crossref

14.

Song J, Han C, Wang K, et al. An integrated static detection and analysis framework for android[J]. Pervasive and Mobile Computing, 2016, 32: 15–25.Crossref

15.

Madni A M, Sievers M. System of systems integration: Key considerations and challenges[J]. Systems Engineering, 2014, 17(3): 330–347.Crossref

16.

Zheng Z, Xie S, Dai H N, et al. Blockchain challenges and opportunities: A survey[J]. International journal of web and grid services, 2018, 14(4): 352–375Crossref

17.

Monrat A A, Schelén O, Andersson K. A survey of blockchain from the perspectives of apps, challenges, and opportunities[J]. IEEE Access, 2019, 7: 117134–117151..Crossref

18.

Yaga D, Mell P, Roby N, et al. Blockchain technology overview[J]. arXiv preprint arXiv:1906.11078, 2019.

19.

Li X, Jiang P, Chen T, et al. A survey on the security of blockchain systems[J]. Future generation computer systems, 2020, 107: 841–853.Crossref

20.

Tasatanattakool P, Techapanupreeda C. Blockchain: Challenges and apps[C]//2018 International Conference on Information Networking (ICOIN). IEEE, 2018: 473–475.

21.

Kumar R, Wang W Y, Kumar J, et al. Collective intelligence: Decentralized learning for Android malware detection in IoT with blockchain[J]. arXiv preprint arXiv:2102.13376, 2021.

22.

Coppolino L, D’Antonio S, Mazzeo G, et al. A comprehensive survey of hardware-assisted security: From the edge to the cloud[J]. Internet of Things, 2019, 6: 100055.Crossref

23.

Zhou H, Wu S, Luo X, et al. NCScope: hardware-assisted analyzer for native code in Android apps[C]//Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 2022: 629–641.

24.

Xue L, Zhou H, Luo X, et al. Happer: Unpacking android apps via a hardware-assisted approach[C]//2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021: 1641–1658.

25.

EBPF - Introduction, Tutorials & Community Resources. (n.d.). EBPF. https://​ebpf.​io/​

26.

What Is EBPF?An Introduction and Deep Diveinto the EBPF Technology. (n.d.). EBPF. https://​ebpf.​io/​what-is-ebpf/​

27.

Agman Y, Hendler D. BPFroid: robust real time Android malware detection framework[J]. arXiv preprint arXiv:2105.14344, 2021.