Four Artificial Intelligence Threats Will Challenge the Cybersecurity Industry

Artificial Intelligence (AI) and Machine Learning (ML) systems are growing increasingly popular, especially with advances in OpenAI’s ChatGPT and other large language models (LLMs). Unfortunately, this popularity has led to these technologies being used for malicious purposes, making AI and machine learning an imminent threat to Security Operations Centers (SOCs). SOC teams must start preparing and building threat models to stay ahead of these emerging risks.

In this blog post, we will discuss four major AI-related threats that your security operations team needs to plan and budget for to stay ahead of the curve.

Understanding AI Today

According to a report published by Statista, the amount of big data generated is growing at a rate of 40% and will reach 394 zettabytes by 2028. One of the main consequences of this data explosion is the emergence of an Artificial Intelligence (AI) ecosystem. The term "AI ecosystem" refers to machines or systems equipped with significant computing power that imitate human intelligence. Today's AI ecosystem includes technologies such as Machine Learning (ML), Artificial Neural Networks (ANNs), robotics, and more.

Artificial Intelligence (AI) and Machine Learning (ML) are two terms that are often interchanged, but their differences are important. AI is comparable to human learning, where new behaviors are adopted without a prior baseline. Conversely, machine learning is a subset of AI that uses predefined algorithms tailored to specific data types with expected outcomes. These are fixed AI algorithms that can learn and make inferences.

Types of Attacks Exploiting AI

AI-Powered Malware

AI-powered malware can identify security vulnerabilities in an organization's systems and exploit them rapidly. This type of attack can lead to a complete network shutdown or allow hackers to covertly exfiltrate sensitive information.

Malware developers are increasingly using AI to evade detection by traditional antivirus tools. Threat actors can use deep learning techniques to create new malware variants capable of bypassing traditional signature-based detection methods. SOC teams must invest in advanced, AI-based malware detection solutions capable of identifying and thwarting these sophisticated attacks.

Figure 1 – Common attack chain following the 14 tactics defined by the MITRE ATT&CK matrix for enterprise.

Social Engineering Using AI

Phishing scams and social engineering attacks are commonplace; however, AI-driven social engineering attacks can be far more potent. Malicious actors can use AI systems to create voice bots or chatbots that mimic real people and manipulate targets into sharing information that can then be used to gain unauthorized system access.

AI makes it easier for cybercriminals to launch complex social engineering attacks where the attacker impersonates someone the target trusts, such as a senior executive or business partner. AI leverages natural language processing to generate content—such as emails or chat messages—that appears authentic. SOC teams must develop AI models capable of identifying AI-generated content and distinguishing authentic messages from potentially malicious ones.

With AI, attackers can launch massive Business Email Compromise (BEC) scams. The process to achieve this is remarkably simple:

• Scraping data from numerous employee LinkedIn profiles to map out the products, projects, and teams those employees work on
• Feeding this information into an LLM
• Generating tailored social engineering content

The result is extremely convincing emails that appear to come from the employees' managers or CFOs. They may even contain specific details about current projects. If a hacker manages to compromise internal company data and feed it into the LLM, they can make the attack even more convincing.

Data Poisoning

According to a Gartner article, data poisoning attacks will pose a significant cybersecurity threat in the coming years. The goal of these attacks is to intentionally inject false data into an organization's dataset, thereby distorting the results of any predictive modeling or machine learning algorithm. Data poisoning is a form of adversarial attack that involves manipulating training datasets by injecting corrupted or "poisoned" data to control the model's behavior and produce false results.

The potential damage of backdoor attacks on machine learning models cannot be overstated. These attacks are not only more sophisticated than simple injection attacks, but also far more dangerous.

By introducing undetected corrupted data into a machine learning model's training set, adversaries can slip in a backdoor. This hidden doorway allows them to manipulate the model's actions without its creators' knowledge. The malicious intent behind backdoor attacks can remain undetected for long periods, with the model operating as expected until specific trigger conditions are met. Taking the necessary measures to prevent backdoor attacks on your machine learning models is essential.

Generating Deepfakes with AI

Given that AI can create convincing imitations of human activities (writing, speech, and images), generative AI can be leveraged in fraudulent activities such as identity theft, financial fraud, and disinformation. AI-generated deepfakes are used to produce content that appears authentic despite containing false information. For example, a manipulated video might depict someone saying or doing something they never actually did, causing harm to their reputation and credibility.

Deepfake technology has been developing for decades, but in recent years, deepfakes have become far more accessible and advanced due to the development of robust, versatile generative models—such as autoencoders and Generative Adversarial Networks (GANs). As a result, it is becoming increasingly difficult for security professionals to distinguish real media from fake. Deepfake technology uses machine learning algorithms to analyze and learn from precise data—such as photos, videos, and voice recordings—and then generate new data that closely resembles the original with subtle alterations.

Deepfake technology relies on Artificial Neural Networks (ANNs) that learn from data to perform tasks requiring human intelligence. Developers use two neural networks to create deepfakes: one generates synthetic data (the generator), and the other determines how convincing that data appears (the discriminator). The generator uses this feedback to refine its output until it successfully deceives the discriminator, creating a Generative Adversarial Network (GAN).

Real-world examples of deepfake technology:

• A deepfake video of former U.S. President Barack Obama delivering a speech he never actually made, created by comedian Jordan Peele.
• A deepfake video of Meta CEO Mark Zuckerberg boasting about controlling the data of billions of people, created by artists Bill Posters and Daniel Howe.
• Videos of Tom Cruise performing various stunts and jokes, created by TikTok user @deeptomcruise.
• The deepfake app Reface, which allows users to swap faces with celebrities in videos and GIFs.
• The deepfake app Wombo, which lets users animate themselves or others singing and dancing in video clips.

Taking Proactive Action to Defend Against AI Threats

AI technologies and machine learning systems are developing rapidly, and their applications within cybersecurity continue to evolve. Security Operations Center teams must actively prepare for the growing security risks posed by AI. SOC teams need to start building dedicated threat models, upskilling their staff, and procuring the right tools to tackle emerging challenges. In this fast-changing technological landscape, SOC teams that are proactive in building resilience against AI-driven threats will be far better positioned to protect their organizations.