An event powered by
Close this search box.

AI is the most advanced frontier of phishing. What we need to know

Phishing’ scammers could train algorithms to attack our e-mail

For experts companies providing Machine Learning technologies must act as the first line of protection against “phishing”


Phishing is a scam carried out by sending an e-mail through which an attacker tries to trick the victim into providing personal information, financial data (credit card number, password for home banking access) or access codes, by claiming technical needs.

An example? We receive an email that appears to have been sent by the bank to induce us to provide information about our bank account. Everyone knows not to click on these messages and to delete them immediately. Yet people keep falling for them.

According to the Anti-Phishing Working Group, around 200,000 new phishing sites appear every month, and the campaigns impersonate more than 500 different brands and entities every month.

The FBI’s Internet Crime Complaint Center then found that US-based phishing victims lost nearly $58 million in 2019 alone. In short, phishing works. Even more so does spear phishing, attacks aimed at a specific person or company, thus highly targeted and personalised. But they could work even better if these messages were composed by Artificial Intelligence algorithms.


The research

At the recent Black Hat e Defcon security conferences in Las Vegas, a team from Singapore’s Government Technology Agency presented an experiment in which they sent 200 targeted phishing emails, some created by agency staff, others generated by an AI-as-a-service platform. Well, the researchers were surprised to find that more people clicked on links in the AI-generated messages than in the human-written ones, and by a fairly significant margin.

Training a phishing algorithm

Why does AI phishing work better? Machine Learning focused on personality analysis aims to predict a person’s inclinations and mindset based on behavioural inputs. By running outputs across multiple services, the researchers were able to develop a pipeline that curated and refined emails before sending them. The results, from what the researchers say, seemed ‘strangely human’.

Easy to use

“Researchers have pointed out that Artificial Intelligence requires a certain level of expertise. It takes millions of dollars to train a really good model,’ explained Eugene Lim, a cybersecurity specialist at the Government Technology Agency, ‘but once you put it on AI-as-a-service, it costs a couple of cents and it’s really easy to use: you just input the text and it outputs the text. You don’t even have to run the code, you just give it a prompt and it will output. This lowers the barrier of entry to a much wider audience and increases the potential targets for spear phishing. Suddenly every single large-scale e-mail can be customised for every recipient”.

Just the first step

As the researchers themselves pointed out, the experiment was only a first step. The sample size was small and the target group was fairly homogeneous in terms of occupation and geographical region. However, the results prompted the researchers to think more deeply about how AI-as-a-service could play a role in phishing and spear phishing campaigns in the future.

Possible defence

The intention is to create mechanisms able to mark synthetic media in e-mails to facilitate the capture of possible AI-generated phishing messages. Yes, but how to recognize them? Researchers note that synthetic media are used for increasingly legitimate functions, such as communications and customer service marketing, so it will be even more difficult to develop screening tools that flag only phishing messages. So? The weapons, despite the impressive human imitation of AI-generated phishing emails, remain the same: education, awareness, instinct and scepticism. 

AI technology providers

Timothy Lee argues that AI-as-a-service providers need to act as the first line of defence, developing terms of use and screening guidelines that determine misuse and abuse. He also suggested that solution providers ensure that the use of its products can be audited and tracked.

sources: /

cover photo:

Maker Faire Rome – The European Edition has been committed since eight editions to make innovation accessible and usable to all, with the aim of not leaving anyone behind. Its blog is always updated and full of opportunities and inspiration for makers, makers, startups, SMEs and all the curious ones who wish to enrich their knowledge and expand their business, in Italy and abroad.

Follow us, subscribe to our newsletter: we promise to let just the right content for you to reach your inbox