Deepfake Deep Dive

Artificial intelligence exponentially increases the volume, value, and effectiveness of fraud attacks. One clear example is “business email compromise” (BEC), also called “CEO fraud”. 

A BEC or CEO fraud is a social engineering attack in which a fraudster assumes the identity of a trusted persona, such as the CEO of a company, to manipulate victims into taking a desired action. These actions could include transferring funds or handing over sensitive information. BECs target specific individuals and are often personalized to the victim, using information the fraudster collects during detailed research. AI-generated deepfakes and other content provide significantly enhanced tools to perpetrate fraud. 

AI-enhanced deepfake CEO fraud causes $25 million loss 

Arup is an engineering firm with 34 offices and 18,500 staff worldwide. In 2024, it was targeted by a CEO fraud which resulted in the loss of HK$200 million (US$25 million). 

An employee in the finance team in Hong Kong received an email purporting to be from the Chief Financial Officer, who was based in the United Kingdom, requesting them to make several payments. Initially, the staff member was hesitant about the legitimacy of the email. The supposed CFO confirmed the payment instructions via a video conference. 

The staff member made 15 transactions totaling $25 million to five local bank accounts. It was only later, when the employee checked with head office, that the fraud was identified. 

So far, we’ve seen this use of AI-deepfake methodology used in frauds before. 

But in this case – it was not just the CFO on the video conference. 

There were multiple other staff present too. 

The target of the fraud recognized many of their colleagues, and because of this they were persuaded of the legitimacy of the call with the CFO. 

But all the staff were deepfakes. 

During the investigation, police assessed that the attacker downloaded videos of the colleagues in advance and used AI to add fake voices. 

In CEO frauds, AI deepfakes are used to target businesses, including companies, suppliers, and business partners. The same methodologies also can be used to create more personalized and credible romance scams, family/medical emergency scams, and other frauds.

Customers need protection too

Many financial institutions provide resources and awareness campaigns to help their customers avoid being targeted by fraud. These institutions gain a commercial advantage by strengthening their relationships with their customers by adding value beyond simply products and services. They also provide protection for the institution and financial system by reducing the likelihood that the institution will process fraudulent transactions. 

There are other reasons institutions should consider enhancing their fraud awareness programs. Recognizing the increase in fraud and the greater resources available to institutions relative to their customers, some regulators hold financial institutions responsible if they do not have adequate measures in place to identify when their customers are being scammed. 

Legal action brought against Citibank for failing to protect consumers from fraud 

The New York Attorney General commenced legal action against Citibank in 2024 for failing to protect and reimburse victims of fraud. The lawsuit alleges that Citibank does not adequately protect its customers from fraud losses because it: 

• Does not have adequate online protections to prevent unauthorized account takeovers 

• Does not respond promptly to fraud claims

• Does not properly investigate fraud claims 

It also misleads account holders about their rights and denies reimbursement if they are victims of fraud. 

As an example, in October 2021 a customer received a text message that appeared to be from Citi and instructed her to log into a website or call her local branch. The customer clicked the link but did not provide any additional information. She called her local branch to report suspicious activity but no action was taken. A few days later, the customer identified that a fraudster had changed her banking password, transferred $70,000 between her accounts, and executed a $40,000 electronic wire transfer. None of these were consistent with her previous account activity. The customer repeatedly contacted the bank but was told her claim for fraud was denied. 

In January 2025, Citi’s motion to dismiss the claim was denied and the legal action continued. The Attorney General stated that this “will allow us to continue our case against Citi to help those whose savings were stolen and ensure the bank follows the law to protect its customers.” 

Weaknesses in fraud detection and prevention measures make institutions and customers more vulnerable to many types of fraud, including those enabled by AI. Enforcement actions like Citi also indicate regulatory expectations of those institutions towards customer protection.

Bank customer coached by fraudster: bank ordered to refund 

The United Kingdom Financial Services Ombudsman ordered a bank to reimburse £100,000 ($134,000) plus interest to a customer who lost money due to fraud. The Ombudsman held the bank responsible because branch staff had indications their customer was a victim of a scam but did not take sufficient action. 

The customer, “Nadia,” received a call from a person stating they were from the police. She was told that staff at her local bank branch had been stealing money from customers and that she needed to move all her money to a “safe account” to protect herself from fraud. Nadia made four transfers over four days of £25,000 (approx. $33,000) each to an overseas account specified by the “police officer.” 

When Nadia made the transfers, bank staff realized they were outside her usual activity and asked her questions. Nadia explained that they were payments for a wedding, following instructions from the fraudster. She said that she was nervous and stressed but because she had been able to answer the questions, the bank staff followed her instructions. 

The UK Financial Services Ombudsman identified that the bank had enough information to identify that Nadia could have been the victim of a scam, even though she had been able to answer the questions. Had staff asked more questions or called the police, the scam could have been prevented. 

While this fraud did not involve the use of AI, it provides an example of how customers can be coached by fraudsters to pass bank checks. With the increasing use of sophisticated AI-enabled frauds, customers are increasingly vulnerable – and financial institutions can and must help. 

Best practices 

Fraud intersects multiple departments within an institution. Many types of frauds are financial crimes or are closely associated: proceeds of fraud are often laundered, or frauds are committed to acquire funds for terrorism or other illicit activities. Fraud can also be seen as an operational risk as it may result from inadequate processes, systems, or human factors. And it closely links with cyber security. 

To enable a more effective response to prevent and detect fraud, three key steps an institution should take are: 

1. Establish a joint and coordinated approach between departments, such as stakeholder working groups that include representatives from all departments with fraud-related responsibilities. This ensures knowledge is shared, minimizing duplication and gaps. 

2. Provide training to staff on how to identify fraud, customized to their role and focused on evolving fraud trends and the use of technology. For example: 

• Branch staff should understand the indications that a customer may be the victim of a scam, how to identify whether the customer has been coached on how to respond, and the action to take. 

• Compliance staff should know how deepfakes can be used to circumvent digital identification and verification so that they can configure technology and manual controls accordingly.

3. Consider a customer fraud awareness program which is relevant to the institution’s customer base, including written resources, email updates, and other formats. For example: 

• Corporate customers could be educated on CEO frauds and how their business might be targeted. They could be encouraged to provide training for their staff, similar to the training a bank provides to its staff. 

• Individual customers could be educated on elder fraud, romance scams, and family/medical emergency scams, among others. They could also be encouraged to learn the signs that their family members or friends might also be targeted so that they can more effectively intervene and support. 

Deloitte estimates that generative-AI enabled fraud losses will increase from $12 billion in 2023 to $40 billion in 2027. By taking more effective joint action, within institutions and with customers and communities, we can more effectively protect the financial system and each other from fraud.