Deepfakes - From Fun Toy to Fraud Tool

Sudhanshu Ranjan is an Associate Analyst II at AML RightSource.

We encourage our team members to share their insights and expertise by contributing to our content library. 

Introduction 

Definition 

Deepfake is a blend of two different words, 'Deep Learning' and 'Fake.’ When deep learning, a subset of Machine Learning, is used for fake or malicious purposes, it results in Deepfakes. To be precise, deepfakes are any media that has been synthesized and digitally manipulated to replace one person's likeness with that of another.

 Rise in popularity and accessibility 

The creation of fake content is not a new thing. Before the internet breakthrough, it usually consisted of fake photoshopped photos, news, etc. As the internet became more accessible to the public, the impact of counterfeit content grew even more. However, it was still identifiable for two reasons, 1) the contemporary technology needed to be more efficient, and 2) it was mainly being created by amateurs in online communities.

However, the advancements and accessibility of technologies, such as AI and Machine Learning, have been very rapid in the last decade, enabling the general public and groups - with good and bad intentions - to quickly learn and contribute their piece to the pool of fake content.

 Exploring Deepfakes 

 How it works 

In simple language, deepfake technology swaps a person's unique identity, such as face and voice, with another's in digital media or recording with the help of advanced computer algorithms. These algorithms require a large amount of data (related to the quality intended to intermix) of the subjects to analyze and learn the minor details of the facial features and vocal patterns to blend these in a way that makes the original unidentifiable. 

 Examples of deepfake applications in entertainment and social media 

 The Deepfake technology is being used extensively in different types of social media circles, such as the following: 

• Memes and Viral Contents – Deepfake is used to create memes that make famous personalities do unusual things to make them contagious. These memes are mainly created for mass humor or satire.
• Advertisements and Marketing - The emergence of deepfake technology has enabled big corporations to run more personalized ads that are easy to scale.  Hiring a celebrity for an ad is very expensive and at the same time limited in scale as one can only make a celebrity do and say a limited amount of things. Using deepfakes, businesses can customize the voice of the celebrity to fit the needs of the ad campaign.
• Political Satire and Commentary - Using the voice and physical movements of influential political figures and distorting them to convey contrary opinions or statements in a satirical way to mock them

 Increasing concerns about malicious use

 Unfortunately, deepfake technology can also be misused to spread false information, manipulate public opinion, or create fake news by altering videos or audio recordings to depict events that never happened or statements that were never made. 

Deepfakes and Financial Crimes  

Overview of financial crimes facilitated by deepfakes 

Now that you understand the possible usages of deepfakes, here are some of the implications in the financial world: 

• Fraudulent Transactions - Deepfakes can create convincing videos or audio recordings impersonating individuals in positions of authority, such as company executives. Scammers can then use these deepfakes to authorize fraudulent transactions or manipulate employees into transferring funds to unauthorized accounts.
• Phishing Scams -- Deepfakes can enhance phishing scams by creating fake videos or audio messages from trusted sources, such as banks or financial institutions. deceiving individuals into providing sensitive information, such as login credentials, personal or financial information, etc., which can be used for identity theft or fraud.
• Identity Theft -- Deepfakes can be used to steal individuals' identities by creating fake identification documents, such as passports or driver's licenses, with manipulated images or biometric data. These counterfeit identities can then be used to open fraudulent bank accounts, apply for loans, or commit other financial crimes under someone else's name. 

A case study highlighting an instance of deepfake-related financial fraud 

In February of 2024, the world learned about a case of a deepfake facilitated scam in Hong Kong of $25 million. Yes, that's right. Let's take a brief look: 

A finance department employee of a multinational corporation in Hong Kong received a deceptive communication from the CFO based in the UK. This message prompted the initiation of a confidential financial transaction. Then, the victim was added to a group video conference in which the fake replicas (created through the deepfake technology) of key executives of the company, including the CFO, were seamlessly interacting with each other - with similar individual gestures and behaviors - regarding the transactions to project its credibility and winning the trust of the employee. The victim then executed a series of 15 transfers totaling $25 million to multiple bank accounts. It was only upon closer scrutiny with the company's headquarters that the elaborate ruse came to light, exposing the depth of the deception.

Detecting and Preventing Deepfake Fraud

Current methods for detecting deepfakes

As the technology and its implications are quite new, its detection techniques, such as visual inspection, source verification, forensic analysis, and others; could be more efficient. But most of these techniques will become futile as the technology grows and gets even more polished and genuine in its creations. The problem with these techniques is that they are outdated and cannot counter the sheer velocity at which deepfake content is being and will be created.

Strategies for mitigating the risk of deepfake fraud in corporate settings

The corporations must now pull up their sleeves and start to have serious discussions to counter any finance or compliance-related shock that they might encounter from the deepfake technology.

Here are a few suggestions:

• Employee Training - Educate employees about the potential risks of deepfakes. Run different learning programs that make them aware of the potential risk when receiving unexpected requests, especially if they involve financial transactions or sensitive information. Encourage them to report suspicious activities or communications promptly and provide channels for reporting potential security incidents.
• Authentication Procedures - Implement robust authentication procedures for verifying the identity of individuals involved in critical communications or transactions. Use multi-factor authentication, secure channels, and secondary verification methods to validate requests.
• Internal Controls - Implement internal controls and approval processes to monitor and review financial transactions, particularly those initiated through electronic or digital channels.
• Anti-Fraud Technologies - Invest in anti-fraud technologies and solutions designed to detect and mitigate deepfake-related risks. Explore the use of AI-driven detection tools, behavioural analytics, and anomaly detection systems to identify fraudulent activities.
• Crisis Response Plan - Develop a comprehensive crisis response plan to address potential deepfake-related incidents or breaches. Define roles and responsibilities, establish communication protocols, and outline steps for containing and mitigating the impact of fraudulent activities.

It is also advisable to continuously monitor the threat landscape and adapt your strategies and defences accordingly. Stay vigilant against evolving deepfake techniques and adjust your security measures as they emerge.

Regulatory and Legal Considerations

Overview of existing regulations and emerging ones related to deepfakes and financial crimes

As of now, there are no federal laws directly related to prohibiting the creation or circulation of deepfake content. However, there have been some serious talks regarding it. In Jan 2024, representatives proposed the No Artificial Intelligence Fake Replicas And Unauthorized Duplications (No AI FRAUD) Act. The bill set up a federal framework to protect individuals against AI-generated fakes and forgeries by making it illegal to create a ‘digital depiction’ of any person, living or dead, without permission. This would include both their appearance and voice.

Many US states have implemented legislation targeting deepfakes, though the details vary state to state. Some of those states are California, Texas, Georgia, New York, Florida, and others. California was an early adopter, passing its law  in 2019. They have also banned the use of AI deepfake content in the election campaign seasons altogether.

I believe that the approach of the government is laudable, but they need to hasten this process and make it more comprehensive and widespread, as the risk posed by the evil usage of deepfakes is increasing every day.

Conclusion

We thoroughly discussed the past, present, and future of the deepfake technology, including its good and bad usage, impact on the financial world, potential crime risks, and many other things.

Deepfake, like any other technology, has many advantages that can help us become more efficient and effective in our goals. However, the technology comes with great challenges. So, the only option we are left with is ensuring it is a net-win for us society. It's about using technology in ways that benefit society as a whole while minimizing potential harm or negative consequences.

Making people in the organization aware of these new technological risks and harnessing serious talks, considerations, feedback, and suggestions among one another will lead to group maturity and a cautious approach in every process, and these will be our defence against all external threats.