When investing in an user acquisition campaign, any app developer or agency wants to make sure 100% of its investment will bring high value users who will soon become engaged customers. What if we were to tell them that over 30% of the overall internet traffic was actually spotted as bot traffic in 2019*? What can they do better now to make sure the impact of their advertising spend grows from 70 to 100% of human-generated traffic? We will share in this article the different kinds of bot traffic that exist on the market and tips on how to spot it and be able to fully evict it.

What is Bot Traffic?

First thing to know is that not all bot traffic is bad, actually 35% of this overall bot traffic is considered as “good”. It is made of feed fetchers, search engine bots, commercial crawlers and monitoring bots. They are considered to be good as they will help a website owner to get a better ranking in the search engines, protect its content to avoid any duplication content on some concurrent sites and make sure websites are running smoothly. 

On the other hand, mobile “bad” bot traffic can be defined as any traffic that is not generated by an actual human, originating from a real or an emulated mobile device, usually in a device farm, a server, operating via emulators, or a malware program. It results in the generation of fake clicks, installs and even post-install events such as registrations or purchases. The well-known mobile measurement platform Appsflyer explains that “these programs constantly refresh their metadata, observe and learn user behavior patterns, later applying them in their activity to go under the radar of fraud protection solutions”. This makes these bad bots extremely hard to spot, as in most cases they are smart enough to learn how an actual person would act and copy the same behavioral pattern to remain undetectable. 

There are 2 types of goals among these fraudsters: the ones who deliberately aim at harming the in-app experience and performed by cyber criminal organizations, and the ones who seek a monetary profit, performed by business organizations or individuals. The latter category, the most widely spread, is generating fake installs or events to get the attributed payment, from a “mobile farm” (having hundreds of real devices on a wall performing actions through a script that they developed) or emulators. They are targeting mobile advertising campaigns offering high payouts, or high possibilities of scale, in order to maximize their revenues. And they have been particularly greedy for CPA campaigns (Cost Per Action), as they have been a new market trend, offering considerably higher payouts than a regular CPI campaign (Cost Per Install). While many app developers or agencies continue to believe that moving from a CPI to a CPA pricing model is safer to prevent fraud and maximize their revenues, our experience has shown otherwise. updating it along the way to make it smarter than anti-fraud solutions. They look very similar to human traffic so are extremely hard to spot unless having a strong anti-fraud solution tailored to be smarter than them.

How to detect Bot Traffic?‍

In 2019, yearly mobile advertising spend has increased by 20% to reach $190 billion**, which means mobile advertisers have been facing a $45 billion loss. This is considerable. If bot fraud remains stable in 2022 (it actually increased by 18% last year), as mobile ad spend is expected to grow to $280 billion yearly, bot fraud would reach an amount higher than $67 billion. While all this money loss can be avoided or at least limited by applying stronger controls and having a smarter approach than fraudsters to counter them in an agile way. For this, the first step is to be able to properly detect this bad bot traffic:

• IP detection. As most bot operators used to rely on data center proxies, it is an easy way to check the IP and compare it with an existing IP blacklist through a provider. But fraudsters became smarter, starting to use Tor exit nodes to remain undetectable so that now IP detection can only spot the least advanced type of bot fraud.
• SDK version. Noticing that installs are coming from a previous SDK version can be a sign of bot traffic. 
• Device information. In the case of emulators, then there are possible ways to identify bot fraud by looking at all the device-related information and see if there is anything that does not match (OS version, device type, browser, etc).
• Events Patterns. Such as high densities of installs that follow identical or programmatic, non-human behavioral patterns. This can be spotted by looking at the timestamp of these events (for example installs that would all arrive in a very specific time frame after the click). This can also be spotted on a post-install events basis, when events such as registrations or purchases arrive during a very similar time frame, or at an unusual time.
• Suspicious user behavior. This can be spotted when comparing with organic installs and events, and how a human user usually interacts within the app. Comparing this organic data of retention & engagement with the data coming from bot traffic would usually result in different rates. Bot traffic would show a  low user retention rate and suspicious events ratio (either too low or sometimes even too high). But fraudsters becoming smarter along the way, by knowing the KPI to reach for a specific campaign, they can also adapt their script to make it look human-like.
• Payment data. Fraudsters can use fake credit card to realize the fake purchases within the app. For this reason, it is extremely important for any advertiser to verify all payment information and user’s credit cards validity.
  

Looking at all these different data points in order to cross reference the metrics became the new normality in this industry, but not all advertisers are yet applying an automated and real time control to evict completely this bot traffic.

‍How to evict Bot Traffic?

There are 3 main brakes among app advertisers to efficiently evict bot fraud: 1) the lack of knowledge about bot traffic and the negative impact on their user acquisition campaigns, 2) the resource and tool to properly spot it in real time, 3) a static approach to evicting fraud that does not take into account the constant evolution of fraudsters’ ways. While we have overcome the first brake in this article, let’s tackle together the second and third issues. 

Manual checks will not allow an efficient barrier against malicious bot traffic, as being too static and arriving too late after the event actually happens. What is needed is the activation of a powerful anti-fraud tool, on the MMP side, but that can also be biased by a conflict of interest, thus adding an extra layer of control is always a much-needed thing. Dreamin has built its own anti-fraud solution, deploying machine learning algorithms to spot behavioral and statistical anomalies in real-time. This detection solution works on the basis of multiple data points collected and analyzed by the AI system at the same time. Once spotted, it is removed and blacklisted, while getting smarter along the way, as new types of bots arise. 

...

*According to the survey from Imperva on “Bad Bot Report 2020: Bad Bots Strike Back”

** According to Statista.com’s survey. Source: https://www.statista.com/statistics/303817/mobile-internet-advertising-revenue-worldwide/