The Impact of Online Bots on Classifieds & Listing Websites

The Global Online Classifieds & Listing Industry

Classified ads and listings, have been around for as long as printed newspapers. There are horizontal and vertical classified websites. Horizontal platforms have no particular specialization and may include a variety of services such as jobs, real estate, autos, and so forth. Vertical classifieds are more specific to services like recruitment, matrimony, automobile, real estate, education etc.

In the US, classifieds first moved online starting in the mid-1990s with the development of Craigslist and eBay. While Craigslist focused on a variety of different listing types, eBay initially grew as an auctions website. Since then, many vertical classifieds have popped up such as Zillow in real estate, LinkedIn in jobs, and AutoTrader in vehicles.

As reported on Statista, the global online classifieds market is enormous with estimated revenues as high as USD 25,572 Mn by 2021. In 2016, revenues in classifieds segment amounted a staggering USD 16,461 Mn. China leads revenue numbers in classifieds sector with a total market volume of USD 3,733 Mn in 2016, followed by US amounting to USD 3,307 Mn.

Bot Frauds on Classified Websites have Direct Impact on Revenues

The lowered cost to build bots for web crawling coupled with readily available web scraping tools has led to a drastic increase in non-human traffic on classified portals and the internet at large. At InfiSecure, we have seen bot traffic on classified websites reach almost to 87% on weekdays. The reason for such a spike in non-human traffic is pretty simple. When the core asset of your business model is classified ads and unique listings, your competitors are always on the run to grab your listings and data and take away your legitimate web traffic. Fraudsters will try to pull your site down with malicious bots that not only scrape your content, but also create fake accounts and fake listings. In essence, online bots are a cause for a number of online frauds on classified portals that has a direct impact on KPIs and business revenue.

The Business Impact of Bad Bots on Classifieds is a Massive Loss

This global lucrativeness and the business model of classifieds makes it one of the most vulnerable industries to web scraping and innumerous associated bot frauds.

1. Web Scraping of Listings
2. The competitive advantage of a classified portal is its unique listings that are updated by users on a very regular basis. Such unique listings are often prone to web scraping. Businesses use mass-scale listing scraping techniques to steal new listings generated on competitors' websites. They then add the stolen listings to their own websites, with or without consent of the end-user. This is done to drive away legitimate traffic to their website. Your classified portal losses out on genuine unique visitors, search rankings leading to lowered brand value and decreased revenues.


2. User Data Theft
3. It gets serious when your user data is stolen by fraudsters for lead generation using automated bot attacks. Having user data brings huge responsibility of protection from hacking attempts, account hijacking, carding fraud and user data theft done via bot attacks.


3. Fake Form Submission
4. Listings and ads via advertisers are a major source of revenue for classified portals. Spam bots submit fake leads that are of no use for sales teams leading to lost advertiser confidence. Fake accounts and fake listings are created by miscreants to pull down your listing website, spoil user experience and hurt your brand image.


4. Skewed Website Analytics
5. Marketing analytics products and a host of data analytics tools are lacking human vs bot traffic identification capabilities, not to mention they do not give you much insight into good and bad bot analysis, the origin of such traffic and what bots are looking at on your classified portal. Since advanced bots can perfectly mimic human behaviour, even when captured as human traffic, distort visitor tracking and reporting, rendering critical business marketing tools ineffective. Non-human traffic that seems to be genuine leads to misinformed business decisions like investing in wrong promotional channels and wrong landing page optimization.


5. Negative SEO Attacks
6. Malicious bots downgrade website SEO ranking as original content is duplicated all over the web without attribution or back links. Your sales team efforts and investment to get unique listings is no more a competitive advantage when your competitors and fraudsters engage is mass scale web scraping to steal unique listings from your classified portal. Now, they enjoy the same or even better search ranking to attract more visitors and capitalize on the new visitors.


6. Slow Site Performance
7. The implications of excessive web crawling and heavy bot traffic goes beyond just online bot fraud and has a direct impact on classified website’s performance. Heavy bot traffic on website strains the web infrastructure and slows the website access for legitimate traffic. This is not a great sign with respect to user engagement and drop rates on your classified portal. It just takes a few seconds for the user to switch from you to your competition just because you were busy rendering the web pages to bots and not genuine users.


7. IT Overhead & High Costs
8. Rampant scraping bots increase server capacity usage, server cost and put financial and cost burden on internal IT teams. This overhead can be perfectly eliminated by blocking scraping bots and controlling excessive crawling that doesn't add any business value.

The impact of online bot fraud on classified and listing websites is nothing but lost customers, lost brand reputation and ultimately lost revenue. Considering the quantum of bot traffic and equivalent global bot fraud losses, it is time that listing marketplaces, both small and big, horizontal and vertical start implementing a real-time bot mitigation strategy.