OWASP Automated Threats

Collect application content and/or other data for use elsewhere.

What is Scraping?

Collecting accessible data and/or processed output from the application. Some scraping may use fake or compromised accounts, or the information may be accessible without authentication. The scraper may attempt to read all accessible paths and parameter values for web pages and APIs, collecting the responses and extracting data from them. Scraping may occur in real time or be more periodic in nature. Some Scraping may be used to gain insight into how it is constructed and operates - perhaps for cryptanalysis, reverse engineering, or session analysis.

When another application is being used as an intermediary between the user(s) and the real application, see OAT-020 Account Aggregation. If the intent is to obtain cash or goods, see OAT-012 Cashing Out instead.

Scraping is also known by terms such as API provisioning, bargain hunting, comparative shopping, content scraping, data aggregation, database scraping, farming, harvesting, metasearch scraper, mining, mirroring, pagejacking, powering APIs, ripping, scraper bot, screen scraping and search / social media bot.

The symptoms of Scraping

  • 1
    Unusual request activity for selected resources (e.g. high rate, high number, fixed period)
  • 2
    Duplicated content from multiple sources in search engine results
  • 3
    Decreased search engine ranking
  • 4
    Increased network bandwidth usage with throughput problems
  • 5
    New competitors with similar service offerings

Sectors targeted by Scraping

  • Education
  • Entertainment
  • Financial
  • Government
  • Health
  • Retail
  • Technology
  • Social Networking

Can InfiSecure prevent Scraping?

InfiSecure is anti-scraping service. InfiSecure can block even the most advanced bots trying to scrape a website. InfiSecureā€™s bot detection engine put basic checks like capping rate of application use per session/user / IP address / device/fingerprint to advanced bot detection techniques like bot fingerprinting.