One of the major ironies of the InfoSec community is that the bad guys too often have more community behind them than the good guys. Ironically, hackers are among the best organized communities with efficient exchanges of information. There is a tight-knit global community of villains behind most malware and malicious hacks, from BredoLab, with an estimated 30 million bots in its network capable of 3.6 billion spam emails a day, to Grum, recently taken down (for now) but at its peak capable of 40 billion spam messages daily with more than half a million hostage computers at its bot command.
Incapsula is calling on the InfoSec community to band together to help solve some of the most common problems we all face on the Web. We believe that, as a community, we can crowd source a solution to fight back to defeat the worst bots and also recognize and reap the often-overlooked benefits of the good bots.
Today we are launching BotoPedia, the first online community-sourced directory that consolidates information about active bots. BotoPedia will enable website owners and operators to identify non-human traffic, allowing them to enforce policies to allow good bots access to the web site while denying access to malicious bots. We agree with Shawn Henry, former FBI director, who recently spoke at Black Hat, saying, “Intelligence is the key to all of this. If we understand who the adversary is, we can take specific actions.”
BotoPedia is a play on the stunning accomplishments of Wikipedia, a free worldwide directory of information on the widest variety of topics imaginable and maintained by a global community of volunteer experts. It’s a simple idea powerfully executed. We can do the same to protect our businesses and organizations on the Web. If we work together as a community, we can crowd source solutions to make the web safer for all of us and to also create a much better Internet user experience.
With BotoPedia, Incapsula is donating much of the proprietary intellectual property that we have developed over many years working with thousands of customers worldwide. We have also created sophisticated lookup and query tools to make it fast and easy to separate good bots from bad bots. In other words, we recognized that the success of our business was built on the pooled knowledge of our own community. We want to share that success and knowledge and invite you to join us. A community gets stronger as it gets bigger.
And we are just getting started!
We’re starting with the 50 most common or dangerous bots in BotoPedia. Our goal is to encourage community contributions to grow BotoPedia to hundreds of bots, and eventually the most comprehensive directory of bots in the world. We’ve designed BotoPedia to be simple to use. Combined with the power of sharing, we think BotoPedia can multiply the impact of pooled community knowledge. We’ll track classification (type of bot), organization (the entity that operates the bot), user-agent (the descriptor used by the browsing software to identify itself), IP address (range of IP addresses used by the bot) and a detailed description of each bot.
Website operators will be able to quickly look up and identify bots by entering any piece of information about the bot and perform a database-wide search to find the bot. Registered users will be able to contribute to BotoPedia and share their comments about known bots and contribute data on new bots. Bot operators will be able to register their bot and provide all the required information that will then be vetted by Incapsula, the sponsors of BotoPedia.
Many people will also be surprised by the good bots that visit their websites. It’s just about as important to know the good bots as it is to block the bad bots. Good bots raise awareness of your website on the Internet and drive more traffic. While everyone recognizes Google’s Googlebot (the most likely bot to visit a web site according to our research), but how many will recognize Soso Spider? That’s the 7th most likely bot to visit a website. We’ll share more of our findings from BotoPedia in the coming weeks.
Meanwhile, join our community. Let’s take back control of our web!