How will we keep up with it all?


Have you ever wondered how much information we are generating? A recent study at the University of California at Berkeley, shows that the world produces between 1 and 2 exabytes (an exabyte is 1018 bytes or a billion gigabytes) of unique information per year, which is roughly 250 megabytes for every individual on the face of the earth. Most of this data is already digital, as only .003% of this total is due to printed documents.

Looking solely at the web, static pages on the web are estimated at over 10 billion. Taking into account all the web-accessible information, such as web-connected databases and dynamic pages, there are roughly 550 billion web pages. The amount of web content is also growing at a phenomenal 100% per year. In terms of storage, it would take 7,500 terabytes ( a terabyte is 1024 gigabytes or 1012 bytes) to store all this web data in one place based on estimated average page sizes.

The numbers are simply staggering, and if not bad enough email is an even bigger content source than the web. Estimates on email range from 610 billion to 2000 billion email messages sent each year. Conservative estimates put the storage for email between 11,000 and 20,000 terabytes.

If your wondering how we will ever sort through all this information of Carl Sagan proportions, your not alone. Without an automated solution, one could imagine becoming buried in useless digital data, as email volumes increase and the web continues to expand. The most promise may come through the use of intelligent software agents, or bots as they are affectionately called, that work on our behalf.

Bots are simply computer programs that act without human intervention to accomplish a task. There are a variety of Bots in use today. You can find numerous shopbots on the web that will scour the web on your behalf searching for the lowest price on a particular item. Many other vendors are touting bots, which will function as virtual sales reps or customer-service agents. These bots come complete with high quality animation, video, and audio. They will chat with potential customers or current customers until they draw a blank and then refer the customer to a real person. Granted they may be crude today, but bots may be the only means to manage this information explosion that is occurring.