Potential investors, journalists, etc. are always asking us at PubSub.com for numbers that will reveal our capacity. Can we handle the load? Well, in part to answer these questions and in part as a bit of an inside joke, we've been tracking what we call the Hyperbole number -- a number designed to sound impressive while also being completely truthful. Our Hyperbole number is computed by multiplying the number of messages we receive by the number of subscriptions we compare them against. (This is reasonable since we compare *every* message we receive against every subscription.) Well, our Hyperbole number usually oscillates between 3 and 7 billion matches per day and is increasing from day to day as we get more subscriptions and find more blogs or data sources (like the SEC filing information). That certainly sounds impressive! Of course, it is only a tiny fraction of what we're capable of. Our matching engine still rarely shows CPU consumption over 5% and then only for a moment or two.
I thought of the hyperbole number today as I read a number of articles and blog entries that were gushing about the tremendous number of "searches" that are processed by Google and other search engines each day. The consensus seems to be that there are about 500-600 million searches entered each day on the Internet and about 250 million of these are handled by Google. Of course, those numbers are meant to impress. On the other hand, if you compare them to our Hyperbole number, it seems that we're currently processing between 6 and 14 times as many "queries" per day as the entire web based search industry! Now, how's that for Hyperbole!
bob wyman
Bob,
So does this mean you have over 10,000 servers ;-)? Sounds like things continue to go well. Going to be adding some more queries soon.
Thanks
Stuart
Posted by: Stuart Watson | February 18, 2004 at 16:22