Google Crawler Simulator

[Paperbag]

This program that I made with my buddy simulates a lightweight Google crawl server (really).

While Google uses a whole custom OS and thousands of servers we use one server and one program.

This does Google does but in a few minutes (depending on the site size)
This program is not available for download.

http://www.youtube.com/watch?v=MTeQ7sLmArg

 

[Zangrefia]

Wow, this is really, really nice. Are you going to release it?

 

[Paperbag]

Possibly never, this could be used wrong as it opens a load of connections to a web server.

I might consider.