crawl and extraction of a site's data

A client needs to crawl a site and save emails on each of the forums on this site. He has a software that can do this for him, but he wants to create a script that does it online. Site is: foromania dot com It's in spanish, but you'll see it's pretty easy to understand. It's a collection of thousand of forums where users post messages. Click on the first one, Arte, then the first or second one. You'll see the email icon at the top of the page when you read a message, or somewhere else. It depends on the forum. So, we need to: - Scan all pages in each forum looking for emails and save the name of the forum from which it links ok? In red and in parentheses you have the number of forums in each category. - We need to crawl emails that can be found in all forums. - The script should allow for detection of duplicates and eliminate them. On the list of a category you see this the list of forums. So we need to visit each forum, crawl emails and save database with category and name of the forum. Or have a folder with each category and inside each .xls file with emails of that forum, with name "the forum itself". Just run them from your server and give me the results. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform Data Extraction