I need a PHP expert developer to write a PHP script that crawls ".uk" websites and records the following information in a MySQL DB:
- the domain name
- all WHOIS records for this domain
- email addresses found on website
- phone numbers found on the website
- top 50 most common keywords used on the site
- timestamp of the crawl
The script must follow URLs to other ".UK" websites and consequently add them to the database for future crawling. The script needs to re-crawl domains which have not been crawled in the past 3 months (based on timestamp of the crawl). The script needs to have a seperate table where certain domains can be blacklisted from crawling.
*** With your bid please tell me which PHP libraries are you planning to use, and also show me an example of your recent PHP web-scrape/web-spider work. Bids without this information will be ignored.
50 freelancers are bidding on average £310 for this job
I am really interested to do this job and get started right away. Can we discuss the project details? Payment after you're completely satisfied nothing advance .
This kind of project is our forte. Our team can definitely complete this for you. This project is pretty straight forward. I just have a few questions for you for discussion