Simple script needed to scrape page source of websites.
1) i will upload notepad of URLS- millions
2) then we will have a blank field to enter the variable we are trying to find on each websites page source code
3) I can export the results to an excel file with a status next to each url FOUND or NOT FOUND
Will use my server
45 freelancers are bidding on average $128 for this job
Hello, I'd be glad to develop a simple script that searches for desired variables in webpage source code. Looking forward to hearing from you soon . Best regards,
Obviously we would have to use regular expressions to match the variables Or do you want an exact match ? let me know and want a simple gui or a web interface ?
For millions of URLs its better to use compiled code in C, that uses threads or epoll for faster requests. I can do the program in Python too, but will be slower.