Build 2 web scraping tools and populate MySQL database

In Progress Posted Sep 28, 2013 Paid on delivery
In Progress Paid on delivery

Stagecoach Software Pty Ltd requires a freelancer to create two automated web scraping tools and a MySQL database suitable to serve as a back end to a forthcoming website to be built on Ruby on Rails.

The first tool needs to run on an automated schedule, initially at a set time every week. The tool is required to scrape a single large web site listing thousands of unique products and then populate a MySQL database with details of the products and prices sought. The details to be scraped appear in a consistent format on the site. The first scrape is required to obtain details for all products whilst subsequent scrapes are required to track new products listed or changes of price on existing products. The freelancer must populate the DB using the scraper as part of the project.

The second tool is required to automatically download and scrape 2 to 4 pdf consistently formatted text files published every week on a different website that list the realized prices for a significant proportion of the products listed in the first website. The second tool is required to populate a further table in the MySQL database with these products and price results.

The MySQL database must then automatically combine the data from the two scraping tools and calculate the difference between the realized price and the initial listed price(s). The DB must be loaded to a host site nominated by me. The scraper tools must be able to be easily activated from the host site and full operating instructions must be provided.

Full details of the sites to be scraped and the required DB columns will be supplied to the successful bidder.

This project will be followed with the creation of a RoR website that will present graphs showing aggregate and individual data obtained from the database and allow user searches. Preference will be given to bids that make that better facilitate this second stage.

MySQL Ruby on Rails Web Scraping

Project ID: #4975478

About the project

14 proposals Remote project Active Oct 1, 2013

Awarded to:

ltravenvw

Hello, I'm an experienced Web developer (6+ years, Zend PHP Engineer certification, Oracle MYSQL developer certification). For the last 2 years I've worked Ruby and Ruby on Rails. I have experience writing content scra More

$1555 AUD in 20 days
(7 Reviews)
3.1

14 freelancers are bidding on average $1128 for this job

taro

I'm interested and would like to know details and see site to scrape and sample PDF document as well.

$1400 AUD in 10 days
(40 Reviews)
7.1
kamosion

Hello. I can make it for you in shortest time. I've made professional tool for Web Crawling and parsing in Python (executable version also - 32 and 64 bits) 1. Multi Threading. (parallel execution) 2. Very Simple Cr More

$1111 AUD in 10 days
(2 Reviews)
3.6
osnofasl

Good evening. I am a computer engineer with most web development made in RoR. I need more details about the places where to scrap data from, but I am ready and able to do the job. Finally, there is another freelance More

$1222 AUD in 20 days
(1 Review)
1.5
vikas21

Greetings, I have been working in the web industry for the last 6 years providing customized designs .I have wide and insight level of experience in web designing and web developing from scratch building of code a More

$1333 AUD in 10 days
(0 Reviews)
0.0
omanasoft

I offer to provide the software that meets your needs. It will be a solution in Java, built on top of my own already developed, demonstrable web-scraper tool (OmanaWIM Tool). The web-scraper tool is designed for comple More

$1250 AUD in 3 days
(0 Reviews)
0.0
PennyAuction2

We are a company expert in WEB SCRAPING with hundreds of clients. This is a sponsorized bid (we had to pay for that) because we are the only company that can do this job. Reply in private to see our demo.

$1111 AUD in 3 days
(0 Reviews)
0.0
amtechconsulting

Hi, We are a team of experience web application developers. We have past experience with scraping projects. We use html parsing framework works to ensure data accuracy and compliance with web standards. Please get i More

$800 AUD in 10 days
(0 Reviews)
0.0
MobileCreative

Hello, We have a team of designers and developers focus on developing and implementing enterprise products and solutions. We have over five years of experience, a passion for providing great service. Since we had si More

$1263 AUD in 20 days
(0 Reviews)
0.0