Find Jobs
Hire Freelancers

Google Trends Perl Command Line Utility -- 2

$30-250 USD

Completed
Posted almost 10 years ago

$30-250 USD

Paid on delivery
The Script will run from the command line. It will take input and perform a series of searches on [login to view URL] I have a Python script that have to be rewritten in perl. This Python script authorise in Google and send request to Google Trends to get csv file. The script should slurp input file with queries into hash and execute search for each query. Input file format: 1 query per line. Lines ending is "\n". This file can contain up to 1,000,000 queries. Each query should be urlencoded if needed. Example: http://www.google.com/trends/explore#q=I%20love%20you The script should be configurable with following internal parameters: [login to view URL] region (a two letter code which can be passed to the search url); “all” by default [login to view URL] date in format YYYY-MM; default value is 2008-1 [login to view URL] date in format YYYY-MM; default value is the previous month [login to view URL] authentication: user email and password. The script will then take the input and perform a series of searches on Google trends for a given query between the start and end date. It will download the data from each of these searches as a csv file (e.g. the export “Download as CSV” button at the top of the search results). Then the script should parse csv file and write results. Parsing should extract year and frequency data only. Resulting file should have number of line equal years search. Each line should start with ‘YYYY,’ followed by frequency data separated by commas. Like: 2008,3,5,8,3,1,….. 2009,0,3,5,1,3,…. Each line should have 53 (52 for weekly data and 1 for year) records separated by commas. The query name should be transformed using Digest::MD5 md5_hex and this will be result file name for given query like ‘e2b5f8691dde034ec7a8b1465b93335e’. The results file for each query should be written using following directory structure ‘./<first 2 chars of file name>’/ next 2 chars of file name>/ file name’. So the file file written as an example above should be kept lile ‘’./e2/b5/e2b5f8691dde034ec7a8b1465b93335e’. Furthermore, the script will need to be logged in to google trends, so it needs some way to handle this. Also, I’m not sure if running all these searches may cause problems with google not liking the large volume of search requests. Error message "You have reached your quota limit" usually indicate this case. So the script may need to overcome this. This could perhaps be achieved by just slowing down the searches. Also, if a rare search term is entered, the script may come across the page “Your terms – blahblahblah - do not have enough search volume to show graphs.” In such a scenario, the query should be written to 'unsuccessful' log file. Queries of successful searches should be written to "done" log file. The script should read these queries from "done" file, add queries from 'unsuccessful' file and do not repeat them. So the script can be stopped at any time and start again from the query next to "done" list. The script should well structured and heavily annotated (including each sub input and output). PLEASE INDICATE IF SIMILAR JOB WAS DONE IN THE PAST.
Project ID: 6138337

About the project

5 proposals
Remote project
Active 10 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I have completed the Perl script. It implements all of the required features. It could use some more testing, such as with a request for interest data over multiple years. The script can login to a Google Account; however, Google's quota is the bottleneck in this case. I have found a way to get HTML/Javascript/JSON output without a login, and using a Tor proxy. This method might be usable for many different proxy IP addresses. The next task will be to implement parsing of the HTML code in order to output the CSV data. Thanks again for working with me on this project.
$30 USD in 2 days
0.0 (0 reviews)
0.0
0.0
5 freelancers are bidding on average $157 USD for this job
User Avatar
Hi, I can develop a command line based Perl script to submit request to google trends, download the csv and parsing the result. Looking forward to start this project asap with you
$222 USD in 5 days
5.0 (26 reviews)
5.3
5.3
User Avatar
Hello. More 20 years programming experience. I need take look on your python script to set real time and price. Regards. -----------------------------------------------------------------------------------------------------------------------------------------------
$155 USD in 3 days
5.0 (8 reviews)
3.4
3.4
User Avatar
Hello, This sounds like a interesting project. While I have not worked on Trends before, all of my past scraping and crawlers were done in Perl only. You can have a look at my past projects to know more. Please share your Python script so I can quickly check the feasibility. Regarsd Vishy
$222 USD in 3 days
5.0 (4 reviews)
3.4
3.4
User Avatar
Hi, I am perl developer and done several perl projects. Let me know if I can help you with this script. I assure you to give fully working script with proper comments. Thanks and Regards, Ritu
$155 USD in 5 days
4.5 (3 reviews)
2.6
2.6

About the client

Flag of SWITZERLAND
RiverEdge, Switzerland
5.0
3
Payment method verified
Member since Oct 4, 2006

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.