Find Jobs
Hire Freelancers

Data Scrape and front end design developer needed for large big data and analytics platform -- 2

min £36 GBP / hour

Closed
Posted over 4 years ago

min £36 GBP / hour

Looking for a talented developer with experience in four key skills: 1) Data scraping from multiple (hundreds) of different websites, with thousands of sub pages of information. Each website is laid out differently however largely contains the same key information I wish to scrape. 2) Application of machine learning to the results of the scraping. This is absolutely crucial to the success of the platform! Further details on this below. 3) Building analytics from the results of the scraping 4) The development of a front end platform that would work with the scrapped data and have a combination of drop downs and free text to bring up search results. Regarding the application of machine learning: The reason this is so important is that for each website I want to scrape, there are between 200 – 2000 sub-pages per website to scrape and allocate different values to. If you scrape a website page with a blurb of information in text format and I allocate different values to that text, I would need machine learning to learn what I have done, and allocate the same values to text that reads the same/contains certain key words and/or phrases from other web pages. For example: Website blurb [login to view URL](754) "Ros is head of Eversheds Sutherland’s Competition, EU and Trade Group and has been a Partner in our commercial practice since 1989. She specialises in cartels, competition investigations, distribution and market studies/investigations, and also advises on all aspects of EU law. Ros acts for major companies in the automotive, consumer, energy, financial services, food, insurance and pharmaceutical sectors. Her clients include Pentland Brands, The Finance & Leasing Association, Volkswagen Group, RWE and Volvo. Ros has been recognised for many years as a leading individual in Chambers Guide to the Legal Profession. She is currently described in Chambers UK as "very engaged, a very smart and practical lawyer with deep commitment to the client" and in Chambers Global as “one of a kind - calm and collected in crisis situations and can cut through a lot of information and get to the crux of the issue". She is also consistently recognised in The World's Leading Lawyers and Legal 500 for competition law and Brexit advice. In March 2017, Ros’ team won the Global Competition Award for “Behavioural Matter of the Year – Europe” for its work on the European Commission’s trucks cartel investigation, which resulted in the first ever EU settlement post-receipt of a Statement of Objections. Ros is a regular speaker at competition law conferences, as well as at sector focused events. Furthermore, since July 2015, she has spoken extensively on the subject of Brexit to numerous organisations including the British Chamber of Commerce in Hong Kong, the Dublin Chamber of Commerce, the Franco-British Chamber of Commerce in Paris and a number of regional Chambers of Commerce. Ros is a member of the Joint Bar Council and Law Society Working Party on Competition Law" Example Machine Learning Values: 1) Main practice area: Competition 2) Sub Practice area: cartels, competition investigations, distribution and market studies/investigations 3) Sector Expertise: automotive, consumer, energy, financial services, food, insurance and pharmaceutical 4) Clients: Pentland Brands, The Finance & Leasing Association, Volkswagen Group, RWE and Volvo 5) Name: Ros = female 6) Title: Partner Example static values: 1) Email address 2) Phone number 3) Location 4) If available - link through to Linkedin profile
Project ID: 23368811

About the project

11 proposals
Remote project
Active 4 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
11 freelancers are bidding on average £38 GBP/hour for this job
User Avatar
Hello, thanks for the detailed requirements. Data scraping, ETL, data analysis, data processing, and data science are my main expertise for more than 8 years. My core language is Python, it's the best for scraping tasks. I have strong knowledge and wide experience with all your requirements. I have my team of data scientists and developers, so in need, we can do a faster turn around of your project. I'm sure I can help and sure that we can find ways of cooperation. I have done hundreds of similar projects for clients, including Fortune 500 and Fortune 100 companies (such as Chevron, Unilever, etc.). I have scraped thousands of sites. We process data from more than 10k sources annually, biggest scrapers get data from 20m pages daily. I have a Master Degree in Math, and I have considerable experience with data analytics for many years I have worked as Head of the Analytical Department of Corporate Clients in one of the biggest Ukrainian banks. Let's set up a call/chat to discuss the project scope in detail. Regards, Vlad
£40 GBP in 40 days
4.9 (56 reviews)
7.7
7.7
User Avatar
Hello, I think most people can do the all the task except task 2. By the way, I can do all of them and I will explain how and why. First, I have a PhD degree in machine learning and AI. I will explain a recent project which is quite similar to yours. In the project we have to crawl various news sites and then extract key information (like amenities, building name, address, link, etc.) automatically. This area is called information extraction and it's very challenging issue because the articles are unstructured and we want to automatically find a structure. Even with this complexity we were able to get an accuracy of 80% in most of the attributes. In your case, the framework would be quite similar but we might need ontologies for some of the attributes that you mentioned in the description. If ontologies are not available then we might have to create one. Besides that, your project will also need semi-supervised or reinforcement learning because we need to constantly improve our model with the changes in data. As you know that there are thousands of ways to represent the same thing in natural language, therefore we can't provide all the possible combinations upfront. For this sake, you need the model to adapt over time. You can see my real estate project in my profile. Please feel free to contact me in case of any query. * This messages is written by mobile device so pardon any mistakes.
£50 GBP in 30 days
5.0 (3 reviews)
5.8
5.8
User Avatar
Dear sir. I read your project description very carefully. I've really rich experience in Machine Learning & Big Data Programming, so your project is very interesting to me. I'm really confident about your project, and very eager to join it. If you give me a chance, I'll do my best to provide wonderful result. I believe this will be a good starting point of the business relation between us. Looking forward to your response. Thank you.
£36 GBP in 40 days
5.0 (3 reviews)
2.9
2.9
User Avatar
Hi I've done a lot of different web scraping projects and should have no issue with this. I am also experienced in multiprocessing so can set the scraper to scan multiple sites at any one time and if needed through proxy servers, so there are no issues with getting blocked. The collected data would then need to be stored. I think MongoDB might be best for this, as it sounds like we are storing documents and some meta data. The ML section can be done with Keras and we can create a model to search the larger number of records based on the ML learning material. I am assuming that after this the data structures will be more structured and thus an RDB be a better storage there, but we can use Mongo if a single DB engine is required. Thanks Marc Nealer A web frontend is not a problem to develop. I can build one in a could of hours with Django
£36 GBP in 20 days
5.0 (1 review)
2.9
2.9
User Avatar
Hello, 1. Experienced in data scraping with python, selenium, beautiful soup, requests 2. Neural networks can be applied for keyword extraction 3. Experienced in Data Analysis and Visualization using python libs 4. Python kivy or .net can be used for frontend. Let me know, if interested. Can provide sample work on this too. Thank you Arya
£36 GBP in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello , I am experienced in both data scraping and machine learning.I believe that with my skills I will be able to produce high quality data throughout this engagement. I also believe that with the vast websites you intend to scrape , my input will be of value. Thank you. Please reach out so that we can get started ASAP.
£36 GBP in 20 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, i'm an expert in highly responsive website with optimale web technologies.I could do the job perfectly. i will work this project with c # desktop application with buttons, progress bar, multithreading and saves the data in a csv file. everything is clear and detailed. Can we discuss more details about this project? then I will provide example data/script for you. I'm at your disposal for any further information. Waiting for your response!
£36 GBP in 40 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED KINGDOM
London, United Kingdom
0.0
0
Member since Jan 13, 2020

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.