Puppeteer - Website Crawling project

€250-350 EUR

Closed

Posted

almost 2 years ago

€250-350 EUR

Paid on delivery

Hi developers, I hope you are all having a lovely day. I need a crawler that crawls websites, most importantly the ads on these websites. The crawler screenshots the website, the ads and saves the target and source urls of the ads. The crawler is optimized for a small list of websites, it should be easy however if new website styles can be added easily. There exists a prototype that already implements these features, so this project will mostly be re-coding the prototype but better and more professional. The prototype is JavaScript, there can be many things copied from the prototype. The new crawler should be coded new and implemented in TypeScript. Also, the finished data is stored in AWS S3 and a database (Preferrably AWS DynamoDB). There should be an API implemented (secured with Bearer Token only) that exposes the crawled data and lets the API consumer add new URLs to crawler. It is very important, that the new crawler is reliable and the code should be as structured and modular as possible, to add logic, if e.g. a website is added. The crawler+API should run in Docker on an AWS EC2 instance. I will provide you with the source code of the app, as soon as we start chatting. Technologies required: - TypeScript - Puppeteer - Node.js/Express - AWS S3, EC2, DynamoDB - Docker Best regards, Florian

Project ID: 33772770

About the project

22 proposals

Remote project

Active 2 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

22 freelancers are bidding on average €499 EUR for this job

@AwaisChaudhry

Hey there, I have 4 years of working experience in web technologies and I have a strong working knowledge of every possible framework which is in the market. In terms of client-end I know: Node.js, Web Scraping, Docker, Amazon Web Services and Typescript Also I have done tons of work on freelancing career. I am waiting anxiously for a positive response from your side. And I have read your requirements as well. Thanks and regards, Awais

€750 EUR in 18 days

4.9

(17 reviews)

7.0

@PoojaRautela417

Hi there, I can write script in Node.js using Puppeteer according to your requirement. Please ping me back so I can start right away. Looking forward to hearing from you soon. Thanks & Regards Pooja Bohra

€500 EUR in 5 days

4.9

(70 reviews)

6.2

@oleksdevr

Hi . I am a fully experienced NodeJS developer. I have strong background in Web crawling using Puppeteer. I’m confident I can deliver high quality result. I’d like to discuss more about the project. I can start your project soon. Thank you.

€500 EUR in 7 days

5.0

(6 reviews)

4.8

@rashidamjad

Hi there, I am Rashid, Hope you are doing great. Firstly about the Puppeteer - Website Crawling project that I have a great experience in it. I am very confident to pull it off once awarded. I am a Full stack developer with a team of experienced developers and designers on my side, together we have 5+ years of experience in Node.js, Typescript, Docker, Amazon Web Services and Web Scraping I aim to establish a long term working relationship with my clients. It would be my honor to have you on board with us. Please let me know what is needed to secure the bid! Thanks for your consideration. Rashid Amjad.

€650 EUR in 15 days

5.0

(7 reviews)

5.0

@Devrits

Hi!, I’ve carefully checked your requirements and really interested in this job. I’m full stack node.js developer working at large-scale apps as a lead developer with U.S. and European teams. I’m offering best quality and highest performance at lowest price. I can complete your project on time and your will experience great satisfaction with me. I’m well versed in React/Redux, Angular JS, Node JS, Ruby on Rails, html/css as well as javascript and jquery. I have rich experienced in Node.js, Typescript, Docker, Web Scraping and Amazon Web Services. For more information about me, please refer to my portfolios. I’m ready to discuss your project and start immediately. Looking forward to hearing you back and discussing all details.. Thanks

€750 EUR in 4 days

4.9

(21 reviews)

4.4

@brananikolic0091

Hi, I am a senior developer with 5+ years of experience. I just have read your description and can start right now. Skill set - languages : Javascript/Typescript, Python, PHP, Golang - frameworks : Next, Nuxt, Nest, Node.js, Express, Laravel, React, Vue, Svelte, Angular, Django, Flask, FastAPI - database: MySQL, PostgreSQL, MongoDB, Firebase - project: AWS, Kubernetes, Asana, Trello, Heroku, Git - others : Tailwnd, Bootstrap, MaterialUI, Web3 I hope that I will be able to share more about my experiences with you soon. Thank you.

€500 EUR in 7 days

5.0

(1 review)

3.4

@dandiuzhenko

Hi, Dear I've read your requirements very carefully and I`m really interest in your project. I have 7+years of great experience in ExpressJS, Node.js. Also I've 3years of working experience in Docker, AWS, Puppeteer. Your success is my success. I would like to build a good relationship with you through the completion of a successful project. I'll do my best to make your project a success. Please contact me so that discuss more regarding project. Then I can start right away and complete your project successfully. Thanks.

€500 EUR in 7 days

5.0

(1 review)

3.2

@EDGESOL

Hi, I Hope your life is going good!! We are experts in Auction Website & Mobile App development I will happy to show you a demo via personal chat. I have read your requirement Puppeteer - Website Crawling project and understood that you are looking for Developer team to develop a Property Auction App & Website with Admin. We have a full team so you will get everything from our side. Designer, Web Admin/API Developer, Web App Developer, Android App Developer, iPhone App Developer, QA (Testing), Project Manager. Let me tell you in this project we will work on everything like, -> Mobile App Design -> Website Development -> Admin Panel- Backend -> API- Web Services -> Android App Development -> iPhone App Development You will get the best price with the best quality of work and service from us than others because we like to keep a long-term relationship with our clients. Can you please share your best time for the chat then we can discuss further and move ahead. Awaiting your reply to move further. Best Regards, Muhammad Fawad

€555 EUR in 4 days

5.0

(1 review)

2.8

@mandionisio154

Hello, Florian After carefully reviewing the experience requirements for the job description, I feel that I am suitable for the job. I have 7+ years of rich experience in full stack development. I am specialized in Node.js, Python/Django for back end and React.js/Next.js and Vue.js/Nuxt.js, Typescript for front end. I am also very familiar with AWS cloud structures such as AWS EC2/S3/Lambda/DynamoDB and databases such as MongoDB, GraphQL, PostgreSQL and Redis. I mastered git for effective version controlling and comfortable with project management tools like Jira, Trello, Pivotal Tracker and etc. I have expert knowledge of Devops such as Docker and Kubernetes. I also have strong communication, good teamwork and excellent problem-solving and troubleshooting skills. After all, I will bring you competitive and relevant result that you're looking for and it might bring us futuristic cooperation in this field. In recognition of this project specification, I will pose the reasonable and receptive budget and timeline that you can buy into. As I am excited about this opportunity, I am thankful if I might have a chance to talk with you via chat. Best Regards Dionisio

€500 EUR in 7 days

3.6

(2 reviews)

2.0

@maksimjevdjic122

Hello, how are you? As a senior developer, I always concern with Security while development. Skill Set: - Basic skill C, C++ I have developed anti-virus, VPN, game engines and so on. Go-lang I have analyzed go-ethereum core code and worked on forking it. - Blockchain development NFT (Non Fungible Token) Marketplace Development Smart-Contracts (Solidity, Truffle framework) Token development based on Ethereum and Binance (ERC20, ERC721, ERC 1155, BEP20, BEP721, BEP1155) DLT Apps & Private Blockchain Development Cryptocurrency Wallet Development for Linux, windows, Mac, Web. - Network Security Linux server management IPsec, SSL VPN development Thanks for reading

€555.56 EUR in 10 days

0.0

(0 reviews)

0.0

@vladyslavmazurok

Hi! I checked you need a highly proved javascript programmer for your project. For the last 8 years, I worked on various javascript libraries like React/Redux, Angular, Node and gained knowledge with payment integration, real-time broadcasting, analytics, source management applications. I can start your project immediately and promise to deliver good result on time. As I have rich experience in Typescript, Amazon Web Services, Node.js, Docker and Web Scraping, I would be a perfect match for this project. Looking forward to hearing you back and discussing all details.

€555 EUR in 7 days

0.0

(0 reviews)

0.0

@ayesha0124

Hi, I can help u as i have done several similar jobs related to Docker, Node.js, Typescript, Amazon Web Services and Web Scraping, I have read the details and furthermore discuss about it, plz discuss with me in detail. Regards

€750 EUR in 8 days

0.0

(0 reviews)

0.0

@lakimo21

Greetings ! I am skilled Frontend engineer . I am experienced in such as Web Scraping, Docker, Amazon Web Services, Node.js and Typescript. I can start right away and want to discuss in detail. Please contact me to discuss more about this project. With regards

€400 EUR in 5 days