Linux web scraping jobs

I need an experienced Linux admin who can help me set up a CentOS server to function as a web, database, and file server. The ideal candidate will be familiar with CentOS and have experience configuring a server with multiple services. Key Requirements: - Install and configure CentOS on a server - Set up Nginx as the web server - Configure the server to also function as a database and file server - Ensure the server is secure and optimized for performance - Provide documentation of the setup and configuration for future reference.

Introduction We are seeking proposals from experienced web scraping and data analysis firms to conduct a comprehensive data collection and analysis project. The purpose of this project is to gather detailed information on menu items, pricing, customer ratings, promotions, and deals from various food delivery mobile apps operating within our city. The goal is to mine this data for actionable insights to inform our advertising, promotional strategies, and competitive benchmarking. 2. Project Scope The selected vendor will be responsible for developing and implementing a solution to: • Subscribe to and navigate through multiple food delivery mobile apps. • Collect data on menu items including descriptions, pricing, customer ratings, and associated promotions or deals....

...information and disease descriptions. Requirements: Programming Language: Preferably Python. Libraries: Experience with web scraping libraries like BeautifulSoup, Selenium, and data processing libraries like Pandas. Output Format: Ability to export data into an Excel file using Pandas or similar libraries. Error Handling: Robust error handling to manage failed requests and parsing errors. Compliance: Adherence to the websites' terms of service and rate limits to avoid being blocked. Project Scope: Input: A list of CRM numbers and a list of CID numbers. Process: Automate the lookup and data extraction process. Output: An Excel file with the compiled data. Skills Required: Web Scraping Python Programming Data Processing Automation Excel File Generation ...

...person within each company. This could be anywhere between 10 to 30 people per company. - The information should be accurate and up to date. - If you have a strategy for obtaining this data, please share it in your bid. Skills/Experience: - Strong research skills - Data entry and data mining expertise - Proficient in Saudi Arabian business landscape - Attention to detail and accuracy - Web scraping - Web search - Leads Results should be in an excel sheet format, outlining the name, position, email address for each contact. Contacts for each company should be grouped together, end file should be in 1 sheet, not separate sheets. Send me: 1- Your strategy in finding those companies and collecting names and emails of relevant employees, 2- Total number of expected V...

Please Sign Up or Login to see details.

I'm looking for an expert to set up Redis in a Rocky Linux podman container. The aim is to ensure that it communicates over TLS using a Let's Encrypt certificate. Key Tasks: - Configure Redis to communicate over TLS - Integrate a Let's Encrypt certificate Ideal Candidate: - Proven experience with Rocky Linux and podman - Strong understanding of TLS and Let's Encrypt - Excellent problem-solving skills The successful candidate will make sure the Redis instance is optimally secured and can communicate over an encrypted channel. I can provide the certificates and build process. podman run --name rediscontainer -it rockylinux:9.3-minimal bash microdnf -y update microdnf -y install redis

I need a professional to scrape names and email addresses from specific public websites. I'm looking to populate a database with details from a range of public websites. This data will be used for lead generation and market research purposes. Ideal skills and experience: - Proficient in web scraping - Familiar with public data extraction - Able to structure data effectively

I need to Extract followers of group members of the link I will provide

I am looking for a web-scraping expert to retrieve both images and text from a specific website for research purposes. The text data will need to be structured and saved in a CSV file, including paths to saved images. Key details to consider: - Knowledge of web scraping tools and techniques is essential to accomplish this task. - Quality control to ensure accurate data extraction and image quality is critical. - Structure page information and freeform descriptions should be made available in a CSV file. - The images should be saved in their original resolution and quality settings. Please provide any relevant experience you have in web scraping, including free and paid services you have used before to help with your work. I would like the proj...

I'm in need of a knowledgeable Linux and AWS expert to assist me with various tasks related to command-line operations, system administration issues, and application support using AEM admin console. Key Requirements: - Strong command-line skills: You should be proficient in using the Linux command line for a variety of tasks. - System Administration: Experience in Linux system administration is essential, as I need assistance with file management, user management, and package management. - AWS Services: Familiarity with AWS services, particularly the AEM admin console, is a must. The ideal candidate for this role should possess a deep understanding of Linux, system administration, and AWS services to provide timely and effective job support.

I need a professional who can help me with scraping data API from a specific website or platform. I'm looking to extract specific data points or information and potentially set up a regular scraping process. Key Requirements: - Proficiency in Web Scraping: I need someone who has experience with web scraping tools and techniques. - Data Mining Skills: The ability to identify and extract the specific data points I require is crucial. - Automation Expertise: If you have experience setting up automated scraping processes, that would be a huge advantage.

Want to extract a list of profiles including name, country, city, email, phone numbers, link to profile for a list of followers for total output should be around 100K profiles that will be used to feed meta audience (customer list)

...but it's not functioning as expected. I need a developer to address these issues, described below. - **Issue Description**: The script is crashing when scraping certain Telegram groups, which is likely due to the fact that it's not respecting selections. The error presented is “ AttributeError: 'MessageMediaWebPage' object has no attribute 'document'” I can provide the script if you want to check how it’s working. I am open to a whole new script being made. So long as it functions in a similar way. If not familiar you can view telethon api - **Expected Behavior**: The scraping script is supposed to collect data that matches specific regex patterns. This means that it's not just supposed to collect all messages or o...

I am on the lookout for a tech whizz who can streamline and boost our Squid and Linux configuration to more efficiently handle a large quantity of proxies, potentially using a /48 ipv6 for each, amounting to a massive number (~65k for a /32 and 512k for a /29). Currently, we are facing: - Proxy connection issues - Resource utilization problems We are working with an enormous number of proxies, way more than 50,000. Our current server is running on Ubuntu 20. Ideal Skills and Experience: - Proven track record with Linux (Ubuntu) server management - Thorough understanding of Squid protocols - Expertise in managing high-volume proxy servers - Solid capability to optimize system resources We need someone who can provide a smooth and superfast experience to our users by re...

I need a script that can scrape the Google Play store specific data from ...result. Key Project Details: - I require the script to scrape app names, descriptions, emails, download counts, and URL of the app. - The scraped data should be compiled into an Excel spreadsheet format for my review and analysis. - deploy the script into my server The Ideal Freelancer: - Proficient in web scraping tools and techniques - Experience working with Google Play store data - Competent in automating data extraction and processing - Ability to deliver a reliable, daily scraping solution - python experts, node.js experts If you think you're the right person for this job, please get in touch with your relevant experience and a brief overview of your proposed approach. The scrip...

I'm on the hunt for a skillful coder proficient in web scraping, specifically for Website A. The goal of this project is for obtaining valuable and significant data for analysis, competitive intelligence, and price comparison. While there's no explicit mention of the type of data to be scraped, I anticipate that a capable offering will cater to a variety of needs such as: - Text content - Image content - Metadata Ideal Experience and Skills: - Proven experience with web scraping - High proficiency in handling large data sets - Comfort in navigating Website A - Ability to collect comprehensive data categories Looking forward to a potential collaboration to achieve this goal!

I need a developer to help me create an automated web scraper. - This scraper should be able to operate on multiple websites. The ideal candidate should have experience in web scraping, data extraction, and automation. The ability to adapt to different types of data and websites is essential. Thank you

We are seeking a skilled developer or development team to create a comprehensive web crawler system. This system should scrape, structure, and update information from web pages, handle large projects, and be accessible via a powerful API. The system should offer user-friendly features for both individuals and businesses and include flexible user management for both customers and employees. Project Requirements 1. Web Crawling and Data Extraction -Develop a system capable of scraping entire websites and extracting data from each page. -The system should divide the scraped information into sections to facilitate easy reading and understanding. -For e-commerce sites, the system should identify and structure the following information: --Product category (including ...

Hi, I need the following scraped. You can use scraping software or script. YellowPages might be the easiest - law firms and lawyers - only name and email - Colorado, USA

I need an experienced Linux admin who can help me set up a CentOS server to function as a web, database, and file server. The ideal candidate will be familiar with CentOS and have experience configuring a server with multiple services. Key Requirements: - Install and configure CentOS on a server - Set up Nginx as the web server - Configure the server to also function as a database and file server - Ensure the server is secure and optimized for performance - Provide documentation of the setup and configuration for future reference.

I need historical stock prices and income statement data, deliver python script to collect this data for an arbitrary stock from google finance.

Hello, This is a $20 project. Looking for a freelancer to develop a python script to retrieve restaurant data from Google Maps using the Maps/Places API. The script should target restaurants that have a phone number and *do not* have a website. The inputs I need to enter are: - Location (latitude and longitude) - Search Radius (in meters) - Ensure the business/place is a restaurant, bakery, etc. Key Data to Retrieve, output in a .CSV: - Business Name - Address - Phone Number - Google Maps URL Thank you, trooper501

I'm in need of a skilled professional to help me create a data mining script. The task involves mining emails, data from a certain website and delivering the data in...and delivering the data in a raw text file format. Key Requirements: - Develop a data miner script that can effectively scrape email data from websites. - Ensure the script can handle multiple sources and is scalable for future needs. - Deliver the mined data in a raw text file format. Ideal Skills: - Proficiency in web scraping and data mining techniques. - Experience with developing scripts for text data mining. - Strong knowledge of scripting languages suitable for scraping like Python/BeautifulSoup, R, etc. Please provide samples of your previous similar projects or portfolio that demonstra...

I'm looking for an Indian Python developer to create a script that can retrieve specific data from Telegram. Key Points: - The script should be able to fetch phone numbers and migration of users from Telegram. The ideal candidate should: - Have a strong background in Python development - Be familiar with working with API's - Have experience in web scraping - Have a good understanding of data privacy and security - Be able to deliver the project on a monthly basis. Looking forward to hearing from Python developers with relevant experience.

I'm looking for a Linux developer experienced in firewall handling and know PHP Apache seriuos developer for long term Key Tasks: Ideal Candidate: - Proficient with Linux environment - Experience in system administration - Understanding of different system tasks

is New York based Ecommerce company involved in processing of online print orders since 2010. Key Responsibilities - We are looking for AWS Linux Web Administrator to set up new and modify the existing AWS EC2 instance running Laravel application on Bitnami LAMP stack. Set up new dev server on AWS or Digital Ocean. Skills and Experience: - Minimum 3 years verifiable experience as Linux Web Administrator in creating and managing existing and new LAMP EC2 instance. Laravel application hosting experience preferred. Duration: Permanent: assignment and based on completed work items

Hello^^^^. I'm using Amazon AWS and would like to request server maintenance. First of all, I would like to ask you to do as much as possible for server security and privacy...root identification and blocking, suspicious process occupation identification and recovery, server hacking analysis, server access history tracking, password recovery and complexity setting, virus and malicious software checking,,data leakage prevention.....etc My current monthly budget is about $50, Please send me a proposal/chat with as much as you can about what you can do to secure your Linux server and I will check them all and get back to you. **You will be required to write a weekly report (anything related to security) to check the security status of the server every week. **Daily server check...

I'm on the lookout for a skilled RHEL Engineer to assist in getting our server on the Certified Hardware List. The role will require expertise in the core language of RHEL. Although I'm unsure if proficiency in C is obligatory, having it would be a welcome advantage. Specifically, ...expertise in the core language of RHEL. Although I'm unsure if proficiency in C is obligatory, having it would be a welcome advantage. Specifically, the Engineer will: - Understand the requirements for RHEL certification - Prepare the server for the process - Perform the tasks and tweaks necessary to meet certification conditions I anticipate that a strong understanding of programming, RHEL, and Linux-based operating systems will be necessary. Experience with server certification will a...

I'm looking for someone who can...for someone who can create a WordPress plugin for scraping manga websites. Plugin Requirements: 1. Grab per chapter by link. 2. Grab full chapter by link. 3. Grab new manga by link. 4. Compatible with various manga themes. 5. Images from the scraped content should be uploaded to s3 storage. 6. Must have 10 sources for scraping (with a guarantee that if one source fails or the website goes down, it can switch to another source). Usage Flow: 1. I create a manga post and fill in the manga link in the metabox. 2. The plugin will automatically fill in the chapter links, scrape HTML images, and create posts. It will also automatically fill in the chapter number and the manga post ID number based on the manga link. Note: Speed during sc...

We are looking for candidate from Hyderabad City only. I'm in need of a skilled system administrator who can manage a small infrastructure across Windows, macOS and Linux operating systems. The key tasks include server maintenance, network troubleshooting, software installation and updates, and configuring our server. Key aspects of the project include: - Supporting multiple operating systems: You should be experienced in working with Windows, macOS, and Linux. - System maintenance: Regular maintenance of our servers is essential for smooth operations. This includes updates and ensuring all systems are secure. - Network troubleshooting: You'll be responsible for keeping our network up and running. Troubleshooting issues as they arise is crucial. - Software installa...

I'm in need of a professional who can consistently extract data from a specific website on a daily basis. Key Requirements: - Data Extraction: Extracting various forms of data from a website. - Ongoing basis: This is not a one-time job. I'm looking for long-term cooperation, with the extraction ideally happening on a daily schedule. Ideal Skills and Experience: - Web Scraping: Proficiency in web scraping tools and techniques is a must. - Automation: Experience with automating the extraction process to make it efficient. - Consistency: Ability to maintain a daily extraction schedule accurately is crucial. Please include in your bid how you plan to approach this sort of ongoing data extraction.

I need a professional who can extract data from a website on a daily basis. Key Responsibilities: - Extracting data from a specified website - Ensuring the data is accurate and up to date - Delivering the data in a specified format Skills and Experience: - Proficient in web scraping and data extraction - Experience in automating data extraction processes - Ability to maintain data accuracy and consistency - Knowledge of relevant data formats for delivery - Strong attention to detail and reliability This is an ongoing project, and I need someone who can commit to daily extractions. Be sure to include relevant experience in your bid.

...higher - Visual FoxPro 9 - Office 2019 Please make sure to include in your proposal any related experience and specific skills that make you a suitable candidate for this project. Project Description I'm currently in a bit of a tight spot. A Windows Server 2016 web application () was recently moved from a US provider to the by a non-professional freelancer. Unfortunately, after less than a week, the system has stopped working. I found out recently he has installed the application in Centos 7 (Linux) instead of Windows against my guidelines. In light of this, I am seeking a highly skilled professional to help me reinstall the system from the copy to the Window Server 2016 environment. The job involves the following steps: 1) Change the OS from Linex to Windows

...that the software is designed to run on Windows OS. Website Accessibility: - Restricted access: Websites that require restricted access or login should be identified and reported separately. The software should skip such pages and indicate them as inaccessible. Ideal candidates should possess: - Proficiency in software development and experience in creating similar tools. - Familiarity with web scraping, data extraction, and bulk processing. - Prior experience in creating Windows-compatible software. - Strong attention to detail and the ability to accurately handle and report inaccessible URLs. As the client, I am open to the choice of programming language(s), as the final product is more important to me. The primary goal is to have a reliable and efficient tool that c...

...of organizations. The ideal freelancer for this project should have: - Experience in web scraping, particularly with LinkedIn data - Ability to extract data at scale - Proficiency in organizing and structuring data for clear analysis - Strong attention to detail to ensure data accuracy The project involves: - Using the provided list of target companies to source employees' data - Identifying individuals who have previously worked at the provided specific companies - Compiling the data in a spreadsheet format for delivery The successful applicant should not only have the required technical skills but also the capability to deliver quality and timely results. Please showcase your experience in web scraping and the relevant tools you intend to use to ens...

I need a Python developer to create a bot that scrapes data from APIs on an hourly basis. - The bot's primary function is to scrape data - The data will be sourced from APIs - The bot needs to run on an hourly basis Ideal candidate: - Proficient in Python and API data scraping - Familiarity with creating bots - Able to ensure smooth hourly operation, possibly using a scheduling library such as APScheduler

I'm in need of a skilled Python coder to assist me in deploying Locally Hosted Language Model (LLM) applications. Here are the details: - **Operating System**: The deployment is hybrid, so familiarity with Windows, macOS, and Linux is ideal. - **Purpose**: The primary aim of the deployment is for text ebedding (vector db) and generation. - **Requirements**: Specifically, I'm looking for functionality related to Language modeling and retrieval augment generation in the LLM applications. - mongoDB should be set up runining a .sh file - docker-compose deploy of the app Essential skills for this project include a strong background in Python programming and experience with deploying applications on different operating systems. Previous work in natural language processing (NL...

I'm looking for a...Facebook, and Instagram and all over the internet Key responsibilities: * Extracting Email leads, Phone number leads, and Social media leads. * Collecting specific data - Name, Job title, Company, and Location, email and phone numbers Ideal freelancers for this project should have: * Extensive experience in data scraping, especially on the platforms mentioned above. * Proficiency in data mining techniques and possessing advanced analytical skills. * Understanding of ethical data scraping practices. * Strong attention to detail - ensuring all data is accurate and up-to-date. * Proven track record of delivering quality leads in a short timeframe. Get in touch if you're up for the task. I value clear communication and timely delivery. Looking f...

For streaming project, linux/ffmpeg expert is required. You must demonstrate previous work and have your environment to show running demos. Desirable communication in Spanish

I need to run TTS wine voices from Linux in docker. I found two tutorials that explain how. Or I need a developer to go through each tutorial and see if you can run the wine gui environment from a docker. Then package that solution as a Docker image I can run locally on lubuntu 22.04. I would do it myself, but I don’t have much experience with docker Requirements: 1 Install balabolka in wine - 2 Install cli balalbolka “bconsole”

I'm in need of an experienced professional to assist with setting up Apache for Cognos12 on Linux. Key Responsibilities: - Perform a comprehensive Apache configuration to ensure compatibility with Cognos12 Required Skills: - In-depth experience with Apache configuration - Proficiency with Linux operating system - Understanding of Cognos12 architecture Moreover, I would like to implement specific directory access restrictions as part of the setup to enhance security. Desired Skills: - Knowledge in implementing directory access restrictions - Awareness of best practices in security for Apache servers

I need a skilled Web Scraping specialist to write code in Python (preferably) that scrapes a specific table from a website for me (Website: ). The table contains WNBA player statistics, The goal is to extract all columns of this table, which is approximately 7 columns and write it to a CSV file (in the same column format). I'm a beginner level coder, and noticed that the table was created with that looks like a Shiny App in R. (I'm not even sure if extracting the table is possible). If it was only possible to use R, then I could potentially work with that. Website: I would like the data to go from the year 2020 to current. Regular Season data. Key Requirements: - Scraping Tables on websites If you're

I am looking to establish a long-term relationship with a talented person to scrap thousands of dating profiles from popular dating websites. This person should have years of experience dealing with the various countermeasures these sites will employ to discourage scraping. The use of proxies may be needed to achieve satisfactory results. The end goal to to have a system capable of delivering 20K - 40K profiles per day.

I need a VRBO scraper to extract property details, pricing, and availability. - The scraper should be built with Python and the output should be in CSV format. - The data points I need include property title, description, amenities, pricing, and availability. - The scraper should be able to handl...and availability. - The scraper should be built with Python and the output should be in CSV format. - The data points I need include property title, description, amenities, pricing, and availability. - The scraper should be able to handle multiple URLs at once. - The code should be well-structured and commented for easy maintenance. Ideal skills: - Proficiency in Python - Experience in web scraping - Understanding of VRBO website structure - Ability to handle and store d...

I need a web scraping professional to create a script to extract specific data from , a popular online marketplace for trading cards, please check attachement to get the idea. Key Requirements: - Scrapping: The primary task is to scrape the following information from the platform: - Card properties 1. Hero score for last 7 days 2. Hero rank for last 7 days 3. Stars 4. Rarity 5. Name 6. Quantity listed to purchase 7. Inflation data 8. Minted count - Buy prices List Price Last Price Order book (price and quantity see picture attached) - Bidding details bids book (bid and quantity see picture attached) - Twitter activity per card View Views+ Likes Likes+ Retweets Retweets+ Replies Replies+ Quotes Quotes+ Bookmarks Bookmarks+

VM CREATION (1 VPS BIND) os CentOS 8 & 9 Stream MINIMAL Direct installation or converted from CentOS 8 Linux. (Recommended) panel add install: nginx server VARNICH APACH very important !!!!!snaphopt image creation just servers working (just os) (without admin panel)!!!I want to be able to do restores, for a new installation of another panel, if necessary.! !!! !! !!!!!! link to server add SSL ADD ALL PHP CGI AND FMP (latest 6 versions)PHP AVAILABLE FOR CUSTOMER FOR ALL SERVER ADD PHP VERSION 7.9 PACHT FMMPEG add latest version change SSH22 port verification example: ns1 . add IP.: add a Mac machine adds IP.: adds mac Example of adding HOSTENAME: my

I'm seeking a Python developer experienced with web scraping. This job involves setting up a Python script that can scrape a popular review site. Key details to extract: - Review text - Star rating - Date of review - Reviewer name - Star-ratings for sub-categories - Review ID - Reviewed person - Reviewed product name The site does not use captchas, so it should be reasonably straightforward. All reviews for a given provider should be scraped. It would be good to have an option to start with the newest and stop at a certain date. The specific review site can be seen in the sample URL in the attached PDF. I can send further details in a private message including screenshots. Preferred frameworks would be Beautifoulsoup 4 / Scrapy if sufficient and Selenium otherwise....

...in NodeRed Monitoring specific "HTML div elements" from different web pages using Node-Red (different flows). Objective: Identify when there is any change in the DIV section of the webpages (every "x" seconds) and notify me of the change. It is not necessary to show what the change was. Just send the hyperlink by Telegram, including the subject of the flow. Requirements: Create a Node-Red project that can detect changes on multiple websites by creating flows. Include 2 example flows to demonstrate how the monitoring works. Provide detailed instructions on how to create a new flow for a different webpages, indicating what needs to be extracted/copied to set up the new flow bymyself in the future. For example, on this web page, by pressing F12, yo...

...data scraping project completed on multiple websites. - TASK: The task involves scraping text content from: Google searches: Commercial Real estate agency Saudi Arabia Commercial Real estate agency UAE Commercial Real estate agency Qatar Commercial Real estate agency Kuwait Commercial Real estate agency Bahrain I have made a start to the spreadsheet with a template for each of the regions which can be continued to be filled in. In the Google searches, just take the first 3 pages with the main agent listings and see what agent names/ email addresses you can find. - DATA REQUIRED: The primary data needed is text content. - TIMELINE: This is a one-time task, with no ongoing scraping requirements. Skills/Experience: - Proficiency in web scraping

I have two websites that need to be merged with a main focus on consolidating information. The current websites are different in design and themes, and I'm looking for a professional who can suc...understand and implement user-centric content organization - SEO's also need to link seamlessly Please let me know your experience with similar projects and your approach to this particular task. Your input on the best ways to consolidate the information and prioritize content is highly appreciated. Website language is: - Linux AMI > OS - PHP > web and back-end - NodeJS > ICS - > ICS & Warehouse App - Ionic v1 > Warehouse App - Apache > web server - Database: MySQL Aurora - AWS hosted, leveraging EC2, RDS and S3 - Front-end is custo...