Scrapy run spider sequentially jobs
I am seeking an experienced Python developer with a proven track record in working with the Scrapy framework. The primary goal of the project is to handle data extraction from websites, dealing particularly with nested or hierarchical data. Key Responsibilities: - Collaborating with the team to define the scope of the project and identify the data requirements. - Design and implement data extraction scripts using the Scrapy framework. - Handle complex, nested or hierarchical data structures. - Ensure the accuracy and efficiency of the data extraction process. Ideal Skills and Experience: - Proven experience in Python development. - Expertise in working with the Scrapy framework. - Strong understanding of web technologies and data structures. - Excellent problem-solvi...
...movie screenplay for a Spider-Man villain that I've named "Venom: Sunset". The story is set in 1389 Tokyo and centers around a misunderstood hero who eventually turns into a villain. Key Elements: - Historical Accuracy: The screenplay should be well-researched and accurately reflect the culture, customs, and events of Tokyo in 1389. - Samurai Fighting Scenes: The plot should include engaging, well-choreographed combat sequences involving Samurai warriors. - Character Development: It's crucial that the protagonist's transformation from a misunderstood hero to a villain is believable and emotionally resonant. Desired Tone: - Dark and Gritty: The overall tone of the screenplay should be dark, intense and gritty. The story should be both compelling and fi...
...of 25 different clues that will create iambiance from intimate for softer songs to very energetic for upbeat songs. See songlist attached. - The design should include a playback button for each song which includes up to 10 cuelists for each section of a song. A document describing where the scene changes occur will be provided separately. - The lighting fixtures include 2 moving heads, 2 U`King Spider Moving Head Light, 2 SLI-FLATSET1, 2 Chauvet COLORSTRIP MINI DMX-512 LED RGB Light Bar, 2 chauvet slimpar64 and 2 Venue thinpar 38. - The design should be created with the assumtion that haze will be used as atmospherics. - The design has to be implemented in Onyx 4.2 for PC. - Accetance for the design is to be demonstarted in a video using the Capture visulizer (
I'm looking for a talented screenwriter to craft a standalone origins screenplay for a movie tentatively titled "Venom: Sunset". The story is set in the Midwest of Texas during the early 1900s and is aimed primarily at teenagers. Key elements that need to be included in the screenplay: - Focus on the origin of powers: The storyl...at teenagers. Key elements that need to be included in the screenplay: - Focus on the origin of powers: The storyline should revolve around the discovery and development of the villain's unique abilities. This is a crucial element that should be explored in depth. - Tone: The screenplay should be crafted with a dark and gritty tone. It should resonate with the angst and complexity of a typical Spider-Man villain, thereby creating a ...
I having a Application project on vitis that i need to run to see the result on zynq zedboard . Since i dont have the board , im looking for someone to run it and instruct me how to set up the zedboard to run the project. Im working on Windows OS
I'm seeking a skilled screenplay writer to create a thrilling and engaging script for a 10-episode TV series that will appeal to an adult audience. This project is a crossover between two iconic Marvel characters, Ghost Rider and Spider-Man, and thus requires a deep understanding of their stories. Key Requirements: - Target Audience: The aim is to create a screenplay that is particularly captivating for adults. The story should be intricate and mature, while still retaining the essence of the original comic book characters. - Tone: The screenplay should have a dark and gritty tone to match the themes of the Ghost Rider character and the general atmosphere of the Marvel universe. It should be crafted in a way that keeps the audience on the edge of their seats, while also provid...
I'm seeking a Python developer experienced with web scraping. This job involves setting up a Python script that can scrape a popular review site. Key details to extract: - Review text - Star rating - Date of review - Reviewer name - Star-ratings for sub-categories - Review ID - Review... All reviews for a given provider should be scraped. It would be good to have an option to start with the newest and stop at a certain date. The specific review site can be seen in the sample URL in the attached PDF. I can send further details in a private message including screenshots. Preferred frameworks would be Beautifoulsoup 4 / Scrapy if sufficient and Selenium otherwise. Please state which one you plan to use. Note: it should be possible to run the script in headless mode (i...
I'm seeking a skilled Senior Python Developer to assist with a project that involves a blend of web scraping and data analysis, with a focus on the following tasks: - Extracting data from multiple websites - Automating data extraction and parsing - Analyzing and visu... with a strong understanding of AI and Machine Learning - Hands-on experience with web scraping, particularly working with diverse data sources - Ability to automate data extraction and processing efficiently - Adept at data analysis, with the ability to visualize and interpret data Although I haven't specified any particular tools or libraries, your familiarity with tools like BeautifulSoup, Scrapy and Selenium would be a bonus. Your ability to recommend and justify the use of specific tools and methods wi...
I'm looking for someone experienced in running projects from GitHub repositories "" to assist me in setting up and running the demo-k6-operator project on my Mac machine. Your guidance should help me to understand how this project works, which is my primary goal. Key requirements: - Expertise in running and configuring projects from GitHub repositories on Mac machines - Solid understanding of the demo-k6-operator project - Ability to provide clear, detailed instructions and guidance for running the project on my machine - Patience and willingness to explain concepts to help me understand the project better The specific challenge we may face is that I'm using a Mac CPU that is NOT Intel, so experience with this kind of set-up would be a significant advantage. Your supp...
We're looking to see how the latests AI rotoscoping tools look compared to one we're building ourselves. Specifically we're interested in Davinci, After Effects and Runway. We'd like someone with access to/familiarity with all of them (at least the background removal function). This is how it would work: - We give you a single clip - You use each tool to remove the background best you can while screen recording each process and making note of how many clicks it took to select the whole object - For controlled conditions, you have 30 seconds to select the foreground in each tool - Export as is (no further clean up) - Then using whatever editing tool you like the most, take these outputs (and one we'll prvide) and create identical vertical videos for each where i...
I'm in need of an experienced web developer with strong shell scripting skills. While the main purpose of the website application is not specified, it's targeted at the general public. Ideal Skills and Experience: - Proven expertise in web application development - Experience in creating websites geared towards the general public - Robust skills in shell scripting - Ability to independently make decisions to enhance website functionalities Note: As the tasks for shell scripting are undefined, a flexible way of working is preferred for meeting emergent needs in the project.
I'm looking for a talented screenwriter who can craft an engaging and action-packed crossover movie screenplay involving iconic characters like Spider-Man, Deadpool, Transformer Autobots, Doctor Octopus, Taskmaster, and Transformer Decepticons. - **Spider-Man Character**: Please create an original version of Spider-Man, diverging from the existing Tobey Maguire, Andrew Garfield, and Tom Holland's Spider-Man. This character should be engaging, relatable, and seamlessly fit into the established Marvel and Transformers universe. - **Tone**: The movie should oscillate between a mix of comedic and serious tones. It should be able to deliver both the light-hearted banter and the emotional gravity that fans expect from these characters. - **Setting**: T...
We are in search of a skilled freelancer proficient in web scraping with Scrapy to extract product data from This project involves scraping the Category page to retrieve product details including name, price, SKU, URL, breadcrumb, currency, and brand. Targeted webpage: Category page of Required data: Product name, price, SKU, URL, breadcrumb, currency, and brand Experience: Prior experience in scraping product-related information from e-commerce sites is essential, with proficiency in Scrapy being crucial for accurate and efficient data extraction. Data Handling: Ability to ensure accuracy and consistency of extracted data. Familiarity with different data formats is advantageous. Dynamic Scraping: Navigation logic for all categories
I'm in need of a unique patch design that represents our squadron/unit with emphasis on the Black Widow Spider. This design should be created with the following considerations: - Representing squadron/unit: The patch should be a symbol of our identity and cohesion. - Iconic Symbol: A Black Widow Spider and Tasmanian devil needs to be the central element of our emblem, We would like a circular Patch with spider legs grabbing the patch from behind with the shape of Singapore and Australia sharing half the centre of the emblem, with a Tasmanian devil wearing retro flight googles crossing over from Australia to Singapore
I'm in need of a professional who can build a tool that scrapes product data from Walmart and exports it into an Excel spreadsheet. Key Requirements: - Develop a web scraping tool specifically for Walmart website - Extract essentials (price, upc, title) Skills and Experience: - Proficiency in web scraping using Python, Scrapy, or similar tools Please let me know your availability and estimated time for completion. This is an urgent project, so a quick turnaround would be highly appreciated.
...The organization will be honoring and presenting Awards to members of the Music Hall of Fame for their contributions to the music industry and culture. Artist Honorees and performances will be Gary’s Gang, The Hues Corporation, Giggles, Gloria Jones, Judy Cheeks, Juan Angel Gonzalez, Rainere Martin, Sharon Brown. DJ honoree and Radio personality Al Bandiero; DJ John Donato, Eddie Muentes aka DJ Spider, Max Sierra aka John Spectre. A posthumous award will be presented to Maria Respeto the original “Giggles” The organization will be presenting the “Icon” Award to Legendary recording artist Freda Payne. Concert Performances: Cyre’, Company B, Lenny Zakatek aka “Gonzalez” and George McCrae celebrating his 50th Anniversary of the world h...
I'm seeking a proven and experienced freelancer to compile the content of some of my blogs into a friendly and enticing ebook on soils and gardening from the ground up! Key Responsibilities: - Compiling the content sequentially from a gardening perspective. Ideal Freelancer: - Exceptional writing skills. - Prior experience in authoring eBooks or comprehensive guides. - Knowledge about various gardening practices. - Ability to write for both beginner and experienced gardeners. All submissions should demonstrate a solid understanding of the project's requirements. Looking forward for a captivating content.
...achieve faster data extraction rates. Responsibilities: Optimize existing web scraping scripts for speed and efficiency. Deploy and manage web scraping scripts on platforms such as Scrapy Cloud, Apify, AWS Lambda, or Azure. Implement techniques to ensure high-speed data extraction (e.g., parallel processing, distributed crawling, efficient request handling). Monitor and troubleshoot scraping processes to ensure uninterrupted operation. Collaborate with our team to understand project requirements and adjust scraping strategies as needed. Requirements: Proven experience with web scraping using platforms like Scrapy Cloud, Apify, AWS Lambda, or Azure. Proficiency in Python only relevant programming commonly used for web scraping. Strong understanding of web scraping techn...
More details: What type of tone do you envision for the "Spectacular Spider-Man" movie? Balanced mix of both How closely should the screenplay adhere to the comic book origins? Moderately, with some creative liberties Which aspect of the story do you want to emphasize the most? All three
I want to enlist the assistance of an experienced web scraping developer, specifically in scraping data from Redfin. I'm primarily interested in the following data points: - Property address - Price - The number of bedrooms - The number of bathrooms This data will be used for personal reasons therefore respectful dealings with ...dealings with the website and its terms of service are obligatory. Here's the kicker, I don't just need you once. I'm wanting the data to be updated on a monthly basis, so I require a solution that's reliable in the long term. Ideal skills for this job are expertise in web scraping, familiarity with real estate websites particularly Redfin, and proficiency in language like Python, BeautifulSoup, Scrapy or similar. Understanding...
The games will be table games for 4/6/8 players. All games will be played via touch screen. Calibration and screen use must be planned correctly All players will play on a single stage, simultaneously and sequentially. Score areas of the games will be recorded on Compact DB. There will be a timer in the games. For example, if the game lasts 20 minutes, it will give a warning to the end user at the 19th minute. Please insert a coin. we are ready for all game offers for these tables. -Text me for a detailed brief for games- All games must be prepared to run on GAMING TABLE. Device used: 43" Inch Touch Table(GAMING TABLE) 40 points PCAP touch CPU : Intel 5 6th Gen 6400T RAM:512GB screen : LG orignal A level screen panel Protect surface: 5mm tempered glass Resolution:38...
I'm programming a sensitive app where I don't want to have full control over the deployment process / cloud account. Therefore I need a reviewer / ops person to take ownership of the deployment process. I need an hour or so of your time every week to: - Review a pull request created by me - Run a deployment pipeline - Confirm project has deployed successfully - Review cloud account and indicate any issues (e.g. unpaid invoice, server downtime, etc.) Estimated effort / time required: 1-2hrs per week. Note: You don't need to be a programmer / specialist. Just technically literate.
I need someone to assist me in setting up the Zabor POS APK on my Windows 10 computer. I would like the APK to run on BlueStacks, an Android emulator. This is for personal use, so it is important that the setup is seamless and runs smoothly. Key Responsibilities: - Install and configure BlueStacks for optimal performance - Download, install, and set up the Zabor POS APK - Ensure the APK is running smoothly on Windows 10 Ideal Skills and Experience: - Strong knowledge of Android emulators, particularly BlueStacks - Experience in setting up and configuring software - Attention to detail to ensure smooth operation Please note that I require assistance with the initial setup and configuration - not just running the APK. Your help is appreciated.
This is a simple C++ command line program with parameters. I don't work with C++ on a daily basis and I need support. The program uses an external SDK, and I have header, lib and dll files. It needs to be connected and configured for everything to work. I present screenshots. This needs to be configured on my computer, I can share the screen remotely.
I'm looking for a skilled React.js developer to help me run the app locally using or deploy it to sever using anydesk. I will provide a great detailed review. @danielhvoigt/how-to-make-your-react-app-open-locally-through-an-index-html-file-413b0405907
...scrapping software quickly to extract text content from both websites and PDF documents. It's crucial that the scraped data is directed into Excel or Google Sheets format for my immediate use. Ideal skills for this job would be: - Proficiency in programming languages like Python that are often used for data scrapping. - Familiarity with web scraping libraries or frameworks, e.g., Beautiful Soup or Scrapy. - Experience using APIs to scrape data if necessary. - Previous work in pulling data from PDFs into readable formats in Excel or Google Sheet. - Ability to deliver the project ASAP. Your attention to these requirements will be much appreciated as I need this task done urgently. Please contact me if you need more detail about the project, and I look forward to seeing your...
I'm looking for a skilled 3D artist/modeler who can bring my 'Spider-Sona' character to life. The character should be designed in a low detail, cartoonish style. Key Requirements: - The character will be used for video games, so the created model should be optimized for real-time rendering. - Full body rigging is needed for the character to be animated in various game scenarios. - The design should be fun and engaging, suitable for a cartoon-themed video game. Ideal Skills and Experience: - Proficiency in 3D modeling software (e.g. Blender, Maya, 3ds Max) - Experience with character design and animation rigging - Prior work in the gaming industry or creating assets for games - A strong portfolio showcasing similar, cartoonish-styled characters. The successful ...
I'm in need of a Python developer to assist me with web scraping. The data I'm after is sourced from...Python developer to assist me with web scraping. The data I'm after is sourced from APIs and once extracted, it needs to be stored in a CSV file. Key Requirements: - Proficiency in Python - Experience with web scraping - Knowledge of working with APIs - Ability to store scrapped data in a CSV file Ideal Skills: - Familiarity with popular Python libraries for web scraping, like Beautiful Soup or Scrapy - Understanding of API integration and usage - Strong data handling and manipulation skills It's a straightforward task that needs attention to detail and adherence to best practices in web scraping. If you have the right experience and are keen to assist, I loo...
Looking for a skilled and creative 2D animator to bring my humorous vision to life. As such, the animation should carry a comedic theme. The animation's expected duration is more than 3 minutes. I am looking for someone who: - Has experience or is comfortable with comedy-genre animation - Can create high-quality 2D animations - Can work on a prolonged animation of over 3 minutes - Can bring originality and creativity to the project. Experience in working on big projects is a plus but not a requirement. However, portfolios demonstrating skills in 2d animation, especially in humor-themed undertakings, will be considered preferentially.
...the beginning of each game session, the AI generates a hidden scenario that sets the context for the tasks. - This scenario is not directly revealed to the players but influences the tasks they receive. 3. **Dynamic Task Distribution:** - The AI distributes tasks to players based on the hidden scenario. - Tasks can be distributed to all players simultaneously or to individual players sequentially. - For example, the AI might instruct Player A to "open a door" by presenting them with a button labeled "Do it." 4. **Player Actions and Scenario Evolution:** - If Player A completes their task (e.g., presses the "Do it" button), the AI then assigns new tasks to other players (e.g., Players B, C, D). - The sequence of task distribution c...
looking for Facebook ad account provider old account with no daily spend limit and ready to run ads no need to warm up
...the beginning of each game session, the AI generates a hidden scenario that sets the context for the tasks. - This scenario is not directly revealed to the players but influences the tasks they receive. 3. **Dynamic Task Distribution:** - The AI distributes tasks to players based on the hidden scenario. - Tasks can be distributed to all players simultaneously or to individual players sequentially. - For example, the AI might instruct Player A to "open a door" by presenting them with a button labeled "Do it." 4. **Player Actions and Scenario Evolution:** - If Player A completes their task (e.g., presses the "Do it" button), the AI then assigns new tasks to other players (e.g., Players B, C, D). - The sequence of task distribution c...
...the beginning of each game session, the AI generates a hidden scenario that sets the context for the tasks. - This scenario is not directly revealed to the players but influences the tasks they receive. 3. **Dynamic Task Distribution:** - The AI distributes tasks to players based on the hidden scenario. - Tasks can be distributed to all players simultaneously or to individual players sequentially. - For example, the AI might instruct Player A to "open a door" by presenting them with a button labeled "Do it." 4. **Player Actions and Scenario Evolution:** - If Player A completes their task (e.g., presses the "Do it" button), the AI then assigns new tasks to other players (e.g., Players B, C, D). - The sequence of task distribution c...
... The program should include the duration each house has been listed on the site, and have it run every hour to capture new listings. Additionally, I would like duplicates to be automatically eliminated. Key Responsibilities: - Scrape for home addresses - Compile addresses in CSV format - Include the duration each house has been listed - Run the program every hour to capture new listings - Automatically eliminate any duplicate entrie - Run CSV file against another data base that has age and name of owner and contact info. Ideal Skills and Experience: - Experience in web scraping and data extraction - Proficiency in programming languages suitable for web scraping (e.g. Python, Scrapy) - Knowledge of CSV file creation and data handling - Ability to automate ...
I need help to run an ad for my digital product. I'm looking for a freelancer who can showcase their previous work before I make a final decision. Your responsibilities will include: - Running a digital ad campaign for my product - Providing insights on the ideal platforms for the campaign - Showcasing your previous work in digital product ad campaigns The ideal candidate for this job should have: - Prior experience in running digital ad campaigns - An understanding of the digital product market - The ability to showcase previous work for my review
I need a talented freelancer to set up my Shopify store and integrate it with Facebook Pixel for running ads. The objective is to optimize the store for effective marketing campaigns. Key tasks include: - Configuring Shopify store: The successful candidate should be adept in customizing Shopify stores, ensuring a seamless purchase journey for customers. - Integrating Facebook Pixel: I require the freelancer to link my Shopify store with Facebook Pixel. This will involve installing the Pixel, creating custom audiences, setting up conversion tracking, and ensuring that the integration is correctly implemented. Ideal applicants should have: - Proven experience in Shopify web design and development, particularly in configuring stores for optimal performance. - Proficiency in setting up and i...
I need a new emblem logo design for my business. The logo is meant to be used primarily for print materials such as business cards and flyers, as well as for merchandise like apparel and mugs. Key requirements: - The logo must be an emblem type - It should be versatile enough to look good in both small (business card) and large (apparel) sizes Ideal skills and experience: - Proven experience in creating emblem logos - Understanding of design principles for print and merchandise - Ability to deliver a design that's both professional and appealing to customers.
I am looking...pk/agent-locator/). Key Requirements: - I need the data to be delivered in an Excel spreadsheet format. - The specific details that I require for each Jazz locator are the Name and Address, along with the Services offered. - The scraping needs to be done only once; it will not be recurring. Ideal Skills and Experience: - Proficiency in web scraping, particularly using tools like Python's Beautiful Soup or Scrapy. - Experience in extracting data from websites with dynamic content. - Strong attention to detail to ensure accurate data extraction. - Previous experience with similar data scraping projects will be an advantage. I am particularly interested in working with a freelancer who is located in Pakistan, though this is not a strict requirement. Thank you for...
We don't want our employees to see the phone numbers of customers on whatsapp and chat with them. We just want to restrict whatsapp usage to making calls only without visibility of the numbers. So best is that if the GUI of WhatsApp Desktop is not visible and we access it using another app. Either from browser or another windows app. Alternative is that we do this with android app
I'm looking for someone to scrape text and image data from a specific website for analysis purposes. Key Tasks: - Scraping text data: Extracting specific textual information from the website - Scraping image data: Gathering images from the website The data is to be used for analysis, so understanding...image data: Gathering images from the website The data is to be used for analysis, so understanding of data structures and data analysis would be beneficial. Ideal Skills and Experience: - Proficient in web scraping techniques, particularly with text and images - Experience with data analysis - Familiarity with tools and technologies that can be used for scraping, such as Python, BeautifulSoup, Scrapy - Strong attention to detail as the data needs to be extracted accurately...
I'm in need of a video editor to enhance my product demo video. The current version is 5 minutes long and needs adding text slides between video clips sequentially. Key Requirements: - The final video should be 5 minutes long. - The text slides to be added should focus on product features. Additional Information: - I do not want any music to accompany the video.
...make a web app that is able to accept any kind of raw text or a pdf and extract raw text from it. So, the initial page can have 2 options for Data Source: Raw Text and PDF (We might add support for other input formats as well in the future). Next, once we have the text available, it should split the text based on the lines. It can split on the full stop character. And then every line must be sequentially given to a ML model. This would be a text classification huggingface model, which can be instantiated via this code: from transformers import pipeline pipe = pipeline("text-classification", model="legal-text-classification/IndianLegalBERT") Feel free to explore the above model yourself. It will give you 15 labels with different scores. If the score o...
I need a Python developer who can create a script to scrape news in ...scrape news in JSON format from moneycontrol at a specified interval and store it in a database. Key Requirements: - Develop a Python script to scrape news in JSON from moneycontrol - Automate the scraping process at a specified frequency - Ensure the scraped news is stored in a database Ideal Skills and Experience: - Proficiency in Python - Experience with web scraping using BeautifulSoup or Scrapy - Knowledge of working with JSON data - Familiarity with databases and data storage - Previous experience in similar projects would be a plus Please note: The project does not have specific requirements for news categories or keywords at the moment. The focus is on extracting news in JSON format and storing it in a...
Im working on a c++ image processing project , and i need to convert my C++ code to Verilog using HLS vitis , then implement it to run on Ultra96v2 Xilinx FPGA board .
I'm looking for ...scraping and data analysis tasks. Key Tasks: - Web Scraping: Proficiency in using Beautiful Soup or Scrapy is essential. The person should be able to extract specific data sets from websites and present them in a structured format. - Data Analysis: Experience with Pandas and NumPy is crucial for this role. The ideal candidate should be able to manipulate and analyze large data sets efficiently. - Automation: Familiarity with Selenium for automating tasks is a plus. What I'm Looking For: - Experience: Please include details of your past projects and your experience with Python in your application. - Specific Libraries/Frameworks: A strong grasp of Pandas, NumPy, Beautiful Soup, Scrapy, and Selenium is highly desirable. Looking forward to r...
I'm currently looking for an expert who can design a data scraping script for me. However, I haven't decided...expert who can design a data scraping script for me. However, I haven't decided on which particular website and kind of data (text content, images, links etc) should be scraped yet. Furthermore, the frequency for data collection hasn't been specified. Ideal Skills and Experience: - Commercial experience with Python or JavaScript - Proficiency in web scraping using tools such as BeautifulSoup, Scrapy, or Selenium - Ability to add flexibility for target URLs, data type, and collection frequency Please note: the project details will be finalized upon discussion. Your recommendations are welcome and appreciated. Only apply if you're comfortable proce...
I am in need of a proficient Python coder who can develop a script to furnish an important function, data processing. More specifically, this task involve...process numerous multimedia data in the form of videos. Desired skills and experience: - Thorough understanding and experience in Python scripting. - Proven experience in working with multimedia data, especially videos. - Familiarity with Google Cloud Platform (GCP). - Knowledge and experience in data processing algorithms and techniques. The final deliverable will be a Python script able to run in an HTML page, which we can then deploy to GCP. The successful candidate should be comfortable in different stages of development, including the setup and running of the script on GCP. Prior experience in similar tasks will be highl...
Looking for an experienced Python developer who can create a script to be embedded within an HTML page and deployed on Google Cloud Platform (GCP). The script should: - Be capable of real-time data processing. - Fetch data directly from a webpage. Ideal skills and experience needed for this role: - Extensive Python programming background. - Experience working with real-time data. - Familiarity with extracting data from webpages. - A solid understanding of Google Cloud Platform for deployment. - HTML integration knowledge. This task requires a developer who is savvy with Python's data processing abilities and adaptability in real-world, real-time scenarios. Ready to take the challenge? We're willing to offer a good budget for the right skills.
