I have done the data scraping program for scraping web sites hosted in shopify.com. When there are new items posted on the targeted web sites, the system will immediately capture the new items information including description, the web link of the items, price, inventory, the variants, size, and color are send to the clients' twitter account, and email account right away. On the message to email account, the user can click on the variants which is a hyper link, and direct it to the checkout page of the produce so that the user can checkout the product as soon as possible.
Moreover, since the user want to obtain the new items information asap, I am using mutithreading which is parallelly running all web site scraping at the same time. The scraping speed of my program that displays twitter message of news items is proved to be a few second faster than others.
For your information, I am currently studying the master degree of software engineering in California State University, and I am specialized in python programming. For data scraping, I normally use ElementTree, and Beautifulsoup. I also use Panadas, and Matplotlib for data analysis, and data visualization. Beside, I use Flask, or Django for web development. I use Raspberry Pi for robotics programming. I use Tkinker, PyQt, and Kivy for GUIs.
Please don't hesitate to inform me for the further questions. Thanks.