Open

big data Parsing and processing

* there are a series of json file each around 15GB

* from each file you take out Unique IP and store in a DB (sqlite)

* then there are 30 csv file contains a column IPs and you see if the any IP in DB match to any IP in DB, if yes in a new column of DB just write (Yes or NO)

Requirment:

optimized code and fast

Bug free

Dynamic style

Skills: Python

See more: big data processing python, free lance big data, free lancer big data, parse complex xml in java, parsing xml performance, xml stream parser, java 8 parse xml, parse large xml files c#, large xml parser, java xml parser benchmark, how to parse a huge xml file, online data processing with storm big data pdf, best big data free lancing website, big data consultant free lance, big data free lance, big data free lance project, big data free lance projects, big data free lancer, big data java free lancing, big data job free lance

About the Employer:
( 1 review ) Berlin, Afghanistan

Project ID: #19622051

6 freelancers are bidding on average $31 for this job

anupkelkar02

I am a Python data science expert with experience in Classification and partitioning,Nureal Networks, Association rules. Also I am a Oracle Certified professional (OCP) with experience in Oracle,MySQL, SQL Server a More

$35 USD in 2 days
(42 Reviews)
6.0
josefN

Hello, I can write a python script for this in one day. I just need a couple of sample files. json/csv What python version do you want to use? how should input?

$50 USD in 2 days
(32 Reviews)
4.9
sanidhyarajawat

I am a professional Python developer and I can complete this task within timeline and in less budget. I have more than 5 years of experience in this industry and I am very good with Linux systems. I also have experienc More

$30 USD in 2 days
(6 Reviews)
2.4
honzucha

HI, I am 15+ years experienced Data Warehouse Architect and Business Intelligence Developer. For further reference check my LinkedIn profile [login to view URL] I will deliver you the python scri More

$30 USD in 5 days
(0 Reviews)
0.0
flowerbed2020

[login to view URL] This is my previous work for my client. you will make sure about your project.

$25 USD in 1 day
(0 Reviews)
0.0
mustafakmal

Hello I can help you with this task. I have been working as a Big Data Engineer and have experience processing data with Spark. Best Regards, Mustafa

$15 USD in 1 day
(0 Reviews)
0.0