This post is over 30 days old. The position may no longer be available

Data Engineer

Cloudwick Technologies , Bangalore · cloudwick.com · Full-time employment · Programming

Job description

We are looking for talented data engineers who are passionate about working with large scale
data processing systems on AWS to help manage the ever-growing information needs of our
clients.


Specific responsibilities include the following :
(Deliverables)
- Create and maintain optimal data pipeline architecture on AWS from a wide variety of data sources
using AWS- Big data technologies (EMR, Spark, Airflow & Many)
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes,
optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Work with stakeholders including the Executive, Product, Data and Design teams (primarily with
client and internal teams) to assist with data-related technical issues and support their data
infrastructure needs.-
- Performing root cause analysis on internal and external data and processes to answer specific
business questions and identify opportunities for improvement-
- Manipulating, processing and extracting value from large disconnected datasets-
- Message queuing, stream processing, and highly scalable - big data- data stores-
- Building and optimizing - big data- data pipelines, architectures and data sets.




Education Qualification And Work Experience:
- B.E., /B.Tech,preferably in Computer Science or Engineering plus 6 years relevant work
experience
or
- Master's degree, preferably in Computer Science or Engineering plus 4 years relevant work
experience
- Direct/Onsite work experience with Client / In-house projects is preferred

What We Seek:
- Candidates should have hands-on experience of using many software/tools such as and related
to:- Smart in problem solving skills
- Strong understanding & experience as Data Engineer with Hadoop, Spark & Hive.
- Good Exposure to handling large & complex data sets
- Experience with AWS cloud services is required: (EC2, EMR, RDS, Redshift)-
- Development of data pipeline and workflow management tools: Airflow, Azkaban, Luigi, etc.
- Object-oriented/object function scripting languages: Python or Java is must.
- Analytic skills related to working with structured and unstructured datasets
- Experience working with Data Scientists is good to have

Apply for this position

Login with Google or GitHub to see instructions on how to apply. Your identity will not be revealed to the employer.

It is NOT OK for recruiters, HR consultants, and other intermediaries to contact this employer