Responsibilities : - Build, maintain and optimize the data pipeline architecture at Zluri - Develop ETL pipelines based on the requirements following best practices, specifications, and standards - Work with project teams provide input and assistance in the design, development, and support of ETL and database components following standard software development lifecycle - Work with the team to ensure that ETL and database components conform to global standards via thorough code reviews - Mentor junior programmers in the team and plan their skill development - Determine the technical needs for the team to work on a project assigned or third-party tools - Documenting the data requirements, data format and tool dependencies - Flexibility and ability to adapt to new technological changes as per project need Requirements : - 5 years of experience in solving multi-dimensional data problems - Knowledge of multiple cloud storage options like S3, Redshift, or Big Query is a must - Should be well versed with My SQL, No SQL - Knowledge of Pandas and Spark is a must.
- Knowledge of working with DBT is a plus - Knowledge of Python, SQL, shell script is a must and Javascript (Node.js) is good to have - Knowledge of streaming services like AWS streams / Kafka / Google Pub/sub is a good to have - Knowledge of Queue systems like Rabbit MQ, SQS and building pipelines using the same is very much needed - Knowledge on Dockers, Kubernetes is good to have - Familiarity with multiple cloud providers (AWS, Azure & GCP) is good to have - Should be familiar with strategies for Real-time analytics and batch processing - Masters or Bachelor's degree in Computer Science or the equivalent in experience (ref:hirist.tech)
Advertisement:
Data Engineering Lead - Etl/python, Bangalore
Free
Data Engineering Lead - Etl/python, Bangalore
India, Karnataka, Bangalore,
Modified November 14, 2024
Description
Job details:
⇐ Previous job |
Next job ⇒ |