Position Title: Data Engineer
Experience Required:3-5+ Years
Employment Type: Full-time
Employment Mode: Hybrid
Location: Bangalore, Pune, Trivandrum. Job Summary:
We are looking for a skilled Data Engineer with hands-on experience in Py Spark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data architects and analysts to ensure the accurate, efficient, and scalable processing of data. Key Responsibilities:
Develop, optimize, and maintain scalable data pipelines using Py Spark and other Big Data technologies.
Work with cloud-native services to ingest, process, and store large datasets.
Implement data processing solutions using Apache Hive, Parquet, and other big data storage formats.
Build end-to-end data pipelines for various use cases including reporting, system integration, and API endpoints.
Ensure data quality by applying data engineering best practices, testing, and validation methodologies.
Collaborate with stakeholders to understand data requirements and develop solutions for effective data management.
Analyse data using SQL to uncover insights and ensure efficient querying and reporting.
Maintain an understanding of data modelling principles and apply them to consumption scenarios. Qualifications:
Bachelor’s degree in computer science, Engineering, or related field.
Min 3-5 years of experience in data engineering, with proven skills in Py Spark and Big Data platforms.
Proficiency in Apache Hive, Parquet, and other Big Data technologies.
Experience with at least two end-to-end Big Data projects involving the ingestion, processing, and storage of data.
Knowledge of cloud-native services, with at least one project experience.
Solid data analysis skills using SQL.
Understanding of data modelling concepts.
Azure Synapse knowledge is desirable.