Role: Data Engineer Location: Hyderabad, Bangalore5 days WFOExperience: 7+ years Responsibilities Knowledge to ingest, cleanse, transform and load data from varied data sources in the above Azure Services Strong knowledge of Medallion architecture Consume data from source with different file format such as XML, CSV, Excel, Parquet, JSONCreate Linked Services with different type of sources.
Create automated flow for pipeline which can consume data i.e may receive file via email or Share point.
Strong problem-solving skill such as backtracking of dataset, data analysis etc.
Strong Knowledge of in advanced SQL techniques for carrying out data analysis as per client requirement.
Required Skills The candidates need to understand different data architecture patterns and parallel data processing.
S/he should be proficient in using the following services to create data processing solutions: Azure Data Factory Azure Data Lake Storage Azure Databricks Strong Knowledge in Py Spark SQLGood programming skill in Python Desired Skills Ability to query the data from serverless SQL Pool in Azure Synapse Analytics.
Knowledge of Azure Dev Ops.
Knowledge to configure any dataset with Vnet, Subnet Networks Knowledge of Microsoft Entra ID, to create App registration for single and multitenant for security purpose.
