Job Description

Design and implement data ingestion, transformation, and movement using Azure Data Factory (ADF), Azure Synapse, and Data Lake.
Collaborate with business stakeholders, data scientists, and engineers to build robust data solutions.
Develop and manage high-volume, real-time, and batch data ingestion pipelines using Azure Data Factory (ADF).
Implement event-driven architectures for real-time data movement and processing.
Develop large-scale data processing solutions using Azure Databricks and Apache Spark with PySpark, Scala, or Python.
Optimize data partitioning, caching, and indexing for efficient performance.
Manage complex transformations and aggregations for structured and unstructured datasets.
Design and implement high-performance data models in Azure Synapse Analytics using dedicated SQL Pools and Spark Pools.
Optimize query performance, workload management, and cost efficiency in Synapse Analytics.
Implement column store indexes, partitioning strategies, and data caching to enhance performance.
Design and manage secure, scalable data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2).
Implement RBAC (Role-Based Access Control), encryption, and data masking to ensure security.
Implement Azure Monitor, Log Analytics, and Application Insights for data pipeline monitoring and troubleshooting.
Optimize cost management, auto-scaling, and performance tuning across Azure services.

Requirements

8+ years of experience in data engineering, ETL development, and cloud-based data integration,
Strong experience with Azure Data Factory (ADF): ETL/ELT pipeline orchestration and data movement.
Azure Databricks: Large-scale data transformation and big data processing with Apache Spark, PySpark, Scala, or Python.
Azure Synapse Analytics: Data warehousing, SQL Pools, Spark Pools, and performance tuning.
Azure Data Lake Storage Gen2 (ADLS Gen2): Secure, scalable data lake architecture.
SQL Server: Advanced T-SQL, stored procedures, indexing, and query optimization.
Power BI, Tableau, or other BI tools for data visualization.
Financial/Banking experience.

Job Tags

Full time,

Similar Jobs

German American Chambers of Commerce

[Rödl & Partner USA] Praktikant - German Intern Job at German American Chambers of Commerce

...NC and Greenville SC fr mindestens 6 Monate eine(n) qualifizierten Praktikanten. Du wirst unsere Abteilung Business Process Outsourcing (BPO) bei den tglichen Aufgaben untersttzen. Das Programm bietet Dir Einblicke und Erfahrungen in die internationale Wirtschaftspr...

Sunstates Security

Flex Security Officer Job at Sunstates Security

...is currently hiring for a Part Time officer in the Gainesville area with weekly pay. This position is for unarmed service only. Flex Officer Position Requirements ~16 guaranteed hours per month of on-site refresher training ~ Various part time coverage...

Tutor Me Education

In-Person High School Teaching Assistant Job at Tutor Me Education

...at Tutor Me Education, you will work as part of a collaborative team to support lead teachers... ...~ Tutoring starts in the 2024-2025 School Year If you are passionate about... ...Previous tutoring/teaching experience highly preferred Ability to make learning fun...

Data Nomad

Work From Home Product Consultant ( Remote) Job at Data Nomad

...the option to participate remotely or in person. This provides a great opportunity for you to earn extra income while working from the comfort of your home. Apply here Directly: Job Requirements: Show up at least 10 mins before the scheduled start time of the...

Еquinox

Club Staff Job at Еquinox

...Job Description We are looking for energetic, creative, and enthusiastic Style Advisors to join our team at EquinoxSports Club San Francisco! This is a great position for a candidate looking to make a significant impact in a growing and dynamic organization....

Sr Azure Data Engineer Job at Zapcom, Dallas, TX

RlR6amI2amg0ZjFadEI2THV4biswVGliaHc9PQ==