Data Engineer

Job Description

Detailed JD (Roles and Responsibilities)
Extensive Experience in IT data analytics projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Cloud Data,Proc, Google Cloud Storage, Composer

SQL concepts, Presto SQL, Hive SQL, Python (Pandas, Numpy, SciPy, Matplotlib) and Pyspark

Design and implement scalable data pipelines using Google Cloud Dataflow

Develop and optimize BigQuery datasets for efficient data analysis and reporting

Collaborate with cross-functional teams to integrate data solutions with business processes

Automate data workflows using Cloud Composer and Apache Airflow

Ensure data security and compliance with GCP Identity and Access Management

Mentor junior engineers in best practices for cloud-based data engineering

Implement real-time data processing with Google Cloud Pub/Sub and Dataflow

Continuously evaluate and adopt new GCP tools and technologies

Extensive Experience in IT data analytics projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Cloud Data,Proc, Google Cloud Storage, Composer

SQL concepts, Presto SQL, Hive SQL, Python (Pandas, Numpy, SciPy, Matplotlib) and Pyspark

Design and implement scalable data pipelines using Google Cloud Dataflow

Develop and optimize BigQuery datasets for efficient data analysis and reporting

Collaborate with cross-functional teams to integrate data solutions with business processes

Automate data workflows using Cloud Composer and Apache Airflow

Ensure data security and complia