Role and Responsibility:

Proven experience in Data Engineering as an individual contributor, including direct interactions with business users, cross-functional stakeholders, and participation in end-to-end development and implementation activities.
Strong hands-on expertise with Big Data technologies, PySpark, Python, DevOps practices, and performance tuning of large-scale data workloads.
High proficiency in SQL along with a solid understanding of data modelling concepts and best practices.
Working knowledge of version control systems (Git), DevOps CI/CD pipelines, and Agile development methodologies.
Deep understanding of distributed computing fundamentals, including RDDs, Data Frames, partitioning strategies, and cluster execution behavior.
Excellent debugging, analytical, and problem-solving skills with the ability to troubleshoot complex data issues

Experience

4–10years of experience as a Data Engineer working in enterprise-level data platforms.
Proven expertise in designing, developing, and maintaining scalable Big Data pipelines using PySpark and Python.
Hands-on experience working with business users and cross-functional teams to convert business requirements into robust technical solutions.
Strong proficiency in SQL with extensive experience in query optimization and relational data modeling.
Experience implementing CI/CD pipelines, Git-based version control, and DevOps best practices.
Worked within Agile environments, participating in sprints, requirement grooming, and technical reviews.
Expertisein performance tuning, including Spark job optimization, partitioning strategies, and efficient resource utilization.
Strong debugging and analytical skills for addressing complex data issues and system inconsistencies.

Good-to-Have Skills

Additional expertise in advanced data modeling practices (conceptual, logical, and physical modeling).
Experience with PySpark code migration, including refactoring, platform upgrades, and performance enhancements

Data Engineer

Job Description