Pyspark Data Engineer

india, Tamil Nadu, Coimbatore

Full–time

Posted on: 3 days ago

Job Title: PySpark Data Engineer

Location: Bangalore

Experience: 5+ Years

Employment Type: Permanent (Full-Time)

Notice Period: Currently Serving 30 Days / Immediate to 30 Days Preferred

About the Company

Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.

Job Description

Must Have Skills:
  • PySpark
  • Informatica BDM (Big Data Management)
  • SQL
  • ETL / ELT Development
  • Big Data Technologies (Hadoop / Spark)
  • Data Modeling
  • Banking / Financial Services Domain

  • Minimum Qualifications:
  • Bachelor’s degree in Computer Science, IT, or related field
  • 5+ years of experience in Data Engineering or related roles
  • Strong hands-on experience with PySpark for large-scale data processing
  • Experience in ETL/ELT development and workflow orchestration
  • Solid experience with Informatica BDM
  • Strong knowledge of SQL and data modeling concepts

  • Preferred Qualifications:
  • Experience with data lake and data warehouse architectures
  • Exposure to cloud platforms or big data ecosystems
  • Knowledge of data governance, security, and compliance (Banking domain)
  • Experience working in Agile environments
  • Hands-on experience with Python-based data engineering frameworks

  • Responsibilities:
  • Design, develop, and maintain scalable data pipelines using PySpark
  • Develop and manage ETL/ELT workflows using Informatica BDM
  • Collaborate with data architects, analysts, and stakeholders
  • Optimize data pipelines for performance and scalability
  • Implement data quality, validation, and governance processes
  • Integrate data from multiple sources (databases, APIs, enterprise systems)
  • Support data warehouse and data lake initiatives
  • Ensure compliance with banking regulatory and security standards
  • Troubleshoot and resolve data pipeline issues and bottlenecks
  • Participate in code reviews, testing, and deployment processes

  • Domain Requirement (Mandatory):
  • Experience in Banking or Financial Services domain
  • Exposure to regulatory reporting systems or large-scale financial datasets

Important Note (Please Read Before Applying)