Lead Developer / Architect – Kafka as a Service

icon
inr null - null undefined/undefined

Full–time

Posted on: 2 days ago

Skills

Job Title: Lead Developer / Architect – Kafka as a Service Experience: 7–10 Years Location: Gurgaon / Pune Role Overview We are looking for a highly skilled Lead Developer / Architect to design and build a Public Cloud Kafka-as-a-Service (KaaS) platform. The role will focus on designing and implementing a robust control plane responsible for provisioning, scaling, and securing multi-tenant Kafka clusters deployed on Kubernetes. The ideal candidate should possess strong expertise in distributed system architecture, Kubernetes internals, Kafka internals, security architecture, and platform reliability engineering. Roles required : System Design, Kubernetes Internals (Operators/CRDs), Security Architecture, and Reliability. This person needs to design how the "Cloud API" talks to the "Kubernetes Cluster" to spin up complex stateful workloads. Job Description: Lead Developer to architect our public cloud Kafka-as-a-Service (KaaS) platform. You will own the design of the control plane that manages the lifecycle (provisioning, scaling, security) of multi-tenant Kafka clusters on Kubernetes. Key Responsibilities: • Core Architecture: Design the KaaS Control Plane to handle cluster provisioning, resizing, and decommissioning. Kubernetes Orchestration: Implement custom controllers or leverage Kafka Operators to manage stateful sets, PVCs, and Namespace isolation. •Advanced Kafka Engineering: Implement Rack Awareness and AZ-based placement strategies to ensure high availability and leader distribution. •Security Design: Architect the mTLS and SCRAM-SHA-512 authentication flows, including Certificate Authority (CA) management per cluster tier. •Resource Management: Design logic for enforcing hard partition limits and Kubernetes Resource Quotas to prevent "noisy neighbor" issues. •Observability: Design the metering pipeline using JMX Exporters and Prometheus to track billing metrics like "Bytes In/Out" and "Storage Usage". Required Skills: • Languages: Expert proficiency in Java or Go (Go preferred for K8s Controllers/Operators). • Kubernetes Internals: Deep understanding of CRDs, Operators, StatefulSets, PVCs, and Resource Quotas. Kafka Internals: Not just using Kafka, but understanding the broker protocol, controller placement, partition reassignment, and dynamic configuration. •System Design: Experience designing multi-tenant, distributed systems with strict SLAs for throughput and latency. Why Join Us? • Opportunity to work on cutting-edge ERP solutions with a global impact. • Collaborative and innovative work environment. • Professional growth and continuous learning opportunities. Share your updated resume along with the following details: • Current Location • Willingness to work in a Hybrid Model • Current CTC • Expected CTC • Notice Period (shorter notice periods will be given preference) Kindly send the above information to careers@acclerotech.com at the earliest. We look forward to reviewing your application. Thanks Chaitra Kamath Head-People Management