Building scalable infrastructure and automating everything. Passionate about cloud-native technologies, Kubernetes, and contributing to open-source projects.
DevOps Engineer II with deep expertise in AWS and GCP Cloud, focused on designing, implementing, and maintaining best-in-class DevOps and DataOps practices. Experienced in leveraging technologies such as Kubernetes, Terraform, Ansible, Jenkins, Crossplane, and ArgoCD, along with modern observability platforms, to deliver scalable, secure, and resilient infrastructure solutions.
Specialized in the Data-DevOps ecosystem, working extensively with Apache Spark, Kafka, NiFi, Metabase, RabbitMQ, DataHub, and other Big Data frameworks. Skilled in orchestrating end-to-end data pipelines, ensuring system reliability, performance optimization, and smooth integration across distributed data systems.
Passionate about automation, observability, and performance tuning, with a strong focus on bridging the gap between data engineering and operations. Dedicated to driving continuous improvement, cost efficiency, and empowering teams to deliver faster, smarter, and more reliable solutions in complex cloud-native data environments.
I believe in giving back to the community. Here are some of my contributions to open-source projects.
Added early validation of SparkApplication metadata.name to ensure it conforms to DNS-1035 label requirements. This prevents applications with invalid names from being accepted and failing later when the controller attempts to create the web UI Service.
Identified and fixed a critical bug where SparkApplication status would get stuck in submitted state due to UI service creation failure caused by invalid names.
Actively working on improving the Spark Operator with additional features and bug fixes to enhance the Kubernetes-native Spark experience.
Check out more of my work on GitHub
K8s-native Spark management platform
Open source contribution - DNS-1035 validator
IoT-based parking management solution
Interactive algorithm visualization tool
Real-time IoT data streaming
Interactive word game
Banking system simulation
Indian Institute of Information Technology, Dharwad
Sanjay Singh Yadav Inter College, Gaya
Oxford Public School, New Delhi
I'm always open to discussing new opportunities, interesting projects, or just having a chat about technology.
India