Manikandan Muthiah

Cloud Data Engineer

Senior Data Engineer · Azure · GCP · Apache Spark · Databricks

Professional Objective

"To be a dynamic person with passion to learn new things and to adopt the new surrounding and able to deliver best results."

Technical Summary

Senior Data Engineer with expertise in PySpark, Python, SQL and exposure to Azure and GCP cloud
Around 9 years of Software industry experience, 8 years of Data Engineering
7 years of experience in Agile workflow model
8 years on Azure cloud services: ADF, Azure Databricks
2 years on GCP cloud services: Dataproc, Composer and BigQuery
Extensive knowledge in Healthcare, Mortgage Insurance, Retail and Airline domains

Technical Skills

Languages

Python SQL PySpark Scala

Cloud Platforms

Azure GCP AWS

Azure & GCP Tools

Azure Data Factory Azure Databricks Dataproc Cloud Composer BigQuery

Databases & Storage

Delta Lake MS SQL Oracle PostgreSQL Hive AWS Redshift

Work Experience

Senior Software Engineer

EMAKINA (EPAM Systems), Doha

Sep 2023 – Present

Duty Free Business Intelligence · Leading Aviation in Qatar

Environment:

Azure DatabricksGCPBigQuery Cloud ComposerPowerBI

Designed scalable ingestion framework transferring data from on-premise to Google Cloud
Transformed raw data into insights for duty-free sales forecasting and food waste optimization
Built PowerBI datasets enabling data-driven decisions for business stakeholders
Orchestrated end-to-end pipelines using Google Composer (Airflow)
Reduced processing time via BigQuery indexing and partition optimization

Senior Software Engineer

EPAM Systems, Chennai

Apr 2022 – Sep 2023

Aviation Inventory Analytics · Leading Aviation in Qatar

Environment:

Azure DatabricksDelta TablesDremio

Built ingestion framework from on-premise to Azure cloud (high availability)
Supported data science team with booking trends, transaction behaviour, Flight 360 metrics
Enhanced performance via Delta table optimizations and query tuning

Senior Data Engineer

EPAM Systems, Chennai

Apr 2022 – Jun 2022

Data Factory for CTC · Retail & Distribution

Environment:

Azure Data FactoryAzure Databricks Delta LakeAzure MySQL

Built file- and RDBMS-based ingestion framework with minimal latency
Applied Spark optimizations: broadcast joins, repartitioning, coalesce, salting
Compacted small Delta files to significantly reduce query execution time

Application Development Senior Analyst

Accenture, Chennai

Apr 2021 – Apr 2022

Home Buyer Transformation · NA Mortgage Insurance

Environment:

Azure Data FactoryAzure Databricks PySparkAzure DevOps

Ingested data from diverse sources into Azure Data Lake via ADF
Designed ADF pipelines integrating Databricks for transformation
Applied PySpark best practices: caching, partitioning, broadcast joins

IT Analyst

TCS, Chennai

Jul 2016 – Apr 2021

Healthcare Data Modernization · US Healthcare

Environment:

MS SQLIBM DB2Hive PySparkAWS RedshiftHDFS

Converted legacy COBOL SQL to optimized Hive queries
Implemented PySpark DataFrames API with broadcast joins and repartitioning
Created Hive tables with bucketing and indexing optimizations

Certifications

🏆

Databricks Certified Associate Developer for Apache Spark

Issued: April 2025 · Expires: March 2027

View Credential

☁️

Microsoft Azure Data Fundamentals

Microsoft Certified

View Credential

🔐

Google Cloud: Implement Cloud Security Fundamentals

Google Cloud Skill Badge

🤖

Google Cloud: Perform Foundational Data, ML, and AI Tasks

Google Cloud Skill Badge

📊

Google Cloud: Manage Data Models in Looker

Google Cloud Skill Badge

View All Badges on Credly

Education

🎓

Bachelor of Engineering (B.E.) — Electrical and Electronics Engineering

Anna University, Chennai

CGPA: 9.13

Latest Articles

Data Engineering

Hosting n8n and Building an English–Hindi Voice Translation Bot

A guide on setting up n8n and creating a voice translation bot using Telegram.

Read on Medium →

BigQuery

Applying CMEK Encryption to BigQuery Dataset and Tables

Security best practices for BigQuery data encryption.

Read on Medium →

Dataproc

Submit job in Dataproc using Composer

Orchestrating Dataproc jobs using Cloud Composer.

Read on Medium →

Products

Side projects I've built and shipped — from personal finance tools to cognitive games.

Live

💰

MyFinance

Personal Finance App · PWA

A real-time personal finance dashboard that brings live market data, investment tracking, and multi-currency finance tools into one clean interface.

Live indices — NSE Nifty 50, Bank Nifty, NASDAQ, S&P 500, Hang Seng, Nikkei & more
FX & Gold rates — USD/INR, QAR/INR, live gold prices in INR and QAR
Investment portfolio tracker with P&L and gain/loss analytics
Cash flow management — income, expenses, and category breakdowns
Insurance policy and subscription management
Secure password vault for credentials

TypeScript React Tailwind CSS Vite Netlify REST APIs

🌐 Live App ⌥ GitHub

PWA

🧠

Brain Games

Cognitive Game App · Progressive Web App

A progressive web app with 500 levels of cognitive challenges across 6 unique game types — designed to sharpen memory, logic, and problem-solving skills.

500 progressive levels with increasing difficulty
6 distinct cognitive game types
Installable as a PWA — works offline on mobile and desktop
Smooth level navigation with sound effects and result overlays
Built with modern React + TypeScript stack

TypeScript React Tailwind CSS Vite PWA GitHub Pages

🎮 Play Now ⌥ GitHub

🔀

SplitIt

Expense Splitter · Web App

A lightweight web app to split shared expenses fairly among groups — perfect for trips, dinners, or any group activity.

HTML CSS JavaScript

⌥ GitHub

🌐

Live Translate

Voice Translation Bot · n8n + Telegram

A real-time English–Hindi voice translation bot powered by n8n automation and Telegram, demonstrating AI workflow integration with LLM pipelines.

n8n JavaScript Telegram API LLM / AI

⌥ GitHub

Projects

📦

GitHub Repositories

Explore my 39+ repositories covering various data engineering topics and tools.

View Code →