Shyam P

Skills

Proficient in Python, PySpark, Scala, Java, and advanced AI frameworks like OpenAI (LLMs), FAISS, and RAG Architecture. Strong background in Databricks, Delta Lake, Presto, Snowflake, Airflow, Cassandra, Kafka, Hive, HDFS, and Elasticsearch

About

Architected and deployed ETL pipelines processing 50TB+ of data daily using Medallion Architecture on Databricks, Delta Lake, Presto, Snowflake, and Airflow.
Optimized Spark-based data pipelines, reducing processing time by 40%, improving efficiency in Credit Analytics, S&OP, Supply Chain, and Logistics domains.
Engineered highly available AI/ML models, enhancing predictive accuracy by 30% in production environments.
Leveraged AWS Cloud (S3, EMR, Glue, Step Functions, Athena, Redshift) to build cost-effective data solutions, reducing cloud costs by 25%.
Designed and deployed real-time streaming solutions using Kafka, Cassandra, and Elasticsearch, handling millions of events per second.
Developed interactive data visualization dashboards in Power BI.