Skip to content
Srujan Rudrapati

I build the data layer, and the AI on top of it.

Data engineer working across Databricks, Azure, and AWS. Lately building LLM systems that run on real data.

Data pipeline from source systems up to LLMs and agentsSource systems feed a medallion architecture: bronze for raw data, silver for cleaned data, and gold for modeled data. Gold feeds a serving layer, and LLMs and agents run on top of it.Medallion architectureSource systemsBronzerawSilvercleanedGoldmodeledServing layerLLMs and agentsRAG, Claude API, MLX

About

I'm a data engineer finishing an M.S. in Data Science, Analytics, and Engineering at Arizona State University. Before that I spent two years at Celebal Technologies building data platforms on Azure Databricks for clients in healthcare, energy, and manufacturing. Lately I build LLM systems that run on real data, like a private, offline AI system for ASU Surplus. I graduate in December 2026 and I'm available early 2027.

Featured work

2026 · Built on the job, in active use

Surplus AI

Private, offline AI system for ASU Surplus

Problem

Surplus staff were typing hardware specs into a spreadsheet by hand, which introduced typos into the inventory. Writing product listings was also manual.

What I built

One connected pipeline: collect specs, build a master spreadsheet, and have AI write the listings, all on a private network that never touches the internet.

    • USB spec collector

      system_profiler on each working Mac

    • Camera scanner

      Vision OCR on the bottom label

  1. Lookup table

    Product, year, screen size, resale lane

  2. Master inventory

    Keyed on serial number

  3. LLM

    Strict prompt: never invent specs

  4. Listings

    Results saved

How a Mac goes from the shelf to a listing. Working Macs report their own specs; Macs that won't boot get their label scanned. Both land in one inventory keyed on serial number.
Internet
No uplink
Gigabit switch
  • Node 1

    10.0.0.1

    Head

  • Node 2

    10.0.0.2

    14B model files still copying

  • Node 3

    10.0.0.3

    14B model files still copying

  • Node 4

    10.0.0.4

    14B model files still copying

  • Node 5

    10.0.0.5

14B model files still copying

Five M1 Mac minis on an isolated gigabit switch with fixed IPs and passwordless SSH from the head node. Nodes 2 through 4 are still receiving the 14B model files.

No uplink on the switch

Wi-Fi off

HF_HUB_OFFLINE=1, so the model software can't download anything

Model server only answers on localhost

Surplus AI
Four layers sit between the model and the internet. The app checks Wi-Fi and internet status live.

How it works

Five-node cluster
Five M1 Mac minis on a gigabit switch with no internet uplink, fixed IPs, and passwordless SSH from the head node. Apple's MLX framework splits one model across several machines with tensor parallelism, so a 14B model can run across Macs that can't hold it individually.
USB spec collector
A Bash script on a USB stick runs system_profiler on each Mac, reads model, chip, RAM, storage, battery health, and macOS version, looks up year and screen size from a reference table, and de-duplicates by serial number.
Camera inventory scanner
For Macs that won't boot, a webcam photographs the bottom label and Apple's Vision OCR reads the model number, EMC, and serial. These are matched against the lookup table to get product, year, screen size, and resale lane (Retail, Auction, or Parts).
Merge logic
Both sources land in one master inventory keyed on serial number, so a camera scan never overwrites real data from a working machine.
Surplus AI app
A Flask server runs two model engines: Qwen 14B across four nodes, and Llama 3B on one node for quick responses. Replies stream word by word. A Listings tab builds a strict prompt from each row's real specs, and a Cluster tab polls each node over SSH every 5 seconds.
Launchers
One double-click starts everything, waits for the engines to respond, and opens the app. Closing it shuts down every node.

Status

3B engine runs now14B cluster engine in progress

Everything works except the 14B cluster engine, which is waiting on model files being copied to nodes 2 through 4.

Stack

  • MLX
  • Flask
  • Bash
  • Apple Vision OCR
  • Qwen 14B
  • Llama 3B

November 2025 · Personal project, built solo

Autonomous Supply Chain Watchtower

Multi-agent LLM system

  1. Input

    Sales history

  2. Agent 1

    Inventory Forecaster

  3. Agent 2

    Risk Analyst

    Tool

    RAG tool

    ChromaDB, shipping-disruption data

  4. Agent 3

    Logistics Coordinator

  5. Output

    Logistics plan

Three agents run in sequence, each consuming the previous agent's output. Only the Risk Analyst has a tool: retrieval over real shipping-disruption data in ChromaDB.

Problem

Turning raw sales history into a logistics plan normally takes several manual steps by different people.

What I built

Three agents running in sequence on the native Anthropic Claude API through CrewAI, each consuming the previous agent's output. The Risk Analyst has a custom tool backed by a retrieval-augmented generation pipeline over ChromaDB, so its output is grounded in real shipping-disruption data instead of model recall. Built with Claude Code.

Status

Personal prototype

Stack

  • Python
  • Anthropic Claude API
  • CrewAI
  • RAG
  • ChromaDB

Other projects

  • · MS capstone, team of four

    Demand Forecasting & Inventory Optimization

    27.7%WAPE vs. 38.5% naive baseline

    Forecasting demand across 250 SKU and warehouse series, with a strict time-based test window. Turned the forecast into a reorder-point policy that cuts simulated holding cost about 28% at comparable service levels.

    Team project with Rakshit Govind Thota, Sanjana Parmareddy Gari, and Yashwanth Sai Tondapu.

    • Python
    • LightGBM
    • Prophet
    • scikit-learn
  • End-to-End CI/CD Deployment Pipeline

    Multi-tier AWS environment in Terraform, parameterized so one codebase deploys dev or prod. GitHub Actions runs tests on every push to main, builds a Docker image, pushes to ECR, and triggers a Terraform deploy.

    • Terraform
    • Docker
    • GitHub Actions
    • AWS
    View Repo
  • · Built as a TA for ACO 430 under Prof. Feng Wang

    WLAN Traffic Forensics Toolkit

    100,000+WPA2-encrypted packets parsed

    Rebuilds the network topology as a directed graph and classifies device types with K-Means clustering. A Z-score detector on top caught a live denial-of-service deauthentication attack.

    • Python
    • Pyshark
    • NetworkX
    • scikit-learn
  • F1 Race Outcome Predictor

    80%top-3 accuracy vs. 73% baseline

    Ranks drivers by win probability, with rolling features shifted forward to prevent look-ahead leakage. Backtested on 44 held-out races: 52% top-1 and 80% top-3 accuracy, against a 43% / 73% baseline.

    • Python
    • scikit-learn
    • HistGradientBoosting
  • Containerized Flask Service on Kubernetes

    Flask API with Postgres and Redis in a multi-stage Dockerfile, run locally through docker-compose and deployed to a local kind cluster.

    • Docker
    • docker-compose
    • Kubernetes (kind)
  • Real-Time Group Chat Application

    Multi-client chat in Core Java. A server handles concurrent TCP socket connections and broadcasts messages, with client and server code kept modular.

    • Java
    • Swing
    • AWT
    • TCP sockets

Experience

  1. May 2026 to present

    Tech Assistant, Surplus Operations

    Arizona State University

    Tempe, AZ

    • Built Surplus AI, a private, offline AI system for ASU Surplus.
    • Built a Power BI dashboard on asset sorting data showing where items stall.
    • Advise university departments and outside buyers on what equipment fits their needs.
  2. August 2025 to May 2026

    Graduate Teaching Assistant

    Arizona State University

    Network Security & Wireless Networks, Glendale, AZ

    • Designed hands-on labs for two 400-level networking courses covering traffic analysis and packet-level debugging.
    • Held office hours for 40+ students.
    • Built and presented the WLAN forensics toolkit to the class.
  3. September 2022 to November 2024

    Associate, Data Engineering

    Celebal Technologies

    Jaipur, India

    A data and AI consultancy. I worked on client engagements across healthcare, energy, and manufacturing, mostly on Azure Databricks.

    • Telehealth client in New Zealand, Microsoft Fabric

      Clinical and operations teams were waiting days for reports. I designed the Lakehouse architecture and coordinated a five-person team. Time-to-insight dropped about 50%, and the dashboards ended up in their weekly operational meetings.

    • A US materials engineering client, AWS

      Ran a production pipeline on Glue, S3, and Redshift handling about 10 million records a day from 7+ source systems. Cut downtime about 30% by tracing recurring failures to root causes instead of patching them.

    • A major US utility, data governance (10 months)

      Their Microsoft Purview catalog was maintained by hand. I built automation on Databricks that pushed entities and custom classifications into Purview on a schedule, plus Apache Atlas lineage pipelines and a business glossary mapped to their governance policies.

    • Medallion architecture on Databricks

      Built bronze, silver, and gold layers plus a serving layer on Azure Databricks and Delta Lake. Wrote the Spark transformations and dbt models.

    • A US medical device manufacturer

      Audited their existing Azure Data Factory pipelines before changing anything, and shipped updates with zero disruption to production.

    • A large FMCG client

      Data governance engineer, using Unity Catalog alongside Microsoft Purview to manage access, lineage, and classification across the Databricks workspace.

    • Production ML pipeline

      Built the Spark and Databricks pipelines feeding a production machine learning model, worked with the data science team on feature engineering, and built the serving layer that delivered model-ready data.

    • A/B testing

      Built the pipelines feeding experiment data, set up test and control assignment logic, and ran the statistical analysis on results.

    • Pre-sales

      Wrote Azure cloud architectures and Bill-of-Materials cost estimates, and presented them directly to prospective clients alongside the sales team.

Skills

Data engineering

  • Python
  • SQL
  • Spark
  • PySpark
  • Databricks
  • Delta Lake
  • Unity Catalog
  • Databricks Workflows
  • dbt
  • Kafka
  • Airflow
  • medallion architecture
  • ETL/ELT
  • dimensional modeling
  • data warehousing

Cloud

Azure
  • Databricks
  • Data Factory
  • Fabric
  • ADLS
  • Synapse
AWS
  • Glue
  • S3
  • Redshift
  • Lambda
  • EC2
  • RDS
  • IAM
  • VPC
  • CloudWatch
  • ECR
  • ECS
Other
  • Snowflake

AI and ML

  • Anthropic Claude API
  • RAG
  • vector databases (ChromaDB)
  • CrewAI
  • agentic workflows
  • MLX
  • local LLM serving
  • LightGBM
  • Prophet
  • scikit-learn
  • forecasting
  • clustering
  • anomaly detection
  • feature engineering

Governance

  • Microsoft Purview
  • Apache Atlas
  • Unity Catalog
  • metadata management
  • data lineage

Infrastructure

  • Terraform
  • Docker
  • Kubernetes
  • GitHub Actions
  • CI/CD
  • Linux
  • Bash
  • Flask

BI

  • Power BI
  • Tableau
  • Qlik

Certifications and education

Certifications

  • Databricks Certified Data Engineer Professional
  • AWS Certified AI Practitioner
  • Microsoft Certified: Azure Data Engineer Associate
  • Microsoft Certified: Azure Administrator Associate

Education

  • M.S. in Data Science, Analytics, and Engineering

    Arizona State University

    Expected December 2026 · GPA 3.96 / 4.0

  • B.Tech in Computer Science

    Rajasthan Technical University

    2019 to 2023 · GPA 3.38 / 4.0

Contact

Graduating December 2026, available early 2027.

srujanrudrapati07@gmail.com

linkedin.com/in/srujan-rudrapati