Summary
Spent the summer working on the data science team. Migrated and rebuilt four production revenue-forecasting models into Dataiku, maintaining or improving predictive performance across millions in revenue planning. After finishing the primary objective ahead of schedule, led a proof-of-concept for AI-driven inventory deduplication, projected to save roughly $200K over third-party vendor options.
Key Projects
- Forecasting & Model Evaluation w/ Dataiku
- Supply Chain Duplicate Detection
- Legacy-to-Snowflake Script Migration
Revenue Forecasting Models
Background / Problem
With the electric grid, there are regional grid operators such as PJM, who oversee supply and distribution to balance the grid. They are the "air traffic controllers" for the electric grid across 13 states and D.C. To do this, locational marginal pricing (LMP) and regulation market clearing price (RMCP) are co-optimized, representing the marginal energy cost and the ancillary price for marginal reliability/risk. LMP is essentially the price of energy, while RMCP is the price of having reserve capacity on hand in case something disrupts the grid, such as extreme weather, a sudden change in wind output, and so on. Because that dynamic can be lucrative, internal models existed to predict revenue across four ancillary service streams — Regulation, Synchronous, Non-Synchronous, and Secondary — and there was real room to modernize both the platform they ran on and how much signal they were able to pull from the available data.
Design Choices & Approach
I started with background research into the domain, then met with the stakeholders who'd built the original models to understand their reasoning and where there was room to improve. Alongside that, I was learning Dataiku, the platform everything was being migrated into.
The migration began with loading and preprocessing the data, then iterating on feature engineering and model selection. Several of the models struggled to find a strong signal in the available data, so I explored ways to get more out of what I had, reworking how time-based patterns were represented, adding lagged signals, and testing what actually moved performance versus what didn't.
I also reconsidered what "better" meant here. The original models had been optimized for R², but this is really an error-minimization problem, so I shifted the primary metric to MAE while still tracking R², MAPE, RMSE, and directional accuracy, with outputs swinging into positive or negative revenue in the millions, the choice of metric mattered more than it would for a typical regression task.
The result was four production models that matched or improved on the originals' performance, now running on infrastructure the business team could actually access and trust. I learned a lot about working with limited data, avoiding leakage, choosing the right metric for the actual business question, and translating technical performance into something stakeholders could act on.
Supply Chain Deduplication
Background / Problem
The supply chain team's master inventory data with internal part numbers, manufacturing info, specs, descriptions, plants, etc. had accumulated years of manual entry, part reuse, and plant acquisitions, resulting in a huge number of near-duplicate entries describing the same thing in different ways. With hundreds of thousands of entries, finding and reconciling duplicates by hand didn't scale.
Algorithms & Technical Details
This was scoped as a proof-of-concept, evaluated against several known categories of near-duplicate cases the team dealt with regularly. I started by manually walking through how a person would identify a duplicate today, both to understand what signal they were actually using and to see what could realistically be automated.
Plain text-similarity approaches on their own weren't reliable as two entries could look nearly identical on manufacturing info and still be genuinely different parts, or vice versa. The approach that actually worked combined several signals into one weighted comparison, including a step that used an LLM to pull structured features out of inconsistent free-text descriptions, which turned out to be the piece that made the biggest difference in accuracy. That fed into a per-pair confidence score, bucketed into ranges the supply chain team could act on directly, with adjustable weighting so the system could generalize across different material types instead of being hardcoded to one.
Getting the weighting and thresholds right took several rounds of working sessions with the supply chain team. "Is this actually a duplicate" turned out to be a more contextual judgment call than it first appeared, which shaped a lot of the design.
Legacy-to-Snowflake Script Migration
Background / Problem
An R script that runs on a monthly job was set up to pull data from the old big data storage, which was no longer being maintained, and needed to be updated to use connections to the new data platform.
Challenges
The update itself did not seem too complex, as the main changes were just switching the old connections to point to the new source, and everything else should remain relatively similar. Though switching the
connections to point to the new source was just the start, as some of the queries had to be updated to account for the new platform's style, meaning syntax changes and removing unnecessary lines as the new platform was more efficient with its
techniques. This led to testing out queries in Snowflake to see what worked and how the queries could be optimized. The real challenge was what came next: validating the outputs. Now, this part didn't seem daunting at first, but two side-by-side
RStudio sessions later and multiple days revealed the complexities of this task. Most of the errors that came up were minor and could easily be resolved, such as handling the new connection returning data in a slightly different format or with the headers in all caps.
To truly validate the new script, I wanted to do side-by-side comparisons of each component that involved a database connection to see that the old script outputs matched the new ones, but I realized that 1. the data was being queried slightly differently
based on dates, so the outputs weren't even comparable and 2. some of the queries were nested into loops with lots of operations (200+ lines, taking hours to run), which I had validated them isolated, but wanted to verify the connection and any resulting data was
compatible with the other operations within the loops and the two scripts still matched. For context, the data being queried was from four tables, each with over 252 billion rows, which made it very challenging to complete thorough checks, especially in big loops and
when dealing with competing resources with other employees using RStudio. All in all, the task taught me a lot about patience, but also about thoroughly validating changes, even if they seem relatively simple at first.
Adventures
Exploring new places and experiences has been a significant part of my journey, offering insights and perspectives that complement my technical work. Some of the professional experiences included: Data & AI Summit, Women's Conference, volunteering at local community events, doing a boat tour of the Central Virginia Offshore Wind site (CVOW), an unofficial intern skyzone trip, learning about other areas such as the Storm Center and distribution planning, multiple coffee chats, and winning the "Dominion Energy Rowing Challenge".