spotify analytics

A project that started as an attempt to reverse-engineer my own music recommendations. It pulls my listening history (top tracks, top artists, recently played, audio features), keeps a running record of it on my homelab, and uses that to experiment with recommendation models of my own. What began as "can I copy what Spotify does" slowly became a way to actually learn how these models work, test them, and point them at my own taste.

why i built it

Music is a big part of my life. It's on constantly, and I care more than is probably reasonable about what I listen to. For years Spotify's recommendations were the quiet thing I leaned on to find new music, and I never really thought about them until I switched to Apple Music and realised how much worse they could be. The suggestions felt random and lifeless next to what I was used to, and it made me wonder how hard it would actually be to build the good version myself.

So I set out to replicate Spotify's suggestion algorithm, with a longer-term idea in mind. If I ever moved my music onto my own catalogue, a future iPod or something running on the homelab, I wanted to bring the recommendations with me instead of being locked to whichever platform happened to own them. I started by building a pipeline to pull my listening history and park it on the homelab, then began experimenting with ways to turn that history into suggestions, KNN, cosine similarity over audio features, and a few others.

Somewhere along the way the goal quietly changed. It stopped being about copying Spotify and became about actually understanding how recommendation models work, comparing them, measuring which ones felt right, improving them, and using them for real. Being curious about my own taste, what it looks like in numbers and what a model thinks I'd like next, turned out to be a much better reason to keep going than beating Apple Music ever was.

the thinking

A couple of decisions shaped it. First, I built everything on my own listening history rather than a generic dataset, and I persisted it on the homelab so it would keep accumulating over time, since real, personal, longitudinal data is the whole point of a recommender that's supposed to know me specifically. Second, I started deliberately simple, distance-based methods like KNN and cosine similarity over audio features, before reaching for anything heavier. I wanted to understand what a basic approach could and couldn't do, and to have an honest baseline before deciding whether more complexity was actually earning its keep.

the analysis

how does it move over time?

Because every run is a dated snapshot, comparing them shows how the top tracks and the feature profile shift week to week, the drift Wrapped can't show.

how it works

The pipeline runs outside-in, from raw listening data all the way to a playlist I can open in Spotify.

Collection. A Flask backend handles the Spotify OAuth2 handshake and pulls my listening data (top tracks, top artists, recently played, and audio features), and a headless daily_snapshot job reuses a stored refresh token to keep pulling on a schedule with no browser. That history lands in PostgreSQL on the homelab (a recent_tracks table, the audio-feature tables, and so on), so it accumulates into the longitudinal record the models depend on.

Feature engineering (models/feature_engineering.py) turns two sources, my listening history and a larger catalogue of candidate tracks, into one shared feature space. It resolves the numeric audio features they have in common, cleans and coerces them, fills or drops missing values, deduplicates, standardises everything with a StandardScaler, and computes recency weights from when each track was collected. The output is a single weighted user preference vector, a profile of what I've actually been listening to lately.

Two recommenders. The cosine baseline (recommenders/cosine.py) scores every catalogue track against that preference vector by cosine similarity and returns the top-k with their metadata. The item-to-item KNN model (recommenders/knn.py) works the other way around. For each track I've listened to it finds its nearest catalogue neighbours in feature space, then aggregates those candidates across all my listened tracks using the same recency weights, with knobs for how many source tracks to use and a minimum-similarity cutoff. Both run end to end through small CLI runners (train.py and train_knn.py) that write ranked recommendations to CSV, with flags for top-k, neighbour count, dedupe mode, and recency half-life.

Using it for real. A backend route (POST /api/recommendations/create-playlist, with model=cosine|knn) pushes a set of recommendations straight back into Spotify as a real playlist, and the React front end has a model-and-parameters selector with an "open in Spotify" button. So the whole loop, from my own listening history to a playlist I can actually press play on, closes without leaving the app.

what i learned

The most useful thing I learned is that the project got more interesting the moment I stopped trying to copy something and started trying to understand it. Chasing "replicate Spotify" gave me a goal, but the real reward was watching a simple model actually surface songs I liked and understanding why it did. It also taught me that a recommender is only as good as the data behind it, which is exactly why hosting my own listening history on the homelab mattered so much.

To be honest about where it stands, the data pipeline and the exploration work end to end, the model experiments are genuinely useful and still evolving, and the dashboard front end is only partially built. It's the project I most want to come back to, mostly because it's the one I actually use.