Skip to content
ale@portfolio:~/portfolio$ cat README.md

portfolio

Every case study here answers one question: measurable outcome.

self-serve-monitoring.mdZapier

Self-Serving Monitoring Data Tool

ShippedTeam: Data Products2 weeks
TL;DR

A no-code monitoring test builder on top of the open-source Great Expectations library, moving the team from reactive firefighting to proactive data quality.

Outcome

  • 2wkPOC to shipped v1
  • ↑teams proactively creating their own tests
Saw It
  • Tables and dashboards were breaking silently.
  • An in-house monitoring solution existed, but required coding skills few data team members or stakeholders had, meaning root-cause delays and missing data reaching production reports before anyone noticed.
↓
Figured It Out
  • Ran internal cross-validation with users to confirm fit with the existing stack before wider rollout.
  • User feedback drove a naming change: "Great Expectations" confused users, so renamed the feature "monitoring test" in the UI.
↓
Did It
  • Attached a self-serve monitoring layer to the existing Data Catalog app.
  • Let any team member add data quality tests and route alerts to their own channels, no coding required, no backlog ticket needed.

My Role

Made the build-vs-buy call to adopt an existing OSS library rather than extend the in-house coded solution, and drove the naming simplification after user testing showed "Great Expectations" as a term was actively confusing non-technical users.

Engineering Collaboration

Integrated directly into the Data Catalog app's existing UI and backend rather than standing up a separate tool, a deliberate choice to reduce surface area, made in partnership with the engineers who owned that app.

Full breakdown

Problem & Risk

Usability RiskValue Risk

Tables and dashboards were breaking silently. An in-house monitoring solution existed, but required coding skills few data team members or stakeholders had, meaning root-cause delays and missing data reaching production reports before anyone noticed.

Customer Discovery & Validation

Evidence

Ran internal cross-validation with users to confirm fit with the existing stack before wider rollout. User feedback directly drove a naming change: "Great Expectations" (the underlying library's name) confused users, so we renamed the feature "monitoring test" in the UI.

Context

Rather than build a monitoring system from scratch, we evaluated the open-source Great Expectations library and launched a POC attached to the existing Data Catalog app UI, a build-vs-buy call made in weeks, not months.

Timeline

git log --oneline⏱ 2 weeks, POC to shipped
kickoff: evaluate Great Expectations, scope POCDay 1
validate: internal cross-user testing against existing stackWeek 1
ship v1: self-serve monitoring live inside Data Catalog UIWeek 2

What We Did

Attached a self-serve monitoring layer to the existing Data Catalog app, letting any team member add data quality tests and route alerts to their own channels, no coding required, no backlog ticket needed.

~ rough sketch, not an actual screenshot ~

+ new monitoring test

table, schema, alert channel

✓ passing

✓ passing

! alert sent

Who Benefits

Non-technical stakeholders can now monitor their own data without waiting for a backlog-prioritized engineering fix. Fewer last-minute failure discoveries, more lead time to fix issues before they hit downstream reports. The Data Products team was onboarded to this project based on their prior success implementing Great Expectations elsewhere.

Next Steps

Technical team evaluates functionality against the previous internal tool to confirm this is the right long-term alternative before wider rollout.