NPR migrated 190+ radio station sites to a modernized third-party Content Management System (CMS.) The vendor provided the basic design, but we added our own enhancements and named the system Grove.

I first conducted a review of secondary data, auditing the prior four months of ticket requests and questions raised in Office Hours. Customization was a big theme, so I explored visual improvements such as color schemes, font alternatives, and module enhancements (above.)

Pros: Readily accessible data, quick turnaround
Cons: Self-selected user base, environment primed for obvious bugs

Continuous discovery

I then pivoted to continuous discovery and invited my team observe a weekly interview with a Grove user. After 10+ interviews, it was clear what users really loved about Grove was how it made their work faster, and they wanted more of that. I plotted out the steps of a common workflow in Grove and identified 13 high-impact wins to save time. (One change saved 6.5 hours/year per person.)

Pros: Direct and recurring access to real humans, encourages team investment
Cons: Requires lead time to operationalize, can lead to slow-forming insights

A learning framework

Our ticketing system was filled with people wanting custom features, but our interviews included people that were quite happy with Grove. I sent out a survey to better understand user groups. 55% of stations participated.

The results helped us craft trainings and support documentation in the short term. In the longer term it gave us a starting point for curating permissions settings based on realistic personas.

Pros: Data at scale, fast reach
Cons: Numbers can lie, tells you what but not why

Ultimately, mixed methods give the clearest picture, so all of the above combined formed a powerhouse of knowledge about our users.