I have yet to find a better way to innovate than simply talking to people. In the world of innovation, we often get trapped in internal meetings discussing abstract concepts, forgetting that sometimes the only feedback that matters is the kind you get on the street.
In 2013, the Obama administration set out to make college more affordable. At the time, there was growing frustration with influential ranking systems, which rewarded exclusivity, lavish amenities, and prestige over actual student outcomes. As noted in The Atlantic’s analysis, the logic was flawed: the fewer students a college accepted, the higher its rank; the more expensive its facilities, the more admired it became.
The administration’s goal was to create a tool that treated students like informed customers, one that revealed what happened after graduation. Were students finishing their degrees? Were they earning enough to pay back their loans? What was the actual return on their investment?
However, building this tool proved far more complex than expected. According to the authors of Hacking Your Bureaucracy, the team immediately hit nuanced questions of context. How do you measure the “value” of a liberal arts degree that pays off slowly over time? How do you compare institutions serving low-income, first-generation students against elite, private universities? And how do you evaluate a theology student pursuing spiritual service rather than a high-income career?
The initial version of the College Scorecard, launched in 2013, fell flat. It offered outdated averages, lacked personalization, and, most critically, failed to include the earnings data the White House had prioritized. While it was intended to be a “torrent of data,” the first attempt lacked the user-centric focus needed to make that data actionable.
The turning point came in 2015 when a cross-governmental team, including the United States Digital Service (USDS) and 18F, was tasked with a total overhaul. The project was led by Lisa Gelobter, the Chief Digital Service Officer at the Department of Education. Instead of retreating into internal meetings, Gelobter’s team conducted a “discovery sprint” to engage real customers.
The team took paper prototypes of a mobile-friendly site directly to the National Mall, interviewing students and parents on the spot. This immediate feedback was a revelation. Some of the team’s favorite features, including a complex comparison tool, was flatly dismissed by students as irrelevant for mobile devices.
The result was a redesigned, mobile-first College Scorecard grounded in how students actually make decisions. The change didn’t happen because the technology got better; it happened because designers finally consulted the people they were trying to help, The shift was profound: they were no longer designing for an abstract customer profile, but for real people with names, faces, and personal aspirations.
While this National Mall test offered a spontaneous and informal approach, consultation for innovation can also be achieved through more formal, regulated frameworks like in Bologna’s neighborhood labs.