What To Do When Users Don’t Want Your Product

Last week, we discussed our research process and our preliminary user segmentation as part of a student project aiding Treasury’s re-launch of USAspending.gov in May. 

Now, after meeting with more than 25 current or potential users of federal spending data, we’ve learned the value of hearing “no” and of digging into our users’ perspectives.

While feedback on what’s not useful is hard, it is valuable. In a government setting, it can save money and help to direct efforts into the right places. 

A Common Need for “More”

Our biggest conclusion is that USAspending.gov is not perceived as actionable by the many existing and potential users of the website that we interviewed.

Despite differences in industry, expertise, and responsibilities, every group we interviewed had broader needs that extend beyond USAspending.gov’s current capabilities.

So, what do users need? What is “more”?

Our interviews helped us understand what “more” entailed for each user group. We used best practices in design thinking—including the KJ method—to distill our research into five takeaways:

1.     Better design is not a solution

  • Our users’ pain points are not rooted in the design of the website.
  • For example, Chris, the VP of Federal Sales at a data informatics startup uses a business intelligence software service that incorporates federal spending data. Since Chris doesn’t access USAspending.gov directly, a website re-design isn’t impactful for him.

2.     Forward-looking data is valuable

  • Vendors to the government want clues into future contracting opportunities. Past expenditure doesn’t easily inform future revenue opportunities, and is thus immediately relevant.
  • For example, Kevin, the owner of a curtain drapery business, wants to re-drape a local Navy office, but notices that office’s drapings haven’t been touched in the last decade. For Kevin, historical data is not actionable. 

3.     Full contracting data tells a bigger story

  • In order to win contracts, users want information linking past expenditures to the relevant contract documentation. This provides insights on upcoming re-bidding opportunities.
  • For example, Tom, the Head of Government Relations at a large technology startup, wants all information at one place. He currently searches through 3-4 government databases, uses a business intelligence service, and networks with competitors in the effort to piece together a full picture of upcoming opportunities.

4.     Primary data sources are the most trustworthy

  • Because USAspending.gov aggregates other agencies’ data, many users we spoke with perceive USAspending.gov as lower quality in terms of accuracy and timeliness.
  • For example, a VP of Product at a large financial data company described the difficulty of reconciling data inconsistencies. He has dealt with cases where multi-million contracting opportunities are missed because the data “didn’t include all the 0’s.”

5.     For existing visitors, USAspending.gov is one step among many

  • Small business advisors and vendors that use USAspending.gov today view it as just one step in larger process—and not something that helps them take the next step in their process.
     

User Journeys

We initially thought procurement was simple.  Below is how we thought vendors to the government researched and won contracts.

Yet, through our user interviews, we learned that procurement is a winding, often circular process. We identified two user journeys:

1.     Large vendors to the government are aided by business intelligence services (part of the “re-purposers” segment), that they use to inform them through procurement process. Because these firms provide synthesized procurement information, large vendors rarely access USAspending.gov directly.

2.     For the Small Business Owner, the procurement process is a daunting multi-step process to navigate alone. Because they are more resource constrained, they turn to government resource centers like the Small Business Administration.

Next steps

All of this negative feedback was surprising to us—and we were puzzled about our next steps.

To start, we recognized the limits of USASpending.gov—an important lesson in preventing potential scope creep. We are expanding user our research into other groups, including housing experts financial institutions and startups focused on transparency.

Finally, we plan to explore ideas based on the needs we heard from procurement-oriented user groups.  Prototyping solutions in a limited time here may be difficult.  Nonetheless, we’re excited for the challenge, and look forward to sharing more in our next blog!

-Anna, Cindy, Maya, Ni, & Will

*All names have been changed to protect the privacy of our interviewees

 

Who Cares About Federal Spending Anyway?

The U.S. Treasury Department is about to re-launch USAspending.gov—a database containing detailed accounts of all U.S. federal expenditures—with the aim of making the government budget more transparent and understandable.

As a team of Harvard students in a Kennedy School field class working with Treasury, our task was to discover the answer to a difficult question: who cares?

Federal spending data seems very important—so much so, that Congress passed the DATA Act, a law requiring that it be transparent and freely available online. Politicians and pundits all talk about federal spending, but our team is on a mission to discover who actually uses this dataset—and for what?

To answer this question, we started with the basics: interviewing and observing existing and potential users of federal spending data. For example, we spoke with these three individuals: 

  • Kaitlin, a product manager at a government-focused data startup looking to build a sophisticated dashboard that shows federal contracting opportunities
  • Mike, the CEO of a small media company that provides video services to government agencies
  • Peter, an advisor at a local Small Business Administration (SBA) branch who is focused on giving his clients the best business advice

Kaitlin, Mike and Peter came from diverse backgrounds and had very different data needs. To make our project effective for Treasury, we knew we had to focus—and fast. But where? We wrestled with this question as part of our user research process.

 Diverging in order to converge

To organize our research, we created five categories of users and developed hypotheses on how each could use federal spending data:

  • Government Vendors: companies that sell products or services to the federal government interested in learning about contracts
  • Information Providers: companies that provide aggregated business or government data insights to other companies looking to add new datasets to their sources
  • Researchers: academics, think tank staffers, and journalists looking to uncover trends, tell stories and advocate policy solutions
  • Congressional Staffers: Capitol Hill staffers interested in comparing how their constituency fares relative to others, or advocating for a budget priority
  • Grant Recipients: state and local government officials looking to better plan and compete for federal grants

As we spoke with dozens of users across these categories, Treasury had one main guiding principle for us: search for scenarios where USASpending.gov can create economic value. With this in mind, we were able focus and know what to listen for, even as we kept expanding the list of users we spoke with.

Key Insights

We’ve come away with three main insights from our user research process:

1. Understand the “hierarchy of needs” for your users: We learned that Government Vendors and Information Providers are the categories most likely to leverage spending data to spur economic activity. For them, information about government contracting is central to business. 

As Mike—the CEO of the media company—told us, a small firm like his does not have the resources to send staff to Washington, DC to lobby government agencies. Although he doesn’t use USAspending.gov today, Mike was excited to explore the new website and how it might inform his sales process. Kaitlin—the product manager—saw potential in linking spending data with other procurement information to enhance her product and help her startup grow.  

2. Refine and redefine your user segments as you learn more: The segments we researched had nuances and differences within them. In the Information Services category, for example, we had very different types of users.

Peter—the SBA advisor—uses federal spending data manually and occasionally:

“I go through four or five government websites to register small businesses before thinking about spending data. When I do get around it, I typically search the website by industry and show the results to my clients.” 

On the other hand, Kaitlin’s product management needs were quite different:

“We pull government data automatically, through an Application Programming Interface (API). To use the new spending information, we would need extremely up-to-date data and an identifier field to tie this dataset to others we are already using.”

3. A business is not a user, but it may have many users within it: We understood the importance of seeing a user as a specific person in a specific role. When we spoke to Sam, a director of business development in the startup where Kaitlin works, his perspective was very different. Rather than features and technical integration (Kaitlin’s focus), Sam was thinking about using spending data to price a contract. Despite working in the same company, his vantage point and data needs were completely different. 

We realized that good user research should focus on a consistent profile and allow us to compare several users with similar roles across organizations. 

Next steps

Our user research got us a preliminary answer to the question of “who cares?”. Now, we will focus on two additional questions: why do they care and how much.

We will continue talking with prospective and actual users of federal spending data—and are eager to turn our research into useful insights, prototypes, and recommendations to aid the U.S. Treasury. Stay tuned as we share more in our next post!

Anna, Cindy, Maya, Ni, & Will

*All names have been changed to protect the privacy of our interviewees.

Making the Case for Spending Data

Imagine the possibilities of unleashing detailed data about federal government expenditures across the United States:

  • Enabling a laundry business owner in San Diego, CA to successfully research and bid on a large, stable contract with the military as she seeks to expand her business
  • Empowering a research and evaluation firm in Durham, NC to compete for a Department of Education grant to determine the efficacy of a federally-funded STEM education program
  • Educating a Seattle city councilmember about the procurement timeline for the expansion of the local VA hospital

Soon, needs and desires like these will be more easily and quickly fulfilled.

In May 2017, the U.S. Department of the Treasury will relaunch USAspending.gov, the federal government’s spending transparency website. It will offer the public a user-friendly financial transparency tool with new capabilities and more extensive data, giving access to timely and accurate information.

The public will soon be able to access a dataset that reveals how the $3.8 trillion federal budget is spent annually across 100+ governmental agencies.

Meet the Team

Where do we, a cross-disciplinary team of Harvard students, come in? Here’s a high-level view of the team members:

  • Anna Ponting brings the perspective of a policy-maker and digital government implementer. She has worked for the City of New York City and consulted for mayor’s offices around the world on digital transparency projects, which will help the team understand the challenges of launching tech projects in government.
  • Cindy Yang has a background in both start-ups and finance, having worked on launching a new vertical at a fintech startup in San Francisco after helping it scale from 30 employees to over 400. Her experience in user research, business development and ability to work alongside regulators will help us engage with potential users and assess demand for financial data.
  • Maya Perl brings expertise in strategy, operations and project management as a former management consultant for organizations ranging from large corporations to community-based non-profits. Her experience driving analytical initiatives will help the team think about the data and metrics needed to make Treasury’s products impactful to users.
  • Ni Xu comes to the project with a background in software engineering and entrepreneurship. He is a former engineer from Apple now on his way to an MBA, and will help us recommend useful tools for developers that leverage USAspending.gov’s new capabilities.
  • Will Long brings experience leading technical teams in projects for startups, international development NGOs, and think-tanks. A Joint Concentration in Computer Science and Government, his expertise will enable the team to analyze and build out use cases with Treasury’s data.

Our mandate

The Treasury Department has spent a lot of time painstakingly preparing this data for public consumption. Now, they want input on how users might best use the data, build on it, and create value.

We’ll take our cumulative skills in market research, data analysis, product management, and software engineering to build out these use cases and help Treasury uncover demand for the spending data. For starters, we’re building hypotheses on various use cases—who could best utilize this data set, what format do they need to digest it in and how could Treasury promote the creation of economic value from this use? Our earliest thoughts center around vendors—unveiling new opportunities for America’s 28 million small business owners who otherwise might have confronted barriers in the government procurement process. That said, we’re still early in our user discovery and research—stay tuned!

An Unprecedented Case of Transparency

Why make spending data transparent? The impetus for the expansion of USAspending.gov was the Digital Accountability and Transparency Act, also known as the DATA Act, signed by President Obama in 2014. The legislation mandated the creation of government-wide reporting and data standards across all types of spending and agencies in an unprecedented granular and standardized level.

The Treasury team, however, is going far beyond data disclosure. Our project leads at Treasury, Christina Ho and Amy Edwards, are so committed to usability that they want to proactively uncover new use cases across a variety of stakeholders. Over the next 12 weeks, we’ll explore this landscape.

Treasury’s Work to-Date

For our project, we’ll be building off of Treasury’s momentum. They’ve already done extensive work by establishing the data standard, incorporating user feedback and setting out initial user research.

The relaunch happening in May will include 400 data elements for every contract, grant, loan, or other governmental financial disbursement, making expenditures comparable and standardized. Since ~75% of federal spending is transferred to state or local governments, users will be able to track the expenditure down to specific projects..

Incorporating public feedback has been a priority to Treasury since the project’s inception, which they’ve achieved through their GitHub site. GitHub acts as a collaboration space for Treasury’s developers to interact with interested members of the public. It’s available to everyone to comment on code, proposed functionality, the data model, or participate in user testing.

Beyond the technical achievements behind USAspending.gov, the Treasury has vigorously adopted user-centered design into their work. They’ve developed four personas, or user types, likely to use this data: taxpayers, recipients of a contract or grant, journalists, and technical users seeking to repurpose the data (see graphic, below). These personas informed the design of the new website roll-out. Our team will build on this work as we consider the tool’s potential users and their needs.

Our next step is to evaluate a breadth of potential users to understand their pain points, desires, and goals when using government spending data. Treasury is especially interested in identifying business use cases and measuring economic impact to demonstrate the value of their effort and promote it as a model to other parts of local, state, and federal government.

Together with the Treasury, we’re flipping the “build it and they will come” approach on its head. In a few months’ time, we hope to leave Treasury with a strategy to go directly to users and help them discover the potential behind the data.

Will Long, Maya Perl, Anna Ponting, Cindy Yang, Ni Xu