Team VA Goes to Washington

As our five-person Harvard student team left a federal government building and stepped onto Jackson Place, a small street adjacent the White House, we remained silent. This was uncharacteristic. Throughout our weekly meetings at the Harvard Innovation Lab, we often struggled to get one person to speak at a time. Whenever guests visited class, we would interrupt their lectures with questions.

Exhaustion may be one reason we fell silent. We had started off the day briefing the Board of Veterans Appeals before a quick lunch and a 90-minute meeting with the VA Deputy Secretary, Sloan Gibson. We then toured the White House East Wing and the headquarters of the US Digital Service. Including a five-minute talk at the beginning of the US Digital Service’s weekly staff meeting, we presented our project three times—all at different locations in town.

But exhaustion only explains a small part of the silence. We kept quiet out of a deep feeling of content. Having left our protected bubble in Cambridge, we all finally had a chance to see the heart of the action. In fact, we were a part of it. We met the public servants who dedicated their lives to working on the same problems we sought to address. We felt honored to contribute to their mission and inspired to see how our work would live past the semester.

We would like to thank the U.S. Department of Veterans Affairs for their incredible support. In particular, we are thankful to Giuseppe Morgana and Mary Ann Brody at the VA for the time they took each week to help us. We would also like to thank Professor Nick Sinai for dreaming up this collaboration and our Teaching Fellow Angel Quicksey for providing her wisdom along the way.

Working on disability appeals at the VA has been the professional highlight of our time at Harvard. We cannot wait until the next time we get back to the action.

Paris Martin, Jane Labanowski, Chetan Jhavervi, Rohan Pavuluri, and Josh Welle

Prototype #3: Form 9 Redesign

This blog post describes one of the prototypes our team built as part of a Harvard Kennedy School of Government course in which we applied human-centered design principles to government processes. Our team has had the honor of working with the U.S. Department of Veterans Affairs on the process veterans endure to appeal disability claim ratings.

What we heard

Paper and snail mail are still the primary way in which many veterans interact with the VA. Since the majority of veterans going through their appeals process are over the age of 40, most of the VA’s ‘customers’ are not digital natives. We heard over and over again in our interviews that the letters from the VA were hard to understand. Ricky, a retired veteran who is now a lawyer himself, observed, “I’m trained to read legal documents, and even I have trouble making heads or tails out of the gobbledygook in these letters.” We learned that there is an effort already underway at the VA Center for Innovation to redesign the letters, so we decided to focus our efforts on another form of written communication, the forms veterans have to complete to file their appeal. We started with Form 9, which is the form veterans complete to start their appeal.

What we built

We made the following changes to Form 9:

• Made the numbers clearer and larger, so veterans could be confident they were completing all the steps

• Inserted short instructions next to each step, rather than referring to a long set of instructions at the end

• Inserted guided number spaces that indicated to a veteran how many digits should be in the numerical fields. For example, 10 slots for a phone number.

• Used white space to more clearly highlight important information

What we learned

Small changes can make a big difference. While none of these tweaks on their own seemed groundbreaking, one of the veterans remarked, “It’s like bumper bowling! I know I’m on the right path.” We were particularly happy that the use of white space led one veteran to notice important information about how the type of hearing they chose might add a significant delay to the issuance of their decision. On the downside, veterans noted that they would still probably ask a veterans service organization for help while filling out the form.

Paris Martin, Jane Labanowski, Chetan Jhavervi, Rohan Pavuluri, and Josh Welle

Prototype #2: Veterans’ Academy

This blog post describes one of the prototypes our team built as part of a Harvard Kennedy School of Government course in which we applied human-centered design principles to government processes. Our team has had the honor of working with the U.S. Department of Veterans Affairs on the process veterans endure to appeal disability claim ratings.

• Veterans liked hearing from someone who sounded like them:

• Name, sounds like a snoozer

• A la carte

What we heard

One of the veterans we spoke with, Dave, compared applying for disability benefits to being dropped in the middle of a maze without a map. “You just have no idea what is around the next turn,” Dave said. The VA trains veterans during their transition trainings, but we heard the quality of the information veterans receive during these trainings is inconsistent, and disability benefits is only one of many topics covered. Another theme we heard during the interviews was disappointment about how impersonal interactions with the VA seem. Veterans were turned off by the fact that most of their communication with the VA was through an intermediary or via impersonal letters.

What we built

We wanted to create a flexible platform where veterans could learn about all aspects of the disability benefits appeals process, and we wanted to put a human face on the VA. Inspired by Khan Academy, an educational platform that uses bite-sized videos to explain complicated topics, we created Veterans Academy. The prototype allows veterans to see a flow diagram of the appeals process, and see short videos on the sections that pique their interest. Our teammate, Josh Welle, who also happens to be a veteran, was the star of the videos. Josh tried to walk veterans through the process in a way that was easy to understand and conveyed empathy.

What we learned

Veterans again reminded us of the importance of branding by picking on the name of the prototype. “Veterans Academy…doesn’t exactly sound thrilling, does it?” But they LOVED the fact that they could learn about the process from a veteran just like them. Mike said, “Hey, when that guy said he was a veteran JUST LIKE ME…my ears perked up. It wasn’t just another bureaucrat.” Veterans wanted the prototype to take a step further, and allow veterans to choose the type of veteran who would guide them through the process. So in the second version of our prototype, we added a gallery of potential veteran guides. Finally, veterans thought they might remember the key messages of each video better if text was included alongside the video.

Paris Martin, Jane Labanowski, Chetan Jhavervi, Rohan Pavuluri, and Josh Welle

Prototype #1: Veterans’ Service Organization (VSO) Finder

This blog post describes one of the prototypes our team built as part of a Harvard Kennedy School of Government course in which we applied human-centered design principles to government processes. Our team is working with the U.S. Department of Veterans Affairs on the process veterans endure to appeal disability claim ratings.

What we heard

After retiring from the Navy, Josh started thinking about applying for disability benefits for the shooting pains in his back, bum knee, and hearing loss. As a combat veteran, Josh was no stranger to navigating tricky situations. But when Josh talked to his fellow veterans about the process for applying for disability benefits, they all said, “GET HELP!” Fortunately, there are dozens of veterans’ service organizations (VSO) that assist veterans like Josh to apply for disability benefits. But when Josh went to find a VSO, he was frustrated that they all seemed to have long waiting lists to see a representative. Worse still, Josh had to go in person to the offices of multiple VSOs and talk to the receptionist at each one to learn the time and date of the next available appointment.

Another veteran we spoke to, Andrew, also complained about the inconvenience of booking an appointment with a VSO. Andrew now works for a bank on a trading floor, where he isn’t allowed to use his cell phone. Every time Andrew wants to meet with his VSO, he has to step off the trading floor to call the VSO. Yet another veteran we spoke with, Anthony, complained about the inconsistency of the VSOs he has worked with: some were great, some were not very good. Anthony wished there was some way to know the quality of a VSO in advance.

What we built

We set out to build a prototype that would make it more convenient to schedule an appointment and compare the quality of VSOs. We called our prototype, “VSO Finder.” The prototype allowed veterans to search for a VSO near them by entering their zip code, compare the ratings of VSOs, view possible appointment windows, and book an appointment with a few clicks.

What we learned

Veterans liked the idea of a more convenient way to book appointments with VSOs. But they wanted to be able to filter based on more than just zip code. For example, when we showed Andrew the prototype, he thought some of the VSOs in his search results (e.g., Vietnam Veterans of America) did not cater to younger veterans like him. Veterans also cringed a little when we referred to VSO Finder as “Yelp for VSOs.” Applying for disability benefits is a much more serious decision than booking a dinner table, and veterans wanted the branding and messaging of the prototype to reflect that. 

Paris Martin, Jane Labanowski, Chetan Jhavervi, Rohan Pavuluri, and Josh Welle

From Insights to Action

Any kid who plays with LEGOs can tell you that there is a thrill to building something new. As our team transitions from user insights to prototyping, we’re learning that the thrill of creation can be even more satisfying when your concepts are informed by deep user insights.

So far in our semester-long course at the Harvard Kennedy School on Technology & Innovation in Government, our team has taken the time to better understand the disability benefits appeals process, interview veterans and synthesize user insights. Now it is time to turn those insights into product ideas that could improve the experience of U.S. veterans who are appealing their disability benefit decisions. Our prototyping journey so far has been split into three phases: 1) brainstorm, 2) prioritize, and 3) test.

1. Brainstorm: No idea is off-limits.

Our team started by listing the five main themes that emerged from our interviews with veterans, and brainstorming solutions. We adopted a few norms proposed by IDEO, a leading design and innovation consulting firm. One of the more unusual norms was to “be visual” by drawing on our post-its as we conceived of the ideas. The little sketches really helped us to convey the idea’s essence and remember each idea long after the session.

Some ideas that emerged were potentially transformative but would require a lot of changes to existing processes. For example, Chetan wondered if we could ‘flip the system’ by having Veterans’ Service Organizations work with veterans to determine their initial ratings, and having the VA double check a smaller portion of the claims. This would bring the VA in line with best practices at private insurance companies, where claims are processed relatively efficiently because only a sample of claims is audited.

Other ideas would really increase the ‘human touch’ of a system that currently feels impersonal. Paris thought of a case manager, a single person at the VA in charge of your case who you could contact through phone or text. Jane wondered if VA raters could create an audio narrative of their decision—like a doctor dictates their notes—and Josh built on this idea by proposing veterans also be allowed to submit audio narratives in support of their claims. Rohan proposed a “Khan Academy for Veterans,” where veterans could learn more about the process through videos, rather than confusing instruction sheets.

2. Prioritize: Narrow the list.

Our brainstorming session generated over 50 ideas. We narrowed the list by focusing on three criteria:

• Impact on the Veteran: Would the idea meaningfully improve the appeals experience of a veteran?
• Burden on the VA: Does the idea decrease the amount of resources the VA would need to process the claims?
• Feasibility of testing: Can we think of a way to test the idea before May?

The pictures below show 12 of our finalists and how they fared across these criteria. We chose three concepts to test:

• Yelp for VSOs: A platform that links veterans to veterans' service organizations (VSOs). Veterans would be able to see services offered, schedule appointments, and connect to other veterans' reviews of the VSOs
• Vets Academy (modeled after Khan Academy): A portal where veterans can access user-generated, expert curated videos on issues throughout the appeals process. Should include a step-by-step overview of the process. This could be a VA resource that is promoted to the VSOs and in transition training.
• Form 9 Redesign: Form 9 is where veterans note their points of disagreement, and choose the type of hearing they would prefer (hearings can be in person in DC, by teleconference, at a regional office, or the veteran can choose to waive a hearing). The concept is to improve the overall look and feel of letters to increase trust, usability, and form effectiveness.

Our whiteboard prototype

3. Test: The rougher the prototype, the better.

A founding member of the U.S. Digital Service, Erie Meyer, visited our class to give us tips on how to ‘hack bureaucracies’ and launch successful technology projects in governments. Erie’s biggest piece of advice was to make early prototypes as ugly as possible. She even offered a personal White House tour to the team with the ugliest prototype! Why? According to Erie, a rough prototype allows testers to be more creative, and encourages testers to think more about the concepts and ideas rather than the design. Rough prototypes also prevent a team from being overly committed to an idea based upon the time and effort it took to create the prototype.

Challenge accepted, Erie. In the next week, our team will create three “ugly” prototypes and test them with veterans.  

Chetan Jhaveri, Jane Labanowski, Paris Martin, Rohan Pavuluri, Joshua Welle

The Power of Post-its

---

After four weeks of meeting with veterans and understanding the VA appeals process, we started the next phase of our journey in the Kennedy School’s new Tech and Innovation course: turning our research into user insights.

Kate Krontiris, an all-star user research expert from the U.S. Digital Service, kicked off this part of our project with a guest lecture. Walking us through her own work helping streamline immigration, Kate showed us how to turn interview notes into actionable user insights. She gave us one piece of particularly prudent advice: use Post-Its.

After Kate’s talk, our team gathered at the Harvard Innovation Lab (iLab) on a Sunday night for a marathon session. Following Kate’s directions, each member of our team chose a specific interview and walked the room through a veteran’s appeals experience.

Jane told us about a veteran who gave up on his appeal because he needed to go to grad school. Rohan described a veteran who only used the internet on his smart phone and couldn’t access his personal information because the VA’s eBenefits system wasn’t yet mobile-friendly. Chetan quoted a veteran going through an appeal: “I want the VA to look me in the eye and deny me, and tell me my airborne operations, or carrying a 70-pound ruck, are not service-connected to my lower back issues”.

As each of us detailed a veteran’s timeline, key quotations, and interview themes, the others in the room wrote down the pain points we heard using a design researcher’s favorite tool: Post-Its. These consisted of brief comments like “Needs more personalized communication” and “Only found information from word-of-mouth.” Once we finished walking through the interviews, we shared what we wrote and identified emerging opportunity areas.

First, we created five distinct clusters on the iLab’s expansive white-board, one for each veteran. Within each cluster, we combined similar pain points into separate columns. It was great to get each team member’s unique insights and takeaways.

We then clustered our Post-Its once again—this time by theme—making sure not to lose any of our ideas along the way. Organizing these interconnected pain points served as the most challenging part of our exercise.

Finally, we labeled each cluster with a distinct theme. Debating succinct terms and phrases to describe each collection of Post-Its, we worked hard to reach a consensus for each heading.

By the end of our session, we found trends across all of our interviews. While the veterans came from different branches of the military and told us different stories, they faced many of the same problems. The process confirmed much of our intuition, but it also revealed issues we had yet to consider.

Through Kate’s Post-It exercise, we learned that design research is equal parts art and science, requiring both creativity and algorithmic thinking. Next stop: turning our insights into a prototype.

Chetan Jhaveri, Jane Labanowski, Paris Martin, Rohan Pavuluri, Joshua Welle

Delving into the Veteran's Disability Appeals Experience

The past few weeks have been exciting for the Team VA—our group of five students in a Harvard Kennedy School field class about tech and innovation.

After the initial problem definition and immersion phase of the project, we began prepping for research. We explored a variety of design research methods under the tutelage of Dana Chisnell and crafted a plan, including a recruiting screener and interview guide.

We then reached out the Boston veterans’ community, as well as our own Harvard and MIT networks to identify participants for research. We were pleasantly surprised by the number of responses from our initial canvassing email, and spent a few days calling folks for a second screen, to ensure we got a good mix of branch of service, age, gender, and life stage. We decided to target a combination of in-depth interviews (30-45 minute conversations, ideally in person) and intercepts (~10-15 minute conversations) to get the broadest swath of perspectives.

Meeting Veterans Where They Are

Our first foray into user-centered design with veterans was a morning trip to the Boston VA Hospital and the JFK Building, to meet as many vets as we could in an environment where they were comfortable. There, members of the team encountered a variety of servicemen, and spent a few hours getting to know them, and hearing their stories. Concurrently, we began holding more in-depth interviews with Boston-area veterans in person, and over Google Hangouts.

Meet Joe B.

We spoke to Joe*, who served in the Army for over ten years, starting as a private and spending the majority of his time in Special Operations from the Rangers to “Delta Force”. Upon his return to civilian life, Joe’s initial claim for disability benefits was rejected because he missed a mandatory appointment due to an overseas assignment. He appealed this claim, and four months later, received an initial disability rating, which he still felt did not reflect his true disability, which he is again in the process of appealing.

Talking with Joe, we began to truly understand the complexity of the process for the veteran appealing a claim, from simple things like misinformation of address, to bigger things like not knowing who to ask for help. Joe stressed that his frustration was not unique, that he had plenty of buddies going through similar barriers to care, and mentioned he knew at least “5 friends who didn’t even file” due to this difficulty.

Kicking it with Kara

We also spoke with Kara*, who served with the Air Force for most of her twenties, and is now living with her husband in Boston, and applying to grad school. Kara was one of the first of her female classmates from the Academy to separate from the service, and was shocked by the lack of information she received during her Transition Assistance Program (TAP) on filing a claim. At the time, she was based overseas, and was cognizant that the amount of time spent detailing disability claims may have varied by region, but she ended up doing the majority of legwork herself, once she had separated. Due to the nature of her service, she suffered both physical and mental afflictions, and felt it was important to get treatment. She credits her extreme organization to her ability to file a claim and receive a disability rating that she felt accurately reflected her injuries.

Kara now works to provide her friends with as much information as possible upon their separation from the military so that they too can access the benefits they deserve. Kara has put together a checklist and spreadsheets that she shares with friends, and had a host of great ideas for how the VA could streamline the process—including better transparency of information and better access to doctors. Kara’s enthusiasm to help our team underscored how much veterans care about each other, and how dedicated they are to improving the veterans’ experience.

Digging into the Data

So far, we’ve spoken to almost twenty veterans about their experiences with disability claims, and the list keeps growing. Our team feels so lucky to have the opportunity to be privy to the narratives of these noble men and women, and as we move into the synthesis phase of the project, we’re looking forward to beginning to design potential solutions. As our portable “war room” fills with post-its, themes and opportunity areas are starting to bubble up… next stop, user insights!!

If you’re interested in learning more about design research, check out this list of great reads to get started.

Chetan Jhaveri, Jane Labanowski, Paris Martin, Rohan Pavuluri, Joshua Welle

*Names have been changed

Serving Those That Serve

Introduction: On January 28th, five Harvard students of various ages and from different degree programs were asked a question: “Who is your most cherished veteran?” They answered: my grandmother, my brother, my friend, my Annapolis classmate, and my teacher.

Our diverse team was huddled around a rolling desk and surrounded by whiteboards at the Harvard Innovation Lab (iLab). It was clear that this class project would be different. The mission: work with the Department of Veterans Affairs’ (VA) to improve the lives of veterans by innovating on the disability appeals process.

This Harvard Kennedy School field study, established by Adjunct Professor and former White House Deputy CTO Nick Sinai, fuses students -- undergraduate techies, Silicon Valley product experts, and policy wonks -- to become government policy hackers. In 12 weeks, our team will work with Washington officials, interview veterans in the Boston region, and leverage MIT/Harvard computer scientists to rapidly design a user-centered solution to help the VA better serve its constituents.

Serving the VA is personal. VA policies matter because the appeal process and its responsiveness directly impact people’s quality of life. Due to modern medicine and better training, the wars in Iraq and Afghanistan yielded over 50,000 wounded veterans (either physical or mental). In wars past, many of those soldiers would have died on the battlefield, instead, they survived. Those without wounds are also transitioning due to force restructuring. At the peak of Operation Iraqi and Enduring Freedom, Army force strength was 570,000 troops and will downsize to 450,000 by September 2017. All soldiers separating will need medical physicals prior to signing the DD 214; however, over time, thousands of vets will need to go on to appeals in the decades to follow service-connected disabilities surface.

Integrity. The commitment to providing a transparent appeals process is about integrity. The contract between the government and an all-volunteer force is sacred. Some injuries do not surface during the claims process because they have not developed, while other claims are not disclosed at transition because of the anxiety associated with transition. No matter when an injury is found, if it is connected with military service, the VA must address the ramifications and determine compensation. Veterans of all stripes-- Army, Navy, and Air Force -- must be afforded the resources to communicate ailments as they surface.

Speed. The appeals process must strip out all distracting and unnecessary steps that deter a veteran from getting treatment. When joint injuries or mental depression emerge after an initial claim is determined, the Veteran is often in need of attention. It can’t wait. However, today, the appeals process is longer than it should be, or could be, if the VA adopted smarter technology.

Trust. The appeals process must provide sufficient feedback and open communication to the veteran so that trust develops between both parties. Some veterans have lost trust in the system because of inconsistent communication between local, regional, and federal offices. One office reported an appeal would take 8 months while another 15, the appeal ultimately took 26.

Next steps. The VA serves the 1% who defend the liberties of the 99%. Under Secretary Robert MacDonald’s leadership philosophy, shaped by time as an Army Ranger and CEO of Procter & Gamble, the VA is putting the veteran at the center of the user experience. Building on the great work of VACI (VA Center for Innovation), we'll be applying a user-centered approach to drive our efforts in this semester-long project.

Chetan Jhaveri, Jane Labanowski, Paris Martin, Rohan Pavuluri, Joshua Welle