Field Study for a Federal Agency’s Intranet
Lesson Learned:
The best insights are found outside of your own office.
Field Study for a Federal Agency’s Intranet
In this article…
Why has NN/g done this for over 20 years?
Project Overview
Type of Project:
Ethnographic Research
(Field Studies & Contextual Inquiry)
Role:
Co-Researcher
Client:
Government Services Agency (GSA)
Date:
May 2019
Location:
Washington, DC
Project Details
The Nielsen Norman Group publishes an Intranet Design Annual report each year to both recognize the top 10 intranet designs and offer best practices in a space where there is little, if any, publicly available information (given the gated nature of most intranet environments). This annual report is made possible due to dozens of contributors who submit hundreds of screens and sites for review every year. In exchange for their submission (and permission to publish their intranets) NN/g conducts free research and offers pro-bono consulting to all applicants.
During this study, I got the chance to facilitate usability studies and a particularly memorable field study, which involved a tornado warning and evacuation to a designated emergency shelter area.
(More on that shortly.)
Updated: Dec 2024
Project Goal:
Conduct usability studies and field studies in order to determine usability of the federal agency’s Intranet.
Why has NN/g done this for over 20 years?
For starters, intranets are a remarkably under-resourced category of digital design. Employee-facing solutions are often seen as less “reputationally risky” than customer-facing solutions, so the temptation to not invest in them is often quite strong. Not to mention, the false-consensus effect often misleads teams into thinking that research isn’t necessary, because “the product is for ‘us.’”
I probably don’t need to tell you this next bit, because bad intranet experiences are everywhere. But, once you’ve experienced a terrible intranet, then you know that the people who build these systems are rarely representative of actual users of the system.
Before we get into the project details, it’s important to reiterate why intranet design is important. Intranets are not just a convenience to employees; they have the potential to be a productivity multiplier or inhibitor, and if done poorly, can cost companies millions of dollars in wasted time.
The real cost of a bad intranet
Bad intranets waste time; and time is money.
To best explain this, let’s look at an example that I’ve personally experienced: the US Army Reserve.
In a typical duty day, the average US Army Reservist must log into at least 4 different gated intranet services (but often, several more):
the virtual desktop / VPN (avg. 10 min in load time to connect)
email
myHRC
milSuite.
It would take an additional 3 clicks per system (with an average load time of about 10 seconds per click) just to get to the login page (and an additional 15 seconds to log in). If you’re doing the math, that’s about 3 min more of logging in. This might sound trivial. However, since these are separate systems, switching between them also results in attentional switching costs (it takes time to remember what the heck you were doing), and also, raises the potential to time out in the middle of the log-in process, effectively starting it over again.
The end result: it often takes 20 min every day to just to log in.
Multiply 20 min x an average wage of $21/hr x the number of Army Reservists (176,968)
= $1,238,764 per day
Multiply that times 39 duty days, and that’s $48,311,780 per year.
… and that’s just the Army Reserve.
If you also factor in the active duty members of the US Army, you would then get:
Daily cost of logging in ($21 * 0.3) x the number of active duty Soldiers (452,689)
= $3,165,654 per day
Multiply that value x the typical number of (non-deployed) duty days per year (240)
= $759,756,960
In total, the US Army wastes a staggering $808,068,740 per year on slow login processes.
Shaving off just 3 minutes per day would save: $158,673,564 per year.
Now, the US Army is an enormous organization, but if I told you a 50-person organization could likely waste $63K per day (over $15 million per year) by having to sift through multiple sources of information, would you invest a few thousand dollars in the research and design of a better intranet?
I’d bet yes.
What our team provided
I was invited to join my colleagues Maria Rosala and Anna Kaley in Washington, DC due to my prior public-sector experience (and general interest in the design of federal applications). As part of the data-gathering process, we did the following:
Stakeholder interviews: We interviewed the primary points of contact and key intranet team members to understand the intranet’s goals, key workflows, and areas of functionality about which the team was hoping to learn. This helped finalize the research plans for our primary research methods.
Qualitative Usability Testing: We conducted 75 min usability testing sessions in a conference room at the client’s site, with 8 different participants from a variety of different roles and levels of seniority. This provided insight into whether or not the intranet supported key workflows.
Field Study (Contextual Inquiry): We “shadowed” several employees to better understand how the Intranet was utilized by those responsible for creating content and those looking for critical information.
The tornado warning
To be brief, I’d been observing a member of of the internal communications team as he worked to submit a story for the Intranet, when, halfway through the session, a tornado warning was sent (via SMS) to both the participant’s phone. Certain areas of the building were evacuated to more storm-resilient areas, and I was able to witness something far more extraordinary: how employees turn to intranets during moments of crisis to find things like emergency shelters and protocols.
I followed the employee for safety reasons, but continued to document the experience in the event something more serious were to happen. I asked the participant for permission to take photos and record via my phone. This ended up being a far more effective strategy than typing and walking, and actually felt less intrusive, ironically.
Luckily, the emergency management elements of the organization were honed well, and broadcasted communication was swift and effective at steering employees toward the right places. And, luckier still, the tornado passed with little incident, and we were able to leave the building the minute the session was scheduled to end.
What we learned
To summarize the key takeaways from the… more “normal” parts of the study.
Clear, simple writing saves time, and boosts confidence. Technical jargon only serves technical audiences, and often results in “side-quests” to learn the jargon (rather than actually complete the task).
Information architecture shouldn’t be an afterthought. Primary, secondary, and tertiary navigation need to be deeply considered. “Popular tools” and “resources” categories tend to become “catchalls” for content that ultimately catch… no-one.
Design systems are not just for customer-facing interfaces. Documentation and guidelines can do wonders in helping onboard and guide Intranet content contributors as they make changes via the CMS.
Well-designed intranets impact more than daily work; they also keep employees informed and well-prepared for extraordinary situations.
Some things that I learned, as a researcher:
Be prepared to take photos during field studies to capture how people supplement or keep track of necessary information.
Be flexible in note-taking strategies (using recordings, photos, or even your phone) to ensure you can balance note-taking and active listening.
Now, can you just conduct a usability test in a conference room and learn a lot of things? Absolutely.
But, seeing how people live and work, in the flesh, will give you far more insight and inspiration to build something that keeps people happy, productive, and most importantly: safe.
•••