Researching internal business processes and forms
UX Research: Contextual Inquiry, Task Analysis
Researching internal business processes and forms
Researching internal business processes and forms
Project summary
Evaluative research of the intranet and PDF forms used internally for the South Carolina Department of Natural Resources
My role
Sole UX researcher
Background
Based on heatmaps, analytics, and survey responses, one of the most used areas of the current agency intranet is the “Forms” section. Feedback from business managers and SCDNR staff identified difficulty finding forms and difficulty filling out those forms as pain points for staff regarding forms.
Overarching question
Why is it difficult for staff to find and fill out internal forms?
Preparing for the study
There were three significant hurdles I encountered while preparing for this study: a significant lack of data, low UX maturity within the organization, and trouble with identifying internal subject matter experts.
Hurdle 1. Lack of data
We started out with little to no data, and none of that data was current. I tracked down what there was and used it as a jumping off point. What we had was:
- an internally conducted employee survey about the employees’ experience with the intranet from a few years prior, and
- an externally conducted employee morale survey that had been conducted a year or two later.
There were no analytics, and it was believed that the intranet could not house analytics at all due to it being a password-protected website. I later found that not to be the case.
Without reliable quantitative data to help identify users' top tasks, my approach was to prioritize problems on the intranet based on stakeholder feedback, existing survey data, and user interviews.
After determining that the intranet was able to support analytics, I prioritized creating and executing a strategy to implement traditional analytics, heatmaps, and session recordings to better prepare us for future studies.
Hurdle 2: Low UX Maturity
With no previous history of UX research when I started, the agency had low UX maturity. This meant that a basic understanding of UX and its importance had to be established with the stakeholders before we could complete the planning phase of the research. I was able to get buy-in from the stakeholders by first identifying who they were, asking and answering questions to build their understanding, then starting a collaborative discussion about strategy and creating a proposed plan and timeline.
Hurdle 3: Identifying relevant experts
In order to ask the right questions during the study, I had to first identify the agency’s internal form Subject Matter Experts (SMEs). Initially, the assumption was that the form SMEs were the form owners (HR and Finance), however that ended up largely not being the case.
This quickly became apparent after meeting and asking about their experience helping staff find and fill out forms. Having relatively little of this experience, they pointed me to the business managers.
The group I ended up with was a mish mash of titles with individuals from different pre-defined groups, but who were all on the same level when it came to experience and familiarity with forms from a form-filler support perspective.
- Questions used to identify form SMEs:
- How often or rarely do staff ask you for help with finding forms?
- Can you finish this sentence for me with what you most commonly hear?What form do I use for ___?
- Where do I find the ___ form?
- How often or rarely do staff ask you for help with filling out forms?
- Which forms do staff ask you for your help on most often?
The answers to those questions varied enough that I realized not all business managers had significant experience helping staff with forms or a high level of familiarity with those forms. I was perplexed. No pre-defined group (whether by department or title) seemed to perfectly fit my criteria. I had to throw away the idea that the type of staff I was looking for would fall into one of these pre-defined groups.
In my search to find these individuals, two approaches helped. One intentional and one a surprise.
- I conducted a search to try to find the right individuals. I reached out to the handful of administrative staff I knew of to see if they fit the criteria.
- The right individuals reached out to me. Making sure that staff were consistently updated with the project efforts by creating an internal communications strategy paid off here. While I was sending out communications to all internal staff to begin the recruitment process for this study, a handful of staff who met the SME criteria contacted me directly.
Once enough SMEs from a mix of divisions had been identified, I was able to use their answers to the above question set to assist in creating a shortlist of forms to be used in the research study.
These staff were non-business manager staff who also had a high level of experience as “helpers” and high familiarity with forms. When I asked if they would be willing to get involved with the study as subject matter experts (SMEs), they were highly motivated to help.
Business goal: Improve the efficiency with which business is conducted in the agency.
- Save the form SMEs time from having to walk users through the forms and their processes.
- Later saw during the observational interviews that a common first or early step would be to ask for help after user spent a significant amount of time trying and encountering roadblocks.
- Later saw during the observational interviews that a common first or early step would be to ask for help after user spent a significant amount of time trying and encountering roadblocks.
- Save employees time by keeping them from having to figure out:
- Who they need to ask to learn about the process
- What they need to do during the process
- What they need to know before even beginning the process
Method
Moderated user interviews & contextual inquiry
- 60 minute, remote sessions with 19 employees
- I aimed for 16 - 24 participants with at least 8 participants per segment (more familiar and less familiar) to allow for more representation among different divisions, years of experience at the agency, hierarchy, and job titles.
- I aimed for 16 - 24 participants with at least 8 participants per segment (more familiar and less familiar) to allow for more representation among different divisions, years of experience at the agency, hierarchy, and job titles.
| Participant | Division | Experience |
|---|---|---|
| P01 | Wildlife & Freshwater Fisheries | 10+ years |
| P02 | Wildlife & Freshwater Fisheries | 3-5 years |
| P03 | Wildlife & Freshwater Fisheries | 20+ years |
| P04 | Land, Water and Conservation | 20+ years |
| P05 | Wildlife & Freshwater Fisheries | 3 years |
| P06 | Marine Resources | 3-5 years |
| P07 | Wildlife & Freshwater Fisheries | <1 year |
| P08 | Marine Resources | 6-9 years |
| P09 | Land, Water and Conservation | 3-5 years |
| P10 | Marine Resources | 6-9 years |
| P11 | Marine Resources | 10+ years |
| P12 | Marine Resources | 6-9 years |
| P13 | Wildlife & Freshwater Fisheries | 3-5 years |
| P14 | Wildlife & Freshwater Fisheries | 3-5 years |
| P15 | Law Enforcement | 6-9 years |
| P16 | Marine Resources | 10+ years |
| P17 | Marine Resources | 10+ years |
| P18 | Land, Water and Conservation | 6-9 years |
| P19 | Land, Water and Conservation | 6-9 years |
WFF: 7 (Wildlife: 4, Fisheries: 3), LWC: 4, MRD: 7, LE: 1, DOA: 0, EO: 0
Supervisors: 11, Non-Supervisors: 8
| Participant | Division | Experience |
|---|---|---|
| P01 | Wildlife & Freshwater Fisheries | 10+ years |
| P02 | Wildlife & Freshwater Fisheries | 3-5 years |
| P03 | Wildlife & Freshwater Fisheries | 20+ years |
| P04 | Land, Water and Conservation | 20+ years |
| P05 | Wildlife & Freshwater Fisheries | 3 years |
| P06 | Marine Resources | 3-5 years |
| P07 | Wildlife & Freshwater Fisheries | <1 year |
| P08 | Marine Resources | 6-9 years |
| P09 | Land, Water and Conservation | 3-5 years |
| P10 | Marine Resources | 6-9 years |
| P11 | Marine Resources | 10+ years |
| P12 | Marine Resources | 6-9 years |
| P13 | Wildlife & Freshwater Fisheries | 3-5 years |
| P14 | Wildlife & Freshwater Fisheries | 3-5 years |
| P15 | Law Enforcement | 6-9 years |
| P16 | Marine Resources | 10+ years |
| P17 | Marine Resources | 10+ years |
| P18 | Land, Water and Conservation | 6-9 years |
| P19 | Land, Water and Conservation | 6-9 years |
WFF: 7 (Wildlife: 4, Fisheries: 3), LWC: 4, MRD: 7, LE: 1, DOA: 0, EO: 0
Supervisors: 11, Non-Supervisors: 8
- Test 3-5 forms identified and prioritized by the form SMEs
- Present a scenario for each form then see if and how participants find and fill out the prioritized forms
The moderated interview and observational usability testing approach allowed for identifying:
- Mental models – Word association, search terms.
- User typologies – Different types of users based on roles in the process and experience.
- Task flows – What steps users take, when, and what questions/obstacles occur along the way.
- Roadblocks or pain points – When/where/why users became frustrated and used a workaround or quit.
- Knowledge gaps – What knowledge did some users acquire after years of experience with business processes that allowed them to more quickly/accurately accomplish their tasks.
With the help of the form advisors, scenarios were crafted for each form without naming the specific form or any words in the form’s title and without indicating that the goal was for them to find/fill out a form.
After a brief interview, participants began the task-portion of the session in which they were prompted with a couple of the scenarios based on which forms the participant had familiarity with, the priority of the forms, and how much time was left in the interview.
Scenarios:
- You have decided to buy a desk for your office.
- You need to attend a conference outside of South Carolina. OR You need to stay a few nights in another SC county to do some work.
- It’s time to document your p-card transactions.
- You are moving from one DNR section to another and before you leave, you need to officially reassign all of your tagged items to the person that will be taking over your former job role.
- You have a purchase order in place for services received. The vendor has provided the services as noted in your purchase order and it is time to pay them.
Recruitment
Participants were selected based on familiarity with the forms, determined by their screener survey responses.
- Participants who had familiarity with more of the forms were prioritized over those who had familiarity with less of the forms since the study allotted time for observing participants attempting to find and fill multiple forms.
Segments
- Group 1: Employees who are more familiar with the forms
Included employees who were sorted into “Familiar with all of the prioritized forms” and “More familiar with most of the forms - Group 2: Employees who are less familiar with the forms
Included employees who were sorted into “A little bit familiar with all of the prioritized forms” - Excluded:
Employees sorted into “Not familiar with any of the forms” and “Not very familiar with the forms”
Form SMEs/advisors
Form owners (Human resources staff, Finance staff)
Business managers
- Group 1: Employees who are more familiar with the forms
Data Analysis
The data analysis process included affinity diagramming, task analysis, and identifying typologies.
Details about specific paperwork are purposefully obscured due to project confidentiality.
Affinity diagram
- Used to identify patterns within each form across different types of users and across the forms to identify overarching themes in pain points and opportunities.
- Different levels of familiarity in users were signified but ultimately the data was categorized by themes in pain points, showing that users experienced the same pain points regardless of familiarity, and that even the Form Veterans struggled through these processes.
Task analysis
- Used task analysis for a precise view of the step-by-step processes for each form and for forms in general, which would help inform decisions about the design of the prototype based on the captured users’ paths, the obstacles they encountered, questions that came up along the way, and choices the users had to make.
- Because the task analysis captures detailed information in a chronological way, it points to what information needs to be clear when, which allows for anticipating user questions.
Typologies
- Used to identify roles that exist within the business processes, which would lend itself to classifying users and setting up permissions/flows specific to them.
Noting details about each participant helped with identifying the different types of roles each user had, and how nuanced those roles could be.
Ex: One participant was a Form Filler, Form Approver, and Supervisor but was still a novice at one of the most used forms in one of the most used processes at the agency.
Findings
The original overarching research question “Why is it difficult for staff to find and fill out internal forms?” was too narrow.
This realization led to the most impactful finding: The complete end-to-end experience –including searching, finding, filling out, submitting, and post-submission steps —is problematic. How participants even got to the point that they realized they needed a form was heavily reliant on how much knowledge they already had.
Regardless of form and process familiarity and experience, participants stumbled through the end-to-end form process.
Despite many participants having years of experience filling out the forms and speaking with confidence about the forms they were filling, the same participants would still stumble while filling out the forms and exhibit confusion and frustration.
Participants across the board encountered roadblocks with PDF technology as well as cognitive challenges when filling out forms. Even participants who expressed that they had many years of practice dealing with the exact forms that were part of the study got frustrated and stumbled when trying to manipulate the PDFs or made errors while filling them out.
This was a surprising finding. We assumed that participants with more process familiarity and experience would not run into the same obstacles that those with less familiarity and experience did, but we were wrong. While more familiar and experienced participants had the upper hand when it came to institutional/process knowledge (and therefore ran into fewer roadblocks), the processes themselves were complex and unintuitive enough to cause them to face significant struggles/confusion anyway.
Final learnings
Because of the participant selection criteria, the majority of the participants had a level of familiarity and experience with the forms and internal business processes to the extent that they were able to speak about them with confidence. However, these users still struggled in the end-to-end process, meaning it would be that much more difficult and time-consuming for employees less familiar with these forms and processes.
In hindsight:
Waiting to send out a screener until after identifying and meeting with the forms SMEs would have saved time on identifying the proper segmentation and narrowing down of recruitment criteria.
The segmentation used in the initial study was based on job role (supervisor vs non-supervisor). And while it is true that supervisors probably use certain forms that non-supervisors might not, for the forms in question, it was unclear whether or not hierarchy was significant enough for segmentation for this study. The selection criteria changed becoming clearer after meeting with the forms SMEs. Instead of being focused on job titles/roles/hierarchy to inform segments, the criteria was focused on form familiarity without regard to job hierarchy/role.
Key design recommendations
- Introduce search functionality to accommodate users with different mental models.
- Provide users with more contextual information about each form.
- Descriptions
- Related forms
- Additional documentation they may need to provide
- Categorize forms in a way that is logical to users.
Current recommendations
Recommendations that are feasible with current in-house resources for a phase 1 intranet release.
Form finding
Interface design improvements:
- Introduce search functionality to accommodate users with different mental models.
- Categorize forms in a way that is logical to users.
Form filling
- Provide forms’ final internal destination to users.
- Make language consistent across all forms. Every form that asks for a certain type of information one way should be asked for in the same way across all forms.
- Identify form fields that are meant to be filled by an internal support service department (ex: HR)
Future recommendations
Recommendations to consider for future phases of the agency’s intranet.
Form process
- Eliminate the PDF system for internal forms.
- Replace the downloadable, printable forms with web-based forms connected directly to databases involved in processing business requests.
- Create official process documentation for core internal processes.
Form filling
- Re-examine information requested in each form.
- Consider whether the same information is requested on different forms that are part of one process.
- Customize what information is visible to what party. Do not request information from the form filler that only the form receiver would know, for example.
- Eliminate the need for requestors to know who to send their requests to.
- Review approval requirements and map accordingly.
- Build as much requested form information into the system as possible.
- Program user information into the system.
- Program accounting information into the system.
See Designing an interactive Forms Library prototype to see how the above learnings were applied to the UX Design strategy and prototype.
© Maria LaRocca 2024