The Department of Benefits and Family Services in the San Francisco government provides access to health care and money to pay for food, among other services. The Data Warehouse Team for the department worked with managers to develop a data dashboard. The purpose was to help frontline teams better manage their daily tasks and track their performance over time. As an internal research and design member the department, I was the lead individual contributor for the project managing and doing (1) research strategy, (2) operations, (3) data gathering, (4) analysis, (5) the creation of deliverables (aided by a visual designer) and (6) presenting the work.
Role: Project Manager & Lead Researcher
Challenge: Supervisors and managers asked their 400+ frontline teammates to use the dashboards. About 40% did not. The Data Warehouse Team wanted to why and what could be improved.
Project Scope: I worked with the team to align on the project’s (1) goals, (2) questions to be answered, (3) timeline, (4) deliverables, and (5) success metrics.
Strategy: The Data Warehouse Team (DWT) were partners throughout the research. I interviewed two of them about their experiences then the three managers overseeing the 400+ frontline teams, a supervisor and and two frontline employees (one who uses it a lot and another who rarely uses it). I asked them to show me how they use it or what they use instead, taking notes and screenshots with their permission. People felt uncomfortable being interviewed with the DWT joining our video call, so I conducted the 30-minute interviews independently. The DWT asked for my notes, which I provided, along with analysis in a report, removing interviewees’ personally identifiable information in all the documents.
Equity Considerations: My sampling strategy included asking employees to participate from different demographic backgrounds and tenure in the agency. I asked if they felt comfortable doing the interview and informed them they could stop the interview at anytime without penalty. Their personally identifiable information was not shared with the Data Warehouse Team. I then shared my research findings with all research participants, asking for any feedback or edits.
Outputs: (1) Three profiles of research participants who were non-managers, (2) a summary of the three managers’ diverging wants and uses of the dashboard, (3) visuals showing alternative presentations of data (thanks to a visual designer on the project team), and (4) a concise report with recommendations.
Impact: Based on the insights, (1) the Data Warehouse Team agreed to ongoing feedback loops to keep improving the dashboard, (2) data is automatically refreshed earlier in the morning before employees start their workday, (3) managers are creating a communication strategy and an onboarding document to guide their frontline teams in using the dashboard, (4) edits have been made to discrete pages (e.g., one refresh link is at the top of a page instead of two), and (5) the team is planning to recheck the number of people using the dashboard a month after all the changes.