Data ethnography is good data science

[Updated 16 August 2022] An incredible amount of effort can be expended by individuals and organizations to store, gather, analyze, share and present data. The proverbial “last mile/kilometer” of this work is understanding how people experience data: their behavioral response to data, particularly performance metrics and its associated results.

Data ethnography is a humanistic and holistic approach to understanding how people experience numbers, evidence and insights.

Many of us champion “data-driven decisions” and yet don’t develop any processes, feedback loops or systematic ways of understanding how people use data and success metrics, accordingly to whose values, for or against whom, and to what ends exactly?

Here are use cases of not having structures and processes in place to encourage employees, clients, regulators and individual members of society to ask, answer and act on these sorts of questions?

“Now digital productivity monitoring is also spreading […] Many employees, whether working remotely or in person, are subject to trackers, scores, ‘idle’ buttons, or just quiet, constantly accumulating records. Pauses can lead to penalties, from lost pay to lost jobs […] But the most urgent complaint, spanning industries and incomes, is that the working world’s new clocks are just wrong: inept at capturing offline activity, unreliable at assessing hard-to-quantify tasks and prone to undermining the work itself […] Crossover defends its practices on its website, saying that […its product WorkSmart is the…] ‘Fitbit of productivity’ […] Two years after helping to build WorkSmart, Mr. Mazzoli started using it. He became awash in anxiety and doubtful about its accuracy. ‘Some days you were just moving the cursor around just for the sake of it,’ he said. The tool was powerful but dangerous, he concluded.” ~New York Times, 15 August 2022

“The two incentive structures under which most AI research is currently organized is the two funding entities. One is the Department of Defense […] And then the second bucket is corporations […] The question is, can we have a research institute that has a completely different incentive structure from the very beginning, and can we organize ourselves like that? […] So our goal is, though, to think about AI that benefits people in marginalized groups. How do we conduct that research? How do we include their voices? How do we make sure that we don’t exploit them and pay them? And my hope is, once we do that, we can arrive at actual research products that are beneficial to the community because their voice is part of it.” ~Timnit Gebru, Marketplace, 16 December 2021

“What were the metrics?” ~Jon Stewart asking the U.S. Secretary of Veterans Affairs, Denis McDonough, about why some illnesses of veterans are considered to be caused by burn pits but not others, 1 October 2021

“Investigation Shows Post Office Managers Changed Employee Time Cards To Pay Them Less […] they’re under a lot of pressure to keep overtime costs down because, you know, the Postal Service is having a lot of financial problems […] they said their bosses’ pay raises – their annual pay raises are tied to whether or not they keep overtime costs down. So that creates an incentive for them to – I guess to pay workers as little as possible.” ~National Public Radio, 2 September 2021

“[…] an idealistic F.B.I. agent, grew so disillusioned by the war on terror that he was willing to leak classified documents — and go to prison for doing it […] Tremendous pressure was put on agents to bolster their squad’s numbers on open or active investigations and informants, which boosted the office’s statistics, resulting in more funding for agents, analysts, surveillance teams and other aspects of the J.T.T.F., which in turn would open more investigations.” ~New York Times, 1 September 2021

The federal government is preparing to crack down on hospitals for not reporting COVID-19 data on a daily basis. That crackdown comes in the form of cutting off Medicare funding to hospitals that don’t comply […one is] concerned that the threat of losing funding might cause hospitals to start making up data to report.” ~National Public Radio, 24 September 2020

“Public reporting of District Health Board’s performance of procedures including elective surgeries, cancer treatment times and Emergency Department wait times, has been axed […] Health Minister David Clark said the targets created “perverse incentives”, particularly in relation to surgery – but the Opposition said there was no evidence to suggest that’s true.” ~Stuff, 2018

“[…] a board member with a strong grasp of accounting felt that he couldn’t speak up about over-valued assets, as the compensation of the whole board depended upon profitability, and he did not want to have everybody against him. The results were catastrophic for the banks, which had to be bailed out from economic collapse in 2008. The very rewards system they had installed had incentivized workers to act contrary to the companies’ best interests.” ~Natàlia Cugueró-Escofet and Josep M. Rosanas, Journal of Business Ethics, 2017