According to DataRobot ‘s “The State of Bias in 2019,” which was released last week, 42% of organizations surveyed reported being “very to extremely” concerned about bias occurring in their organizations.
Copyright by www.datanami.com
This number jibes with another finding from the DataRobot survey: that 38% of the organizations surveyed reported they use “black box” systems that offer no insight into how it makes decisions.
The juxtaposition of bias concerns and black box systems is enough to warrant serious questions about the direction compaines should take with their , according to John Giannandrea, Apple’s senior vice president of and strategy.
“If someone is trying to sell you a black box system… and you don’t know how it works or what data was used to train it, then I wouldn’t trust it,” DataRobot quotes Giannandrea as saying in its report.
The survey indicates that organizations are aware of the potential pitfalls and are actively working to mitigate it. DataRobot found that 64% of survey respondents say they’re “very to extremely” confident in their ability to identify bias.
The survey found some interesting differences among US and UK companies. While UK companies were more concerned about “technical bias” in the algorithms than their US counterparts (47% compared to 34%, respectively), the survey suggests that US companies were more concerned about “emergent bias” than UK firms (35% versus 22%, respectively).
The survey indicates that organizations are aware of the potential pitfalls and are actively working to mitigate it. DataRobot found that 64% of survey respondents say they’re “very to extremely” confident in their ability to identify bias.
The survey found some interesting differences among US and UK companies. While UK companies were more concerned about “technical bias” in the algorithms than their US counterparts (47% compared to 34%, respectively), the survey suggests that US companies were more concerned about “emergent bias” than UK firms (35% versus 22%, respectively).
The survey defines technical bias as bias “arising from technical limitations, (e.g., algorithms that perpetuate bias.” Emergent bias, meanwhile, is bias “arising in the context of using the system when there’s misalignment between the user and system design, (e.g., only seeing content that confirms an already-held belief and not seeing content that conflicts).” The survey also looks at human bias.
DataRobot had a theory for the discrepancy between the bias concerns of UK and US firms. “It’s likely that because UK companies operate under tighter regulatory climates, they are more cautious about the technical aspects of ,” the company says.
Compromised brand reputation and loss of customer trust was the number one concern as it relates to bias, the survey found. But the survey found other concerns, including mismatch with personal ethics, loss of employee trust, and legal penalties.
The report also found that the majority of systems in use (69%) are developed by third-parties. About 70% of organizations are using to automate fewer than 20 business functions, while 19% are using it to automate 49 or fewer, and 10% are automating more than 50 business functions. […]
0 Comments