Business and government organizations are rapidly embracing an expanding variety of artificial intelligence (AI) applications: automating activities to function more efficiently, reshaping shopping recommendations, credit approval, image processing, predictive policing, and much more.Like any digital technology, AI can suffer from a range of traditional security weaknesses and other emerging concerns such as privacy, bias, inequality, and safety issues. The National Institute of Standards and Technology (NIST) is developing a voluntary framework to better manage risks associated with AI called the Artificial Intelligence Risk Management Framework (AI RMF). The framework\u2019s goal is to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.The initial draft of the framework builds on a concept paper released by NIST in December 2021. NIST hopes the AI RMF will describe how the risks from AI-based systems differ from other domains and encourage and equip many different stakeholders in AI to address those risks purposefully. NIST said it can be used to map compliance considerations beyond those addressed in the framework, including existing regulations, laws, or other mandatory guidance.Although AI is subject to the same risks covered by other NIST frameworks, some risk \u201cgaps\u201d or concerns are unique to AI. Those gaps are what the AI RMF aims to address.AI stakeholder groups and technical characteristicsNIST has identified four stakeholder groups as intended audiences of the framework: AI system stakeholders, operators, and evaluators, external stakeholders, and the general public. NIST uses a three-class taxonomy of characteristics that should be considered in comprehensive approaches for identifying and managing risk related to AI systems: technical characteristics, socio-technical characteristics, and guiding principles.Technical characteristics refer to factors under the direct control of AI system designers and developers, which may be measured using standard evaluation criteria, such as accuracy, reliability, and resilience. Socio-technical characteristics refer to how AI systems are used and perceived in individual, group, and societal contexts, such as \u201cexplainability,\u201d privacy, safety, and managing bias. In the AI RMF taxonomy, guiding principles refer to broader societal norms and values that indicate social priorities such as fairness, accountability, and transparency.Like other NIST Frameworks, the AI RMF core contains three elements that organize AI risk management activities: functions, categories, and subcategories. The functions are organized to map, measure, manage, and govern AI risks. Although the AI RMF anticipates providing context for specific use cases via profiles, that task, along with a planned practice guide, has been deferred until later drafts.Following the release of the draft framework in mid-March, NIST held a three-day workshop to discuss all aspects of the AI RMF, including a deeper dive into mitigating harmful bias in AI technologies.Mapping AI risk: Context mattersWhen it comes to mapping AI risk, \u201cWe still have to figure out the context, the use case, and the deployment scenario," Rayid Ghani of Carnegie Mellon University said at the workshop. \u201cI think in the ideal world, all of those things should have happened when you were building the system.\u201dMarilyn Zigmund Luke, vice president of America\u2019s Health Insurance Plans, told attendees that, \u201cGiven the variety of the different contexts and constructs, the risk will be different, of course, to the individual and the organization. I think understanding all of that in terms of evaluating the risk, you've got to start at the beginning and then build out some different parameters.\u201dMeasuring AI activities: New techniques neededMeasurement of AI-related activities is still in its infancy because of the complexity of the socio-political ethics and mores inherent in AI systems. David Danks of the University of California, San Diego, said, \u201cThere's a lot in the measure function that right now is essentially being delegated to the human to know. What does it mean for something to be biased in this particular context? What are the relevant values? Because of course, risk is fundamentally about threats to the values of the humans or the organizations, and values are difficult to specify formally.\u201dJack Clark, co-founder of AI safety and research company Anthropic, said that the advent of AI has created a need for new metrics and measures, ideally baked into the creation of the AI technology itself. \u201cOne of the challenging things about some of the modern AI stuff, [we] need to design new measurement techniques in co-development with the technology itself,\u201d Clark said.Managing AI risk: Training data needs an upgradeThe management function of the AI RMF addresses the risks that have been mapped and measured to maximize benefits and minimize adverse impacts. But data quality issues can hinder the management of AI risks, Jiahao Chen, chief technology officer of Parity AI, said. \u201cThe availability of data being put in front of us for training models doesn't necessarily generalize to the real world because it could be several years out of date. You have to worry about whether or not the training data actually reflects the state of the world as it is today.\u201dGrace Yee, director of ethical innovation at Adobe, said, \u201cIt's no longer sufficient for us to deliver the world's best technologies for creating digital experiences. We want to ensure that our technology is designed for inclusiveness and respects our customers, communities, and Adobe values. Specifically, we're developing new systems and processes to evaluate if our AI is creating harmful bias.\u201dVincent Southerland of the New York University School of Law raised the use of predictive policing tools as an object lesson of what can go wrong in managing AI. \u201cThey are deployed all across the criminal system,\u201d he said, from identifying the perpetrator of the crime to when offenders should be released from custody. But until recently, \u201cThere wasn't this fundamental recognition that the data that these tools rely upon and how these tools operate actually help to exacerbate racial inequality actually and help to exacerbate the harms in the criminal system itself.\u201dAI governance: Few organizations do itWhen it comes to AI governance policies, few organizations are doing it. Patrick Hall, scientist at bnh.ai, said that outside large consumer finance organizations and just a few other highly regulated spaces, AI is being used without formal governance guidance, so companies are left to sort out these stormy governance issues on their own.\u201dNatasha Crampton, chief responsible AI officer at Microsoft, said, \u201cFailure mode arises when your approach to governance is overly decentralized. This is a situation where teams want to deploy AI models into production, and they're just adopting their own processes and structures, and there's little coordination.\u201dAgus Sudjianto, executive vice president and head of corporate model risk at Wells Fargo, also stressed top-level management in governing AI risk. \u201cIt will not work if the head of responsible AI or the head of management doesn't have the stature, ear, and support from the top of the house.\u201dTeresa Tung, cloud first chief technologist at Accenture, emphasized that all businesses need to focus on AI. \u201cAbout half of the Global 2000 companies reported about AI in their earnings call. This is something that every business needs to be aware of.\u201dAs with other risk management frameworks developed by NIST, such as the\u00a0Cybersecurity Framework, the final AI RMF could have wide-ranging implications for the private and public sectors. NIST is seeking comments on the current draft of the AI RMF by April 29, 2022.