NIST wants a dictionary of AI development – DNyuz

NIST wants a dictionary of AI development

Leaders at the National Institute of Standards and Technology reiterated the need for a common lexicon of terms relevant to artificial intelligence systems and machine learning models as the technologies continue to advance and be used globally.

Speaking during an Athens Roundtable event on Friday, NIST’s associate director for emerging technologies at the Information Technology Laboratory Elham Tabassi discussed both the need for and challenges of gathering a broad, diverse community response to standardizing technical language.

Tabassi and other officials have been working on developing federal policy that looks to measure and account for a broad range of societal risks posed by emerging technologies like AI. This includes developing metrics and standards for hazards such as bias and discrimination. She noted that marrying recommendations from scientific, quantitative professionals with those that have more social sciences backgrounds is challenging — but necessary — to do for democratically-developed standards for AI.

“It’s difficult, but really important to make sure that we’re all having the same understanding of terms such as bias,” Tabassi said. “If you’re applying it for [the] application of AI in healthcare versus [the] application of AI in autonomous vehicles or [the] financial sector.”

While NIST helms the development of standards and accompanying measurements, Tabassi said that reaching consensus on specific language and terms governing the usage and development of AI systems is a priority over developing relevant metrics.

A common set of technical standards will also help researchers develop better methods to evaluate various components of AI systems, such as for trustworthiness and efficacy. Tabassi said that this includes evaluating a given system not just for technical robustness, but societal robustness as well.

She added that future evaluation standards should emphasize where and how a human user is involved with an AI system in grading its trustworthiness.

“We all agree that AI systems are more than just a computer algorithm that has built them,” she said. “So we have to figure out the new metrics and methodologies that take that into consideration.”

Following President Joe Biden’s executive order on AI, NIST has been given a slew of new responsibilities in federal AI regulatory and research. Tabassi stated that the agency is responsible for developing guidelines to evaluate AI system capabilities, along with cybersecurity guidance. It also has to create test environments and operate the US AI Safety Institute.

The post NIST wants a dictionary of AI development appeared first on Defense One.

Loading...