NTIA seeks comment on availability of model weights in highly capable AI systems

The AI EO, Executive Order 14110, directs NTIA to provide a report to the President considering the “potential benefits, risks, and implications of dual-use foundation models for which the model weights are widely available, as well as policy and regulatory recommendations pertaining to those models.” The RFC catalogs use cases and opportunities for dual-use foundation models, including democratizing access to technology and scientific innovation, providing transparency about AI systems, and promoting competition. It also acknowledges potential harms, including the danger of expanding malicious actors’ access to and understanding of these technologies. Against this backdrop, NTIA asks commenters to describe the current landscape, illuminate technical and ethical tensions related to the technology, and suggest potential policy, regulatory, or legal interventions for effective governance.

Defining "Open" and "Widely Available"

The AI EO defines “model weights” as “numerical parameter[s] within an AI model that help[] determine the model’s output in response to inputs.” The RFC notes that “‘openness’ or ‘wide availability’ of model weights are [] terms without clear definition” and that “[t]here are gradients of ‘openness,’ ranging from fully ‘closed’ to fully ‘open’” and seeks input on how it should define “open” and “widely available” with respect to model weights and highly capable AI models. It asks for comparisons between capability improvements in closed and open models over time.  The RFC also seeks comment on potential licensing and distribution mechanisms that could be used to limit or expand the availability of “open” AI models.

Examining the Risks and Benefits of Widely Available Model Weights

The RFC asks stakeholders to weigh in on the potential risks and benefits of making model weights available for highly capable AI systems. It also asks commenters to contextualize the risks of “open” models as compared with closed models and with other software and information systems.

Technical Tools for Minimizing the Risks and Maximizing the Benefits of Widely Available Model Weights

The RFC seeks input on the safeguards that can be put in place to maximize the benefits of widely available model weights while minimizing any risks. The RFC asks what sorts of model evaluations could be used to identify risks and benefits. It also asks about security measures, the prospect of developing future safeguards, and the ability to regain control, restrict access, or limit the use of an AI model.

Legal and Business Considerations

The RFC asks whether and what parallels there are between open model weights and open-source software, open data, and other “open” initiatives. It asks how widely available model weights might impact competition dynamics in the broader economy and within specific sectors, like healthcare and education. The RFC seeks input on whether intellectual property issues, such as licensing, could influence competition, benefits, and risks, and whether standardization would be necessary to protect interoperability between different foundation model licenses.

Mechanisms for Managing Risks

The RFC also seeks input on any potential regulatory models, either voluntary or mandatory, that could maintain or increase the benefits and/or mitigate the risks of widely available highly capable AI models with open model weights. NTIA also asks about the different kinds of regulatory structures that could address both the large scale of these foundation models but also the ever-declining amount of compute resources needed to fine-tune and retrain them, as well as whether standards or practice should be different for government as opposed to private industry.

Next Steps

The RFC is an opportunity for stakeholders to provide input on a key component of U.S. AI policymaking efforts. NTIA is accepting comments until March 27, 2024.

Authored by Katy Milner, Mark Brennan, Ryan Thompson, and Ambia Harper.