In June 2025, the UN Working Group on Business and Human Rights released a landmark report titled “Artificial Intelligence Procurement and Deployment: Ensuring Alignment with the Guiding Principles on Business and Human Rights” (A/HRC/59/53). The report warns that the expanding use of AI systems by both governments and private actors often proceeds without adequate attention to fundamental human rights. Key concerns include threats to privacy, risks of discrimination, and the absence of meaningful accountability. Urgent reforms are needed to address these gaps. This post explores the report’s central findings, highlights the technical risks that contribute to human rights violations, examines governance shifts and doctrinal trends, and considers the responsibilities of states, businesses, and other stakeholders in ensuring that AI deployment is human rights aligned.

Human Rights Issues with AI Procurement

One of the most striking observations in the report is the lack of human rights due diligence (HRDD). Many procurement processes for AI systems do not include systematic impact assessments to evaluate potential risks for affected communities. This omission allows harmful tools to be adopted without sufficient safeguards. The second concern is bias and discrimination. AI systems often amplify existing social inequalities, particularly when they rely on unrepresentative datasets or opaque design structures. This can result in unfair outcomes in hiring, credit scoring, policing, or welfare distribution. The report also identifies transparency and oversight gaps. Individuals impacted by AI decisions often have no clarity on how those decisions were reached, no effective appeal mechanisms, and no guarantee of vendor accountability.

Finally, the report highlights that those vulnerable and high-risk contexts, such as healthcare, policing, justice, or welfare—tend to have the weakest safeguards, even though the consequences of errors in these areas are most severe.

Technical Concepts & Risk Areas

The Working Group underscores that understanding the technical foundations of AI is crucial to identifying where risks emerge. One central issue is bias in datasets and model training. If historical data reflects discriminatory practices, AI trained on such data can reproduce or even intensify those biases. Another concern is explainability and interpretability. Many AI models operate as “black boxes,” making it difficult for individuals or institutions to understand how decisions are made. This opacity undermines accountability and obstructs meaningful redress. The report also emphasizes risk classification and the regulation of high-risk systems. Tools used in predictive policing, medical diagnostics, or social welfare allocation are inherently high risk and should be subject to enhanced safeguards, including independent audits, continuous oversight, and human review. Finally, contractual levers and regulatory requirements play a critical role. Procurement contracts should impose technical obligations such as transparency standards, data protection measures, and rights of audit to manage risks downstream in the AI lifecycle.

Normative Shifts

The report signals important governance trends. First, there is a move from reactive to proactive governance. Human rights considerations should be embedded at the procurement stage, rather than applied retrospectively after harms occur. Second, the emphasis is on embedding rights in contracts and legal frameworks. Beyond voluntary guidelines, governments must adopt binding standards and insert enforceable obligations into procurement contracts to hold vendors accountable. Third, the report calls for stakeholder consultation and inclusivity. Affected communities, particularly marginalized groups, should have a voice in procurement processes to ensure the systems deployed are fair and locally relevant. Lastly, there is an emphasis on risk regulation and classification regimes. Recognizing that AI systems vary in their potential harm, the report stresses that higher-risk systems must be subject to stricter compliance and monitoring.

Implications for Stakeholders

The findings carry important implications for multiple actors. States and governments must integrate human rights due diligence into procurement laws, mandate human rights impact assessments, and establish oversight bodies to monitor compliance. Vendors and private sector actors are expected to design AI tools that incorporate fairness, explainability, and accountability from the outset. They must also accept contractual obligations that bind them to uphold human rights standards throughout the lifecycle of their systems. Civil society and affected communities play a vital role in demanding transparency, challenging biased outcomes, and ensuring that procurement processes reflect diverse perspectives. Their engagement ensures that AI deployment respects dignity and equity. Technical standard bodies and regulators must develop clear frameworks for risk classification, audit standards, and explainability requirements. They also need to work towards interoperable governance regimes across jurisdictions to address the global nature of AI technologies.

Bottomline

The UN report makes clear that the procurement and deployment of AI are not neutral, technical processes but moments of profound ethical, legal, and political significance. When human rights considerations are sidelined, AI amplifies harm and entrenches inequality. Conversely, when procurement and deployment are aligned with human rights principles, AI can become a tool of justice, accountability, and inclusion. Ensuring this alignment requires collaboration across governments, businesses, civil society, and regulators. Procurement must be recognized not simply as a purchasing exercise but as an ethical commitment. As the report emphasizes, embedding human rights into AI governance is not a barrier to innovation but the foundation of a fair and resilient digital future.