Bridging the AI Governance Gap: From Policy to Practice

Author: Denis Avetisyan

New research explores how internal collaboration can move AI governance requirements from abstract regulation to concrete implementation within software development teams.

A collaborative workshop serves as the central component of a translation pipeline designed to transform the requirements of the EU AI Act into actionable strategies, prioritizing implementation based on both practical impact and developmental effort within existing workflows.

This paper details a participatory action research approach to address the ‘last mile’ challenge of integrating AI governance principles – including those outlined in the EU AI Act – into everyday workflows.

Despite increasing attention to AI governance frameworks, translating regulatory requirements into practical software development remains a significant challenge. This paper, ‘Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration’, investigates how collaborative workshops within an AI startup can bridge this gap, revealing that successful implementation hinges on aligning compliance with existing development priorities and fostering a sense of shared ownership. Our analysis of practitioner perceptions-ranging from recognizing synergies to viewing requirements as administrative burdens-demonstrates that genuine engagement is key to avoiding superficial compliance. Ultimately, how can organizations move beyond performative governance and cultivate a culture where ethical AI development is intrinsically valued and collaboratively achieved?

The EU AI Act: A Pragmatic Look at Compliance

The European Union’s AI Act signifies a landmark effort to govern artificial intelligence through a risk-based framework, compelling developers and deployers to prioritize transparency and accountability. This legislation moves beyond self-regulation, establishing legally binding requirements for AI systems categorized by their potential risk to fundamental rights and safety. High-risk AI applications – those used in critical infrastructure, education, employment, and law enforcement – face the most stringent regulations, including mandatory risk assessments, data governance protocols, and ongoing monitoring. Crucially, the Act doesn’t prohibit AI, but rather aims to foster trustworthy AI by demanding clear documentation, explainability, and human oversight, ultimately shaping a future where innovation aligns with ethical considerations and legal compliance.

The incoming EU AI Act places significant burdens on developers, demanding far more than simply functional code. Achieving compliance requires meticulous documentation detailing every stage of an AI system’s lifecycle – from data sourcing and model training to validation and deployment. Beyond documentation, robust risk management practices are essential; developers must proactively identify potential harms, implement mitigation strategies, and continuously monitor systems for unintended consequences. This isn’t merely a matter of ticking boxes; it necessitates a fundamental shift in development workflows, demanding resources and expertise many organizations currently lack and creating immediate, practical challenges as they strive to navigate this new regulatory landscape.

To maintain legal standing within European markets, organizations deploying artificial intelligence systems must move beyond reactive compliance and embrace a proactive approach to the EU AI Act’s requirements. This necessitates establishing internal protocols for meticulous documentation – detailing datasets, algorithms, and intended uses – alongside comprehensive risk management frameworks that identify and mitigate potential harms. Failing to anticipate and address these stipulations isn’t simply a matter of fines; it could result in the prohibition of AI applications, hindering innovation and market access. Consequently, businesses are compelled to integrate compliance into the entire AI lifecycle, from initial design and development to ongoing monitoring and updates, ensuring sustained adherence to the evolving regulatory landscape and fostering trust in their AI-driven solutions.

Translating Policy into Practice: A Collaborative Approach

A formalized ‘Legal-Text-to-Action Pipeline’ provides a structured methodology for translating the broad requirements of the EU AI Act into actionable technical and operational tasks. This pipeline necessitates a phased approach, beginning with detailed analysis of each article and recital to identify specific obligations. Subsequent steps involve decomposition of these obligations into discrete, measurable requirements, followed by assignment to responsible teams and the establishment of key performance indicators for monitoring compliance. Crucially, the pipeline must incorporate feedback loops to address ambiguities in the legal text and to adapt to evolving interpretations from regulatory bodies, ensuring a dynamic and responsive implementation strategy.

The efficacy of a ‘Legal-Text-to-Action Pipeline’ is significantly enhanced when integrated with Collaborative Workshops involving technical teams. Research indicates that direct engagement of these teams in interpreting and operationalizing regulatory requirements-such as those outlined in the EU AI Act-fosters a shared understanding of compliance obligations. Analysis of workshop participation patterns reveals a correlation between active engagement and successful translation of legal text into actionable technical implementations. This collaborative approach cultivates a sense of ownership over the compliance process, improving both the quality and speed of implementation, and ensures that technical solutions accurately reflect the intended legal requirements.

The implementation of the EU AI Act necessitates a strategic prioritization of compliance efforts, best achieved through an Impact-Effort Matrix. This tool categorizes requirements based on their potential impact on organizational risk and the estimated effort required for implementation. High-impact, low-effort items are addressed first, followed by high-impact, high-effort items requiring detailed planning. Conversely, low-impact items, regardless of effort, receive lower priority. This approach directly supports the collaborative workshop process by providing a framework for technical teams to focus discussions and resource allocation on the most critical areas, thereby maximizing efficiency and ensuring compliance efforts are aligned with organizational priorities and risk profiles.

Workshop participants categorized fourteen strategies based on their perceived impact and effort, revealing that transparency and human oversight were seen as 'quick wins', performance evaluation as 'big projects', and technical documentation, data governance, and risk management as potentially low-impact, high-effort endeavors (detailed in Table 1). — Workshop participants categorized fourteen strategies based on their perceived impact and effort, revealing that transparency and human oversight were seen as ‘quick wins’, performance evaluation as ‘big projects’, and technical documentation, data governance, and risk management as potentially low-impact, high-effort endeavors (detailed in Table 1).

Technical Foundations for Regulatory Adherence

Comprehensive technical documentation is a core requirement for demonstrating compliance with the EU AI Act, particularly regarding transparency and accountability. This documentation should detail the system’s design, development process, capabilities, and limitations. Utilizing a standardized model like the C4 Model – which focuses on context, containers, components, and code – facilitates a structured approach to this documentation, ensuring all relevant aspects of the AI system are clearly articulated and auditable. Estimates indicate the development of this foundational technical documentation requires approximately 8 person-hours, representing a relatively low-effort investment for achieving a critical component of regulatory adherence.

Robust data governance practices are foundational for AI system compliance and responsible AI development. These practices encompass policies and procedures designed to ensure data quality – accuracy, completeness, consistency, and validity – throughout the data lifecycle. Data security is maintained through measures aligned with standards like ISO 27001, an information security management system, which specifies requirements for establishing, implementing, maintaining, and continually improving an information security management system. Responsible data handling involves adherence to privacy regulations, minimization of data collection, and transparent data usage practices, all crucial for demonstrating compliance with frameworks like the EU AI Act and building user trust.

Langfuse delivers observability for Large Language Model (LLM) applications by providing tools for monitoring and auditing system behavior. This capability is essential for demonstrating compliance with regulatory requirements, specifically concerning the traceability and explainability of AI systems. Implementation of Langfuse, as demonstrated through our workshop outcomes, requires approximately 32 person-hours, encompassing setup, integration with existing LLM pipelines, and configuration of key monitoring metrics such as input/output data, latency, and error rates. The platform facilitates detailed logging and analysis of LLM interactions, enabling verification of system performance, identification of potential biases, and reconstruction of decision-making processes for audit purposes.

Sustaining Responsible AI: Governance and Oversight in Practice

The proactive implementation of AI governance frameworks is becoming essential for organizations navigating the complexities of artificial intelligence. These frameworks, increasingly aligned with internationally recognized standards such as ISO 42001, move beyond simple compliance to embed responsible practices throughout the entire AI lifecycle. A robust governance structure defines clear principles for development and deployment, establishes well-defined processes for risk management and accountability, and facilitates ongoing monitoring and evaluation. This systematic approach not only mitigates potential harms – including bias, privacy violations, and unintended consequences – but also fosters innovation by building trust with stakeholders and ensuring AI systems consistently align with organizational values and ethical considerations. By prioritizing governance from the outset, developers can demonstrably showcase a commitment to responsible AI, paving the way for wider adoption and societal benefit.

To foster genuinely responsible AI, a dynamic approach to governance is essential, and ‘Insider Action Research’ offers a powerful methodology for achieving this. This involves integrating researchers directly within AI development teams, allowing for real-time observation and analysis of the practical challenges encountered when implementing ethical guidelines and governance frameworks. Rather than relying on external audits or retrospective assessments, this embedded approach enables researchers to identify governance gaps as they emerge, facilitating immediate intervention and iterative improvement. By working alongside developers, researchers gain a nuanced understanding of the contextual factors influencing decision-making, and can collaboratively design solutions tailored to specific project needs. The continuous feedback loop inherent in this process moves beyond simple compliance, cultivating a culture of proactive ethical consideration and ensuring that governance mechanisms remain relevant and effective throughout the AI lifecycle.

Effective implementation of artificial intelligence demands a commitment to human oversight and transparent interaction disclosure, fostering both trust and accountability in these increasingly complex systems. This isn’t merely an ethical consideration, but a practical one; by ensuring human involvement in critical decision-making processes and clearly communicating when an interaction is with an AI, developers can align technological outputs with fundamental values. Recent assessments indicate that verifying the adequacy of existing AI interaction disclosure practices – confirming systems clearly identify themselves and their limitations – requires a modest investment of approximately two person-hours, a relatively small expenditure for a significant gain in public confidence and responsible innovation. This proactive approach moves beyond simply having governance to demonstrably showing responsible practices, strengthening the relationship between technology and society.

The pursuit of AI governance, as this paper details, often feels less like crafting policy and more like a frantic last-mile sprint. It’s a pragmatic reality that elegant frameworks, however well-intentioned, will inevitably collide with the messy, unpredictable nature of production systems. As Edsger W. Dijkstra observed, “Simplicity is prerequisite for reliability.” This resonates deeply; the collaborative workshops detailed in the study aren’t about imposing abstract principles, but about simplifying compliance by aligning it with developers’ existing priorities and fostering team ownership. The research underscores a vital point: architecture isn’t a diagram, it’s a compromise that survived deployment. Everything optimized will one day be optimized back, and a flexible, participatory approach is the only path to sustainable implementation, especially given the evolving landscape of regulations like the EU AI Act.

What’s Next?

The enthusiasm for ‘engaged AI governance’ feels…predictable. Any framework built on workshops and ‘team ownership’ will inevitably discover that production systems operate under constraints the workshops never anticipated. The paper correctly identifies the ‘last mile’ as problematic, but the true distance remains uncharted. It isn’t merely about translating regulation into code; it’s about translating intentions into reality when faced with urgent bugs, shifting deadlines, and the fundamental human desire to bypass anything resembling ‘friction.’

Future work will undoubtedly focus on scaling these collaborative approaches. The unspoken question, of course, is whether anything called ‘scalable’ has truly withstood sustained, real-world load. The EU AI Act casts a long shadow, and compliance tooling will proliferate. However, the most interesting developments won’t be in the tooling itself, but in the inevitable workarounds, the bespoke exceptions, and the quiet acknowledgement that ‘governance’ is often a post-hoc rationalization for choices already made.

It would be…refreshing to see research that explicitly embraces the messiness of implementation. Better one thoroughly understood, painstakingly maintained monolith of compliance than a hundred loosely coupled microservices each claiming to be ‘AI ethical’ but demonstrably failing to interoperate. The field seems determined to invent new complexity; perhaps the next step is to learn to tolerate, even appreciate, the elegance of simplicity.

Original article: https://arxiv.org/pdf/2604.21554.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

The EU AI Act: A Pragmatic Look at Compliance

Translating Policy into Practice: A Collaborative Approach

Technical Foundations for Regulatory Adherence

Sustaining Responsible AI: Governance and Oversight in Practice

What’s Next?

See also: