Trusting the Machine: Building Verifiable AI for Emotions

Author: Denis Avetisyan

A new framework aims to make affective AI systems demonstrably transparent and trustworthy by securing explanations on a blockchain.

This paper introduces ‘Immutable Explainability,’ an architecture leveraging fuzzy logic and blockchain to ensure verifiable and auditable explanations for affective computing systems.

Despite advances in affective computing, a critical gap remains between increasingly sophisticated AI and the need for transparent, trustworthy decision-making-particularly in sensitive applications. This paper introduces ‘Immutable Explainability,’ an architecture designed to address this challenge by combining interpretable fuzzy logic with the tamper-evident security of blockchain technology. Our approach generates auditable explanations for affective AI inferences, anchoring them on a public blockchain to ensure verifiable and immutable records. By establishing a foundation for self-sovereign identity and data control, can this framework pave the way for a new era of responsible and user-centric affective AI?

Fragile Explanations and the Pursuit of Certifiable AI

Many contemporary Explainable AI (XAI) techniques, despite offering insights into model decision-making, prove surprisingly fragile when subjected to even minor alterations in input data. Researchers have demonstrated that adversarial perturbations – subtle, often imperceptible changes – can dramatically shift these explanations, leading to inconsistent or misleading interpretations. This susceptibility to manipulation doesn’t necessarily indicate a flaw in the AI’s core logic, but rather highlights the inherent instability of post-hoc explanation methods. Because these techniques analyze a trained model to generate explanations, they often lack a direct connection to the original training process and can be easily “fooled” by inputs designed to exploit this disconnect. Consequently, reliance on such explanations can erode trust, as stakeholders may reasonably question the validity and reliability of insights generated by a system prone to yielding inconsistent or easily manipulated justifications for its actions.

As artificial intelligence evolves beyond readily understandable models, the reliance on explanations after a decision – known as post-hoc interpretability – is proving increasingly inadequate. These retroactive justifications are susceptible to adversarial manipulation and fail to guarantee consistent reasoning. A fundamental shift is therefore necessary towards designing AI systems with inherent transparency, generating explanations that are verifiable and immutable by design. This requires building explanations directly into the AI’s architecture, ensuring they are a guaranteed output of the system’s core logic rather than an approximation. Such a proactive approach, focusing on certifiable AI, is crucial for establishing genuine trust and accountability as AI increasingly impacts critical applications, moving beyond simply understanding what an AI decided to why it made that decision with absolute certainty.

Multimodal Emotion Recognition: A Foundation for Robust Analysis

Multimodal Emotion Recognition (MER) integrates data from multiple sources – specifically audio and text in this study – to improve the accuracy and robustness of emotion detection. Utilizing both acoustic features derived from speech and semantic content extracted from transcribed text provides complementary information not available from a single modality. This approach addresses the limitations inherent in relying solely on vocal prosody or textual keywords, as emotional expression is often nuanced and conveyed through a combination of verbal and non-verbal cues. The fusion of these modalities aims to create a more comprehensive and reliable representation of the speaker’s emotional state, leading to improved performance in emotion recognition tasks.

The system incorporates automatic speech recognition (ASR) performed by the Whisper model to convert audio input into text. Crucially, Whisper ASR also outputs a confidence score for each transcribed word or segment. These ASR confidence scores are then utilized as weights during multimodal fusion. Lower confidence scores indicate potentially inaccurate transcriptions, resulting in reduced weighting of the textual modality relative to the audio features. This dynamic weighting scheme allows the system to prioritize more reliable input streams, improving the overall accuracy and robustness of emotion recognition, particularly in noisy environments or with unclear speech.

Acoustic feature extraction utilizes signal processing techniques to derive quantifiable characteristics from the audio input, including prosody, spectral features, and voice quality metrics. These features represent the paralinguistic aspects of speech relevant to emotional state. Simultaneously, text-based emotion analysis employs Natural Language Processing (NLP) methods – including lexical analysis and sentiment scoring – on the transcribed text generated by the Automatic Speech Recognition (ASR) system. This analysis identifies emotionally charged keywords and phrases, as well as the overall semantic orientation of the utterance. The outputs of both acoustic feature extraction and text-based emotion analysis serve as the primary inputs for the subsequent multimodal fusion stage, where the information is integrated to determine the overall emotional state.

Fuzzy Logic: A Pathway to Transparent Inference

Fuzzy Fusion integrates data derived from both audio and text analysis to generate a composite emotional state representation. This process involves mapping the outputs of each modality – acoustic features such as pitch and intensity from audio, and sentiment scores and keyword analysis from text – onto fuzzy sets defining emotional categories like joy, sadness, anger, and fear. By combining these fuzzy sets using logical operators – typically AND, OR, and NOT – the system creates a nuanced profile that accounts for the contributions of both modalities. This allows for a more flexible and accurate assessment of emotional state compared to relying on a single data source, as it can handle ambiguity and partial information inherent in natural communication. The resulting fused representation is not a simple average, but a weighted combination determined by the specific fuzzy inference rules implemented within the system.

The Fuzzy Logic Inference Engine utilizes a Mamdani-type Fuzzy Inference System (FIS) to process inputs and generate outputs representing emotional states. This system operates through four key stages: fuzzification, rule evaluation, aggregation, and defuzzification. Fuzzification converts crisp inputs – such as values derived from audio and text analysis – into fuzzy sets defined by membership functions. Rule evaluation then assesses the degree to which each rule’s antecedent is satisfied, typically using minimum or product t-norms. Aggregation combines the outputs of all activated rules into a single fuzzy set. Finally, defuzzification translates this aggregated fuzzy set into a crisp output value, often employing the centroid method to determine the center of gravity of the resulting membership function. This step-by-step process allows for a structured and interpretable reasoning approach within the emotional analysis framework.

A heuristic Natural Language Processing (NLP) pipeline was implemented to provide an initial assessment of the system architecture’s viability before deploying the computationally intensive fuzzy logic engine. This pipeline utilizes rule-based sentiment analysis and keyword spotting to generate emotional state estimations from textual input. The resulting data serves as a performance baseline against which the fuzzy logic-based system is compared, allowing for quantitative evaluation of the benefits of the more complex model. Specifically, metrics such as precision, recall, and F1-score are calculated for both systems across a standardized dataset, enabling a direct comparison of their respective accuracies and response times.

Decentralized Auditing: Immutable Records for Trustworthy AI

The architecture incorporates blockchain technology as a foundational element for a decentralized auditing subsystem, fundamentally altering how AI inferences are tracked and validated. Rather than relying on a central authority to log decision-making processes, each inference-the step an AI takes to reach a conclusion-is recorded as a transaction on the blockchain. This creates a permanent, tamper-proof record accessible to authorized parties, fostering transparency and accountability. Because the blockchain is distributed across numerous nodes, no single point of failure exists, and the integrity of the audit trail is maintained even in the face of malicious attacks or system failures. This decentralized approach not only ensures verifiability – allowing others to independently confirm the AI’s reasoning – but also builds trust in the system’s operations by removing the need for blind faith in a centralized entity.

Blockchain anchoring establishes a permanent and unalterable record of an AI system’s decision-making process, functioning as a digital fingerprint for each inference. This technique involves cryptographically hashing the system’s reasoning steps – including input data, intermediate calculations, and the final output – and then recording that hash on a blockchain. Because blockchains are inherently tamper-proof and distributed, any attempt to modify the recorded reasoning would necessitate altering the blockchain itself, an extremely difficult and computationally expensive undertaking. This creates a robust immutable audit trail, allowing for independent verification of the AI’s logic and fostering transparency. The resulting record isn’t simply a log of outcomes, but a detailed history of how those outcomes were reached, crucial for accountability, debugging, and building trust in complex AI systems.

Decentralized auditing, facilitated by the integration of blockchain technology, fundamentally shifts accountability in artificial intelligence systems. This approach allows for a publicly verifiable record of an AI’s decision-making process, fostering increased trust and transparency. A recent implementation of this system demonstrates practical viability, achieving a recall of 43.8% and a weighted F1 score of 0.429 when evaluated on the Spanish MEACorpus 2023 dataset – a benchmark for complex reasoning tasks. These results suggest that blockchain-based auditing is not merely a theoretical concept, but a functional tool capable of enhancing the reliability and trustworthiness of increasingly sophisticated AI applications.

Each recorded inference and audit event within the system necessitates a transaction on the Sepolia Testnet, incurring a gas cost of 47,000 gas units per transaction. This cost represents the computational effort required to validate and permanently record the event on the blockchain, ensuring immutability and verifiability. While seemingly minor in isolation, the cumulative gas cost becomes a critical factor when considering the scale of continuous AI reasoning and auditing; developers must carefully optimize the frequency and data volume of recorded events to maintain cost-effectiveness without compromising the integrity of the audit trail. This trade-off between detailed record-keeping and operational expenses is a central consideration in deploying blockchain-based auditing systems for AI applications.

Towards Verifiable and Trustworthy AI: A Blueprint for the Future

The system’s current reliance on heuristic methods represents a foundational step, deliberately chosen for its interpretability and ease of verification during initial development. However, the underlying architecture is intentionally scalable, designed to accommodate the integration of more sophisticated models, such as Neural Networks, without sacrificing its core principles of transparency and accountability. This modular design anticipates future advancements in artificial intelligence, allowing for increased complexity and performance while retaining the ability to cryptographically anchor the inference process. Such adaptability ensures the framework can evolve alongside the field, ultimately enabling the creation of verifiable and trustworthy AI systems capable of handling increasingly nuanced and challenging tasks, and maintaining a clear audit trail throughout.

The architecture underpinning this work intentionally moves beyond the narrow application of emotion recognition, representing a foundational step towards broadly applicable, verifiable AI. Researchers designed the framework with modularity and abstraction as core principles, allowing its principles of transparent inference and cryptographic anchoring to be adapted to diverse fields such as medical diagnosis, financial modeling, and autonomous vehicle control. This generalizability stems from the system’s focus on how decisions are made – by meticulously documenting the inference process and securing it cryptographically – rather than what decisions are about. Consequently, the framework isn’t limited by the specifics of any single application; instead, it offers a blueprint for building AI systems where accountability and reliability are paramount, regardless of the domain.

The pursuit of accountable and reliable artificial intelligence is significantly advanced through the synergy of transparent inference and cryptographic anchoring. Transparent inference allows for the complete tracing of an AI’s decision-making process – revealing how a conclusion was reached, rather than simply presenting the output. This is then fortified by cryptographic anchoring, which creates an immutable record of the input data, the AI model’s state, and the resulting inference. This combination establishes a verifiable audit trail, enabling independent validation and ensuring that any alterations to the system are readily detectable. Consequently, this framework doesn’t merely offer predictions; it provides demonstrable proof of their origins and integrity, fostering trust and enabling responsible deployment of AI across critical applications.

The pursuit of ‘Immutable Explainability’ necessitates a rigorous simplification of complex systems. The architecture proposed directly addresses the opacity often inherent in multimodal fusion, striving for a verifiable audit trail. It recalls Edsger W. Dijkstra’s assertion: “It’s not enough to have good intentions; you need good tools.” This paper’s combination of fuzzy logic and blockchain isn’t merely about technical innovation, but about constructing those tools-mechanisms for ensuring that affective AI’s decisions are not simply interpretable, but demonstrably trustworthy. Clarity is the minimum viable kindness; a transparent, immutable record of reasoning offers precisely that.

What Lies Ahead?

The pursuit of ‘Immutable Explainability’ – a concept bordering on oxymoronic given the inherent fluidity of both affect and logic – exposes a deeper unease. The architecture presented isn’t merely about securing explanations; it’s an admission that explanations, in the context of affective AI, require securing. The field has, for too long, treated post-hoc rationalization as sufficient. This work suggests that isn’t enough. The core challenge remains: can a system truly demonstrate understanding, or only convincingly simulate it, and then retroactively justify its actions?

Future work must confront the limitations of current multimodal fusion techniques. Combining data streams doesn’t inherently yield insight; it merely amplifies noise if the underlying representations are flawed. Furthermore, the application of blockchain, while elegant in theory, demands careful consideration of scalability and energy consumption. A truly robust system will require minimizing on-chain data while maximizing verifiability. The intersection of self-sovereign identity and affective AI also raises ethical questions; the potential for manipulation based on emotionally-derived profiles is substantial.

Ultimately, the question isn’t whether an AI can feel, but whether its explanations are demonstrably linked to a coherent, auditable internal model. The goal, then, isn’t perfection, but a ruthless pruning of complexity. Code should be as self-evident as gravity. Intuition is the best compiler; and until we can build systems that reflect that principle, ‘trustworthy AI’ will remain a beautifully-phrased aspiration.

Original article: https://arxiv.org/pdf/2512.11065.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/