Unlocking Hidden Physics from Data: A New Approach to Thermal-Fluid Modeling

Author: Denis Avetisyan

Researchers are leveraging machine learning to automatically discover and refine governing equations from operational data, paving the way for more accurate and efficient system simulations.

A trained Dyad UDE demonstrably improves performance on unseen data, achieving a 3% reduction in loss and indicating enhanced generalization capabilities.

This work details a semi-automated workflow, built within the Dyad environment, for model discovery and calibration using Universal Differential Equations, sensitivity analysis, and symbolic regression applied to transportation refrigeration systems.

Accurate calibration of dynamic models is often hindered by simplifying assumptions that neglect crucial physics within complex systems. This challenge is addressed in ‘Scientific Machine Learning-assisted Model Discovery from Telemetry Data’, which proposes a semi-automated workflow, Dyad, for augmenting physics-based models with data-driven symbolic expressions. By integrating Universal Differential Equations, sensitivity analysis, and symbolic regression, the method demonstrably improves the predictive performance of thermal-fluid models-specifically within transportation refrigeration units-through an engineer-in-the-loop design process. Could this AI-assisted approach represent a paradigm shift in how we build and refine digital twins for increasingly complex engineering systems?

Bridging Theory and Observation: The Limits of Traditional Modeling

The operation of seemingly mundane systems, such as those maintaining temperature-sensitive cargo during transport, is deeply rooted in the laws of physics. Heat transfer, thermodynamics, and fluid dynamics all interact in intricate ways to dictate performance. Consider a refrigerated container: maintaining a consistent temperature requires a precise balance between cooling capacity, insulation effectiveness, and external environmental factors. These systems aren’t simply about ‘keeping things cold’; they involve complex interactions between conductive, convective, and radiative heat exchange, all influenced by the specific properties of the cargo itself. Understanding these underlying physical principles is crucial, yet often insufficient, as real-world conditions introduce complexities that traditional models struggle to fully capture, necessitating innovative approaches to reconcile theory with observed data.

Conventional physics-based models, though rigorously defined by fundamental equations, frequently encounter challenges when applied to real-world scenarios. The precision inherent in these models is often undermined by the unavoidable presence of data imperfections – sensor noise, measurement errors, or simply incomplete information about system parameters. This disconnect arises because these models typically assume ideal conditions, failing to adequately account for the inherent variability and uncertainty present in operational environments. Consequently, predictive accuracy diminishes as the complexity of the system increases, and the model’s ability to generalize beyond controlled laboratory settings is severely restricted. The resulting limitations necessitate the development of techniques that can effectively integrate imperfect data, bridging the gap between theoretical precision and practical performance.

The inherent gap between the precision of physics-based models and the messiness of real-world data necessitates a shift towards refinement strategies informed by observation. While fundamental physical principles accurately describe system behavior, incomplete or noisy data streams often lead to performance discrepancies when these models attempt to predict outcomes. Recent work addresses this challenge by integrating data-driven techniques to calibrate and enhance model predictions, as demonstrated in a study of transport refrigeration. Specifically, researchers applied these methods to a two-zone cargo box model, achieving a measurable 3% reduction in the loss function – a key indicator of predictive accuracy. This improvement highlights the potential of combining theoretical understanding with empirical evidence, paving the way for more reliable and efficient predictive capabilities in complex systems.

Model Discovery: A Hybrid Approach to System Identification

Model Discovery is a system identification technique that iteratively refines pre-existing physical models using observational data. This semi-automated process does not construct models de novo, but rather leverages existing knowledge – expressed as a base model – and adjusts its parameters or functional forms to better fit observed behavior. The method typically involves a search algorithm that explores modifications to the base model, evaluating each candidate model’s performance against the data using a defined loss function. The degree of automation varies, often requiring expert input to guide the search and validate the resulting refined model, distinguishing it from fully automated system identification approaches.

Universal Differential Equations (UDEs) facilitate the integration of machine learning components into existing dynamical system models by treating the model’s governing equations as a neural network. This allows the system to learn directly from data and modify the original equations through the addition of learned functions, effectively expanding the model’s representational capacity. Specifically, UDEs utilize automatic differentiation to compute derivatives of the learned functions and incorporate them into the dynamical system’s equations, [latex] \frac{dx}{dt} = f(x, t) + \Theta(x, t) [/latex], where [latex] f [/latex] represents the original system and Θ is a neural network-based correction term. This process doesn’t replace the underlying physics but rather refines it based on observed data, enabling the system to capture complexities not explicitly accounted for in the initial model.

Model Discovery seeks to enhance predictive performance and the ability to accurately forecast outcomes on unseen data by integrating established physical models with machine learning components. This hybrid methodology capitalizes on the interpretability and established theoretical foundations of physical models while leveraging the data-driven adaptability of machine learning. Evaluations on a designated test dataset demonstrate a 3% reduction in the loss function when using the Model Discovery approach compared to relying solely on the original physical models, indicating improved accuracy in predicting observed behaviors.

This dyad model combines user-guided steps with automated procedures, allowing an engineer to iteratively refine a system by accepting or rejecting proposed changes.

Unveiling Hidden Structure: Symbolic Regression for Enhanced Interpretability

Uncertainty Decomposition Estimators (UDEs) are frequently employed to refine the accuracy of machine learning models by applying corrections derived from neural networks. However, these neural network-based corrections typically manifest as complex, non-linear functions that are difficult for human experts to analyze and understand. This lack of transparency hinders the ability to validate the reasoning behind the corrections or to gain insights into the underlying relationships driving the model’s predictions. Consequently, while UDEs can improve predictive performance, the resulting corrections often function as “black boxes,” limiting trust and hindering the potential for knowledge discovery.

Symbolic Regression operates by identifying mathematical expressions that best represent the relationships within a dataset, effectively replacing complex, often opaque, Neural Network Corrections with human-readable equations. This process doesn’t simply approximate the correction; it actively discovers the underlying functional dependencies. The resulting symbolic expression, typically composed of elementary mathematical operators and variables, provides a direct and interpretable representation of how the correction modifies the initial UDE output. This allows stakeholders to understand why a correction is being applied, rather than simply observing its effect, and provides insights into the data’s inherent structure. The identified equations can take forms such as [latex]y = ax + b[/latex] or more complex non-linear relationships, depending on the data’s complexity.

The integration of Symbolic Regression facilitated a reduction in model complexity by identifying and eliminating redundant parameters within the Neural Network Corrections. This process, coupled with subsequent Sensitivity Analysis to validate the impact of retained variables, allowed for model parsimony without a statistically significant decrease in predictive accuracy. Quantitative evaluation on the test dataset demonstrated a 3% improvement in the loss function, indicating that the simplified model maintained – and marginally improved – performance compared to the original, more complex architecture. This outcome confirms the efficacy of Symbolic Regression as a tool for both interpretability and efficiency gains in UDE-corrected models.

Symbolic regression successfully derived concise expressions-using only five input variables and two parameters-to represent the underlying relationships within the data.

Dyad: An Integrated Environment for Acausal Modeling and Machine Learning

Dyad represents a novel environment designed to bridge the gap between traditional system modeling and the rapidly evolving field of machine learning, specifically utilizing Unscented Derivative Equations (UDEs). This integration isn’t simply a juxtaposition of tools; it’s a cohesive framework where system dynamics can be defined and analyzed using established modeling principles, then directly leveraged to enhance or be enhanced by machine learning algorithms. By enabling a bi-directional flow of information, Dyad facilitates scenarios where prior knowledge embedded in system models informs the training process, or where machine learning algorithms refine and optimize those same models. This synergistic approach moves beyond conventional methods, offering the potential for improved accuracy, robustness, and interpretability in complex systems-a capability demonstrated through a 3% improvement in loss function during initial testing.

Structural Simplification represents a core innovation within Dyad, addressing the challenge of computational expense often associated with complex system models. This feature employs symbolic manipulation to proactively reduce model order and eliminate redundant parameters before any numerical simulations are performed. Rather than relying solely on computationally intensive methods to approximate solutions, Dyad’s approach directly minimizes model complexity, leading to faster simulations and improved interpretability. By identifying and removing unnecessary elements, Structural Simplification not only enhances computational efficiency but also mitigates the risk of overfitting, particularly when integrating with machine learning techniques like Unique Differential Equations (UDEs). This pre-processing step allows researchers to focus on the essential dynamics of a system, ultimately contributing to more robust and insightful modeling outcomes, as demonstrated by the 3% improvement in loss function achieved within the Dyad framework.

The Dyad framework leverages JSML, a novel acausal declarative language implemented within the Julia programming environment, to streamline the process of model definition and analysis. This language allows researchers to express system dynamics in a concise and intuitive manner, abstracting away many of the complexities associated with traditional modeling approaches. By enabling a more direct translation of conceptual models into executable code, JSML facilitated a workflow that demonstrably improved performance; specifically, a 3% reduction in loss function was achieved through models constructed using this integrated system. This improvement highlights the efficacy of combining declarative modeling with the computational power of Julia, suggesting a promising avenue for advancements in both acausal reasoning and machine learning applications.

The Dyad UDE was trained using configurations B and E within a two-zone box model and subsequently evaluated on a unique, proprietary trailer setup.

Refining the Model: Addressing Artifacts and Ensuring Fidelity

The creation of mathematical models from data, while powerful, isn’t without potential pitfalls; the model compilation process itself can inadvertently introduce what are known as ‘Dummy Derivatives’. These terms, appearing within the symbolic expressions that define the model, are mathematically valid but lack any basis in the physical system being represented. Essentially, they are artifacts of the modeling process-spurious correlations mistaken for genuine relationships. These derivatives don’t correspond to actual physical phenomena and can distort the model’s predictions, hindering its ability to accurately simulate or extrapolate beyond the training data. Identifying and eliminating these meaningless terms is, therefore, a crucial step towards ensuring model fidelity and preventing inaccurate interpretations of the underlying science.

Sensitivity analysis serves as a vital diagnostic step in model discovery, effectively pinpointing and eliminating spurious terms – often termed ‘dummy derivatives’ – that arise during symbolic regression. This technique systematically assesses how changes in input parameters affect model predictions, revealing whether specific terms contribute meaningfully to the underlying physics or are merely artifacts of the compilation process. By quantifying the impact of each term, researchers can confidently remove those with negligible influence, ensuring the final model accurately represents the true relationships governing the system. This rigorous refinement process is crucial for building predictive models that are not only accurate but also interpretable and physically meaningful, ultimately enhancing trust and reliability in the discovered equations.

Ongoing research prioritizes the development of automated systems for identifying and correcting modeling artifacts, a refinement poised to significantly accelerate the process of Model Discovery. Building on initial successes that yielded a 3% reduction in loss function, these advancements aim to move beyond manual sensitivity analysis. The goal is to create a self-correcting framework where the model itself flags and resolves inconsistencies, ultimately enhancing both the efficiency and reliability of generated equations. This automated approach promises to not only reduce computational costs but also minimize the potential for human error in complex modeling scenarios, paving the way for broader application and increased confidence in the resulting predictive capabilities.

Sensitivity analysis, visualized as a heatmap of the Jacobian matrix, reveals the influence of the top seven inputs-including model-generated dummy derivatives denoted by the “\_t(t)” suffix-on system behavior.

The pursuit of accurate thermal-fluid models, as detailed in this work, necessitates a holistic understanding of system behavior-a principle echoing Bertrand Russell’s observation: “The whole is more than the sum of its parts.” The semi-automated workflow presented, employing Universal Differential Equations and symbolic regression, doesn’t merely seek isolated equations, but rather aims to capture the intricate interplay of variables governing transportation refrigeration. Each optimization, as the article implies, introduces new complexities; a truth Russell subtly acknowledges in the inherent difficulty of truly understanding any complete system. The process reveals that structure, in this case, the underlying physics and relationships, fundamentally dictates behavior over time, necessitating a careful consideration of the entire system, not just individual components.

What Lies Ahead?

The pursuit of model discovery, as demonstrated by this work, continually circles back to a fundamental question: what are systems actually for? The demonstrated semi-automated workflow, while successful in its domain, merely addresses the ‘how’ of uncovering governing equations. It does not, and cannot, define the appropriate level of model complexity, nor the very purpose for which that model will be used. A more accurate thermal-fluid model, calibrated with impressive fidelity, remains useless without a clear articulation of the optimization target – is it energy efficiency, precise temperature control, or perhaps lifespan prediction?

Future effort must shift from simply finding equations to evaluating their utility within a broader systems context. The current paradigm, reliant on sensitivity analysis and symbolic regression, is powerfully adept at identifying relationships within observed data. However, it inherently struggles with the inclusion of prior knowledge, or the explicit representation of uncertainty. True progress demands a more holistic approach, one that seamlessly integrates data-driven discovery with established physical principles and a rigorous accounting of epistemic limitations.

Simplicity, it should be remembered, is not minimalism. It is the discipline of distinguishing the essential from the accidental. The field would benefit from a renewed focus on structural understanding – recognizing that the behavior of any complex system is dictated not by isolated equations, but by the intricate interplay of its constituent parts. The next generation of model discovery tools must therefore prioritize interpretability and transparency, allowing practitioners to move beyond mere prediction and towards genuine understanding.

Original article: https://arxiv.org/pdf/2603.15943.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/