---

Mathematical models have become indispensable sources of information on which technical and business decisions are based. It is therefore vitally important for decision-makers to know whether relying on the predictions of mathematical models is justified. When properly used, numerical simulation can be a major corporate asset. However, it can become a major corporate liability if the reliability of predictions is not guaranteed.

Resource Allocation

Project management is responsible for allocating resources to numerical simulation and physical experimentation. Consider the two extreme cases: (a) If the decision is not to use numerical simulation, just rely on experimentation, then management is adopting the trial-and-error methodology that the Wright brothers had to use. (b) If the decision is not to do experiments, and just rely on finite element modeling, then management will risk repeating the costly mistakes of the F-35 program [1]. Therefore, the correct balance depends on the justified degree of confidence in the predictive performance of numerical simulation.

Simulation Governance

Simulation governance is a managerial function concerned with the assurance of reliability of information generated by numerical simulation. The term was introduced in 2011 and specific technical requirements were addressed from the perspective of mechanical design in 2012 [2]. At the 2017 NAFEMS World Congress in Stockholm, simulation governance was identified as the first of eight “big issues” in numerical simulation.

A plan for simulation governance has to be tailored to fit the mission of each organization or department within an organization. We consider three types of mission in the following. A summary of the main points is presented in the table below.

• If the mission is the application of established design rules, then the goal is to verify that a quantity of interest F does not exceed its allowable value F_all. Simulation governance is concerned with standardization of recurring numerical simulation tasks through the creation of smart applications. Smart applications, also called “simulation apps”, are expert-designed in such a way that the use of those applications does not require expertise in numerical simulation. The preservation and maintenance of corporate know-how and institutional knowledge are among the important objectives of simulation governance. The productivity of newly hired engineers significantly increases if routine simulation procedures are standardized so that applications consistently produce certifiable results. Economic benefits are realized through improved productivity and improved reliability.

Table 1. Mission-dependence of numerical simulation tasks.

• If the mission is the formulation of design rules, for example, to establish allowable values for a new material system, then the plan should focus on the collection, maintenance, and documentation of experimental data, management of solution and data verification procedures, revision and updating mathematical models in the light of new information collected from physical experiments and field observations. Economic benefits: Substantial savings through reduction of the number of tests on the sub-component, component, sub-assembly, and assembly levels.
• If the mission is to support condition-based maintenance (CBM), then the goal is to determine the probability that the number of cycles to failure (N_f) is smaller than a given number of cycles (N). The main activities are: Collection, maintenance, and documentation of fatigue data and unit-specific information on service data, standardization of recurring analysis tasks. Economic benefits: Substantial savings through improved disposition decisions.

Observe that solution, data, and code verification are common to all three types of mission. Validation and uncertainty quantification are performed in model development projects.

Recognizing that technology advances and the available information increases over time, planning must incorporate data management and systematic updates of simulation practices so as to take advantage of new data, ideas, and technology. Model development projects are open-ended: Validation and definition of the domain of calibration is conditional on the available data. Since the available data increases over time, and new ideas are likely to be proposed, there will be opportunities to revise and update mathematical models. No one has the final word on model development [3].

Questions for Management

Managers in charge of numerical simulation projects are expected to develop evidence demonstrating that reliance on predictions is justified. They should consider the following issues:

1. Does the management team have the expertise needed to set the goals for numerical simulation and define the technical requirements?
2. Does the engineering team have the technical expertise and software tools required to effectively conduct numerical simulation projects?
3. Are mathematical models being defined independently from the way numerical approximations are obtained? – A common mistake is to conflate model definition with its numerical solution, as in “finite element modeling”.
4. Are the errors in numerical approximation properly estimated, controlled, and reported?
5. Physical testing is necessarily tied to the specific test conditions. Generalization to a larger set of conditions requires a mathematical model. Testing without a plan to generalize the results does not make sense. – Are physical testing projects properly planned, executed, and analyzed? 
6. Are the experimental data properly documented and archived?
7. How well are the various professional skills needed for successful execution of numerical simulation projects coordinated?
8. Are mathematical models properly calibrated and their domains of calibration properly defined and documented?
9. How are new data incorporated into model updates when the data lie (a) within the domain of calibration and (b) outside of the domain of calibration?
10. Are the procedures of verification, validation, and uncertainty quantification (VVUQ) properly and consistently applied?
11. Have opportunities to improve design workflows through standardization been fully explored?
12. What is the estimated economic value of your current numerical simulation activities? – Without simulation governance, that value can be a large negative number.

---

References

[1] Szabó, B. and Actis, R, Planning for Simulation Governance and Management: Ensuring Simulation is an Asset, not a Liability. Benchmark, July 2021, pages 8-12.

[2] Szabó, B. and Actis, R,”Simulation governance: Technical requirements for mechanical design,” Computer Methods in Applied Mechanics and Engineering, vol. 249, pages 158-168, 2012.

[3] Szabó, B.  and Babuška, I. “Methodology of model development in the applied sciences,” Journal of Computational and Applied Mechanics, vol. 16, no. 2, pp. 75-86, 2021 (open source).

---

Related Blogs:

• A Memo from the 5th Century BC
• Obstacles to Progress
• Why Finite Element Modeling is Not Numerical Simulation?
• XAI Will Force Clear Thinking About the Nature of Mathematical Models
• The Story of the P-version in a Nutshell
• Why Worry About Singularities?
• Questions About Singularities
• A Low-Hanging Fruit: Smart Engineering Simulation Applications
• The Demarcation Problem in the Engineering Sciences
• Model Development in the Engineering Sciences
• Certification by Analysis (CbA) – Are We There Yet?
• Not All Models Are Wrong
• Digital Twins
• Digital Transformation
• Simulation Governance
• Variational Crimes
• The Kuhn Cycle in the Engineering Sciences
• Finite Element Libraries: Mixing the “What” with the “How”
• A Critique of the World Wide Failure Exercise
• Meshless Methods
• Isogeometric Analysis (IGA)
• Chaos in the Brickyard Revisited
• Why Is Solution Verification Necessary?
• Variational Crimes and Refloating the Costa Concordia
• Lessons From a Failed Model Development Project