By Dr. Barna Szabó, ESRD Co-Founder & Chairman

All major engineering organizations use numerical simulation in support of decisions pertaining to design and design certification. The question of how well numerical simulation is managed is being addressed by the various stakeholders with increasing urgency. It is now generally recognized that numerical simulation, properly managed, can be a major corporate asset but poorly managed, or not managed at all, can become a major corporate liability.

What Is Simulation Governance?

By its definition in Wikipedia: “Simulation governance is a managerial function concerned with assurance of reliability of information generated by numerical simulation. The term was introduced in 2011 [1] and specific technical requirements were addressed from the perspective of mechanical design in 2012 [2]. Its strategic importance was addressed in 2017 [3] [4]. At the 2017 NAFEMS World Congress in Stockholm simulation governance was identified as the first of eight “big issues” in numerical simulation.

Simulation governance is concerned with (a) selection and adoption of the best available simulation technology, (b) formulation of mathematical models, (c) management of experimental data, (d) data and solution verification procedures, and (e) revision of mathematical models in the light of new information collected from physical experiments and field observations [5].”

Why Is Simulation Governance Important?

To appreciate the fundamental importance of simulation governance, please consider the following scenarios.

Scenario #1

A senior executive (SE) of a Fortune 500 corporation, in charge of a multi-billion dollar program, came to suspect that something was very wrong with the way finite element modeling was being done. There were large discrepancies between the outcomes of physical tests and the outcomes predicted by engineer-analysts. This made it necessary to do more testing than originally planned, resulting in substantial delays and cost overruns.

Getting reliable and timely correlation between predicted and realized test results was particularly important because new materials were being introduced for which design rules had to be developed, yet the discrepancies between predicted and realized outcomes were substantial, to say the least.

The SE invited outside consultants to address his concerns. He also invited five senior engineers to the first meeting. These engineers were responsible for finite element modeling in one way or another. Each of them had impeccable academic credentials and decades of experience.

The consultants explained the difference between finite element modeling and numerical simulation. They conveyed (in as diplomatic a way as they could) that the erroneous predictions were caused by the now obsolete practice of finite element modeling which can produce reasonable results in structural calculations but is not suited for strength calculations.

The engineers were visibly upset by the suggestion that the method they have been using for years is now obsolete. Their strong opposition to any substantive change in current practices became obvious. What do you think the SE should do?

Scenario #2

Your corporation grew over time by acquisition. Each acquisition brought with it a corporate culture, engineering manuals, software, Excel-based calculations and the like. Just to see what would happen, you gave a problem to three engineer-analysts and you got three different answers. What do you think you should do?

Scenario #3

You feel overwhelmed by the number of software products used by your engineering staff. You would like to streamline the workflows and initiate a vendor-reduction program. Your senior engineers agree with your goals but do not agree on what streamlining means and what criteria should be used in the vendor-reduction program. What will you do?

Scenario #4

A flight-critical component of a rotorcraft is to be tested in a fatigue experiment. A finite element model predicted that the probability is 90% that failure will occur between 100 and 250 thousand cycles. The failure occurred at 80 thousand cycles. If you were the engineer in charge of the design, what would you do?

Scenario #5

Your organization decided to introduce a new type of composite material for which design rules are not yet available. You were assigned to set up a program for the development of design rules. Your senior analysts proposed three different ideas on how the failure criteria should be formulated. How will you decide which one to use?

Scenario #6

The cost of unscheduled maintenance is much higher than the cost of scheduled maintenance. You would like to justify postponing maintenance actions relating to safety critical components to scheduled times. How would you go about establishing guidelines?

Scenario #7 – The JSF Story

In 2011 and 2012, two high-ranking US military officials commented on the (costly) lessons learned from the Joint Strike Fighter (JSF) project. From JSF program chief Vice Admiral David Venlet (AOL Defense, 2011):

JSF’s build and test was a miscalculation…. Fatigue testing and analysis are turning up so many potential cracks and hot spots in the Joint Strike Fighter’s airframe that the production rate of the F-35 should slowed further over the next few years… The cost burden sucks the wind out of your lungs.

And, from Gen. Norton Schwartz, Air Force Chief of Staff (Defense News, 2012):

There was a view that we had advanced to a stage of aircraft design where we could design an airplane that would be near perfect the first time it flew. I think we actually believed that. And I think we’ve demonstrated in a compelling way that that’s foolishness.

What do you think should have been done differently? What steps did your organization take to avoid similar problems in the future?

Scenario #8

You asked your principal vendor of finite element analysis software; what tools are provided to support solution verification? The response was that the software had been tested against all NAFEMS benchmark problems and the correct solutions were obtained. Should you be satisfied with this answer? Have you communicated your technical requirements to your software vendors? Is solution verification among your technical requirements? – Should it be?

Planning For Simulation Governance

The concept of simulation governance is easily grasped, however formulation of a plan for simulation governance for a particular organization is not simple at all.

Recognizing that technology advances and the available information increases over time, planning must incorporate data management and systematic updates of simulation practices so as to take advantage of new data and technology.

The preservation and maintenance of corporate know-how and institutional knowledge are important objectives of simulation governance. The productivity of newly hired engineers rapidly increases if routine simulation procedures are standardized so that applications consistently produce certifiable results.

A plan for simulation governance has to be tailored to fit the mission of each organization or department within an organization:

• If the mission is application of existing design rules, then the goal should be:
  • Standardize recurring numerical simulation procedures through the creation of smart applications. Smart applications, also called “simulation apps” are expert-designed, in such a way that the use of those applications does not require expertise in numerical simulation.
  • Economic benefits: Improved productivity and improved reliability.
• If the mission is formulation of design rules, then the plan should focus on:
  • Collection, maintenance and documentation of experimental data.
  • Management of solution and data verification procedures.
  • Revision and updating mathematical models in the light of new information collected from physical experiments and field observation.
  • Economic benefits: Substantial savings through optimization of design and certification procedures.
• If the mission is to support condition-based maintenance (CBM), then the main activities are:
  • Collection, maintenance and documentation of field data.
  • Update calibration records based on unit-specific information.
  • Standardize recurring analysis tasks.
  • Economic benefits: Substantial savings through improved disposition decisions.

Implementation

The first and perhaps the most challenging problem is to overcome widespread misunderstanding of what numerical simulation is. Most managers and many individuals who present themselves as experts in numerical simulation confuse numerical simulation with “finite element modeling” or “numerical modeling “. Those are outdated concepts, responsible for much of the disappointing results.

Existing simulation and data management practices have to be reviewed and evaluated in terms of efficiency and reliability. When a need for change is identified, a business case and consensus among the stakeholders have to be developed.

Metrics that measure the economic value of numerical simulation have to be established.

References

[1] Szabó B. and Actis R. Simulation governance: New technical requirements for software tools in computational solid mechanics. International Workshop on Verification and Validation in Computational Science University of Notre Dame 17-19 October 2011.
[2] Szabó B. and Actis R. Simulation governance: Technical requirements for mechanical design. Comput. Methods Appl. Mech. Engrg. 249-252 158-168, 2012.
[3] Meintjes J. Simulation Governance: Managing Simulation as a Strategic Capability. NAFEMS Benchmark Magazine, January 2015.
[4] Imbert J-F. Simulation Governance – Building confidence, a key dimension of simulation strategy. NAFEMS World Congress NWC15 San Diego, June 2015.
[5] Oberkampf WL and Pilch M, Simulation Verification and Validation for Managers. NAFEMS, 2017. ISBN 978-1-910643-33-4.

Request a Simulation Governance Briefing

Would you like to request a Simulation Governance briefing with our team of experts? Complete the form below and we’ll be happy to reach out with more information: