As a young engineer I had just completed a lot of book learning and of course turned to my books for solutions once on the job. I have witnessed a similar focus on computer tools as 3D CAD and FEA. Both are valuable tools but you must understand the problem before the tools are applied.
During my early days as a mechanical engineer in the auto industry the Japanese where taking market share at an alarming rate. Motown was scrabbling for an explanation. Statistics became the jargon of the day and Dr. Deming a living legend. The company offered an applied statistics course titled Design of Experiments which was taught by the coauthor of the text. The coauthor was of the PhD type but taught us a practical lesson I have carried for over 35 years and will share with you shortly. First some background.
One method of testing vehicles back then and perhaps still today is to have a test track with permanent potholes and other brain jarring features. Technicians would drive the cars for many laps accumulating about 28,000 miles in a relatively short time. The cars were then taken apart and all the components checked for wear and damage. Now 28,000 may not sound like much but if you drive at just the right speeds over a road paved the cobblestone (in their natural form) you can get the car and its content to resonate and that included the un-expecting engineer going for a ride-a-long. That was only one part of the course the car travelled. Most engineers did not do ride-a-longs on a regular basis.
The engineer teaching the class told us that there was no real correlation between the damage the cars sustained during the 28,000 mile test and wear from normal use. But tests of this type did identify weaknesses that could be addressed.
This is where the mystery starts. There were two identical tracks. One located in Michigan and another in Arizona. Though there was no correlation between the test results and the real world use, correlation between the two test tracks was expected. The cobblestones and the steel lined pot holes were the same for both tracks as were the test procedures and driver training. So what could be the cause for this variance?
Michigan is cold and damp and Arizona hot and dry. Was it the natural environment? Was the salt used during winters in Michigan getting onto the test track? All the obvious possibilities were eliminated. Then the statistical tools were brought to bear with no better success than the climate study. I shudder to think of what tools and resources that would have been employed today because of their abundant desktop availability and the WWW.
Well when all else fails and the boss wants an answer, you need to abandon the comfort of the office, get on a plane, and see things firsthand. Now if you know anything about psychology, people perform differently when being watched. To learn the real or typical behavior of people you need to observe from “behind the bushes.”
The mystery had a very simple explanation. Those who were thousands of miles from watchful eyes decided the cobblestones and steel lined potholes hurt their backs so avoiding them was a better alternative.
This is a true story as told to me. The point of presenting here is that the analytical tools we have are a marvel but can truly be a diversion from reality. The tools are only good when the correct input is provided. The tools today are so easy to use and data output so impressive we may not stop and think if the input we provide is correct. Even back in the dark ages of my beginnings a lot of time went into analyzing something that only hiding behind the bushes would reveal. So before all the analysis goes into play make sure you understand what is being analyzed. GIGO was true then and true now.