Comparing Two Wiebull Distributions

Folks,

Let’s look at Patty’s last day of class …

As she was driving north to teach her statistics class, Patty was sad to see her stint at Ivy U come to a close. She was even more nervous about her meeting with Dean Howard after the class.

Before she knew it, she was standing in front of the class, to start her last lecture on Weibull analysis.

“Are there any questions before I begin?” Patty asked.

Patty nodded to Megan Ramsey.

“Professor, last time we talked about when a few samples don’t fail in a test that they are often censored in Weibull analysis. You mentioned that many people probably think it is good that some samples don’t fail. However, if the samples did fail at a later time it increases the scatter and would make the data worse. I’m not sure I understand that concept, as the scale has increased and the the top 10% of the samples would have a much longer life,” Megan summarized.

“Megan has an interesting point. Let me put both the censored (blue) and hypothetical data if the fails came later (red) on one graph. Discuss it for a while with those sitting next to you and see if you can conclude which data are better,” Patty suggested.

About three minutes went by and Patty called the class to order.

Megan was the first to raise her hand, which Patty acknowledged.

“After discussing it with Pete, (there were a few rolled eyes and soft whistles as everyone knew that Megan and Pete were an ‘item’) we concluded that the censored data (blue) is better. The most critical reason is that it predicts the smallest number of fails at a lowest number of cycles. We think this will always be the critical concern in reliability,” Megan answered.

“Precisely! This reason is why unfailed samples are not an endorsement to superior reliability. The censored data predict twice as many cycles – at a 5% failure rate. It is almost certainly misleading,” Patty said.

She chuckled a little then said, “If you want to impress someone in a job interview, discuss this topic.”

Patty didn’t know it, but one of the reasons the students like her as an instructor was her experience as an engineer. The many professors at Ivy U were brilliant, but few of them had actually been an engineer or managed a manufacturing process.

“OK, we have one last topic: how to tell if two Weibull distributions are statistically different,” Patty said.

“Let’s look at Weibull plots of stress test failures of alloys 5 and 6,” Patty said.

Prashant Patyl raised his hand.

“Yes, Prashant,” Patty acknowledged.

“Well. Alloy 6 (red) has a slightly higher scale and steeper slope, suggesting it is better, but it would be hard to say if it is statistically significantly better,” Prashant answered.

“Precisely,” Patty answered.

“Let’s try the plain old two sample t test,” Patty went on and showed a boxplot of the data.

The class chuckled a bit, as this test would be considered much more mundane than Weibull analysis.

“The t test shows that there is only a 30% confidence that the means are different. Just by visual inspection, the boxplot (below) suggests as much. So it would be hard to argue that the data are different at a 95% confidence level,” Patty elaborated.

Her comments resulted in much lively discussion about the normality of the data, if the mean a reasonable metric for comparison, and other perspectives and other related topics.

The ending of the class was very upbeat, so Patty was feeling an emotional high, until she remembered that she had to meet with Dean Howard. With trepidation, she headed toward his office. As she headed in, she was shaking a little.

“Professor Coleman, it’s great to see you,” Dean Howard said with enthusiasm and warmth.

Patty still couldn’t get used to being called “Professor,“ but she had checked on the Ivy U website and she was listed as a “Visiting Associate Professor.” They even had a webpage for her. She thought the photo they used made her look too old.

Before she could answer, Dean Howard got to the point.

“We have really been impressed with the teaching job you have done. The students were especially appreciative of your teaching style,” Dean Howard started.

“Thank you,” Patty said, her relief palpable.

“It appears that Professor Harlow, whom you are filling in for, will require a longer recovery than thought. In addition, we need a course on manufacturing processes. The bottom line is we want you to join the faculty to help us with these courses,” Dean Howard continued.

Patty nearly swooned.

“But sir, I don’t have a Ph.D.,” Patty responded.

“Our plan is that you have done such significant work at ACME, that you don’t need to do a thesis. We want you to take four courses while you teach. After successful completion of these courses, we will award you a Ph.D.,” Dean Howard went on.

Patty was so stunned she didn’t know what to say. She was silent for a while.

The Dean continued, “We can’t quite match your salary at ACME, but we can come close. I have already discussed the situation with Mike Madigan. He is supportive, but said the decision is obviously up to you. What do you think?”

Patty’s mind was spinning. Rob was getting his Ph.D. here, so that would help.

It was as if she was outside of her body looking and she saw herself say, “I would love to.”

They talked for 10 more minutes about some of the details and Patty relaxed a little. It occurred to her that she had not discussed it with Rob yet. Oh well. She expected that he would be supportive.

As they were wrapping things up, Dean Howard appeared to want to discuss a different topic.

After a few minutes of additional discussion, Patty left with a smile on her face.

Epilogue

Pete, as usual, always knew what was going on. He had never felt so depressed. He and Patty were a team. They had traveled all over the world solving electronics assembly problems and she was abandoning him to got to Ivy U! He was also nervous. He wasn’t that thrilled with the other people he thought likely to be his new boss. So, with head hanging, he shuffled toward Patty’s office.

“Hey, Pete! It’s great to see you!” Patty said cheerfully.

Pete got all choked up and didn’t know what to say. Finally, he mumbled with a shaky voice, “You’re leaving.”

“So are you!” Patty responded. “Assuming you want to be the Senior Research Associate for Manufacturing Processes at Ivy U.” They are even offering you 10% more than you make here – and the benefits are great,” she finished.

“Right after my offer, Dean Howard asked if I knew someone who could fill such a position, so I immediately suggested you. Apparently my endorsement was enough to land you the job, if you want it. Don’t screw it up,” she teased.

Patty, The Professor, and Pete in one location. Only time will tell what new adventures await.

Cheers,

Dr. Ron

 

Sample Size is Important in Weibull Analysis Too

Some time ago I posted on “The Curse of the Early First Fail” and “Interpreting Weibull Plots.” Both of these posts related to using Weibull analysis to make sound engineering decisions.

Recently, a reader asked if sample size is important in Weibull analysis. It is interesting that few who do Weibull analyses discuss the effect of sample size. So, let’s do it now. Consider Figure 1. This figure shows Weibull analysis used to compare cycles to fail for Alloy 1 and Alloy 2. Considering that the slope of each curve is about the same, most people would say that since the scale for Alloy 2 is greater (1320 versus 1172), Alloy 2 is superior. But, is the difference statistically significant? By using a simple Two Sample t Test, we can analyze the data and find that there is only a 62% confidence that Alloy 2 is better than Allot 1. Flipping a coin gives us 50% confidence, so this result is not encouraging. Four samples is seldom enough to make a confident engineering decision.

Figure 1. A Weibull plot of Alloy 1 and 2 with only four samples.

If we perform the experiment again with 20 samples, we get the Weibull analysis as shown in Figure 2. Note that although the scale parameters have not changed too much, the shape parameters have changed significantly. The original 4 sample test is just not enough to really lock in on the real shape numbers for the samples. By also performing a two sample t test on the 20 sample data, we now find we have a 99.6% confidence that Alloy 2 is superior to Alloy 1. So, with 20 samples we can confidently say that Alloy 2 is superior to Alloy 1.

 

Figure 2. A Weibull plot of alloys 1 and 2 with 20 samples.

What is the minimum sample size for your test to be confident in the result? It can vary quite a bit and only by analyzing the data with a t test, after the experiment, can you know for sure. But my experience would suggest that you should never have less than 10 samples, and preferably 15 or more.

Cheers,

Dr. Ron