Tag Archives: SPC

Calculating Confidence Intervals on Cpks

Posted on by

Let’s look in on Patty, it’s been awhile.

Patty was looking forward to sleeping in.  Normally she was up very early, sometimes before 5:30 am, after usually getting to bed too late, so she was looking forward to an alarm set at 7:45 am. The kids were off from school and Rob was taking them skiing, so all agreed a 7:45 am wake up time was reasonable.  Since she had no early meetings, her scheduled 9 am arrival at her Ivy University office was also in the cards.

Patty was sleeping soundly when she heard her seven-year old twin sons shouting, “Mom! Dad! Come quickly.”   At the same time, their two-year old beagle, Duchess, started barking.

Her heart pounding, Patty raced to the racket now being produced by this energetic trio.  As she arrived she saw her sons and Duchess looking out of their back window to see a beautiful female deer eating from their bird feeder, just 30 feet away. The entire family was involved in a bird counting exercise and had noticed, several times, that the bird feeder was “wiped out” overnight. This mystery was now solved.

The entire family agreed that it was hard to be angry at the doe, as deer are such beautiful creatures.

Figure 1.  A Female Deer at the Bird Feeder at Patty’s House

 

It was 6:15 am and it didn’t seem to make sense to go back to bed.  So, Patty stayed up and was off to Ivy U in less than 30 minutes.

Patty had a rather light week as she had guest speakers for her two lectures.  However, she was sitting in for one of the engineering school’s senior professors later in the day.  This fellow prof had asked her to sub for him as he was called to an emergency meeting overseas.  Her topic was manufacturing processes; one with which she felt very comfortable.  But, she had to admit to being a bit nervous sitting in for one of Ivy U’s most famous professors.

As was her usual practice, Patty checked her email first.  After going through the first 5 or 6, she saw an email with the subject header, “Ivy U Professor Wins Prestigious Queen Elizabeth Prize for Engineering.”  As she opened the article, she was stunned as she saw a photo of the professor for whom she was substituting later in the day.  The article went on to explain that this prize was like the “Nobel Prize” for engineering.

As she finished her emails she was relishing the thought of having a less hectic day and week ahead.  Maybe she would even have time to read the Wall Street Journal during a relaxing lunch.  Suddenly, her phone rang, startling her a little.  She picked up the receiver to hear a familiar voice.

“Professor Coleman, this is your most faithful student Mike Madigan,” Madigan cheerfully said.

Madigan was CEO of ACME at large electronics assembly contractor. Patty worked at ACME before becoming a professor at Ivy U. Her husband, Rob, and sidekick, Pete, were also ACME employees, but were now all at Ivy U.  Pete was a research assistant and Rob was just becoming a research professor.  Although they all enjoyed their time at ACME, they were much happier at Ivy U.  All three had a part-time consulting contract with ACME and Madigan was typically their main contact at their former employer.

“Mike! What’s up?” Patty said cheerfully.

“We are evaluating a new solder paste and I’m concerned we might make a mistake if we switch,” Mike responded.

“How so?” Patty asked.

“Well, we agreed that consistency in the transfer efficiency (TE) of the stencil printed deposits was the most important criteria,” Madigan began.

“That sounds reasonable as most of our past work has shown that a consistent TE is a strong determinant of high first-pass yields,” Patty responded.

“Right! But the difference between the pastes is only two percent. The old paste has a Cpk of 0.98 and the new paste 1.00,” Mike went on.

“I sense there is more to the story,” Patty suggested.

“Yeah. The new paste has a poorer response to pause,” Madigan said.

“Yikes!” Patty almost shouted.

Patty had shown, time and time again, that poor response-to-pause in the stencil printing process can hurt productivity and lower profitability considerably.

“My sense is the two percent difference in Cpk, might not be significant,” Mike suggested.

“Mike, I think you are on to something. What printing specs were you using and how many samples did you test?” Patty asked.

“The lower TE spec was 50% and the upper 150%. We tested 1,000 prints,” Madigan answered.

“Let me do some homework and I’ll get back to you,” Patty said.

“One problem. Can you get back by 3 pm today? The new solder paste supplier is coming for a meeting at 4PM and is pressing us,” Mike pleaded.

“OK. Will do,” Patty said, sighing a bit.

“There goes my somewhat relaxing day,” she thought.

It was a good thing she had already prepared her lecture and that it was scheduled for 4:30PM.

For several hours Patty thought and searched through some textbooks on statistical process control.  Finally, she came upon the solution to the problem in Montgomery’s Introduction to Statistical Quality Control.

“Perfect!” she thought.

She did finish early enough that she could read the WSJ over lunch, marveling, as always, that she was the only person her age that enjoyed reading a real newspaper.

She called Madigan at 3 pm.

“Mike, I think I have your answer.  I found a formula to calculate the confidence intervals of Cpks,” Patty started.

“And the answer is?” Madigan asked expectantly.

“The Cpk 95% confidence interval on the new paste is 0.95 to 1.05, however the old paste is 0.93 to 1.03,” Patty began.

“So, even I can sense that they aren’t different,” Mike commented.

“Yes, since the confidence intervals overlap, they are not statistically different,” Patty agreed.

Figure 2. The Confidence Interval of the Cpk on the New Paste is 0.95 to 1.05.

 

They chatted for a while and Madigan asked if Patty could join the first 20 minutes of the meeting by teleconference.  It was a bit close to her lecture start time, but she agreed.

Patty had met Madigan’s son at West Point when she visited there to be an evaluator for a workshop two years ago.  She decided to ask how he was doing.

“Mike, how is your son doing at West Point?” she asked.

“Thanks for asking. He is now a Firstie and was in the running for First Captain, but he just missed it.  It’s a good thing he takes after his mom,” Madigan proudly responded.

“Wow! That’s great,” Patty replied.

“I have to admit though, my wife and I are a bit nervous. He has chosen armor as his branch and there is a good chance he will see combat sometime in his career,” Madigan responded with a bit of concern in his voice .

They chatted for a while more and Patty was touched to see so much humanity in Mike Madigan.  He seemed much changed from his gruffness of earlier years.

Cheers,

Dr. Ron

As always, some of this story is based on true events

 

Cpk is Still King in Evaluating an SMT Solder Paste Printing Process

Posted on by

Folks,

If you think about it, to evaluate any process you typically want to know its precision and accuracy. Look at the dart players in the Figure 1 below. The yellow player has good precision, but his accuracy is off. The green player has such poor precision, it is hard to tell if his accuracy is good. The yellow player will typically be easier to correct, as she just needs to change her aiming point.

Figure 1. The yellow player has greater precision. She only needs to change her aiming point.

 

 

 

 

 

 

 

 

 

 

Recently I was asked to evaluate several solder pastes to determine which printed better. We used transfer efficiency (the volume of the stencil printed solder paste “brick” divided by the stencil aperture volume) as the evaluation metric, expressed in percent. So 100% would be the target. The lower specification limit we choose was 50% and the upper specification at 150%.

Figure 2. Data from Pastes A and B.

 

A good result would be an average of 100% with a “tight” distribution. The “tightness” of the distribution being determined by the standard deviation. Figure 2 shows data from two pastes. Note that Paste A has an average of 100% and a standard deviation of 16.67%, whereas Paste B has an average of 80% and a standard deviation of 30%. Clearly, Paste A is superior to Paste B in both accuracy and precision. But what is the best way to express this difference? Is there one metric that will do it? Cpk is the answer.

Cpk is one metric that is sensitive to both the accuracy and precision. Cpk is defined as:

 

 

 

Where x is the average and S is the standard deviation.

Using these equations, we see that the Cpk of Paste A is 1.0, whereas the Cpk of Paste B is 0.333. Note that Paste B has a significant number of data points (about 17%) outside of the specification limits, however, Paste A has almost no data points out of specification.
So when evaluating most processes, Cpk tells it all!

Cheers,
Dr. Ron

 

Conclusion of In Electronics Manufacturing, Does Cpk =1 Yield 66,800 DPM?

Posted on by

Patty, Rob, and Pete were quite sure they understood the confusion in the Cpk = 1 issue, but wanted to make sure they discussed it with the Professor.  After a brief chat with him, they called ACME CEO Mike Madigan from The Professor’s office.

“Professor, it’s great to speak with you again,” Madigan began.

The all exchanged pleasantries, with the Professor thanking Madigan for his financial support of Ivy U through the ACME Corporation.  In a few moments the discussion turned to the Cpk = 1 issue.

“Tell me what you amazing intellectuals have figured out,” Mike chuckled.

“We all thought the article that the vendor referred to had a great discussion on statistical process control (SPC)”, Patty began.

“We especially liked the discussion on the difference between a process being in ‘control’ and ‘capable,’” Rob added.

“But, what about 66,800 ppm equals a Three Sigma process?” Mike implored.

“As we know, Motorola started the ‘Six Sigma’ movement,” the Professor began.  “They defined ‘Six Sigma’ quality has having a Cp of 2 and a Cpk of 1.5.  True mathematical Six Sigma is Cp=Cpk=2.  Their definition, with a Cpk = 1.5, allows for a shift in the mean of 1.5 Sigma.  The adage that ‘Six Sigma’ equals 3.4 ppm defects comes from this definition.  Because of this shift, most of the defects are on one side of the distribution.  By the way, true mathematical Six Sigma is about 2 defects per billion,” he went on.

“It seems a little like cheating to me,” Madigan added.

“Me too. I think they wanted something sexy sounding, like ‘Six Sigma,’ but knew they couldn’t really achieve less than 2 ppb defects, so they created the 1.5 sigma shift of the mean,” Pete chimed in.

“I’m sure that others agree with Pete, but that is where the world of ‘Six Sigma’ is.  Unfortunately, it can create confusion – as in the case at hand,” the Professor responded.

“So how does it relate to the 66,800 defects per million equaling a Cpk of 1 and a Three Sigma process?” Mike asked.

“Pete has done the most work on this. Let’s let him answer,” the Professor suggested.

“If you apply the 1.5 Sigma shift of the mean to process capabilities, we get the table below,” Pete said.

 

 

 

 

Note that the Cpk level for 66,800 dpm is 0.5 not 1 and the true process level is not Three Sigma, but 1.5 Sigma.  Admittedly the Cp level could be 1, but Cpk is a precise calculation and the graph from the paper in question (reprinted below) has it wrong.  The values they list for Cpk are the Cp values.  This is the mistake your vendor made by using this chart. ” Pete said.

 

 

 

 

 

 

 

 

 

“The graph below shows the situation for the vendor.  Distribution A has a Cp and Cpk =1, where as distribution B has a Cp = 1, but a Cpk of only 0.5.  The 1.5 Sigma shift for B is also shown.  The vendor’s data are similar to B, with its the 66,800 dpm..  It is improtant to note that Cp alone tells nothing about the defect level,” Pete went on.

“Pete, please tell Mike about the spread sheet you made,” Patty suggested.

They had signed onto Webex, so Pete gave a limit demo.

“By entering the spec limits, as well as the mean and sigma of the data, it will calculate Cp, Cpk, the sigma limit of the process, and the process dpm,” Pete said.

 

“Oh, and you can enter the dpm and it will estimate the Cpk and process sigma level,” Pete went on.

“Quite impressive,” Madigan summed up. “I assume it is OK if my team uses it?” he went on.

“Sure,” Pete said, beaming a little.

Math was never Pete’s strong suit. But, being at Ivy U, he had recently taken a statistics and calculus class. He had a strong sense of accomplishment after creating this useful spreadsheet.

For those who would like a copy of Pete’s spreadsheet, send me an email at [email protected].

The False Positive Paradox

Posted on by

Folks,

Let’s check in on Patty…

Patty was intensely preparing a lecture on Bayes’ Theorem. She always felt that this theorem was the most profound in probability and statistics. She remembered a real application, when her best friend took the Tine test for tuberculosis before she got married – and tested positive. The test claimed to be 99.9% accurate in identifying someone with TB. Her friend was devastated to find out that she apparently had this ancient, dreaded disease. Further investigation uncovered that the 99.9% number was more accurately stated as, “if you have the disease, this test will pick it up 99.9% of the time.” There was an important number not told: false positives. This rate was 5%. With so few people having TB, a 5% false positive rate would indicate that almost everyone that tested positive for TB, would be a false positive, hence not have TB. So it was, much to the relief of many, with her friend. This situation is an example of the false positive paradox.

While Patty was deep in thought, she was startled by the sound of her phone ringing. She looked at the area code and exchange and knew it was from her old company, ACME. She picked up the phone.

“Professor Coleman,” Patty answered. She liked the sound of that.

“Hey, Patty! It’s Reggie Pierpont!” the cheery voice declared.

Patty’s heart sank. Reggie was an OK guy, but he always got involved in things he didn’t understand and often convinced management to pursue expensive and ineffective strategies. He was that persuasive.

“Reggie, what’s up?” Patty said half-heartedly.

“Well, Madigan insisted I call you before we order some new testers. I think it is a waste of your time, but I’m following orders,” Pierpont said.

“What are the details?” Patty asked.

“We have a contract to produce one hundred thousand Druid mobile phones a week. We are confident our first pass yield is greater than 99%,” he began.

“Impressive,” Patty said with sincerity.

“I want to order some testers that identify a defective phone in a rapid functional test with 99.9% certainly. The testers are very expensive, so Madigan wants a sanity check before buying them. The other important info is that we get a huge penalty from the customer for any defected phone we ship,” Reggie continued.

“Well, with a large penalty, 99.9% is the right number. What do you do with the units the tester determines are defective?” Patty asked.

“Well, it is a good thing yields are high. The phones are so complex that we have quite a drawn out process to find the defect and fix it. Just finding a defect can cost $5 to $10 dollars in burdened labor, but, considering the value of a phone, it’s worth it. Like I said, it’s a good thing yields are high so we don’t have too many units needing this procedure,” Pierpont continued.

“What about false positives by the tester?” Patty asked.

“Shouldn’t be a problem, remember the tester is 99.9% accurate,” Pierpont answered.

Patty knew that Pierpont was missing her point, but she didn’t want to embarrass him……too much.

“Reggie, from what you told me, if a unit is defective the tester will catch it 99.9% of the time. What I am asking is, if a unit is good, how often does the tester say it is bad? This situation is usually called a ‘false positive’,” Patty responded.

“Well, it would be 100 – 99.9 or 0.1%,” Pierpont replied.

“That’s the percentage of bad units that would be called good. These units are often called ‘escapes.’ The only way to determine false positive rate is by a test, you can’t determine it from the 99.9% number,” Patty went on.

There was silence at the other end of the phone.

“What do I need to do to get the false positive number?” Reggie asked.

“You need to test about a 1,000 known good units and see how many the tester says are bad,” Patty said.

“I’ll do that with the loaner tester the tester company is letting us use and get back to you,” Pierpont replied.

Patty hung up the phone. She thought it interesting that Pierpont’s problem was so closely related to both Bayes’ Theorem and her friend’s false positive with the Tine test.

Two days went by and Patty, Rob, and Pete had just returned from lunch with the Professor. They would all meet with him quite often to discuss technical problems they were having. So, they offered to treat him to lunch.

As she walked into her office, Pete spoke up.

“Did Reggie Pierpont ever get back to you?” Pete asked.

“No, maybe I’m off the hook,” Patty chuckled.

At that instant, her phone rang. It was Pierpont.

“Hey, Reggie! What’s up?” Patty asked with more enthusiasm than she felt.

“Well, the tester says 5% of the good units are bad, I think you are going to tell me this is a problem,” Peirpont began.

“What if you run them through the tester again?” Patty asked.

“That IS running them through two or more times! If we run them through just once, it was 7%,” Reggie sighed.

“Well, let’s look at the numbers. You are making 100,000 units a week, with a 5% false positive rate that’s 5,000 units. Your yield loss is 1% or 1,000 units. So, you will have about 6,000 units the tester will declare as bad when only 1,000 really are. These numbers are off a little bit. Bayes’ Theorem would give us the precise numbers, but these are very close. Since your process to analyze fails after the tester costs at least $5 per unit, you will be losing $25K per week due to false positives,” Patty elaborated.

“Time for a new strategy,” Pierpont sighed.

Patty and Pete agreed to help Pierpont work with the tester vendors to develop a better strategy.

Epilogue

Patty and Pete helped Pierpont develop an effective test strategy working with a tester vendor. Neither Patty nor Pete had known Reggie well before… but, after this joint effort, they grew quite close. Reggie became quite engaged in the process and seemed to learn quite a bit. Patty was able to use some of the data in her classes.

A few weeks later she got a beautiful card in the mail. She opened it. It read, “Dear Patty, Thanks for all of your help. We wouldn’t have made it without you and Pete helping us with our testing strategy. Best Regards, Your faithful student, Mike Madigan.”

Patty got a little choked up.

Cheers,

Dr. Ron

Weibull Analysis of Solder Joint Failure Data II

Posted on by

Folks,

Last time we introduced Weibull analysis. Let’s derive the relationships needed to calculate the slope, beta, and characteristic life, eta.

 

F(t) is the cumulative fraction of fails, from 0 to 1. By choosing Ln(t) as x and LnLn 1/(1-F(t) as y, we would expect a straight line. See the derivation above. It can be shown graphically that this fact is so. So if we plot F(t) versus t on logarithmic graph paper, the slope of the line will be beta. To determine eta, let t=eta, in the first equation below. The result is F(t) = 1-e-1 = 0.632. So the time at which 63.2% of the parts have failed, is eta, the characteristic life.

Let’s consider some data comparing SAC305 and SACM (SAC105 with about 0.1% manganese) BGA solder balls in thermal cycle testing. The primary test vehicle employed was a TFBGA with NiAu finish mounted on PCB with OSP finish. SACM is a new breakthrough soldering alloy that has better drop shock resistance than SAC105 and comparable thermal cycle performance to SAC305. The data follow. The first column is the sample number, the third and fifth columns are the number to thermal cycles to fail for SAC305 and SACM. The second and forth columns are rank of the sample number. One would think that the first number in the second column would be 100*(1/15) =6.67%, as it represents the cumulative percent of samples failed, but a slight correct factor is needed. By plotting the log log of rank as shown above (LnLn1/(1-F(t)) vs log of cycles at failure, we get the Weibull plot. The slopes of the best fit line is equal to beta and the number of cycles at rank = 63.2% is eta.

Fortunately software like Minitab 16 does the plotting and calculating of beta and eta automatically. The results are below:

We see that the shape (beta) for SAC305 is 1.76 and that of SACM is 6.09, the scale or characteristic life (eta) is 1736.8 and 2016.8 respectively.

These results are a strong vote of confidence for SACM. Its steep slope (high beta) suggests a tighter distribution, with more consistent solder joints and its characteristic life (eta) is also slightly greater.

I plan on teaching detailed workshops on this topic. I will keep you posted.

Cheers, Dr. Ron

An SPC Rx

Posted on by

It was Wednesday evening and I had just finished a brief pitch on applications of SPC to a group of 20. I was followed by Jim Hall, who spoke of process mapping using SIPOC.  So did these folks have solder paste under their fingernails, or wave solder flux stains on their shirts, or, perhaps, a solder preform or two stuck in their pant leg cuff?

No — none of these souls would have had any of this type of trace evidence of electronic assembly on their person. You see, they were all medical doctors and students at Harvard’s famed medical school (see image below).  (I hope it is OK that the proud dad shares that my daughter Jessica is a colleague of these folks.) 

Jim and I were presenting to the doctors, because they are interested in process optimization in the healthcare industry. The event was hosted by Dr. Andy Ellner.  He is a professor and doctor at the medical school and is a focal point for these process improvement efforts. I was introduced to him in the summer of 2009 by Dartmouth’s new President Jim Kim.

In November 2009, Jim, our colleague Larry Parah, and I facilitated Andy’s team in dramatically improving the prescription refill process in Brigham and Women’s Hospital Clinic.  Jim and I plan on working with Andy in similar efforts over the next year or two.

The most striking thing that Jim and I left with on Wednesday evening was how profoundly interested these doctors and students were in healthcare process optimization. The Q&A session lasted nearly an hour.

Ah, yes, would that our many colleagues in electronic assembly were as interested in optimizing their processes!

Cheers,

Dr. Ron