Statistically Significant vs. Practically Significant in SMT Data Collection

Folks,

Let’s assume your company has decided that transfer efficiency (TE) is the key metric in determining solder paste quality. Transfer efficiency is the ratio of the volume of the solder paste deposit divided by the volume of the stencil aperture. While you agree that TE is an important metric, you are a little troubled with the recent results in a solder paste evaluation. Two out of 10 pastes are fighting for the top spot and it looks like TE will be the deciding metric. Paste A had a TE of 99.5% and Paste B had a TE of 99%. So management wants to go with paste A. You are troubled because paste A has a poor response-to-pause. If it is left on the stencil for 15 minutes or more the first print must be discarded. This weakness may result in 30 minutes or so of lost production time in a 3-shift operation.

However, the TE test results showed that the TE of paste A was statistically significantly better than paste B. You think about this situation and something doesn’t make sense — 5% and 99% are quite close.

You dust off your statistics textbook and review hypothesis testing. Then it hits you, with very large sample sizes, means that are closer and closer together can be statistically significantly different.

The data show that paste A has a mean of 99.5% and a standard deviation of 10%, whereas paste B has a mean of 99% and also a standard deviation of 10%. The sample sizes were 10,000 samples each. These large sample sizes are important in the analysis. The standard error of the mean (SEM) is used to compare means in a hypothesis test. SEM is defined as the standard deviation (s) divided by the square root of the sample size (n):

So as the sample size increases, the SEM becomes smaller or in statistics lingo “tighter.” With very large sample sizes, this tightness enables the ability to distinguish statistically between means that are closer and closer together. This situation was not a concern with sample sizes of less than 100, however with the modern solder paste volume scanning systems of today, sample sizes greater than 1000 are common.

Figure 1 shows the expected sampling distribution of the mean for samples with a TE of 99.5% and 99.0% and a sample size of 100, both have a standard deviation of 10%. Note that to your eye you do not see much difference. However, with the means and standard deviations the same and sample sizes of 10,000 the sampling distributions of the mean are clearly different in Figure 2.

The reality though, is that there is no difference in the results in Figure 1 and 2. The tiny difference in the means (0.5%) may be statistically significant with a sample size of 10,000, but is it practically significant? Would this small difference really matter in a production environment? Almost certainly not.

Figure 1. Sampling distribution of the mean for a sample size of 100.
Figure 2 Sampling distribution of the mean for a sample size of 10,000.

So, with large sample sizes, we need to ask ourselves if the difference is practical. For TE, I think we can be confident that a difference of 0.5% is not practically significant. But, what if the difference was 2% or 5%? Clearly, experiments should be performed to determine at what level a difference is significant.

With the case discussed above, I would much prefer the paste that has a 99.0% TE and a good response-to-pause.

Cheers,

Dr. Ron

Dispelling The ‘Five Ball Rule’

Michel writes:

Dr. Ron, when if comes to SMT printing of solder paste, why do some people use the five-ball rule for rectangular apertures and the eight-ball rule for circular apertures?

Michel:

The “Five Ball Rule” is another metric that SMT assembly industry leaders believe, but it is difficult to find its origin. It states that when selecting a solder paste, five of the largest solder balls should be able to fit across the width of the smallest rectangular stencil aperture. See Figure 1a for a 0.2mm wide rectangular aperture.

Typically, the largest solder ball diameter is assumed at the 90th percentile. See Figure 2. So, in this example, a type 4 solder paste would fit the five ball rule as the largest solder ball is 0.038mm. Five times 0.038 is 0.190mm, just a little less than the aperture width of 0.2mm. It should be remembered that this is a “rule.” not a “law.” So let’s say you had 4.5 balls across the aperture with instead of 5, it would most likely be OK. 

Figure 1. A comparison of the Five and Eight Ball Rules

Figure 2. Solder Powder Sizes

A generation ago, the advent of circular apertures to support BGA and CSP packages necessitated a new “rule.” Figure 1b shows why the five-ball rule is inadequate for circular apertures. Although five type 3 solder balls fit along the 0.275 diameter, off the diameter, there is not enough room for many solder balls.  Hence, an insufficient amount of solder paste would be printed.

For the same aperture, if a type 4 paste is used, 7 or 8 solder balls span the diameter and the amount of paste printed would be much closer to the volume of the aperture.

For a little more on this topic, see a past post.

Cheers,

Dr. Ron

Submit an Abstract to SMTA Pan Pac

Folks,

This coming February will be my third SMTA Pan Pac. Pan Pac is a very enjoyable and rewarding conference. It is small enough that you can get to know all of the speakers, yet large enough that there is a full venue. For those of us in the northern part of the US, it is also a nice break from the winter weather. The first time I went I was surprised that it wasn’t very expensive. For this coming conference, air tickets from Boston are as low as $600 and the hotel is about $200 per night.

The conference will be held on the “Big island” of Hawaii. If you come early or stay late there are many interesting attractions, including the active volcanoes and the Mauna Kea Observatories. So for sure come to the conference, but why not submit an abstract to be a speaker? If interested in submitting an abstract go to this site.

Cheers,

Dr. Ron

A view of part of the Big Island.

Selecting Reflow Oven Length

Folks,

You are putting in a new assembly line to assemble some large boards for which your company just received a three-year contract. The boards are 45cm long and you expect the cycle time from the component placement machines to be 40 seconds per board. Your boss is pressuring you to get another 5-zone oven, as they are cheaper and take up much less space than a 7- or 10-zone oven. But, you are concerned that a 5-zone oven may not have the capacity that is needed to keep up with the component placement machines. Let’s make some calculations and see if your concerns are justified.

Table 1 shows some typical reflow oven metrics:

Let’s assume that you will be using a typical modern SAC solder paste. By studying the reflow profile above, we see that the amount of time needed in the heated zone is about 4.5 min. or 270 sec.

So if we choose the 5-zone oven the belt speed will be:

Belt Speed = BS= Heat Tunnel Length/Time in Heated Tunnel = HTL/Time = 180 cm/270 sec. or 0.66 cm/sec

The component placers will be presenting a 45cm board every 40 sec., so the belt speed needs to be:

BS = Board Length/Cycle Time = BL/CT = 45cm/40 sec = 1.125cm/sec

So clearly a 5-zone oven won’t work. What about a 7-zone oven? Let’s calculate the belt speed for this oven.

BS = HTL/Time = 250cm/270 sec. or 0.926cm/sec

Now we can see that the 7-zone oven won’t do the job either.

How about the 10-zone oven? Let’s see if the belt speed is greater than the 1.125 cm/sec needed.

BS = HTL/Time = 360cm/270 sec. or 1.33cm/sec

Success! Since 1.33cm/sec is greater than 1.1125cm/sec, this 10-zone oven will work. The extra belt speed will permit a small amount of spacing between the boards. Let’s calculate what it will be:

BS = (BL + Spacing)/CT = 1.33cm/sec => BL+ Spacing = BS x CT => Spacing = BS x CT – BL

Spacing = 1.33cm/sec. x 40 sec – 45cm = 53.2cm – 45cm = 8.2cm

To summarize: For our 45cm board that has a cycle time of 40 sec., we need a 10-zone oven with a heated tunnel length of 360cm. There will be an 8.32cm spacing between the boards in the oven.

If you would like an Excel spreadsheet to make these calculations send me an email at rlasky@indium.com.

Cheers,

Dr. Ron

SMT Workshop Pre-Test

Folks,

Six months ago …

Patty had just finished an all day workshop on “Common Defects in SMT Assembly and How to Minimize Them.” The workshop seemed to go really well, and many of the 35 or so attendees thanked her for a great learning experience.

After most of the people filed out of the room, two approached her as she was disconnecting and packing her laptop.

“Dr. Coleman, that was a great workshop. But, I do have one question. You used a term all day that I wasn’t familiar with, ‘SAC’,” a 35-year-old process engineer commented to her.

While saying this, he presented his business card that referred to him as a “Senior Process Engineer.”

Patty was trying to recover from this shock, when the second similar looking fellow asked, “And what are ‘OSP’ and ‘eutectic’.”

After explaining these three terms and exchanging a few pleasantries, the two senior process engineers walked out of the room and bade Patty farewell. As the room became empty, Patty settled into a chair.

“How can this be?” she thought. She was stunned that people with enough experience to be called “senior process engineers” would not know these terms.

Today 6 AM …

Patty was jogging back to her house in Woodstock, VT, when she spied a beautiful red fox. Neighbors had reported seeing the fox numerous times. People believed that the fox was nesting. In addition, a black bear had been sighted by everyone in her family over the past few weeks. Add all of this to the family of deer and the rafter of turkeys in her neighborhood and it was quite an experience for Patty, Rob, and their sons.

The fox, however, created a new problem. Patty and Rob had bought their twin sons a Yorkshire puppy, Ellie, about a year ago. At 6 pounds she could be dinner for the fox, so, unfortunately, they could no longer let Ellie out by herself.

Figure 1. Ellie the Yorkie after a big day. Sadly she has to be watched when she goes outside of Patty’s house, due to the local predators.

By 7:30AM Patty was in her office. She was giving a workshop in two weeks at a local chapter meeting in Boston and decided to create a pre-test to give to the attendees so that she could assess their current knowledge. Patty planned on having the students grade each other’s exams and on working the exam in as a leaning experience at the start of the workshop. By assessing the results of the pre-test, she wanted to make sure she didn’t use acronyms they don’t understand, and to also explain topics that the students might not be familiar with. As she was working on the questions for the pre-test, Pete walked in.

“Hey, Professor C, how goes it?” Pete asked.

“I’m preparing a pre-test for the workshop I’m giving in a few weeks,” Patty replied nonchalantly.

“I remember you talking about doing it a month or so ago. Seems like a good idea to me,” Pete responded.

“I’m ,glad you approve,” Patty said wryly. “I just finished it. Do you want to take a look at it?” she continued.

Patty printed out a few copies and handed one to Pete. They both looked at it for a few minutes, in silence.

Finally, Pete commented sheepishly, “Aaa, Patty your joking, right?”

“Why do you say that?” Patty asked, a little annoyed.

“It’s just too easy. Everyone will get 100% and you won’t get any information,” Pete opined.

Patty then reminded Pete of her experience 6 months ago.

“OK. Maybe you have a point. But, I still think it’s too easy,” Pete concluded.

“I’ll tell you what. How about a bet? If the average pre-test grade is above 70%, Rob and I will take you and your new crush, Mary, out to Simon Pearce. If it is 70% or less, you treat us,” Patty teased.

“It’s a bet,” replied Pete quickly.

The Pretest:

  1. What does the letter “S” in SAC stand for?
  2. How much silver is in SAC305?
  3. What is the approximate melting point for SAC305 solder (+/- 4oC)?
  4. Solder paste is approximately how much (by weight) metal (+/- 5%)?
  5. What is not a current common defect in SMT?
    1. Head-in-pillow
    1. Pad cratering
    1. BGA Ball Matting
    1. Graping
  6. Which is a closest to typical stencil thickness?
    1. 5 microns
    1. 20 mils
    1. 5 mils
    1. 20 microns
  7. Which is closest to a typical lead spacing for a plastic quad flat pack (PQFP)?
    1. 0.1mm
    1. 0.1mil
    1. 0.4mm
    1. 0.4mils
  8. Which has finer solder particles, a Type 3 or 4 solder paste?
  9. What does OSP stand for?
  10. Place an arrow at the eutectic point of the tin-lead phase diagram below.

Epilogue (two days after the workshop)

Patty arrived at Ivy U and couldn’t wait to see Pete. She went to his office but he wasn’t there. Finally, she found him in the machine shop helping four students with a project that required some additive manufacturing.

“Hey, Pete! When are you and Mary going to treat us to our dinner?” Patty teased.

“Don’t tell me the average was less than 70%,” Pete grumbled.

“Forty-three point zero eight to be exact,” Patty punctuated.

Figure 2. The Pretest Scores

“Yikes!” Pete exclaimed, rubbing the back of his neck. “I guess you were right.”

“It really helped me to take things slowly and explain all the terms. I think I helped the students much more than usual,” Patty explained.

“Rob and I both agreed, we are ordering the most expensive meal that Simon Pearce has,” Patty joked.

At that Pete let out a deep groan.

Dr. Ron note: All of the events in this post are true. How would you do on the pretest?


Become a Part of Patty and The Professor!

I have enjoyed writing the Patty and the Professor blog for about 10 years now. I’ve written about numerous real-life electronics assembly examples that I have encountered in my career, all disguised, of course.

To continue keeping things real, and to keep my readers involved, I am inviting you to submit an authentic story from your career. That’s right! You’re being invited to submit an idea, story, or experience that can be built into the Patty & The Professor series.

Your experience will help many other electronics assembly practitioners resolve their issues and avoid problems.

So, get your thoughts together, then shoot me an email at rlasky@indium.com. Share the details of your experience or observation. I may ask a few questions to help me comprehend the full story. Then, I will write up the segment and let you read it before posting. You will be credited, of course.

Bonus: You will also receive either a Dartmouth hat or coffee mug (similar to, not exactly like, those pictured below)!

Contact me if you are interested in submitting a story. I look forward to hearing from you!

 Cheers,

 Dr. Ron

Self-Driving Cars Decades Away Means More Electronics Will Be Needed

Folks,

Recent articles have added to the confusion regarding when fully autonomous vehicles will become common. One suggests that they around the corner with this quote:

“Alphabet plans to launch a self-driving service later this year, while GM Cruise has targeted the introduction of a similar service in 2019. Ford has that it expects to put self-driving vehicles into commercial service by 2021.”

So it sounds like autonomous vehicles will be here this year or next. But wait, here is a counter article. This article points out the many issues to be resolved before fully self-driving cars are launched. Consider this one quote from the article:

“There’s still a lot to be worked out. There are scenarios where the car will have to break the law in order to proceed. One common scenario is, you’re driving down a two-lane highway—one lane each way—and there’s a Fed Ex truck in front of you, parked on the curb. You can’t go around it without crossing the double-yellow line. Are you going to allow the car to break the law? Now, you’re getting into a whole different set of rules, regulations, and even morality decisions.”

These two perspectives were brought home to me recently when I was on a review board for a student projects course, Technology Assessment, taught by friend and colleague, Eric Bish. One of the projects was to assess the viability of bringing fully autonomous vehicles to market by 2021. Reviewing this project helped to clarify the dichotomy between the two perspectives discussed above.

It ends up that the efforts of Alphabet, Ford, and GM are to be launched in very controlled environments. They will only be used in well mapped out routes, with good lane markers, no construction, on days with good weather etc. Note also that the first quote refers to a self driving service, not private autos.

Having an autonomous vehicle that can completely replace a human is still (many?) decades away. There are just too many issues such as the FedEx scenario envisioned above that need to be resolved. I believe that over time, more and more such issues will be discovered and push the date of such vehicles even farther in the future.

Even if, on the whole, early autonomous vehicles are safer, accidents like the one in Phoenix earlier this year, will put a spotlight on autonomous vehicles that will further delay their full advent.

What does all of this portend for the electronics industry? I think these issues will require more electronics and sensors than many believe, so in a sense it is good news for the electronics industry.

Cheers,

Dr. Ron

The Professor’s second visit to ACME … continued

.

“Well what should we do Professor?” John said weakly. 

“Clearly, not shut the line down over the lunch break,” The Professor responded quickly. 

“We can’t!” said John, “The operators are all friends and they count on having lunch together.” 

“How much are they paid per hour?” asked The Professor. 

“Ten dollars,” replied John. 

 “You can pay them $15 per hour and still make more profit if they keep the line running over the lunch break,” The Professor opined. 

“Fifteen dollars per hour for the lunch time or the entire 40 hour week?” John asked nervously. 

“For the whole week,” was The Professor’s reply. 

“I find that hard to believe,” John shot back.

“Consider this,” said The Professor. “Your line is up only 9.7% of an 8 hour shift, that’s only 47 minutes. Today you lost 95 minutes over the lunch hour. You may be able to increase your uptime to greater than 15% by keeping the line running over lunch. I modeled your business with ProfitPro3.0 cost estimating software. Your company will make millions more per year if you keep the lines running over lunch. I have worked with other companies to make this change; it is really easy with a 30 minute lunch period. If 5 people normally run the line, you have just one stay back during lunch. That way each person only misses the regular lunch break once a week.”

John thought optimistically, “There is such a thing as a free lunch.”

“Now, let’s talk about what we can do to double the uptime from the 15% we will get by running the lines over lunch,” said the Professor.

Patty listened to all of this in amazement. The Professor was helping ACME more than she thought possible.

Next steps? Yes, John will keep his job. But, what is The Professor’s plan to get uptime to 30% or more? And, we still haven’t learned where Patty will go to dinner.  Stay tuned for the latest.

Cheers,

Dr. Ron

Dr. Ron note:  As surprising as this may seem, this story is based on real events. The uptime numbers and improvements are from real examples. Any company that can achieve 35% or more uptime can compete with anyone in the world, even in low labor rate countries. Sadly, few companies know their uptime or have an urgency to improve it.

Best Wishes,



The Return of Patty and the Professor: Uptime Part 2

Folks,

For the next few weeks I plan to repost some of the first Patty and the Professor episodes. As I visited several facilities, some of them in other industries, I found that uptime is as vital a topic as ever. Although these facilities were tracking a few metrics, uptime was not one of them.  I estimated they were little better than ACME in the following vignette. Let’s all be committed to measuring and improving our processes uptimes. Now on to Patty and the Professor.

Two weeks passed quickly and The Professor returned to ACME. Patty met him at the door. “Professor, it’s great to see you,” Patty said with enthusiasm. “We collected the uptime data in real time on a laptop, no one has seen that results yet. We wanted it to be a surprise,” said Patty. The Professor suggested that he go out on the shop floor to observe the manufacturing activities until shortly after lunch. He pointed out  that his observations may help to understand the uptime results.

The morning seemed to drag for Patty, she was very anxious to see the resets of the uptime data. She bet Pete a dinner for two that the uptime would not be more than 50%. If she wins, Pete and his wife will treat her and her boyfriend Jason to dinner at the restaurant of her choice.

Around 1:30 p.m. The Professor suggested that he was ready for the meeting. Patty had written a simple Excel macro to perform the calculations for the uptime. She only had to push a button and he whole room would see the result in a moment, as Patty connected her laptop to a projector. There was tension in the air, friendly wagers had been made, but the entire process team realized that their reputation was on the line.

When the number emerged on the screen, John, the manager’s face became ashen. Pete’s visage was redder than two weeks ago. John thought, “I should be fired. How could I manage this team for five years and not know that our uptime was only 9.7%.” Patty was thinking about her choice of restaurants.

“How can we be so bad?” John asked The Professor. The Professor responded, “The good news is that there are tremendous opportunities for improvement. After observing the operations out on the floor this morning, I think we can get the uptime to greater than 40%.” Pete shot back, “You’re kidding, only 40%?”

“I’ve only seen two operations that have greater than 45% uptime, and I’ve been to over 150 facilities worldwide,” answered The Professor.

“Where do we start?,” asked John.

“How about lunch?” beamed The Professor.

“We just had lunch!” Pete groaned.

“No, no Pete,” The Professor chuckled, “I mean how lunch is handled out on the line. Lunch costs the company more than 1½ hours of production in an eight hour shift. That’s nearly 20% of the entire shift.”

Now John was a little agitated. “Professor, lunch is only 30 minutes. We purposely have a short lunch period to avoid the line being down for a long time,” John said with a note of annoyance.

“John, this is true, but I watched what the operators did. Lunch is supposed to start at 12 noon, but the operators turn the line off at 11:40 a.m. They don’t get back to the line until 12:40 p.m. and it takes them more than 30 minutes to get the line running again. Today, the line was not running until 1:15 p.m. It was down for 1 hour and 35 minutes,” stated The Professor.

John thought again, “Yes, I should really be fired.”

Will John keep his job? What restaurant will Patty choose for dinner? What should be done about lunch? Where are all of the other hours lost? Stay tuned for the answers to these and other questions.

Cheers,

Dr. Ron

The Return of Patty and the Professor

Folks,

I teach a course at Dartmouth on manufacturing processes: ENGM 185. In this course, I use many of the chapters from “The Adventures of Patty and the Professor.” This book started as a series of posts on this blog and the posts ended up being gathered into the book. It’s hard to believe that the first post was nearly 10 years ago.

I think most students that have read “The Adventures of Patty and the Professor” have a sense that the vignettes in the book are exaggerated, even though I point out that I have attempted to make them as close to real events as possible. Recently, one of my grad students, Amritansh (Amro) Varshney, had a chance to see some of the real world of manufacturing. After Amro returned to Dartmouth, we chatted and he shared that not only do the stories convey the sense of how poor some manufacturing operations are run, but, in some cases, the realities are worse!

In light of this epiphany, I decided to repost some of the original episodes from the book for a new generation of readers. As you share Patty and the Professor’s experiences, remember they are strongly based on real events. I hope you enjoy the “Adventures!”

Business was good at ACME. Even in these challenging times, the company’s three assembly lines could not keep up with demand. John, the manager of the assembly lines, decided to request the funds for an additional assembly line. A member of his team, Patty, suggested he might want to consult “The Professor,*” before getting a new line. The Professor taught a course on line balancing that Patty took at the SMTAI conference last summer. Line balancing is an important part of optimizing productivity in electronics assembly. A balanced line ensures that the component placement process, usually the “constraint,” is the fastest possible by assuring that each placement machine spends the same amount of time placing components. If any machine is waiting for the others, assembly time is being wasted. In a sense, line balancing is an application of Goldratt’s Theory of Constraints. John remembered that when Patty applied what she learned from The Professor, throughput increased 25%. Unfortunately, Patty did not attend The Professor’s other class on “Increasing Line Uptime.”

John decided to have a chat with Patty about The Professor. “Patty, why do you think I should consult with The Professor, about getting a new line?”

“Well John, perhaps with some effort to improve our uptime, we wouldn’t have to buy another line,” said Patty.

“Patty, that’s a good point,” said John.

Patty contacted The Professor and he agreed to fit ACME into his busy schedule. Upon his arrival, The Professor was given a tour. As part of the tour he was shown the process that ACME used to minimize changeover time between jobs. The Professor appeared to be impressed. After the tour, The Professor asked if a brief meeting could be held with the engineers and managers to discuss the situation.

“What is the average line uptime?” The Professor asked the assemblage. There was some hemming and hawing, finally Pete, the senior process engineer replied, “I’d say at least 95%, we work our fannies off out there.” There was a murmur of agreement from the 9 or 10 people in the room. Finally John spoke up, “Professor, what is your definition of uptime?” The Professor responded, “Simply the percent of time an assembly line is running.” Pete again responded that 95% was the right number.

The Professor asked for some production metrics and performed some calculations on his laptop. In a few moments he commented, “From the data you gave me, I estimate that your average line uptime is about 10%.” Upon hearing this, Pete became red in the face, especially after Patty whispered in his ear, “I told you so.” The noise in the room became so loud that John was concerned he might have a riot on his hands. The Professor asked to speak and John, in a booming voice, asked for calm.

“Let’s not become angry, perhaps my calculations are off. Why don’t we measure the uptime for a few weeks to be certain.”

“How do we do that?” asked Pete, his face still crimson.

“Each day one process engineer will go out to the lines every 30 minutes. If the line is running, he will put a 1 in an Excel® spreadsheet cell, if the line is not running a 0 will be entered,” responded the professor.” It was agreed that this will be done and The Professor will be back in two weeks.

Will Pete’s red face return to normal? Will the line uptime be 95%? Will Patty and Pete ever be on speaking terms again?  Stay tuned on May 27 for the next episode.

Cheers,

Dr. Ron

* The Professor, as he is affectionately called by his many students, is a kindly older man who works at a famous university. Few know his real name. The Professor is an expert in process optimization.