Details on study#2 methodology, and the real world

Re: Details on study#2 methodology, and the real w

MikeW

>Hi Lyn,

Please don't misunderstand my comments deep down in the other thread. It was more in response to the other posting.

Let me say that your studies are the most rigorous *and* thorough to date of any I have found. Ever. Period. And I might add, unless *you* take these studies even further, I doubt anyone else will ever beat them for the qualities mentioned above.

I was just hoping to make clear that the issue of repeatibility by non-interested parties or unrelated testors is the main issue.

In other words: Do you believe that I could obtain the same model planes from different sources and replicate your findings to some degree of correlation to yours?

I have not attempted to emulate your test methodology independitely, with no other guidance than a list of said methods. The skeptic in me says I would not be able to replicate your findings across the board or even maybe in a specific test subject to any satisfactory degree of correlation.

If a significant number of test subjects vary beyond what would be an acceptable margin of deviation, it would be inaccurate to label the tests as non-subjective, i.e., scientific.

But, if over a broader independent study using the same methods a significant group had acceptable deviation, then my findings would be the exception.

That was the point to the other post. I'm just more awake now .

Lyn, I find your work fascinating and I hope you continue. I would hope my comments (nonvalidated comments I might add) do not ever discourage you.

I look forward to the day I would be able to test your conclusions for my own interests.

Your brother in the pursuit of blood, sweat and shavings,

Mike

Re: Details on study#2 methodology, and the real w

Lyn J. Mangiameli

>Mike,

I was sincere, you bring up good points, ones that folks should seriously consider, even more so, I must add, when someone just tosses off an opinion based in their experience.

My comments above are an attempt to show how we can move in the right direction, mindful of the legitimate concerns you raise. Alas, we will never come very close to methodological perfection, not only based on the resources an amateur can bring to bear, but because of the varying nature of wood itself. As I rather uninventively have said before and elsewhere, one never planes the same wood twice.

A couple of specific replies to your last post:

"In other words: Do you believe that I could obtain the same model planes from different sources and replicate your findings to some degree of correlation to yours?"

The closest data I can provide for you is to compare my current results to the earlier studies results, and to compare different planes of the same general type to each other. With respect to the former, I got almost identical results to the prior study, when planes and setting were similar with one glaring excetion. The Knight Japanese Bladed Infill Smoother performed far worse in the second study than it did int he first. I think this may be related to the drying out of the Bloodwood wedge, but I didn't replace the wedge for lack of time and a desire for comparable set up between the two studies. With that exeption, the planes performed very, very similarly between the two studies. Sort of Test-Retest reliabilty.

The more important data, I think, has been that virtually identical planes, of different manufactures, have performed almost identically. The S&S and the Shepherd Infills are virtually identical in every respect except who built them. Add to this that I also built a 45 degree Shepherd from a kit. So I had a 45 and 50 S&S, a 45 and 40 degree factory built Shepherd, and I had a 45 degree Shepherd that I built. The results, when averaged over the many wood samples and multiple raters were virtually identical. Now maybe this can be explained that my measures were lacking in resolution, but I would note that these same measures were quite good at distinguishing small differences in effective cutting angle. I'm left to conclude that the results are likely reasonably generalizable, at least with respect to planes of a different type being tuned and used by the same person. Taking off from this, I think the main variance in test reports and user experiences is going to be a result of the wood samples and the tuning and adjustment capabilities of the user, more so than in differences with manufacture.

Another fair point of Mikes:

"The skeptic in me says I would not be able to replicate your findings across the board or even maybe in a specific test subject to any satisfactory degree of correlation."

I'd love to see some of my investigations reproduced, whether or not they replicate the findings. Frankly, so much of my intent has been to offer methodologies that individuals or small groups could utilize if they so choose. My guess is that the relative rankings would be quite similar, thought the individual ratings would surely vary if for no other reason that the different woods involved. But, we won't know if no one ever tries.

Mike again:

"If a significant number of test subjects vary beyond what would be an acceptable margin of deviation, it would be inaccurate to label the tests as non-subjective, i.e., scientific."

I couldn't agree more. Part of the reason why I went to a lot of effort this last time to include multiple samples from a plane type, multiple testers (two still being quite miserable, but you can't imagine how much trouble it was to include even one additional rater-he had a heart attack in the middle and testing was delayed for months), and multiple samples of wood.

Skipping down:

"Lyn, I find your work fascinating and I hope you continue. I would hope my comments (nonvalidated comments I might add) do not ever discourage you.

I look forward to the day I would be able to test your conclusions for my own interests."

Mike, again, Your skepticism is appreciated, in large measure because it looks to ways to validate or invalidate the skepticism in an objective manner. I just wish more folks would consider all information along these lines. What we need is a network of interested individuals and groups, sharing similar and improving methodologies, to generate the "N" which would increase the validity of the findings. In the end, I think I could do a dozen more studies and it would still come down to some folks saying they don't mean anything because only I did them. There are a lot of studies out there that could be really of some value, not only looking to see if mine are replicable, but expanding to look at such things as quality of surface finish vs effective cutting angle vs distance planed.

We obviously don't need this stuff to work wood, but if we wish to advance tool design and our own techniques of tool usage, I think this kinc of information is invaluable (and I might add, rather influential). I know that when I returned to woodworking as an adult, it took me quite a while to be able to seperate out accurate information from pontification. So much of what I try to do, is provide the sort of information I had wished for when I started out (and still do).

Re: Details on study#2 methodology, and the real w

MikeW

>Hi Lyn,

I hope that others will at least think about the information presented, especially in this thread. It is a concise apologetic and clearly shows your passion, your intent and your purpose.

I for one will be glad when the day comes and I am able to review the completed work, to test its theorem, for my own edification.

Take care, Mike

Re: Details on study#2 methodology, and the real w

R.J.Whelan

>Lyn,

Your tests are so far superior to the ones I did back in the Badger Pond days that I'm almost embarrassed to call what I did a "plane comparison".

I had a really gnarly piece of cherry that was as difficult to plane as any timber I've ever seen (except for doussie); I'd hone the irons and and optimize settings (breaker, frog, etc) to the best of my ability and then use each plane in succession (with a trip through the 12" sniper in between each plane so they all started with pretty equal material); then I'd make a subjective judgement based on final surface with no regard for ease of use, etc.

I really appreciate the extremes to which you go to obtain accurate, useable data.

- rj -

BTW - the cherry test board started out at about 100mm thick and is now down to just under 35mm - when it gets a little thinner it's going to become the top for a jewelry box.

Re: Details on study#2 methodology, and the real w

Greg Sloop, Portland Oregon

>

Let me say that your studies are the most rigorous *and* thorough to date of any I have found. Ever. Period. And I might add, unless *you* take these studies even further, I doubt anyone else will ever beat them for the qualities mentioned above.

I was just hoping to make clear that the issue of repeatibility by non-interested parties or unrelated testors is the main issue.

In other words: Do you believe that I could obtain the same model planes from different sources and replicate your findings to some degree of correlation to yours?

Thanks for clarifying that Mike. Perhaps I do misunderstand where you're coming from... :)

I'm no statistician, but let me think out loud here.

You might not be able to produce identical results - not only because of differences in how you scale and utilize the test article (questions and responses) but perhaps it will be mostly an issue of "power." (I suppose it's not "power" exactly, or at all, depending on your knowledge of statistics.) You may see less or more magnitude response in your (or someone else’s) study vs. Lyn's but I suspect, given my own experience in differing total cut angles, that you'd see the same trends Lyn does.

Would the results be exact? Perhaps, perhaps not. But trends (delta/changes) are often of much more value than absolute values. If a number of studies came to the same trend results, then one could really start to bank on the trend conclusions.

Finally, let me make one more statement about "factual" vs. subjective statements. This is NOT directed at you Mike...

Ironically, I think Lyn's viewpoints are a lot more easily criticized because they review facts and observations. If I disagree with Lyn, I can disagree on the facts without impugning Lyn much, if at all. (To some degree, you're criticizing the author of the study, but often it's more their execution of the study you disagree with, which isn't terribly personal.)

Let's take the alternative: Someone simply throws out their opinion, without any basis for said opinion given. Now to criticize this second position, how do you go about doing it? If you object you are directly attacking the judgment and opinion of the opinion holder. How can I criticize their opinion without demeaning them personally?

Lastly, without some attempt at non-subjective observations and factual points, how can one discuss the merits of one viewpoint vs another. It all ends up degrading to the level of; "You're a dope." "Am not!" and I don't think that's very helpful. It's also, IMHO, not fair to the person attempting to work in less subjective observations and facts - the fact less, subjective argument demeans the discussion. (And I'm not completely focusing on the last round of this to any degree. Every time the discussion about the merits of one system vs. another comes up, I see far too much completly subjective opinion come out. It becomes virtually pointless to even discuss it because as I said it degrades in to a and "it is" "it's not" argument which benefits no one.)

Cheers,

Greg

Re: Details on study#2 methodology, and the real w

Joe Rogers, Northern Virginia

>And...which plane will you ultimately decide to use on it? I respect your experience and judgement. If you haven't made a choice yet, that is a valid reply also:-)JR

Plane studies, Science and thoughts of Feynman

bill tindall

>The participants on this forum have on many occasions attempted to apply scientific principals to issues of interest to woodworking, and often with considerable success. These plane studies are yet another example. Greg's thought reminded me of one of the many famous Feynman lectures, this one on "What is Science" A passage from this lecture is particularly germane to this plane discussion. He is speaking about the accumulation of knowledge by the evolving human race................

"So there came a time in which the ideas, although accumulated very slowly, were all accumulations not only of practical and useful things, but great accumulations of all types of prejudices, and strange and odd beliefs.

Then a way of avoiding the disease[the strange and odd beliefs] was discovered. **This is to doubt that what is being passed from the past is in fact true, and to try to find out ab initio again from experience what the situation is, rather than trusting the experience of the past in the form in which it is passed down.** And that is what science is: the result of the discovery that it is worthwhile rechecking by new direct experience, and not necessarily trusting the [human] race['s] experience from the past. I see it that way. That is my best definition. "

It is therefore the nature of scientists to question all results/facts. The questioning is not personal, and it should be expected. It is also the nature of scientists to enjoy defending their hypotheses and then go for a beer with their advesaries. But the process can be alarming to by-standers.

Re: Plane studies, Feynman and learning

Lyn J. Mangiameli

>Ah, Feynman was such a wonder.

Not many know that when I conducted the first study, I really expected that angles above 50 degrees weren't going to make any difference. The data said otherwise, so I've gone with that.

When I started the second study, I kind of took it for granted that the results of the first study would be refined but replicated. They have been up to around 60 degrees, then some planes begin to deteriorate in performance and others go on to yielld even better results at even higher angles. The last thing I anticipated was the how the trend would not hold for all plane types. But again, I go with the data.

Every study (and review) has brought me new knowledge and a new understanding. They have affected the way I work and the planes I personally choose. No better example of this is than to go to Roger Nixon's Traditional Tools site and look at my comparative review of the LV and L-N LA BU planes. You'll see there that at that time, I didn't even conceive of them being used in a high effective cutting angle configuration, let alone have a clue they would eventually become some of my favorite smoothers.

Re: Details on study#2 methodology, and the real w

Lyn J. Mangiameli

>There really is truth in the line that we reach higher standing on the shoulders of those who have gone before us.

And I really know what you mean about using up some of that wood. Near the end of the testing, I finally used up my favorite test piece of Blackwood Acacia. I actually felt a little bad about it. I should have done what you are planning to do.

Re: Details on study#2 methodology, and the real w

Ben Knebel

>Hi Lyn;

Maybe it's late and my mind is in stall mode but I'm confused by the #'s you have in front of the planes--spent all day programming toolpaths in the CNC--makes you go crosseyed.

I've excerpted your infill plane list:

Infill Planes:

#2 S&S (45 & 50)

#6 Shepherd (#2 @45, #1 @ 47.5, 50, 55, 60)

#1 Ray Iles (47.5)

#1 SMT (later confirmed with second SMT)

#1 Knight Japanese bladed infill

For instance #6 Shepherd (#2 @ 45....etc

What do the #6 and #2 mean?

Regards

Ben

Re: Details on study#2 methodology, and the real w

MikeW

>Hi Ben,

"For instance #6 Shepherd (#2 @ 45....etc

I think those are quantities of each listed plane and their bedded angles.

But Lyn will clarify when he sees your Q.

Mike

Re: Details on study#2 methodology, and the real w

Ben Knebel

>Duh--I think you may be right. Told you--too many tool paths --too many numbers--brain sputters and stalls.

Regards

Ben

Re: Details on study#2 methodology, and the real w

Derek Cohen (in Perth, Australia)

>RJ

I used the same procedure when I examined the influence of the chipbreaker (this was shortly after the LN aftermarket chipbreaker came out - Article on the Traditional Tools website) and the article on Bevel up planes (in the Articles section of this website).

My favourite piece of perverted and sadistic Jarrah was used over and over. The good thing about this process is that it is repeatable at home. The problem if anyone else wants to replicate it is that I will have to offer them room and board for a while (grin). Anyone care for a trip downunder?

Regards from Perth

Derek

Re: Quantities

Lyn J. Mangiameli

>

Down Under... 

MikeW

>

Re: Details on study#2 methodology, and the real w

Matti Kuikka

>Lyn,

Encouraged by your attitude I put for consideration my 0.02€ worth.

What you are trying to measure is ultimate(ly) quality that appears in a changing medium, wood, which japanese believe even to have a soul. It is not a countable quantity. When one is approaching frontiers with ever finer tools, measuring differences becomes progressively more difficult what with chatoyance, smoothness as seen or felt etc. My wife even senses that a well planed piece feels warm to touch if it is not oiled. You will get to a point where individual perceptions are the difference. As the results are really fine one feels that magic that makes woodwork so fascinating. My point is that in this situation one can not and should not avoid emotive arguments as the scientific method is to be seen as serving a different search, if I understand your aim correctly.

I myself prefer your rigorous research attitude over intuitive approaches to this question and the results have been quite intersting and helpful but I still don't understand why my japanese high class plane works better than a self-made woodie with a Hock blade and produces a result that is perceptibly finer and more beautiful though I can't measure the difference (even in thickness of shaving).

Respectfully,

Matti

P.S. In the snapshot are the first shavings from the first self-made plane saved as my wife insisted( no, I didn't ask if they are to be saved as a proof of foolishness ;>) ) I put it here as an aside remembering one of your earlier posts.

Re: Details on study#2 methodology, and the real w

R.J.Whelan

>Joe .... I was speaking/writing in a more general vein. I have not had the opportunity to use any of the really fine planes used in some of these test - my favorite is still the Clark and Williams coffin smoother .... rj

Still a great option!

Joe Rogers, Northern Virginia

>