Setting up a chisel test - ideas for discusssion
Derek Cohen (in Perth, Australia)
..then i would conclude that they are equivalent, which to me at least, would be a pretty interesting result. i am hoping this test will provide some relevant data to prospective chisel buyers, a group to which i belong. there is a lot of hype about the steels and the performance of these chisels but very little comparative data available. are they only 3% better, or are they 3X better? i, for one, would like an answer.
Hi Lars
There are a number of aspects to consider here.
The first (to get it out the way) is the nature of research - I write this for a general audience, not you specifically. Research, technical or otherwise, to be useful - read "scientific" - must be reproducable by others. This means that the methodology is laid out for others. Further, that the method used to test one variable (= chisel) must be the same for all variables.
It is with this last item that we start to run into problems (or not - so how do we deal with it?). The issue is that each steel has an optimum method for preparing the blade's edge. Some steels are recommended to have a 25 degree cutting angle others 30 degrees. If we do this, are we comparing apples with apples? We may decide to go with the manufacturer's recommendation because that is what they sell the chisels with, and that the comparison should be based upon what you get at a vendors' counter. I would support that. I know that Bill would argue otherwise, that a higher cutting angle represents a weakness in the steel. My argument would be that a higher cutting angle has pros and cons - what would be anticipated would be poorer paring and better impact resistance/chopping - and that these would be reflected in the results of the testing, per se.
The second variable, one under discussion, is the method of sharpening. Not all steels are recommended to be sharpened the same way. Whether we agree or not, we need to present the steel in such a way that it is optimised for a comparison. This was a problem area for me in the 2008 (?) FWW chisel test. I recall all blades were ground and honed to 25 degrees, laminated Japanese bench chisels and A2 LN chisels included. So, waterstones, oil stones, diamond paste or mesh - we need to settle on the importance of this and accept/reject that differences in sharpening media could in leave us with an uneven playing field.
Further, one size does not fit all when it comes to honing media. Using only diamond may achieve a sharp edge for all steels (nothing abrades better than diamond), but an important variable is omitted if we do not concurrently assess the ease at which a steel is honed on their chosen media since not everyone has or will wish to use diamond. One possible research project could simply be the steels edges under a microscope after being honed on different media.
A third issue is that with chisels we are dealing with a tool that has two major functions, paring and chopping. Abrasion resistance and impact resistance, respectively. Are the results of the tests going to emphasise both areas, or a composite score, as a guide to "best", "preferred", "winner" or "loser"? Placing this is the real world, consider Japanese chisels: bench chisels are designed for chopping, and slicks are designed for paring. Ideally, one owns a set of each type to meet the range of tasks in woodworking. By comparison, the offerings of LN, LV and AI (to name just three) are dual purpose chisels. So ... do we create three distinct categories in this research project: impact resistance, abrasion resistance, composite score?
Note that Lee Valley has done considerable testing on the above with several steels: http://www.pm-v11.com/TestingIntroduction.aspx
Variable #4: The age and use of a blade. Do we include new or used blades? There are again pros and cons. New blades ensure that the hardness of the steel has not been disturbed by excess heat as a result of grinding, but good (?) used steel may have worn past that point of brittleness which many chisels experience when new. Or we may get new and just use them for a while. How long? Then what about vintage steels - how do we determine whether they are representative of the point they started out in life?
I have two views here. The first is a repeat of something I said to Warren about 18 months ago when completing my own testing of steel types, that is, the selection excluded chisels that were not available new to buyers. This meant that vintage laminated blades were not included.
Testing completed:
http://www.inthewoodshop.com/ToolReviews/ChiselBladeTesting-5Steels.html
http://www.inthewoodshop.com/ToolReviews/ChiselParing.html
More recently I wrote that it would be an interesting comparison, modern versus vintage steels, if only to determine whether we are moving forward with the quality of our tools, or not. This could be the rationale for including a small selection of vintage steels.
That brings up item number five: how many chisels can one have in a test before it all becomes unwieldy - 5, 6, 9, 12 .. ?? I once compared 9 different steels for Lee Valley, and I must tell you that this was exhausting!
Affecting item five is item six .... do you choose to do a "blind" assessment or not. The LV testing above was blind, that is, even LV did not know the steels I was testing until later on. They were only identified with a number, and only one person knew that (probably asked for a raise before letting others knows which was which!
). I think that we have no chance of this type of testing, and this then places an emphasis on the methodology for collecting reliable results.
Item seven - what does one consider a reasonable method for testing the chisel steel? I presented a method for abrasion resistance and for impact resistance in my experiments above, as well as how I collected results, but others will have different ideas and these need to be heard for an agreed methodology. The overall factor is repeatability (both for inter-tester, and for between chisels). My testing was not a good method in one respect - the force I used was not controlled for others to repeat. Having seen the LV set up, I know they controlled for both. e.g. http://www.pm-v11.com/TestingImpact.aspx
Having written all this I wonder how much is really relevant! The fact is that there is enough testing to accept how the different steel types compare with one another. Again, see the LV results. This only partly tells the story. At the end of the day the best chisel is still going to be the one that you feel a one with. That is, the handle and its feel in your hand, the heft and balance of the total chisel ... and for some the quality of the edge may come in last! Of course, where there is a tie in these factors, the steel begins to mean more.
Over to all of you ...
Regards from Vegas
Derek