Towards a model for writing tool reviews.
Derek Cohen (in Perth, Australia)
>It is my intention to post reviews of LV tools in the near future and I am concerned about the reception these will receive. I have recently taken possession of the Bevel Up Smoother and Scrub Plane, plus I am in the process of putting together a more formal review of the LV Honing Guide, which will include tips for users (since all I have posted to date have been impressions). These tools were all given to me by Lee Valley with no strings attached other than to provide them with feedback. While they do not vet my reviews, it is understandable that some will question the objectivity of a review written under these circumstances.
The question I raise is what do I need to do to gain the trust of forum readers, that is, that the information I offer be accepted as objective and reliable? This is partly rhetorical since I hope to make a few suggestions for discussion, but I do want to hear the opinion of you my friends on this forum.
One issue that must be dealt with first is what my motivation is for writing reviews in the first place. The answer to this is partly that I have a intellectual interest in the design of handtools and a scientist’s pursuit of knowledge, and partly that I enjoy the comradeship that comes from discussion that is centred on a common topic. Long before I was offered planes for testing by Rob Lee, I was reporting my experiences on woodworking websites. I did this for the same reasons most of you do so – it is fun. And for formal reviews to continue, it must continue to be fun.
My own background is essentially that of a scientist. By profession I am a clinical psychologist, and work involves making decisions about the information my clients and other health professionals give me. The issue of objectivity is central to everything I do. I am constantly assessing and checking information, always looking at it from a different perspective. This is the nature of the job, and the nature of my professional training.
The issue of bias came up a while ago, and several of us agreed that it was not possible to be without a bias. What is important is to recognise this bias and do what you can to control for it. Am I biased about Lee Valley – you bet I am. I believe that they are one of the best things that have happened to the woodworking world, especially weekend warriors such as myself. They are innovators in the area of design and committed to producing high quality handtools at an affordable price. This is not advertising copy on their behalf – it is how I feel about this company based on my experiences to date. Ever since Rob Lee offered me a LV LA Jack to compare with the Stanley #62 I restored (see Articles for a copy of the paper I wrote), I have been a convert to the bevel up design. Back then I predicted that it was the coming force in plane design. I am excited to be able to participate a little (very much on the very far periphery) in the ongoing development. Am I biased? Are you kidding?
So now the question is, what do I do to bring matters back to an objective level? I know that I will not learn anything new unless I do so, and be incapable of passing on anything useful to others. I need to introduce rules into the reports I write, rules that are fundamental to the scientific pursuit of knowledge. These are the rules I have come up with:
Rule 1: All methods must be capable of being replicated by others. In practical terms, specify the tools used, how and to what level the blades were sharpened, the timber used and, as best as possible, the techniques used.
Rule 2: Record all the results in objective format, that is, in a numerical system or photographic illustrations that make it easy to interpret and gain meaning from the data.
Rule 3: A scientific study requires a comparison of different systems (e.g. handplanes). The “control” system needs to be something that has known, predictable results (e.g. a commonly used handplane). One cannot draw fair conclusions about the tool under investigation unless a comparison is made with a control. Without the control, one is only able to offer a subjective observation.
The bottom line is that results must be demonstrated clearly that they are measurable or identifiable to the satisfaction of the reader. I believe that it is only then that reviews will be considered objective.
It is possible to write fantastic reviews without consciously following the above rules. Alf is one of those rare, natural writers who is able to convey a clear message in her reviews, and these are a wonderful mix of humour and illustration. It would be such a waste to ignore her talents here. There are, no doubt, other amateur writers who could do an equally good job given the opportunity.
My interest in writing this is to open the topic up for discussion. It is to the benefit of all of us that we have reviews and discussion – this is just a hobby for most of us, but we do get so passionate and emotional.
Regards from Perth
Derek