Est. 1998 — 27 years of woodworking knowledge
For the record...
Ted Owen, Moderator
>the policy here on WC always has been, and so long as I'm Moderator will continue to be, negative comments regarding products are welcome and appreciated so long as genuine and true. Whether the manufacturer of the product is an advertiser is not relevant.
Personal attack is not welcome and will bring response and, if necessary, action.
No one has ever been deemed a "problem" for factual reporting.
Best, Ted
Objective, comparitive and subjective reviews.
John K in Hastings, MN
>There are a lot of things in your proposal that are (or should be) standard for comparative reviews. Getting representative (random) test samples, similar setup, etc. All these things make the later tests more informative as they reduce the variableness associated with test setup.
For instance, with wood off the same board, you could pull a plane with a 5# sandbag on top and count the number of spots of tearout per foot. That would be an objective measure of tool performance, although not necessarily representative of actual performance in use. That should also be measured, but this brings operator skill and other variables into play.
HOWEVER, much of what a tool is comes down to subjective thoughts. How does it look? How does it feel in your hands? How can you describe the feel left behind on a piece of wood? Would I buy it again? Would it be my "go-to" if I had them all? Would it be the one I'd choose if I was on a limited budget? All these things are probably MORE important to me, as the difference in performance for top-line tools are probably not significant.
The subjectivity extends to the criteria used to determine "winners". The recent FWW review of wiping finishes is a good example, the criteria was skewed towards things like "water resistance" that some finishes were not designed to do. How much weighting is given to each test or subjective criteria? Do you have a "best" and "best value" at minimum? Are only items in the same price range included? Are vintage tools that have been tuned included?
You may get the idea from the above that there is no perfect review, and I would agree with that. Do the best you can, put it out there, and let us talk about what you concluded.
I love reading them all!
Re: For the record...
John K in Hastings, MN
>Thanks for the clarification. There is one board that no longer exists that had a more active "purge" out to protect their advertisers. Other boards, you disparage their favorite company and you're persona non grata.
The reviews I've seen here have been honest, enlightening and discussion provoking- which is great to see.
Re: Towards a model for writing tool reviews.
Glenn
>This whole thing about your real motives, integrity, etc for how you write your reviews is crazy. Take Frank Klausz and David Charlesworth differing views on honing guides. Frank will tell you that one is not needed and David says they are very helpful. Now is someone going to accuse David Charlesworth of being in cahoots with the honing guides companies because he recommends honing guides?
I have seen fine craft people have very differing opinions on pull stoke and push stroke hand saws. Do we accuse one or the other on being bias or in cahoots with a particular tool manufacturer? The list over differing opinions on the same tools goes on and on.
The real thing here is your review, or anyone else who writes a review, will never be the final work on that tool. You may have a positive experience with a particular tool and I might hate it or vice versa so who is wrong and who is right?
I would recommend you write your review however you want and do not stress over some else questioning your integrity. If someone does not like the tool reviews posted here then don’t read them, wait until Consumer Reports publishes a review on the plane, saw, etc. you have been eyeing. I would personally prefer a more subjective review. Fact is if you write a real scientific review I could probably never exactly repeat what you did in my shop and, frankly, would never want to. I wood work for fun and I have a difficult enough, though enjoyable, time cutting air tight dovetails and would not want to add to my woodworking woes by trying to repeat some exact tool setup so I can verify someone’s tool review.
My two cents.
With all due respect to Rob and LV
Dan in NorCal
>Dear Rob,
While I have been an extremely satisfied customer of LV and lust after all your planes, I also have reservations about the practice of disseminating free tools and have these tools reviewed by the recipients. I believe you when you state “We send out planes (and other product) to a select group of people to solicit feedback on our designs”. However, you cannot deny that there are commercial benefits in the exposure of these products when they are reviewed in a popular medium, especially written by “experienced, articulate, and open-minded people” that you target for tool feedbacks. Whether intentional or not, giving a person an expensive item for free often endear the recipient to the giver, and it is human nature to want to return the favor by being positive about the gift.
The pharmaceutical companies understand this and have all sorts of ingenious marketing strategies to ingratiate themselves to their main dispensers – physicians. Many of these strategies have since been outlawed, but there are many gray areas that still exist. I am NOT implying that this is a ploy used by LV, but if you stand by your statement that:
“Frankly - I could care less if any of the people we work with ever post reviews...as long as they keep sending us their comments”,
Then why not have these testers sign a nondisclosure that they will only communicate their feedback of the products to LV and not publicly? This will eliminate any sense of impropriety. After all, most LV products are truly exceptional and should generate plenty of accolades from paying customers on public forums.
Dan
Guess I can't keep out of this one...
Scott Burr in Ben Lomond,CA
>OK I'll try to keep it simple here. I would say anyone here doing a reviewe here NOT say where one received said such item under review. After all we pretty much know where they (who) come from anyway.
Does it realy matter? I guess it does to some.
Sure brings the rats out from the woodwork doesn't it. I kinda have to agree with Adams view here (somewhat).
Things like this bring out the best and worse of most everyone here. It's open bait for some people here to start a personal/political battle, debate, whatever... You know who you are too...
Ya all know what I mean? Let's just review the facts in a logical way, and let the rest of it go.
Re: Towards a model for writing tool reviews.
paul womack
>The question I raise is what do I need to do to gain the trust of forum readers,
Nothing. If some people need Descartean levels of "proof" in a forum review, they're most likely not a receptive audience anyway.
I'll take any and all information that anyone takes the trouble to post. And I'll use the obvious internet caution (i.e. bear in mind source, track record etc) while I do it.
BugBear
Re: Towards a model for writing tool reviews.
paul womack
>What I have seen lately has been allot of Veritas tool reviews
That's because Lee Valley have been launching tool faster than anyone else.
Much faster.
There was a time when the new LN tools where each reviewed (quite extensively) as soon as they came out.
John Gunterman, over on USENET, seemed to get early delivery by special arrangement.
People get excited over new tools, and want to talk about them. This seems natural to me.
As an amateur writing for my own (and hopefully others) enjoyment, I'm DAMNED if I'm going to perform full blown, "controlled", comparative banks of experiements if I just want to talk about a new toy.
BugBear
Derek, Lyn etc. /free gifts
Matti Kuikka
>If anybody finds a way to earn some real money by writing a review for the price of any tool in LV or LN catalaogues he must be a very, and I mean very, fast worker indeed to get a decent $/h. As the publishing is for a knowledgeable crowd I don't suppose you could get by without at least putting some time in actual use of the tool in question so just having fast writing fingers isn't enough.
I think that a picture with workers hands with the tool would be more interesting and informative than knowledge about how the tool was acquired/got.
My 0.2€
Matti, who likes to read the reviews and especially discussions that arise.
Re: Towards a model for writing tool reviews.
Todd Stock
>One thing that sticks with me to this day from my years in technical and operational testing was the number of times that dedicated, technically savvy, operationally talented scientists and engineers with millions of dollars in lab equipment got things flat wrong, while line mechanics and pilots gutted out a better answer.
However much jargon we wrap our results in, I seldom see much in the way of meaningful quantitative results when it comes to hand tools. I'm not certain that this is a bad thing, given the tendency for techno-geeks such as myself to suck both the art and the fun right out of things. That said, the qualitative impressions are useful to me, and I continue to read with interest the reviews here... with a loaded salt shaker at hand.
To restate the obvious, I'm not criticizing the motiations or integrity of our reviewers (however prickly these notes and their authors may get), but rather ask whether we can get much beyond subjective impression with the limitations of time, equipment, and the hundred or so variables of interest in most hand tool operations.
Offered FWIW; I'll follow your progress with interest.
Have you lost your marbles?
CONGER - The Irish diaspora in Munich
>Pay attention folks…. internet lore should be treated as it is; an expression from someone (?), located somewhere (?), who is following his / her own aggenda. The onus of discerning what is true / correct lies solely with the reader, and with no one else.
Reviews take many forms.
A simple review is just an opinion. Experience allows us to discern the reviewers who are likely to be critical and objective. Admittedly, it does help if a reviewer does report any influences that may colour his / her opinion, such as a the source of the reviewed object. To be fair, many reviewers on this forum do report such influences, either explicitely or implicitely.
A comparative review, where the reviewer documents the relative differences between two or more like objects is a substantially better thing. Comparative reviews impart a much higher credulity, but are more complex and difficult to produce. This should bet he minimum level offered by professional (magazine) reviewers. Some of the reviews offered on this forum are at this level. I am eternally thankful to, and admire, those persons who invest the effort and time to produce comparative reviews.
A referenced review, where the reviewer relates his / her results to a recognised measured standard, is the best review form. Referenced reviews are seldom possible for ww. tools, as there are few ‚standards’.
I enjoy the reviews published by the Dereks, Alfs, Lyns, and others of this forum. I tend to believe those reviews that express both praise and criticism, more than those which do not.
I have never bought a product based only on any one review in this, or any other, forum. I rely also on my own judgement, and often on the current capacity of my wallet.
Re: With all due respect to Rob and LV
Rob Lee
>Dan -
One of the reason we wouldn't use an NDA to gag reveiwers - is that I don't suspect most of them would go for it. I know I certainly wouldn't work under those rules.
People either have ethics - or they don't. No amount of rule-making or legal wrangling will change that. That people are suspicious and mis-trustful of others is their problem - and I won't cater to it. I don't buy from people I don't trust - and don't see why anyone else would either.
Cheers -
Rob
Rule 1. The review is not done for profit
Ian in Sydney
>Derek
I appreciate your concern, and being aware of how the car industry markets its wares, I think that the rules for internet forums like this should be:
Rule 1. you must not be paid for writing the review. (Getting the tool as a gift or for a steep discount on retail doesn't count as payment.)
Rule 2. you must not present sales material prepared by the manufacturer as either your own work or the results of your own testing. (of course, the manufacturer's specifications can be quoted freely and if you say "hardened to Rc62", I don't expect you to independently verify the value.)
I don't think there needs to be any other rules. A not for profit review posted on a forum like this is just a personal opinion shared with others. Appart from the absence of body language it's no different from a discussion in the pub.
How objective or subjective you are is for the reader to judge.
Ian
Why the reference to the car industry?
Well, in conjunction with the free trip to some exotic location to try out a new model, it fairly common for the "press kit" to include pre-written reviews and stories. Your editor wants 300 words on the car — the manufacturer has already written a 300 word story for you, you just need to add your own byline.
(replace "&" with "and" to email)
Re: With all due respect to Rob and LV
Dean in Burlington
>Rob,
I also noticed that you only solicit people who have in the past gave LV planes great praises on this forum. If you are looking for a constructive criticism/review....why don't you send one to Dan Clermont? He seems quite honest to me.
Reality check
mike recchione
>We're talking about tools here. Hand tools. Even the most expensive of them costs a few hundred bucks, most substantially less. If Lyn or Alf or Derek or anyone else induces me to spend a couple of hundred bucks on a plane by writing an inaccurate and biased review, well, it's the kind of thing that won't happen more than once - I'd never believe another word the bogus reviewer said. On the other hand, my experience has been that their reviews are extremely useful. I'm willing to risk spending a couple of hundred bucks on something less than glorious if it means getting the continued (or resumed) benefit of their observations. It certainly wouldn't be the first time I bought something that I later found not to be what I wanted/expected. So, for me, the loss of the benefit of these reviews is a much greater price to pay than the risk of wasting a little money on a tool that I might not love.
Having said that, I think of tool reviews as more like movie or book reviews than like scientific studies. I don't care about objective comparisons of one book with another, or how well one writer made use of symbolism or metaphor or whatever. All I care about is "Reviewer X loved this book, and I almost always agree with him." And I don't think it takes buying a whole bunch of books to figure out whether or not this sympatico exists - you can often tell by the tone of the review, the things the reviewer chose to dwell on, his/her writing style, etc.
One's experience of a tool is very subjective. It's more important for me to develop a feeling for how well a reviewer's opinions match my own than to know how many swipes on a plank of red oak it took to cause the edge of a chisel to show a nick visible only via electron micrography. Things like style of work, body type, hand size, even philosophy - maybe especially philosophy - are much more important to me than trying to reproduce the methodology and tone of an undergraduate mechanical engineering lab report.
The scientific approach is, IMHO, much more appropriate for something like LV's R&D department. I'd expect a manufacturer to do his homework before putting something on the market. Once a tool is designed and produced, I'm much more interested in how a fellow woodworker with whom I feel a kinship in style and priorities perceives it.
My two cents.
P.S. to Rob Lee: I am known as a very open-minded, articulate, objective, enthusiastic, energetic and outspoken individual. You can ask any of my friends. That is, you could if I had any. In any case, I feel duty calling me. I'd be happy to provide my address if you email me... :-)
another point of view
Adam Cherubini
>Do you skeptics feel every reviewer should have to purchase a whole bunch of tools he or she doesn't need and may not like inorder to conduct a "fair" review?
So forget about the added bonus that LV gets when reviewers help them develop designs. Buyers want to read tool reviews. Manufacturers need to spring forth with free tools if they want their tools reviewed. Right? I don't have the money to buy 6 smoothing planes. I already have a smoothing plane I like.
Again, I think you should expect bias of all sorts. We're all biased. In my mind, comparison reviews are better than Derek's first impression-type reviews. Derek is really comparing his expectations with what he's seeing. He doesn't tell us about his expectations. As he writes more and as we get to know his bias better, these sorts of reviews will become more and more meaningful. Its just like a movie reviewer. If you know the reviewer's taste, than his reviews are helpful regardless of whether his taste is yours or not.
With the greatest of respect to all my friends here, I think its naive to think bias is based on whether a guy pays for a tool or not.
I think free tools for reviewers are a huge help to all of you including you complainers. You're benefitting from a process you disagree with.
Adam
P.S. I'm working on an article about hatchets and I wish I could provide you guys with my feelings about the hatchets currently on the market. But I simply don't have the $1000 I'd need to buy a bunch of fancy hatchets. And what would I do with them afterwards?
Shameless!
mike recchione
>Adam, your fishing expedition for a bunch of free tools was transparent, shameless and beneath you. And besides, you forgot to mention how openminded, objective, articulate, etc. you are. :-)
BTW - if you really don't know what to do with those extra hatchets you get, well, I might be able to help you out...
Re: Derek, Lyn etc. /free gifts
Greg Sloop, Portland Oregon
>I'd normally avoid writing on this subject others have covered it pretty well already, but I have to agree with this statement.
I'm a consultant and my billing rates are not trivial. I can only imagine what Lyn's rate per hour is.
For the time spent simply taking the pictures, resizing them, uploading them to my server (which incidentally took more than a Gig of traffic, again at my expense) and writing even the fairly brief things I did, I could have very easily bought one of the smoothers. That doesn't include any of the time actually using the smoother in an odd-ish way so I could give everyone a "better" comparative review.
I wasn’t even aware the smoother was mine to keep at the time I wrote and posted the review. I assumed I’d need to send it back to Rob when I was done. When Dan Clermont implied, IMHO, such ugly motives etc, I inquired to Rob about the status of the tool, asking what the cost would be should I want to keep the smoother, which I did.
His reply?
“Nope - we didn't want you to send back the plane - and it IS a second, which we won't sell. The offer should have been the plane in exchange for your frank feedback...that's it.”
---
Lastly, let me put to bed one other item.
If you want to influence my bias artificially, better than sending me a free tool, just ask my opinion. I’m way more flattered by someone wanting to know what I think about a subject and appearing at least to really care. That’s way more “influence” than a tool that cost way less than the time I used to talk about it – and it’s a bias you’ll never see disclosure about.
The accusations of graft and bias are baseless, IMHO, and further there’s really no defense. I liken it to the Salem Witch trials.
“We think you’re a witch. To verify it, we’re going to tie this huge rock to your body and hurl you in the nearest body of water. If you sink, well, ‘oops’ we were wrong you weren’t a witch. But if you float, then ‘by gum it’ we’re going to kill you proper this time.” For the witch, it was a pyrrhic victory to drown and prove their innocence, and floating was no walk in the park either.
As I said earlier, the eyes of the looker will see what the looker wills them to see.
Cheers,
Greg
I don't appreciate that, Mike
Adam Cherubini
>When I mentioned reviewing hand forged hatchets, I did so with the utmost integrity. Despite the fact that I'm am biased toward hand forged hatchets, I would in no way allow that to effect my conclusion that hand forged hatchets, like those expensive presentation hatchets Todd makes for example, are in fact superior to factory made hatchets.
Of course the only way to know for sure is to compare the balance, edge retention, and head design of SEVERAL smith made, hand forged hatchets, (including of course one from the John Anderson forge in Williamsburg) with say a control group...like say... the entire product line of Gransfor Bruks hatchets, including the model 1700 with the integrally forged handle socket.
On a serious note: I didn't see your post before I posted mine. As usual, you said what I wanted to say (regarding book or movie reviewers) more succinctly. I try to read every post before I add my own. In the future, I'll be sure to refresh my browser, deleted my 4 paragraphs and write in their place "ditto". Talk about serving the greater WC community!
Adam
Rob Lee, David Charlesworth, Todd Hughes, Lyn, etc
Patrick Gibbons, Houston
>The etc. includes Pam, Joel, Denis, and every other person I haven't named because I'm writing this fast and because it would take too long. Your opinions, observations, comments, experiences, choices, biases, etc,etc, fascinate me. I love it. I'm not paying you to give me objective information. I'm not paying you to give me anything. You're not like Consumer Reports that I rely on for objective info. I buy their product as needed. Rob Lee's outfit has taken not one, but two braces back from me because I thought they broke too easily. Did I stress them beyond their specs? Probably. Not a problem with the Guys From The Great White North. So I guess when I got caught up in the rapture of reviews of their large shoulder plane it was because of all these sinister so-called reviews I read here. When reality set in that it wasn't a proper fit for me The Guys From the Great White North took it back, no problem. I have insisted on and gotten to talk to Joel at Tools For Working Wood every time I wanted and ate up his profits from my little purchases with excessive phone charges he had to pay on his toll-free line. That's so typical. You know how New Yorkers are quick to take advantage, especially of us naive Southerners. David Charlesworth is too fussy with all his rulers and cambers and filing this and tweeking that. Brits are all alike. Maybe when I've been doing this another 30 years as a pro I'll understand why. I love the argument over this whole issue. Why? Because you have become my friends. We sit at a cyber-bar and talk. I complain to my wife about some of you. I get mad at some of you. I still come back.
But we don't have cyber-beer... ;-)
Denis Chénard, Orléans, Ont.
>
Consumer Reports
Pam Niedermayer - Austin, TX
>I really admire the job CR does, but find it less than useful for most of my purchases. It seems to me they provide good guidance for the average user of a given product, but not for advanced users. So in areas that I have some expertise, such as photography, woodworking, computers, motorcycles, I find their advice not useful, actually irritating. But in those areas in which I'm an average consumer, say buying washing machines or cars, they are great.
Pam
PS Who thinks the same way about National Geographic. If I know anything about the topic, I know more than they're going to tell me.
Re: Consumer Reports WAAY OT
Adam Cherubini
>Pam,
My favorite characterization of CR was a ficitious auto review comparing the porsche 911 with the Ford taurus.
CR: Both cars have 6 cylinders, but the Ford seats 4 and gets better gas mileage.
I always look forward to what CR reviewers will say about the trunk space in a Viper or some such!
This has absolutely nothing to do with this thread or woodworking. I just thought it was funny.
Adam
Don't you agree that the Taurus was a Best Buy? :)
Pam Niedermayer - Austin, TX
>
Objectivity
Bill Houghton, Sebastopol, CA
>It can be argued that objectivity in determining performance of a particular object only exists within a subjective context. Objectivity assumes measurement, but you first have to determine what to measure, and that measurement requires judgment as to the purpose of the object. Judgment, while one hopes it is informed, is not necessarily objective, but includes the test designer's concepts of the critical factors to measure, assumptions made about the purpose of the tool, etc.