Old versus New: Blind Taste Test

[Jeff Mando]

Another factor rarely mentioned in blind testing is that some people are attracted to vintage instruments and some will only play a new instrument -- regardless of tone.

[JonZ]

Science and subjectivity: too many variables renders such tests invalid. Always fun reading, though.

Which variables are not being adequately controlled?

The studies measure what they claim to measure: expert musicians’ perception of an instrument in a blind test.

[Michael H Geimer]

Assuming a comparable test with mandolins: Even if we could control all aspects of a performance from instrument to instrument, what meaningful difference would exist? I say none, because once we change the player, or the pick, or the strings we would very likely create greater tone changes than the new vs. old experiment. So while we might measure a sonic difference in such a test, it is one without distinction in practice.

[Bob A]

Which variables are not being adequately controlled?

The studies measure what they claim to measure: expert musicians’ perception of an instrument in a blind test.

Don't confuse subjective with objective.
Aside from setup, temperature, humidity, acoustic properties of the auditorium, skill of the player, specific factors that influence the player, skills of the other auditors, string selection, pick/bow factors, player's time to explore the nuances of the instrument, there's also, in some of these "tests" variables introduced by less than forthright individuals who might profit from one instrument over another in the test. People have been known to mess with bridge/soundpost position to cause slight changes in the sound of an instrument, in order to influence the instrument's tonal qualities.

Then the player's conditions; ability, personal preference, state of health, time of day, selection of music to be played, and on and on.

There's no way to define a subjective metric. There's no way to select for a single variable. The situation is awash in complexities. You can't really speak of such things as musicians' perceptions as "measurable".