Journal Club: Plastic or Metal? Saxophone pad resonators
Brief summary: Fun discussion about study design and randomization as it relates to crossover study designs. The article is a study of musician perceptions on a kind of modification to saxophones called “resonators.” This study was introduced to me by my friend Allan Walkey, a pulmonologist, health services researcher, and a pretty good sax player as well. Thanks to Gary Scavone, the senior author of the study who joined us at the end to give his insights!
Guests: Allan Walkey (AW) – https://www.bumc.bu.edu/pulmonary/about-us/people/allanwalkey/
Gary Scavone (GS) – https://www.mcgill.ca/music/gary-scavone
Article Links:
http://www.music.mcgill.ca/caml/lib/exe/fetch.php?media=publications:eveno_pads_aaa_2015.pdf
Plastic or Metal? Pad resonators blind test | Inside the saxophone
PC: I loved this study, which was presented in two separate posts online and as a full paper. I thought it would be fun to start journal club with something non-medical. I’ll explain my positive response to it. The two things I liked most about this study were:
- The question is incredibly clear and the practical importance is obvious. Do I need pad resonators for my saxophone and if so, what kind?
- The elegance of studying the question on multiple levels.
To expand on the second point, there is “objective” data (measurements of impedance, amplitude, and efficiency) and there is “subjective” data on musicians assessment of “brightness.” To translate it into the biomedical realm, it was like a paper where a convincing trial outcome is also supported by a strong and plausible biological mechanism. What was your main reaction?
AW: OK, first a little (OK, a lot of) background. What are resonators? When each key on a saxophone opens and closes, it changes the note. A closed key uses a leather pad on the bottom to make a seal so no air escapes and the note is strong, in tune, and doesn’t squeak. If you imagine a sax with closed keys as a metal tube that conducts sound, you can also imagine that the material on the bottom of the closed keys might matter in terms of the quality of the sound being produced. Additionally, you might expect a ‘dose response’ in which the more keys that are needed to produce a note the greater the effect on the quality of the sound. “Resonators” are material placed over the center of the leather pads on the bottom of the keys and are designed to reflect the sound from the closed key back through the horn. More accurately, they are ‘sound reflectors’ than resonators. You can buy resonators in brass, nickel, silver, gold, plastic, or use no resonator and go with all leather on the bottom of the key.
About 15 years ago I graduated medical school, started residency, and was excited to buy myself a present for completing my intern year: my first professional-grade alto sax. After some initial research I bought a Yamaha unlacquered ‘custom Z’ sax (saxophones are typically coated in a coat of lacquer to maintain a shiny finish, which can change the sound) in order to try to get the purest most free-blowing sound unencumbered by the vanity of lacquer. So I was a little disappointed when I saw that my new sax arrived with plastic resonators on the pads, rather than ‘pure brass’. My dream of having the purest sax sound in history was foiled by this unnatural plastic underneath my keys! From that moment, I dreamed of the day that I would place metal resonators on my horn to allow it to realize its full brass potential.
Fast forward to today. After 15 years of wear and tear (on me and my sax), my pads were literally torn up, the springs rusty, and my sax was in need of an overhaul. In an overhaul, a sax is disassembled, cleaned, oiled, and the pads and springs are removed and replaced with new fresh parts. It’s a big, expensive job that takes a sax repair person probably a week of work. But I saw this as an opportunity – while my sax was disassembled, (for about $150 extra) I could have new metal resonators placed to finally achieve my brass dreams.
Would it be worth the expense and changes from my 15 years of familiarity with my plastic resonator sax to make the change?
PC: Ahh. Now I know what a resonator is! So this was the question that brought you to the article. Two questions – 1) how did you find it? And 2) what did you think of the article?
AW: 1) I just Googled “do sax resonators make a difference” or something like that. And the first hit was this article called “plastic or metal”. It was posted in the website of Syos, a company that makes custom sax mouthpieces, and I thought it was going to be an advertisement for a new line of resonators. But it wasn’t. It was a summary of a “real” research study.
2) This was a “practice-changing” study for me. It used a mixed-methods approach (similar to what we use in our own research center) of quantitative analysis and qualitative evaluation of stateholder perceptions to get a full “picture” of the effect of different resonator materials on sax sound quality. It used randomization, blinding, and a within subjects crossover design to minimize both random and systematic bias. It had clear figures and presentation of results. Finally, the results seemed unambiguous. In some ways, the “negative” result here reassured me that this study was not biased by commercial interests.
PC: Some interesting points there. Let’s talk about the study design, and specifically as it relates to the results in this figure.
To fill in some detail, you have four different saxophones with three outcomes. Saxophone 38 is the “control” (no resonators), sax 40 has metal resonators, and for some reason both sax 37 and 39 have plastic resonators. Each saxophone is then graded according to three measures by 13 musicians. It’s interesting to think how this could be translated into a typical randomized clinical trial. The way I see this, the saxophones are like people, and the resonator status is like the treatment or exposure. The ratings from each of the 13 musicians are the study outcomes. So, if you look at it this way, it’s not quite as good a study as it could have been, because the randomization is done only with respect to the outcome measurement, but not respect to the saxophone and the “treatment.” Do you agree, or do you see it differently?
AW: Interesting. I saw it differently. To me the musicians were the subjects (their perceptions are being measured), the 4 different saxes were the treatments or exposures (plastic, metal, or none), and the outcomes were the 3 different perceptions: brightness, evenness, and ease of play. So I viewed this as a 13 subject, randomized, blinded, crossover study in which the treatments were identical saxes with different resonator types. For these perception outcomes, the scale is 0-1, and the best sax had to get a 1 and worst a 0. They repeated the measurements for each player. In the full manuscript there are lots of additional results, including intra and interindividual concordance ratings for each criteria, results according to sax experience, as well as repeated measures anova for main results. There’s some interesting results hidden there as well.
PC: Got it. My point is that technically they are not “identical” saxes. In this study, the specific saxophone and the exposure (resonator type) are perfectly confounded. But I think you are saying that, if one assumes that the saxes are identical, then this isn’t a problem. But if that’s not a safe assumption, then I say this is a potential problem, and if it would be easy to swap out resonators (is it?), it would have been better to randomly assign the resonator to different sax/musician pairings. So, if one were to distill this down to a single concept, it is that the purpose of randomization is to address confounding that cannot be handled in other ways, but that proper randomization must be done with respect to your exposure. In other words, the randomization must intervene between your exposure and your study subjects, which is why it is important to decide whether we think the study subjects are the saxophones or the musicians here. Capiche? I drew up my conception of three ways of viewing the study (shown below). The first one is my idea of your view, the second and the third are alternate ways of assigning labels of study subject and treatment/exposure. The interesting thing about the third one is that, in this study, the people are essentially viewed as the equivalent of laboratory equipment – they are the measurement device that gauges the effect of the different resonator types. My contention is that the most accurate way to represent the study is option 3, but that in reality none of these is the optimal study design (which I would be happy to expound on). But I would say that if one is willing to assume that the saxophones are truly exchangeable, then I think options 1 and 2 are both defensible. The problem with 3 is that the intervention is not located in its proper place between study subjects and the exposure of interest. Thoughts?
AW: Right, an individual sax = resonator type. So there is potential confounding by sax. In the investigator’s defense, they use the same exact model, with consecutive serial numbers, and then test them for acoustic qualities at baseline and there seems to be no objective difference in sound before they switch out the resonators. But there’s still a risk of confounding by intrinsic sonic differences to each sax. It’s not easy to switch resonators. One way to reduce the risk of confounding by sax further would be to have 2 saxes for each resonator exposure because it would be unlikely that random variation in sax sound would correlate between 2 saxes. They do this only for plastic resonator, but results are reassuring since they are similar for the 2 plastic resonator saxes. I’m having a little trouble with your figure. It seems strange to me to say the study subjects are the resonators. It’s like saying the study subjects are the drugs under study. Yes, resonator is the study subject, but the subjects that are subjected to the treatment (resonator) here are the musicians. The sax here is just a delivery device for the treatment (resonator).
Let’s make an analogy: You want to know if inhaled albuterol (plastic resonsator), inhaled ipratropium (metal), or inhaled saline (no resonator) result in different perceptions of breathlessness for people with asthma (musician). In a random order (to account for confounding by order effects), you have 13 asthmatics use the albuterol inhaler, the ipratropium inhaler, saline inhaler, and another albuterol inhaler and they rate breathlessness on a scale. Now you could say – wait! How do we know that the inhalers themselves don’t work very differently to deliver the medication! Which is a good point, but we can be reassured by prior testing of the inhaler and that the same factory made them all, etc.
So our causal diagram to me, is as follows:
If we assume that confounding by musician differences is taken away by the within-subject design (i.e., exposure of all musicians to all resonators and saxes), that differences between saxes is negligible (like differences between the same model inhaler), and order effects are taken away by randomized order, then there should be negligible confounding to this study.
Thoughts?
PC: I like it. I actually completely agree after hearing you explain it that way. I think the key for helping me understand it is to key in on the musician as the subject (natural), which then highlights the crossover nature of the trial (everybody experiences every resonator), and also highlights that the randomization is related to the order in which they experience the resonators to account for that potential bias, as your diagram illustrates. So to return to the original diagram, #1 is the right way to think of the study, and saxophone is theoretically a perfect confounder, but if one is willing to assume it isn’t then it is a good study! One could also imagine a way in which the relationship between sax and resonator could be randomized so as to avoid that bias, but perhaps that is not a matter of practical importance. On that note, let’s hear some thoughts from Gary Scavone, the senior author of this study.
GS: I helped organize and supervise this study, so I can add a few perspectives on the design:
The saxophones were loaned for free by Yamaha for the study. As mentioned in the paper, they were consecutive serial numbers off the manufacturing line of a well respected company and their tolerances seem to be quite low, so they are about as similar to one another as could be expected. That said, when I was looking to buy a new Selmer saxophone back in 1989, I tried 6 different horns in a shop in Bordeaux and finally settled on one that “seemed” the best match for my priorities. It is hard to compare horns quickly, as you have to set one instrument down, take the mouthpiece off, put it on the next instrument, and so on. I remember thinking that there were some slight differences between those saxophones but nothing super significant. For sure, metal saxophones made on manufacturing lines are significantly more similar to each other compared to instruments made from wood and by hand (such as violins), due to natural variations in the materials. When we received the 4 Yamaha saxophones, I tried them all and could not detect any particular differences between them. My feeling was that either Yamaha has better tolerances than Selmer and/or the manufacturing process has become more consistent compared to the 1980s.
More importantly, in my opinion, is that all 4 saxophones came with plastic resonators. Thus, we had to send 2 of them to a shop to have the pads changed. That was the biggest weakness of the study because we cannot say for sure that extra variations between the horns were not introduced during that process. We even had an issue with leaks after the new pads were installed and had to take one of the horns to another shop to get it adjusted. I don’t know how this limitation could be minimized, but it was an issue.
That all said, when I played the 4 saxophones myself, the horn without resonators was very noticeably “darker” in tone quality, while the other 3 were relatively similar to each other. I was not one of the subjects in the study but I was not surprised by the results … this was one of the clearest perceptual studies with musical instruments that I have supervised. We have done many other studies with violins and it has normally been hard to get consensus on the results due to large variations in preferences.
PC: Thanks Gary. One of the things that strikes me is that, as much as we try to put all of the relevant information into our scientific papers, I rarely speak to the authors of a study directly without getting some useful insight into matters of practical importance. It’s interesting to hear your point that, on the scale of things that you have studied in this way, the effect of the resonators was large and pretty clear. With respect to saxophone as a potential confounder, it sounds to me like it’s possible but not particularly likely based on your and Allan’s previous experiences and perceptions of variation between saxophones. During the course of our discussion we touched on issues related to the design and analysis of crossover studies, and I’d like to return to this in a future post if we can get some of my statistician friends to join, but I think this is a good place to end our current discussion. Again, really appreciate your time and input here Gary, and congratulations on the work you and your group did on this very interesting and well done study.
So Allan, let’s end with the real world implications of this work on an end user. This all started with your question, “Would it be worth the expense and changes from my 15 years of familiarity with my plastic resonator sax to make the change?” So what did you end up doing based on reading this study?
AW: I discussed putting on metal resonators with my sax repair guy. His opinion was also that the material of the resonator did not seem to be very important, but thought size might matter (for resonators). He sent me links to many different choices in different textures and finishes. Depending on the type of resonator, a switch to metal would add between $75-150 to the cost of the overhaul. Because I was unsure if or how new resonators would change my sound, I elected to stick with my old plastic ones.
PC: And there it is. Science in action. Enjoy your overhauled sax Allan!
CDNM - Castaldi Group 