Evaluating New Products

As body piercers, we share the common experience of having been interrupted midway through our (hopefully) well-considered aftercare instruction by our client’s friend, sibling, or family member who declares: “Just use ‘Product X’, it worked for me.”  There’s a problem, though: We know Product X.  We have checked it out.  Not only has the bulk of the industry agreed that Product X is inappropriate for piercing aftercare, but the Product X website says you shouldn’t use it on body piercings.  None of this matters to the person who has successfully healed a piercing with Product X:  “Well, it worked for me.”
This is a problematic perspective, because an individual’s personal experience is powerful “evidence” to that individual. Explaining to them that they healed in spite of, not because of, a particular product is an uphill, and often losing battle. The problem is, personal experience is flawed. Our brains are wired to make the most of positive results and the least of negative results. When piercers try to evaluate a product, we can't help but "like" or "dislike" it ahead of time, usually based on marketing or who introduced us to the product to begin with. This skews our perception of the results we see to the side we want. There are a ton of cognitive biases at work when we attempt to evaluate a new product, but when we try something out, we are bound to have our assumptions prove correct.  This is a cognitive bias called the confirmation bias “the tendency to search for or interpret information in a way that confirms one's preconceptions” (1). This is exactly why so many studies are “blind” or “double blind”. Scientists try to eliminate biases like experimenter's bias and confirmation bias.
Most body piercings will heal because people’s bodies are built to heal wounds. Before the advent of modern medicine, we healed regularly through prayer and quackery. Before the advent of prayer and quackery, we simply healed like any other animal heals. A body piercing isn't a difficult wound to heal considering homo sapiens sapiens had to hunt dangerous animals, get injured in the process, and heal to survive.
Recently, a new aftercare product was discussed on the popular Body Modification Learning Forum.  I was pleased to see the lion’s share of my colleagues approach the new product skeptically.  “Is it FDA approved?” “May I see a Material Safety Data Sheet?” “What evidence do you have that this product is better than what is currently available on the market?” were all questions discussed with representatives for the product.  My peers wanted evidence that the product worked.  The representative for the product suggested that we try it, and see if it worked first hand.
This got me thinking about the idea of first hand experience and how flawed it is. 
How does one evaluate something like an aftercare product? A skin prep? A hard surface disinfectant?
The short answer is this: you don't.
The long answer is this: you can't.
Evaluating a product like an aftercare product requires money. The kind of money, regrettably, piercers tend not to have. It takes scientists. Control groups. Blind studies. Clinical evaluations. Large amounts of data. Peers to review that data. A piercer cannot do this.
An example: a piercer suggests to her clients that they clean their piercings with a new soap on the market, for our purposes we will call this fictional soap Placeboderm. The makers of Placeboderm have marketed heavily to the piercing community, boasting the health benefits and germ killing abilities of their product. Our fictional piercer buys into the hype, and suggests this new soap to her clients.
After one month, and 100 piercings later, our piercer is loving Placeboderm! Piercings seem to be healing better! Fewer problems! Her clients like the packaging and the smell of the soap. All is well, exactly as she had hoped.
Here is the problem: our piercer doesn't have all the data. Our piercer never asked for clinical studies on how effective Placeboderm is on injuries similar to body piercings  (a draining puncture wound, for instance). Had she asked, she would have gotten more marketing but no studies.  Again, paying scientists to accurately evaluate a new product is expensive, and if we as piercers do not demand it, it won’t be done.
Our piercer was never in position to really evaluate her new aftercare. She never saw the piercings that didn't heal well when using the soap. The clients with the worst reactions didn't follow up, got frustrated and removed their piercings, or went to other shops and got different aftercare. Even then, our piercer only did 100 piercings. For this intellectual exercise, let's assume Placeboderm actually is bad for healing piercings. It prevents 2% more of the population from healing well than the aftercare laid out in the APP “Suggested Care for Body Piercings” pamphlet.  Let's also assume, 90% of piercings would heal with the APP aftercare. Our fictional piercer would have to keep amazing records, get honest follow up, and be able to accurately assess what problems were caused by soap versus everything else, and still only 2 of the 100 clients she pierced would heal worse than the APP aftercare. I don't think our piercer has the capacity to discover this minute difference in aftercare issues. I don't think I do either. Even if the opposite is true, and Placeboderm is a fantastic improvement on the standard APP aftercare pamphlet, our piercer doesn't have the resources to really establish if a new soap is the basis for this improvement or not.
Not to mention, that these piercings weren't pairs. Our fictional piercer didn't do 100 sets of ear lobes, with identical jewelry, using soap on one side and then using APP aftercare on the other.
Add to that how small the sample size is... well you can see where this is going.
Chemicals for hard surface disinfection, skin prep, and aftercare simply cannot be evaluated by body piercers. We need the help of scientists and doctors, as well as government oversight. Does this mean that piercers will have a tough time innovating industry specific products? Absolutely. Is this a problem? No. 
We need to see this as it is: a blessing. There are some hoops that are really tough to jump through, and those hoops are tough to jump through for a reason. When I use a hard surface disinfectant on my instrument stand, my client is (unknowingly) happy that this disinfectant was government approved, tested for several months or years by the company who made it, and slowly adopted by a skeptical medical industry, followed by the piercing industry.  We consistently strive for legitimacy:  jumping through the right hoops, taking our time, and waiting for evidence only lends to our legitimacy.
Piercers understand that there are aspects of their job that they cannot evaluate. The best example of this is autoclave sterilization. We use sterilization indicators, sterilization integrators and regular spore tests to double and triple check the efficacy of our sterilizers. We accept that we cannot “like” the way our sterilizer works, we need scientists and engineers to provide us tools that can allow us to trust the way our sterilizer works.
As of late, I read a lot of piercers talking about how they “like” certain chemicals. I've been guilty of saying things like that myself. What we actually mean is we trust certain chemicals. This is an important distinction. Trust needs to be earned. When a new product is introduced, it is always best to be skeptical: make every product earn your trust. 
Step one, get approved for the use intended for your chemical by the government agency your particular product is regulated by. There are, indeed, products that do not require government approval.  When a product tries to “side step” FDA approval, for example getting a product approved as a soap when the people making it intend it to be used as a skin prep... this should draw our strongest skepticism, not our best apologetics.  
Step two, publish studies about your product.  Ideally, have published studies comparing your product to other products that compete with it. An example of this would be a peer reviewed study of the healing of a specific kind of wound with several different aftercare treatments.  
Step three, market your product. 
It really is that easy, yet all too often, we settle for step three alone.  
It's after these criteria have been met that we piercers can discuss what we “like” or “don't like”. We need to be able to discuss and debate the merits of certain products, and without the evidence to back up an argument, we truly are just making noise.  When piercers are skeptical of products, it is body piercing at its best.  We should demand data.  We should demand rigorous science.  Over time, this will actually ease the introduction of industry specific products, as we will have proven to the companies that want to sell us something that we need more than marketing, more than appeals to our biases, and more than popular spokes people for our industry to embrace their product.