GOED's New Zealand Oxidation Paper Published
A new scientific paper authored by GOED and some of its members that attempted to replicate the findings from a paper in 2015 by Albert et al has been published in Scientific Reports. The original paper controversially claimed nearly all fish oil supplements in New Zealand did not contain the EPA and DHA stated on the label and were excessively oxidized. The GOED study found that nearly all — 96% — of the products tested complied with regulatory limits for oxidation for edible oils and 91% complied with label claims about EPA and DHA content.
The GOED paper also assessed the products against GOED’s more stringent Voluntary Monograph standards and found that 72% of products in the market met these self-imposed limits for Peroxide Value (PV), 86% met the p-Anisidine Value (pAV) limits and 91% met EPA and DHA claims on the labels. While there is clearly still room for improvement, the results are in line with what GOED has observed in its randomized testing of products from around the world.
Notably, this is the fourth study that has attempted to replicate the results of the first paper in the NZ and Australia markets, and none could do so. GOED’s paper contains the largest sample size of products from these studies and is unique because nearly all the products were analyzed by multiple labs after being blinded to avoid any confirmation bias or method execution issues. The table below shows the differences in compliance between GOED’s study and those of the original study.
Background
In January 2015, a group of researchers from New Zealand and Australia published a paper, also in Scientific Reports, claiming that only 17% of the fish oils in the New Zealand market met PV limits and only 9% of products met their EPA and DHA label claims. The study was promoted heavily to the media in New Zealand and Australia, but also to the New York Times, and ultimately led to a decline in fish oil sales in New Zealand. The results seemed vastly different to what industry members typically observe, and the results were quickly questioned by GOED, Natural Products New Zealand, The Omega-3 Centre and other industry groups.
GOED decided to try to replicate the study by buying every fish oil it could find from online pharmacies in New Zealand and testing each at multiple labs to confirm each lab’s results. After the test results showed the clear majority of the industry was compliant, GOED sent the results to Scientific Reports and urged the journal to correct the paper by Albert et al and/or ask the authors to retract it. The authors did ultimately correct some of the typos in the paper, but stood by their results. After one year, GOED had not heard back from Scientific Reports regarding further action on the paper and decided to publish its results in the same journal, since Scientific Reports’ policy is to welcome studies that are substantial critiques of work that appeared in the journal. By this point, several of the products had passed expiration and GOED re-tested these to see if the extended time within the labeled shelf life could explain the results in the original paper (it did not).
GOED’s results stand alone as the largest sample of tests on New Zealand fish oil products, but also because products were tested at more than one lab in a blinded manner to confirm each test. However, the Therapeutic Goods Administration in Australia also tested 15 products in their local market because of the 2015 paper’s claims and found 100% compliance with Australian regulatory limits, but the results were unpublished and only presented at a conference. In addition, The Omega-3 Centre conducted its own tests on a sample of 10 products from Australia and similarly found 100% compliance with regulatory limits and published the results in Nutrients. More recently, a group from another New Zealand university used a new Raman spectroscopy method to assess EPA/DHA levels and oxidation in 11 products and found 100% compliance with regulatory limits, although this method is still experimental.
GOED’s paper is also unique in that it retested the products that were in their last year of shelf life again one year later because Albert et al had proposed that the excessive oxidation they observed may have contributed to the reported degradation of EPA and DHA. Instead, GOED's study found that increases in oxidative status were minor over the course of the year (and still within limits) and there was no difference in EPA or DHA content after one year.
Evaluating the Differences
In their original paper, the authors listed several potential reasons they felt the supplements could be oxidized:
• Manufacturers could have used inaccurate methods to estimate label content
• Manufacturers relied on data from extractors, rather than testing on their own to see if EPA/DHA levels declined due to oxidation during transport, encapsulation, packaging and storage.
• Seasonal variations in fatty acid composition of fish is not reflected on labels
• Poor quality control during manufacture
• Batch-to-batch variation
In the media and additional papers, the authors have proposed some additional possibilities:
• Weak regulations allow the sale of oxidized fish oils as supplements (JNIM)
• High temperature heating of oils (J Nutr Sci)
The GOED study addresses many of these issues because the participating labs used a variety of validated methods on finished products pulled from retail shelves. It of course does not rule out batch-to-batch variations or seasonal variations in the fishery, but basic knowledge about the supply chain would place a very low likelihood on these options. Many in the industry have claimed that the original authors used the wrong methods, but the authors used European Pharmacopoeia methods for oxidation and a documented method for fatty acids. It’s worth noting that the method for fatty acid analysis is not recognized by any pharmacopoeia, regulatory or monograph body for marine oil or EPA/DHA-rich oil analysis. This method was originally developed to analyze the fatty acid composition of biological tissues. To GOED’s knowledge this method has never been validated for use in EPA/DHA-containing marine oils.
GOED believes that there may have been issues in the execution of the methods that yielded the unexpected results in the original paper. It does not appear from their paper that any of the researchers had significant experience in lipid analysis, nor was the lab accredited for such analyses. It probably is not expected that an academic lab would pursue accreditation or validate its proficiency, but potential issues with the execution of the methods have never been acknowledged by the authors as a possibility.
The authors also claimed that their results were largely consistent with other papers that had reported 11-62% of products exceeded GOED oxidation limits. This is an important point. Certainly, a finding that 11% of the products are above Monograph limits is in line with GOED’s findings, but many of these results are not comparing similar study designs. Many publications apply limits meant for unflavored oils to flavored oils in their adjudication. Others use inapplicable limits for oxidation and claim these are the bar for rancidity. GOED has compiled the primary (PV) and secondary oxidation (pAV) results from these papers together in the table below, and adjusted the results to compare the rate of compliance with appropriate and applicable limits on a like basis. A second table also highlights which study designs utilized criteria that are important to consider before drawing blanket conclusions about an entire industry. These tables refer specifically to oxidation analysis and not to the fatty acid analyses.
* N/A indicates either that pAV was not analyzed in the study or that unflavored oils were not differentiated from flavored/formulated oils and their results could not be separated out.
*Studies receiving a ‘Y’ stated that they either randomly selected products or used some systematic construct to select products for testing, like acquiring all products available in a market.
Where To From Here?
Our industry should welcome criticism — there are countless examples where it has improved the industry from product quality to sustainability. However, serious questions were raised with the original study by Albert et al and, in fact, blanket allegations were levied at the industry. For the future, we would like to see academic critics of industry not overlook some of the points we have made:
• Inflammatory headlines that impugn the reputation of an entire industry have no place in scientific publications without independent confirmation
• Products should only be assessed against appropriate criteria and limits, and these limits (like the GOED Monograph limits) should be properly described
• Greater understanding of the value chain would help create better hypotheses about underlying causes of product quality
• Taking a step back and looking at oxidation in the context of the broader diet is important because exposure to oxidative products from fish oil is insignificant compared to other edible oils like vegetable oil or olive oil
It is also important, however, that industry not overlook some of the points made by Albert et al, including:
• Consumers get little guidance from supplement brands on how to choose high quality products
• There are no limits established on oxidation for fish oil, or any other edible oil, based on safety and this research should be explored
• Tests for specific oxidative compounds are not easily affordable, so PV, pAV and TOTOX are the defaults, but work should be done to explore other areas
• Industry should work cooperatively with critics in academia to determine if issues exist and work to solve them
Overall, this issue has been characterized by hyperbole that has been played out and fueled by the consumer media. GOED thinks important issues have been raised and it has driven us to fund research into many of these questions. We would welcome the opportunity to conduct multi-lab testing with any critics and are willing to fund their participation in laboratory proficiency testing programs like those of AOCS and NIST to ensure that their results are reliable. Lastly, we also invite them to engage our industry and present their suggestions on how industry can improve. GOED is willing to give them a vehicle to do so.