I have touched on this topic briefly in some of our other posts, [e.g. 1, 2 & 3] but following a recent discussion I decided to explore the topic of irreproducible results further and encourage conversation. During a recent discussion I was told that researchers with a well-established quality system in place were considering dropping it because funders of research did not take it into consideration when awarding grants. Laboratory quality systems help to ensure accuracy, reliability, and timeliness of reported test results. At NFMRI we ask researchers to identify any quality systems including ISO, GMP and GLP that they have in place. There are many reasons why quality systems can be helpful to the translation of research including; confidence in the robustness of methodologies and results, regulatory requirements, attracting potential collaborators, and the training of researchers who may wish to be more competitive with their career opportunities and move into industry. Whilst there are many reasons why research results may be irreproducible the introduction of quality systems can assist to increase the ‘research quality’ of the studies being undertaken. ‘Research quality’ in this case refers not to the importance of the discovery, but the robustness and usability of the results. This focus can be lost when funders of research focus only on “sexy” publishable research and not on other aspects of quality. There are examples of important published discoveries that have been retracted at a later point, as the research could not be reproduced. As it is not a requirement to publish whether or not quality systems are in place and complied with, few academic laboratories have such systems in place. It is unlikely that fraud is a significant issue but that other factors are responsible. One contributing factor may be the focus on publications as discussed “Looking back through the microscope”. In November 2104, Nature Journal published a special issue on the “challenges of irreproducible research” including how they are addressing the issue. Funders of research should also consider how they too influence policy and administration decisions that support research. The global need to address the issue of irreproducible results is clear. In June 2015, the Boston Globe reported on a study that claimed $28 billion is spent in the United States each year on preclinical research that can’t be reproduced by other researchers. This study examined around 24 other studies and concluded that around 53% of all preclinical studies have errors that mean they are not reproducible. 
Earlier studies found that only 20–25% of published data associated with new drug targets were consistent with their in-house findings and the landmark Amgen study found that only 6 of 53 preclinical cancer papers could be reproduced in the company’s labs. The impact of non-reproducible papers is further exacerbated when the hundreds of future articles and citations are considered. Funders of medical research should remember that researchers and their institutions will put extra effort on what is measured and rewarded. If the primary focus is increasingly on headline grabbing new discoveries, then the direction is clear. Worst-case scenario is that translation becomes obsolete if funders of research only reward the opening chapters on a story of discovery and the confidence of potential collaborators is eroded. “Do you know if you’re asking the right questions, including those beyond research, to support the delivery of community benefits?” Join the conversation online and at our conference ‘putting rubber on the road’ on September 9/10th 2015 at the Australian Maritime Museum.