Pilot Study of the utility of text mining and machine learning tools to accelerate systematic review and meta-analysis of findings of in vivo research (360G-Wellcome-201752_Z_16_Z)

Firstly we will convene an expert panel to establish "required" and "desired" performance thresholds for the performance of text mining and machine learning tools. Then, for each of the three tasks of identifying and retrieving relevant publications, extracting meta-data from identified publications, and extracting outcome data from relevant publications we will (1) where not already performed, conduct a systematic review to identify all candidate approaches; (2) implement the most promising approaches using existing systematic review datasets; and then (3) prospectively validate these approaches in ongoing systematic reviews. These tasks will be conducted by a team which brings together expertise in text mining and machine learning as applied to systematic review (Thomas, Ananiadou) and in the conduct of systematic reviews of in vivo data (Sena, Rice, Macleod), supported by external collabortators. The development datasets are (1) a systematic review of in vivo studies in neuropathic pain, (2) a systematic review of in vivo publications from leading UK institutions, and (3) a selection of in vivo publications describing different outcome measures curated on the CAMARADES database. For the validation datasets we will use a systematic review of in vivo models of depression. For each, we will ascertain the sensitivity, specificity and where relevant the accuracy of the text mining/ machine learning approach, and the reduction in human work (eg number of articles needed to screen) possible whilst maintaining performance at the "desired" threshold.