BMC Bioinformatics
Tools
Share this article

Email updates

Keep up to date with the latest news and content from BMC Bioinformatics and BioMed Central.

Open Access Correction

Correction: A linear classifier based on entity recognition tools and a statistical approach to method extraction in the protein-protein interaction literature

Anália Lourenço1, Michael Conover23, Andrew Wong4, Azadeh Nematzadeh23, Fengxia Pan5, Hagit Shatkay467* and Luis M Rocha23*

Author Affiliations

1 Institute for Biotechnology and Bioengineering, Centre of Biological Engineering, University of Minho, Braga, Portugal

2 School of Informatics and Computing, Indiana University, Bloomington, IN, USA

3 FLAD Computational Biology Collaboratorium, Instituto Gulbenkian de Ciência, Oeiras, Portugal

4 School of Computing, Queen’s University, Kingston, ON, Canada

5 Microsoft Corp, Redmond, WA, USA

6 Dept. of Computer and Information Sciences, University of Delaware, Newark, DE, USA

7 Center for Bioinformatics and Computational Biology, Delaware Biotechnology Institute, University of Delaware, Newark, DE, USA

For all author emails, please log on.

BMC Bioinformatics 2012, 13:180  doi:10.1186/1471-2105-13-180


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2105/13/180


Received: 14 June 2012
Accepted: 12 July 2012
Published: 27 July 2012

© 2012 Lourenco et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Correction to A. Lourenço, M. Conover, A. Wong, A. Nematzadeh, F. Pan, H. Shatkay, and L.M. Rocha."A Linear Classifier Based on Entity Recognition Tools and a Statistical Approach to Method Extraction in the Protein-Protein Interaction Literature". BMC Bioinformatics 2011, 12(Suppl 8):S12. doi:http://10.1186/1471-2105-12-S8-S12 webcite.

Correction

While reproducing the experiments that we have previously conducted as part of the Article Classification Task (ACT) of the Biocreative III Challenge (BC3), we discovered two errors in our reported results:

1. When computing the performance of two of our four classifiers (VTT3 and VTT5)on the test data, information from class labels was indirectly utilized. This accidental contamination occurred via the additional named entity recognition (NER) features included in these two affected classifiers. Therefore, the performance we previously reported for these two classifiers on test data is higher than it should be. The problem only applies to the test runs under the two classifiers VTT3 and VTT5. Performance reported on the training data for all classifiers and on the test data for the other classifiers remains correct and was not affected by this issue.

2. The values of the area Under the interpolated Precision and Recall Curve (AUCiP/R) performance measure for the test data were reported lower than their true and correct values. This occurred because the official BC3 evaluation script uses the classifier confidence values only if the appropriate variable is checked, which we did not previously do.

Tables 5, 6, and 7 of the original paper [1], which included the affected results, have now been corrected and are attached below.

Table 5. Performance of the submitted classifiers over the test data

Table 6. Summary statistics and variation of the performance of all runs submitted to ACT on the official BC3 gold standard, including our original and our corrected runs

Table 7. Performance of top 20 reported runs for the ACT in BC3

The above issue does not affect any of the results reported for the Interaction Method Task (IMT), nor those reported in tables 1–4 of the ACT.

The corrected results do change some of the conclusions we have drawn in the original paper regarding the ACT, as follows:

1. There is a substantial improvement in the ranking and classification of articles relevant to protein-protein interaction when using the ABNER NER tool [2] over abstracts; this can be seen by comparing the performance of VTT0 (no NER tools) with VTT1 (using ABNER) in Table 5. However, there are only minor gains in performance by applying the additional NER tools NLProt [3] and OSCAR 3 [4] to abstracts; this can be seen by comparing the performance of VTT1 (using ABNER) with VTT3 (using ABNER, NLProt and OSCAR 3) shown in the corrected Tables 5 and 7.

2. Including partially available full-text NER data as reported in the original paper [1], does not lead to classification improvement. Indeed, it hinders the performance of the VTT classifier. As can be seen in the corrected Table 5, VTT3 (without full-text NER features) outperforms VTT5 (with additional full-text NER features extracted with ABNER and the PSI-MI ontology [5]) on all performance measures except accuracy. Therefore, instead of the approximately 3% improvement, which we previously reported, including such full-text data actually leads to a 3-5% drop in performance.

3. Our linear classifier VTT5, which uses abstract and full-text NER features, is not the top classifier and does not outperform the best classifiers submitted to BC3. Our top classifiers are VTT3 and VTT1, which perform at approximately the same level (see Table 5). These two simple, linear classifiers obtain an overall competitive result well above the mean and the 95% confidence interval of the performance of all submissions to BC3 (see corrected Table 5 and 6). However, as can be seen in the corrected Table 7, using the rank product of the four main performance measures, these two classifiers rank 19th and 20th among the 59 runs submitted to BC3,including our own original and post-challenge runs. Based on these results, our team ranks 6th among those participating in the ACT task.

Along with the original submission [1], we provided a URL to demos including all data used in the challenge; the errors reported above were reflected in the demo code. At the same URL, we now provide updated demos, in which the above errors are all corrected (http://cnets.indiana.edu/groups/casci/piare webcite).

References

  1. Lourenço A, Conover M, Wong A, Nematzadeh A, Pan F, Shatkay H, Rocha LM: A Linear Classifier Based on Entity Recognition Tools and a Statistical Approach to Method Extraction in the Protein-Protein Interaction Literature.

    BMC Bioinformatics 2011, 12(Suppl 8):S12. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  2. Settles B: ABNER: an open source tool for automatically tagging genes, proteins and other entity names in text.

    Bioinformatics 2005, 21:3191-3192. PubMed Abstract | Publisher Full Text OpenURL

  3. Mika S, Rost B: NLProt: extracting protein names and sequences from papers.

    Nucleic Acids Res 2004, 32:W634-W637. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  4. Kolchinsky A, Abi-Haidar A, Kaur J, Hamed AA, Rocha LM: Classification of protein-protein interaction full-text documents using text and citation network features.

    IEEE/ACM Trans Comput Biol Bioinform 2010, 7(3):400-411. PubMed Abstract | Publisher Full Text OpenURL

  5. Chatr-aryamontri A, Kerrien S, Khadake J, Orchard S, Ceol A, Licata L, Castagnoli L, Costa S, Derow C, Huntley R, Aranda B, Leroy C, Thorneycroft D, Apweiler R, Cesareni G, Hermjakob H: MINT and IntAct contribute to the Second BioCreative challenge: serving the text-mining community with high quality molecular interaction data.

    Genome Biol 2008, 9(Suppl 2):S5. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL