Critical Evaluation of Product Ion Selection and Spectral Correlation Analysis for Biomarker Screening Using Targeted Peptide Multiple Reaction Monitoring

Abstract

Introduction

Tandem mass spectrometry (MS/MS) has emerged as a cornerstone of proteomic screens aimed at discovering putative protein biomarkers of disease with potential clinical applications. Systematic validation of lead candidates in large numbers of samples from patient cohorts remains an important challenge. One particularly promising high throughout technique is multiple reaction monitoring (MRM), a targeted form of MS/MS by which precise peptide precursor–product ion combinations, or transitions, are selectively tracked as informative probes. Despite recent progress, however, many important computational and statistical issues remain unresolved. These include the selection of an optimal set of transitions so as to achieve sufficiently high specificity and sensitivity when profiling complex biological specimens, and the corresponding generation of a suitable scoring function to reliably confirm tentative molecular identities based on noisy spectra.

Methods

In this study, we investigate various empirical criteria that are helpful to consider when developing and interpreting MRM-style assays based on the similarity between experimental and annotated reference spectra. We also rigorously evaluate and compare the performance of conventional spectral similarity measures, based on only a few pre-selected representative transitions, with a generic scoring metric, termed T_corr, wherein a selected product ion profile is used to score spectral comparisons.

Conclusions

Our analyses demonstrate that T_corr is potentially more suitable and effective for detecting biomarkers in complex biological mixtures than more traditional spectral library searches.

Introduction

Clinical proteomics depends on the identification of putative biomarkers that correlate with disease status and the subsequent confirmation of differential abundance of candidates in large numbers of patient samples. Currently, tandem mass spectrometry coupled with high performance liquid chromatography (LC-MS/MS) has become the prevailing technique for both high throughput shotgun identification and subsequent verification of potential biomarkers (1–3). Associated software tools and statistical measures have likewise become essential for interpreting the resulting large spectral datasets so as to accurately determine the identities of the corresponding cognate proteins (4).

In general, two types of automated computational procedures have been developed to reconstruct peptide sequences from MS/MS spectra. The first class of techniques attempts to solve this problem by interpreting the entire set of product ions recorded in the experimental spectra. Such tools can be further divided into two major subcategories. De novo methods, such as the PEAKS algorithm (5), attempt to deduce either part (i.e., sequence tags) or the complete peptide sequence based solely on the observed spectral features based on finding the informative b and y ion fragment series characteristically embedded in experimental peak lists. Conversely, database search tools utilize scoring algorithms to compute the similarity (closeness of fit) between each experiment spectra and theoretical representations of peptide product ion patterns generated in silico as a pragmatic solution for large-scale proteomic profiling. For instance, the popular SEQUEST program (6) uses a correlation function to compute the spectral similarity of predicted patterns of b and y ions to the experimentally observed peaks; its variants have been recently developed (7, 8) that improve search speed by novel algorithms. Determining the confidence associated with a tentative match is paramount. For example, the Mascot database search program (9) assigns a score based on the probability of obtaining a close match by chance alone. Hybrid approaches (10, 11) have recently been introduced combining the merits of de novo and database search techniques. However, information about specific ion intensities is not exploited in the scoring functions of most database-search algorithms. While software tools, such as MassAnalyzer (12), can predict ion intensity patterns in silico, this has not been widely implemented in common database-search routines. Uncertainty in such intensity predictions may bias a spectral correlation or render the score meaningless.

The second school of peptide identification tools is based on the direct comparison of experimental spectra. The rationale is that spectra are representative, and hence should generally be highly similar for the same peptide even if obtained independently, assuming the data are produced by analogous instruments under similar conditions. In this paradigm, a compendium of annotated reference spectra is first established, based either on synthetic peptides or recombinant proteins or from available high confidence identifications. Then, unassigned spectra are compared to the library and subsequently identified based on a highly similar match. Pairwise similarity between two experimental spectra can be computed as the normalized inner product (13) or variants such as the correlation coefficient (14). Popular tools in this class include Pep-Miner (15), MS2Grouper (16), X! Hunter (17), and BiblioSpec (18). Such methods have the advantage of offering fast, and potentially more reliable, identifications. The matching process has even been extended to document or verify instances of posttranslational modifications (19, 20).

Unlike clinical discovery studies, wherein large numbers of proteins must be identified, usually in a few biological specimens, biomarker validation is typically restricted to evaluating a subset of promising leads across many samples. Multiple reaction monitoring (MRM) is particularly well suited to this focused form of proteomic detection, since it offers the potential for high throughput quantization in a clinical context (21). In a typical MRM screen using a triple quadrupole instrument, the first mass analyzer is used to preselect one or more target peptides of interest based on predefined precursor mass-to-charge (m/z) ratios. These molecules are then fragmented in a second analyzer, resulting in multiple product ions. In a typical MRM experiment, only a few b/y product ions are usually transmitted by the third analyzer to the detector. These reporter ions are often selected with the goal of maintaining a high duty cycle required for accurate quantization in mind. Consequently, protein identifications are based solely on detecting these signature transitions. The relative intensity profiles of these selected fragment ions could likewise potentially be incorporated into a descriptor. In this respect, we (22, 23) and others (24) have also introduced an LC-MS/MS-based screening procedure, termed targeted peptide monitoring (TPM) (22, 23) or product ion monitoring (PIM) (24), that combines selective precursor data acquisition on an ion trap instrument with spectral library comparisons to achieve highly sensitive and specific protein identifications.

Computational tools have been developed to facilitate such MRM/TPM pipelines. For example, the software suite TIQAM (25) was introduced to assist in the selection of suitable signature ions to assay target peptides of interest. A public spectrum library (http://thegpm.org) also allows for searching of signature ions in existing protein spectra based on intensity and uniqueness measures. However, defining which transitions (precursor–product ion combinations) are in fact optimal for screening clinical samples depends on diverse empirical criteria, which are often not rigorously evaluated in a relevant biological context. Here, we study various practical and statistical issues relating to this problem. As a recent study (26) has established the advantages of database-dependent peptide identifications based only on correlating b/y ions, we have explored this same direction from a quantitative MRM/TPM assay perspective, proposing an effective, generalizable “T_corr” (for Transition correlation) scoring metric. The potential practical advantages and general feasibility of applying such a scoring mechanism for routine biomarker validation is addressed.

Methods

Tandem Mass Spectrometry of Synthetic Reference Peptides

We acquired experimental reference spectra for 384 distinct synthetic peptides (JPT technologies, Berlin, Germany) using the TPM concept of continuous targeted precursor isolation and collision-induced dissociation fragmentation using predefined inclusion lists in three complementary manners:

First, we performed continuous direct infusion of individual peptides in a linear ion trap LTQ instrument to capture a comprehensive collection of spectra at three different charge states (+1, +2, and +3) over a 3 m/z isolation window. Two microliters of each crude peptide (0.2–2.0 pmol) was individually injected using an EASY-nLC autosampler/HPLC system (Proxeon, Odense, Denmark) with a nano-electrospray source at an isocratic flow rate of 600 nL/min using an unpacked 100 um I.D. microcapillary fused silica column (Polymicro, Pheonix, AZ, USA) with an ∼10 m tip opening generated using P-2000 laser puller (Sutter, Carlsbad, CA, USA). The LTQ was programmed to monitor precursors generated by the peptides in each of three charge states (+1, +2, and +3) to a maximal range of 2,000 m/z. Total runtime for each peptide was 8.2 min, with 5 min of spectral recording in centroid mode. For each peptide, about 800∼1,000 spectra were produced. The resulting MS/MS spectra of 384 peptides, acquired in ∼52 h of data acquisition, were used to construct a reference spectra library.

Second, an analogous TPM dataset was acquired using targeted LC-based MS/MS, however, monitoring only the +2 charge precursor m/z. A microcapillary column was packed with Luna 3 µm reverse-phase media (Phenomenex, Torrence, CA, USA) over a length of 10 cm. Sample loading and organic gradient chromatography [98% buffer A (95% water, 5% acetonitrile and 0.1% formic acid) to 80% buffer B (95% acetonitrile, 0.1% formic acid, in water) over 45 min] at a flow rate of 300 nL/min was driven by the Proxeon EASY-nLC system. Spray voltage was at +2.5 kV. Twelve precursors of 12 peptides were monitored in parallel over a 1- to 2-s duty cycle, with an average chromatographic peak width of 6- to 15-s FWHM. MS/MS spectra, 6227, were acquired for 12 different peptides mixed together and injected for one targeted LC/MS/MS experiment.

Third, we performed TPM-style LC-MS/MS in the same way as described above after spiking the sets of 12 peptides (approximately 800 fmol) into a tryptic digest of a mouse embryonic stem cell cytosolic extract (5 ug total protein) serving as a complex biological background.

The MS/MS spectra were all mapped to a sequence database composed of the 384 peptide targets by SEQUEST (v2.7).

Precursor–Product Ion (Transition) Correlation Analysis

Given a list of presumably prominent product ions T = <M₁, M₂, ,M _n >, a spectrum can produce an intensity vector f = <I₁, I₂, ,I _n >, where M _{i i} are the m/z and intensity of ith selected daughter ions. To determine the intensity, the peak list is partitioned into a sequence of equal-sized bins (1 Da by default). Each peak is assigned to the bin covering its m/z. If a bin contains multiple peaks, the intensities are summed up. For a given product ion, the intensity is the total intensity in the associated bin. The pairwise correlation between two spectra s1 and s2 under the given product ion series is formally defined as:

(1)

where f1 and f2 are the intensity vectors for spectra s1 and s2 under predefined transition list, respectively.

Other similar statistics for pairwise correlation have also been proposed to calculate the spectral pairwise similarity. For instance, correlation coefficient is also an effective alternative. However, although correlation coefficient has the merits to return value of 1 when two vectors have a perfect linear relationship, even with certain base shifts, our previous study (14) suggests that, in practice, it is only marginally better than the dot product. In this current work, we chose the inner product for implementation of the spectral correlation. The reason is mainly twofold. First, it is much more computationally efficient. Second, this statistic and its variants have already been widely adopted in many widely accepted software engines, such as SEQUEST (6), X!Tandem (27), and spectral comparison tools (13, 15, 16). As the peak intensities for different ions may vary over a few orders of magnitude, square rooting transform was applied to stabilize the intensity variance (14) as a preprocessing step. Mathematically, the T_corr can be deemed as a special version of dot product-based pairwise spectral similarity. The main difference is that the latter is based upon ions across the entire m/z range, while T_corr is limited to preselected product ions. For the sake of brevity and to avoid confusion, the conventional dot product similarity of whole spectra is referred to as S_corr hereinafter.

Moreover, the well established X_corr scores of SEQUEST (6) is also a special version of inner product of two spectra. As SEQUEST is a database search tool, the X_corr is based on the matching between the experimental spectrum and a theoretical one predicted for each candidate peptide. The X_corr formula is described as:

(2)

Where, τ is a displacement value, i.e., the spectrum is displaced by adding τ Da to the m/z value of each peak. Empirically, the displacement values ranges from −75 to 75, which leads to best results for an experiment. To speed up the computation, it is implemented via fast Fourier transform analysis. In this study, we mainly compare results of these three distinct correlation measures.

Monte-Carlo Simulations to Generalize the Performance of T_corr

In a spectrum library search, typically a pool of candidate peptides falls within the predefined mass error tolerance to a given precursor. The discriminating capability of a correlation score therefore reflects the distributions of true- and false-positive matches. To obtain such data, we used the collection of spectra acquired for the synthetic peptides that were correctly identified by the search engine SEQUEST. For each annotated peptide, multiple spectra were available. Therefore, pairwise comparison of all these representative spectra offers a population of correlation scores for true positives, denoted as C_s. Likewise, spectra corresponding to the different peptides with approximately the same precursor mass were used to derive background correlation scores, denoted as C_d.

In general, and as expected, scores (using either T_corr or S_corr) in C_s were higher than those from C_d. To generally evaluate the discriminating capacity, especially when the size of candidate peptide pool varies, receiver operating characteristic (ROC) curves were inferred from the underlying C_s and C_d distributions. Assuming that the size of candidate peptide pool for library search with precursor mass constraint is N, for a given cutoff correlation score S_c, sensitivity was calculated by:

(3)

where , respectively. Similarly, the specificity was computed as:

(4)

where , with random selection.

As there is not straightforward way to model C_s and C_d in closed form, the theoretical computations (Eq. 3 and 4) are difficult. Hence, a Monte Carlo simulation (28), used in our previous study (14), was adopted to estimate the values. Briefly, as Monte Carlo methods are based on iterative random sampling, they are mostly suitable when deterministic results are computationally infeasible or prohibited. In our case, a group of scores are randomly drawn from the two populations C_s and C_d. Such procedure repeats for sufficient times (in our cases 5,000 iterations). Then, the above probabilities are approximated by the observed frequencies satisfying the above conditions (Eq. 3 and 4) and considered to be the estimates of sensitivity and specificity.

We also considered the impact of noise in the spectra. Given the null hypothesis that a T_corr score is achieved by spurious matching to noisy peaks (i.e., a false positive), it is necessary to estimate the significance of the measurement. For this purpose, we generated stochastic intensities for the transition ions, either based on experimental spectra or artificially, to produce a control T_corr population, C_R, mapping to noise. Based on this distribution, corresponding P values were estimated for a given T_corr value s, proportional to the C_R above a given threshold.

Impact of Transition Ion Set Sizes on T_corr Performance

It is evident from Eq. 1 that when more transitions are matched (i.e., more ions detected for monitored transitions), the correlation scores tend to be higher, given that all other conditions are the same. As is well known, peptide fragmentation is a scholastic process. Although some approaches, such as PepHMM (30), MassAnalyzer (12), apply sophisticated models to unveil the intrinsic relations between different ions, most popular database search tools, such as SEQUEST (6) and MASCOT (9), adopt simplified models, which assumes that matching of ions are independent events. In this study, we also assume that the observation of specific transition ions follows a binominal distribution, with probability p to detect any individual product ion. Hence, the probability to detect exactly k ions out of total n product ions is:

(5)

Then the cumulative probability of observing more than k ions is:

(6)

A detailed table of such cumulative distributions can be found in (29). Although the correct peptides have a higher expectation value of p, other false-positive peptides in the database or library also have a chance to match such product ions. This work also provides an empirical study on how the sizes of selected transition sets can impact the T_corr performance (More details can be found in the following “Experimental Results” section.).

Experimental Results

Constructing a Reference Spectrum Library

A critical question for biomarker detection is establishing a statistical significant cutoff value for distinguishing biologically correct transitions from potentially spurious false-positive correlations. To produce a rigorous statistical framework, we first constructed a high-quality reference spectral library composed of three independent annotated datasets, each consisting of thousands of representative MS/MS spectra for 384 distinct synthetic peptides derived from a variety of mammalian transcription factors that were analyzed individually by targeted TPM ion trap fragmentation (see “Methods” section).

The first dataset (DIMS) consisted of about 800 to 1,000 MS/MS spectra acquired for each peptide at three precursor charge state (+1/+2/+3) through direct infusion nano-ESI-MS/MS³. We used this dataset to construct a spectrum library. For the second dataset, batches of 12 synthetic peptides were mixed together and the +2 charge state precursors subsequently fractionated using microcapillary scale reverse phase chromatography and analyzed by MS/MS over a 45-min chromatographic gradient. We used this dataset to investigate the specificity of fragment-ion intensity patterns across an entire LC-MS/MS experiment. To create the third “test” spectral dataset, we increased the complexity of sample by spiking the 12 peptide sets into a complex mixture consisting of a tryptic digest of a mouse embryonic stem cell cytoplasmic fraction. We used this dataset to evaluate the specificity needed to confidently identify specific target peptides.

The spectra of the first dataset were subsequently mapped against a sequence database composed of the 384 peptide targets using the standard SEQUEST search tool. To generate a reliable core reference set, we picked the top ten highest-scoring positive spectra, sorted according to the SEQUEST X_corr score, for each individual peptide. We then computed the pairwise correlation scores of spectra corresponding to different (nontarget) or the same (target) peptides. To derive discrimination scores to empirically estimate the false discovery rate, all matches to different (nontarget) sequences were deemed false positives, while matches to same (target) sequences were deemed true positives.

Transition Feature Selection and Correlation Analysis

A key issue for achieving reliable peptide identifications by targeted MS/MS is the appropriate number and types of transition ions chosen for monitoring. We reason that more inclusive lists will generally result in better performance, at least in terms of accuracy, since empirically the chances of observing an equivalent number of product ions with false positives will be substantially reduced with longer lists. Nevertheless, the leading edge of T_corr for true positives diminishes quickly when the number of false positives becomes large (see Eq. 2 and 3 in the “Methods” section). Hence, for our subsequent analyses, we also examined more closely how the number and choice of reporter product ions influence detection sensitivity and specificity, producing some empirical guidelines for transition ion selection.

The first dataset (see the “Methods” section) is used to construct a spectrum library and derive T_corr and S_corr populations. The pairwise comparison was conducted among spectra of the same peptides. As b/y ions are typically most abundant, the full list of b/y ions was used for the T_corr computation. We gauged the relative effectiveness of these two measures by cross-validation and by investigating their performance at peptide detection when applied to complex biological mixtures, with the expectation of detecting the targets as well-defined chromatographic peaks with characteristic retention times.

Overall, we obtained correlation scores for all the tandem mass spectra in the first reference (DIMS) dataset. Histograms of the resulting correlation scores were generated for spectra matching the correct target peptides (Fig. 1a, b, right panels) and those that mapped to a different (nontarget) peptide (left panels). We observed a much higher frequency for correlation values >0.8 for the spectra (S_corr) or corresponding transition intensities (T_corr) matching the correct sequence than for nontarget matches. The maxima of the distributions were also at ∼1.0 and 0.9 for correct cognate sequences using T_corr and S_corr, respectively, versus only 0.25 for nontarget matches. Interestingly, the histogram of the false-positive T_corr matches was much broader compared to that for S_corr, suggesting less effective discrimination. However, T_corr had a far higher incidence for nearly perfect correlation values for true-positive matches than was seen with S_corr.

Fig. 1

Histogram of correlation scores for pairwise spectral comparisons. A subset of high quality spectra from 384 synthetic peptides were compared pairwise. a The histogram of T_corr scores generated for comparisons of different (left panel) or the same (right panel) peptides. b Analogous S_corr histograms

As T_corr is a special version of S_corr theoretically, we next investigated correlation between T_corr and S_corr metrics in the real application. As seen in a scatter plot analysis of the S_corr and T_corr results based on the spectrum library (Supplementary Figure 1). Supplementary Figure 1a shows the very clear correlation (correlation coefficient 0.879) between the values obtained for true-positive spectra. Conversely, as shown in Supplementary Figure 1b, while the values obtained for false-positive matches were consistently lower, the overall correlation coefficient was only 0.492. It implies that when true positives are identified, S_corr and T_corr have similar values; whereas they can be very different for false positives.

As described in the “Methods” section, for the real application of library search, a one-to-many comparison is conducted (i.e., a pool of candidate peptides with the same precursor mass constraints must be searched). Depending on the search space, the sensitivity and specificity can vary considerably. Therefore, we conducted Monte Carlo simulation to estimate the performance with the search space increases. To make a fair comparison of the discriminating capacity for true and false positives, we derived ROC curves for T_corr and S_corr separately using different pool sizes (10, 100, and 1,000 spectra). Performance area-under-the-curve values are shown in Table 1. T_corr gradually outperformed S_corr when examining high-quality spectra as the size of candidate peptide pool increased.

Table 1 Performance comparison of S_corr and T_corr as determined by ROC area under various sizes of candidate peptide pool

Statistical Modeling to Estimate T_corr Reliability to Noisy Spectra

We next estimated the statistical significance of T_corr scores in the presence of noisy peaks. In other words, we consider the circumstance that all the peaks from an experimental spectrum for selected ions are spurious. Currently, there are different approaches to model the intensity patterns of noisy fragment ion peaks. For instance, PepHMM (30) relies on a uniform distribution, while PeakSelect (31) assumes a Gaussian distribution. In order to make a solid conclusion, we tested three different schemes. First, we examined our experimental spectra to generate an empirical noise intensity distribution after removing the main (b/y) product ions in the peak lists from consideration. The resulting skewed histogram of observed noise intensities is shown in Fig. 2 (with relative values linearly re-scaled to a [0,1] range). Clearly, the intensity distribution from this set of experimental MS2 spectra does not follow either normal or Gaussian distribution. Therefore, we artificially generated markedly different normalized noise intensities with Gaussian and uniform distributions (data not shown). Then, for each peptide in the library, we repeatedly computed T_corr scores for the corresponding reference spectra to the artificially generated ion intensity series by each of these noise models. Figure 3a shows the resulting distributions of T_corr values. Given such a distribution, P value for a specific T_corrX against noisy spectra is the proportion of T_corr above the threshold X. Figure 3b illustrates the P values derived using each of these three models. Despite the overall differences in the curves, the T_corr based on randomized intensities were unlikely to achieve high scores (>0.7), making T_corr relatively immune to noisy spectra, boosting the confidence of peptide identification.

Fig. 2

Histogram showing the frequency distribution of “noise” peak intensities based on recorded m/z ratios not associated with primary product b/y ions. The ions were derived from DIMS high-quality reference spectra

Fig. 3

T_corr score distribution for mapping noise to reference spectra and the significance of T_corr. Three different intensity modeling for noise were used to compute the randomized the T_corr. a The empirical frequency of T_corr for each condition. b The P values of T_corr based on the above randomized T_corr distributions, respectively

T_corr Application to LC/MRM/MS

One potential application of T_corr is for peptide identification by MRM using a standard triple quadrupole instrument. In this scenario, only predefined product ions are examined as signature transitions. We have established in a previous study (32) that the overall intensity patterns of MS/MS fragment ions acquired on a linear ion trap (LTQ) are typically highly similar to CID patterns acquired in MRM mode on a triple stage quadrupole. Consequently, the key issue is to choose an optimal transition set, which is unique or most distinct for a given target peptide. Although rules to derive robust transitions remain uncertain, software tools, such as TIQAM (21) and P3 (http://proteome.gs.washington.edu/software/P3/), generally select large y ions (above half the mass of the precursor), since these are typically most pronounced. For instance, certain ions may experience multiple fragmentations, producing smaller satellite ions hence, high m/z ions are supposed to be more distinct.

In order to validate this strategy, we tested correlation performance using either four y ions, or four b ions, or a combination of the two as target transitions for correlation analysis. For the purpose of comparison, we first chose the smallest ions above the half precursor mass threshold in the respective ion series. In the same manner as we did for the full b/y ion series, we derived the score distributions and generated ROC curves after Monte Carlo simulations. Table 2 provides a summary of the results. Indeed, as expected, ions in the upper m/z range appeared to be more discriminating, with y ions outperforming the b ions.

Table 2 T_corr performance using predefined subsets of b and y ions

On the other hand, these standard selection criteria entirely ignored the actual intensities of the fragment ions. Low m/z but nevertheless high intensity peaks may also have the potential to serve as discriminating ions. Hence, as an alternate comparison, we selected the subsets of four b and/or y ions with the most intensity as targets and conducted similar experiments in the same manner. The results shown in Table 3 confirm that selection of topmost intensive ions as transitions is capable of outperforming the standard selection criteria (c.f. Tables 2 and 3).

Table 3 T_corr performance using either the four most intensive y ions only, or the four most intense ions observed (regardless of b or y ion status)

Figure 4 provides a more general comparative view of how increasing the size of transition ion sets gradually improves the performance in terms of ROC area. This sort of analysis can be used to determine the trade-off between the number of transitions monitored and the size of candidate peptides in the library to achieve satisfactory sensitivity and specificity.

Fig. 4

T_corr performance using various numbers of highest intensity b and/or y product ions. The ROC area-under-the-curves were estimated by Monte Carlo simulation based on the reference DIMS library spectra

Cross-Validation to Spectra Acquired by LC-MS/MS

Ultimately, since we are interested in developing clinical MRM-type assays, we examined the effectiveness of the T_corr metric using a second batch of 6,227 MS/MS reference spectra obtained by LC-MS/MS-based profiling experiments. We randomly mixed batches 12 of the synthetic peptides in equal volumes and analyzed these on a linear ion trap mass spectrometer that was configured to continuously capture MS/MS spectra on doubly charged precursor ions over a chromatographic elution time of 45 min. As a control, the spectra were first categorized into two groups by mapping against a sequence database of the 384 target peptides using the SEQUEST search algorithm. The first group consisted of spectra that mapped to an appropriate batch target, which were presumed to be true positives, while the rest were considered false positives and put into the second group. The two sets of spectra were then correlated using both T_corr and S_corr against the reference spectra library built from the first (DIMS) dataset.

Figure 5 shows a histogram of correlation scores obtained for these two groups, while Supplementary Figure 2 provides a Venn diagram of the overlap obtained based on from results obtained with T_corr, S_corr, and the SEQUEST search engine. Although S_corr identified more putative correct targets, the discrimination between the true- and false-positive scores was far more pronounced with T_corr than for S_corr, or X_corr (SEQUEST) for that matter.

Fig. 5

Cross-validation. The 6,227 spectra were separated into two groups for each search engine, depending on whether they were assigned to one of the target 12 peptides. a, b shows the score histograms for true and false positive matches, respectively

To analyze the performance of T_corr in a more biologically meaningful context, we re-performed the same entire analysis after spiking in 12 synthetic peptides into a complex background consisting of soluble mouse embryonic stem cell cytoplasmic proteins. Figure 6 and Supplementary Figure 3 demonstrate highly similar overall patterns, both in terms of coverage and sensitivity, despite that the slightly diminished discriminating performance of the three correlation metrics.

Fig. 6

Comparative analysis of three different search engines. The analyses involved 5,541 spectra generated by LC-MS/MS of batches of 12 synthetic peptides spiked into a complex biological sample. a, b Histograms of the score obtained for properly matched and incorrectly identified spectra, respectively

Cross-Validation to Spectra Acquired by MALDI Mass Spectrometry

In essence, T_corr is based on the assumption that major product ion patterns are most reproducible. Therefore, its performance is comparable to, or often even better, than full spectra comparisons (i.e., S_corr). As a further independent validation of effectiveness, we also tested a batch of 64 MS/MS spectra generated using a MALDI ToF-ToF instrument, using our LTQ DIMS spectra for the reference ion signatures. T_corr and S_corr correctly identified 56 and 50 of the targets, respectively. In addition to higher accuracy, Table 4 provides statistical evidence supporting the advantages of T_corr- over S_corr-type spectral comparisons. First, calculation of a T test shows that the mean score of the T_corr population is significantly higher from that obtained using S_corr. Moreover, T_corr was more robust as the mean difference of T_corr values between the correct (i.e., same) and different (mismatched) peptides was larger than that with S_corr, implying a more distinct separation between true and false positives.

Table 4 Statistics comparison for T_corr/S_corr over the DIMS and MALDI spectra

Discussions and Conclusion

Clinical proteomics remains critically dependent both on the successful discovery and the subsequent verification of protein biomarker correlation with a disease phenotype under study. Validation, in turn, depends on fast, accurate, sensitive, and robust analytical methods. Currently, few rigorous reports have been published on how to optimize MRM-transition selections, and existing software tools, like TIQAM (21), use heuristics to pick product ions using fixed statistical criteria, regardless of the actual experimental spectra.

To alleviate such a dilemma, we have introduced T_corr, a similarity metric for determining spectral similarity to facilitate peptide identification via various modes of MS/MS. Although spectra correlation-based library searching has becoming popular in proteomics studies, most existing tools correlate entire peak lists. In contrast, we explore the feasibility of applying robust correlation methods based on observing much smaller sets of signature ions. This functionality is critical for the general suitability for MRM- or TPM-style targeted mass spectral analyses, in contrast to standard MS/MS spectral comparisons, due to the far more limited ion species coverage obtained by MRM/TPM assays optimized for high throughput protein screening.

In a series of objective experiments, we have established that high intensity transitions, regardless of the actual product ion identity, are a particularly effective alternative approach for peptide identification in the domain of spectral comparisons. Based on our results, we argue that ad hoc choices of most significant ions may have advantage over such existent strategy. Such an evaluation is particularly important, since ideally, one wants to reduce the number of transitions used in an MRM-experiment in order to assign and maintain high duty cycle to achieve reliable LC peak area quantitation.

Our results also demonstrate a consistent, albeit modest, enhancement of performance using T_corr versus the spectral standard dot product, across different experimental scenarios in three benchmark datasets. Though ROC performance may be low when only a few product ions are monitored (i.e., <4), particularly if the candidate peptide search space is large, T_corr performance is not significantly degraded when all putative y and b ions are considered. Hence, we conclude that our correlation algorithm can handle complex samples in a fairly robust manner, while detecting peptide compounds with added confidence. This characteristic of maintaining good performance with biological specimens has two major benefits: (1) biomarkers can potentially be detected with greater sensitivity due to reduction in possible interference by atypical or unspecific fragment ions and (2) analytical run times can be shortened as they need not necessarily have to be extremely highly resolved to establish stringent pattern matching. Short run times, with accurate results, are crucial for high-throughput clinical sampling of human tissue or body fluids samples from large patient cohorts.

The T_corr correlation distributions of true positive and false positive differ significantly when matching all spectra against the reference library of 384 peptides. The high frequency of true-positive hits at correlation scores 0.98–1.0 suggests that we have achieved almost perfect matching. Such correlations cannot be achieved when including all fragment ions, as with S_corr. Even though this dataset is fairly small, the intensity pattern of a predefined selection of fragment ions seemed to be highly specific to each of the 384 different peptides in the database. This is especially interesting from the perspective of dealing with potentially overlapping (i.e., isobaric) precursor ions, since on the computational level, we treated the transition sets as being acquired simultaneously.

In this work, we used a subset of the highest scoring spectra, based on a standard SEQUEST database search, as references for our library comparisons. We have shown that other spectra derived from these same peptides usually have high pairwise spectral similarity in S_corr and particularly in the T_corr metric. In practice, noisy spectra can produce lower scores in library searches (10). However, we note that biased selection of reference spectra (i.e., pristine fragmentation patterns) may not generalize well, potentially resulting in degraded sensitivity in real world applications (i.e., with less than ideal MS/MS scans). Recent studies (11, 16) have proposed methods to derive more robust representative compilations from a group of related spectra mapping to the same peptide. We are currently conducting studies to determine whether such an integrative procedure can be imported to good effect with our T_corr-based library search strategy.

Like other database search approaches, the performance of T_corr may not be satisfactory when the search space is large, i.e., there are a sizeable number of peptides sharing the same precursor m/z ratio. Figure 4 illustrates such a trend. The fundamental reason is that the false positives have greater chances to get high scores, regardless of scoring scheme, when the candidate peptide pool increases. On the other hand, additional information about peptide-specific properties, like chromatographic retention time, can alleviate this problem by substantially shrinking the search space. Recently, accurate retention time prediction (33–35) has been documented, and knowledge of retention time (36) is becoming a useful tool in peptide identification. Our additional experimental observations, as shown in Supplementary Fig. 4, demonstrate that peptide retention times are highly reproducible across different LC/MS runs. Our ongoing efforts aim at exploiting such information to boost the accuracy of T_corr-based peptide identification and quantification.

In summary, this work demonstrates that empirically driven signature ion selection criteria combined with the T_corr correlation measure can be an effective approach to peptide identification through spectral library searching. Although we have based our conclusions on an analysis of spectra derived by ion trap TPM, our study also provides a solid rationale for applying T_corr to targeted biomarker tracking by MRM on a triple quadrupole, which produce spectra that are even more selective and therefore potentially less affected by product ion interferences. Our results of the DIMS experiment underline that specificity of matching intensity patterns of selected fragment ions is extremely high, when high-quality reference spectra are used. Nevertheless, we expect that including retention time, we can substantially, provide satisfactory specificity and sensitivity. Moreover, these highly specific proteomic data based on tandem mass spectrometry can be utilized for biomarker discovery and especially biomarker verification in respect of early stage detection of critical diseases such as cancers in complex sample matrices, where highest sensitivity is needed.