Determination of temporal reproducibility and variability of cancer biomarkers in serum and EDTA plasma samples using a proximity extension assay

Abstract

Background

Proximity extension assay (PEA) is a novel antibody-based proteomic technology. Sparse data have been published concerning the matrix effect of serum vs. ethylenediamine tetraacetic acid (EDTA) plasma and the reproducibility of results obtained using PEA technology.

Methods

We analyzed samples with the PEA-based 92-plex Olink® immuno-oncology (I-O) assay. To estimate the matrix effect, we analyzed paired serum and EDTA plasma samples from 12 patients with biliary tract cancer. To evaluate the reproducibility, we used data from 7 studies, where 6–8 serum samples from patients with pancreatic cancer were used as bridging samples on 3 versions of the panel over a 2.5-years period.

Results

For the study of serum vs. plasma, 80 proteins were evaluable. The mean serum to EDTA plasma ratio ranged from 0.41–3.01. For 36 proteins, the serum and plasma values were not comparable due to high variability of the ratio, poor correlation, or possible concentration effect. For the bridging samples, the mean intra-study inter-assay coefficient of variation (CV) ranged from 11.3% to 26.1%. The mean inter-study CV was 42.0% before normalization and 26.2% after normalization. Inter-study results were well correlated (r ≥ 0.93), especially for studies using the same version of the panel (r ≥ 0.99).

Conclusion

For 44 of 92 proteins included in the Olink® I-O panel, the variation between results obtained using serum and EDTA plasma was constant and results were well correlated. Furthermore, samples could be stored for several years and used on different versions of the same PEA panel without it effecting results.

Introduction

There is an increasing interest in developing panels of cancer biomarkers for diagnostic, prognostic, and predictive applications [1, 2]. Attention has shifted from single biomarker measurements to using multiple biomarkers in combinations to achieve better discriminatory capabilities [3,4,5]. This shift has been aided by the development of new technologies. A novel proteomic technology for biomarker research is the antibody-based proximity extension assay (PEA). This method enables simultaneous measurement of many proteins (from 92 to 384) at low abundance [6]. The PEA technology is based on protein detection by pairs of antibodies linked to DNA oligonucleotides (probes). When a pair of antibodies binds to a target protein, the probes are brought into proximity, hybridized, and extended by a proximity-dependent DNA polymerization event, followed by quantification using real-time polymerase chain reaction (qPCR) [6]. Today more than 500 publications have reported use of this method in biomarker studies, including more than 90 cancer biomarker publications [7,8,9,10,11,12]. One of the most widely used PEA panels in cancer research is the 92-protein immuno-oncology (I-O) panel from Olink® proteomics (Uppsala, Sweden) [10,11,12].

Biomarker studies are often based on sparse and valuable clinical samples from biobanks used in multiple studies. Furthermore, it may not always be possible to obtain the same blood sample matrix from patients in different studies. Two of the most used blood-derived products for proteomics are serum and ethylenediamine tetraacetic acid (EDTA) plasma. Although both matrices can be analyzed using PEA technology, results obtained with the two sample types cannot be directly compared. The only published data quantifying the variation between protein levels in serum and EDTA plasma samples using the Olink® I-O panel are based on matched samples from only 4 healthy subjects [13].

Another methodological problem is the comparison of biomarker results obtained in analyses performed using different array plates over several points in time. When using PEA technology, the proteins are measured as normalized protein expression (NPX) values which are relative quantifications [6]. Therefore, it is not possible to compare NPX values obtained using different plates without normalization using bridging samples. The same bridging samples might be used for several years, enabling researchers to compare the results obtained in different studies.

In this study, we used the PEA-based I-O panel from Olink® and evaluated the variation and correlation between protein results in serum and EDTA plasma samples in patients with cancer. We also assessed the temporal reproducibility and the effect of storage using bridging samples analyzed in several studies over 3 years using different versions of the I-O panel.

Materials and methods

Study cohort and sample preparation for sub-study of differences between protein levels in serum and plasma samples

We analyzed matched serum and EDTA plasma samples from 12 patients with advanced biliary tract cancer included between October 2020 and May 2021 in the CHOCA study—an ongoing prospective observational open-cohort biomarker study of patients with biliary tract cancer. The study was approved by the Regional Danish Ethics Committee (approval number: H-3–2014-055). The samples for serum and plasma were drawn simultaneously prior to first line palliative chemotherapy. The blood samples were drawn into 8-ml serum tubes (Vacuette® Tube 8 ml CAT Serum Separator Clot Activator) and 9-ml EDTA tubes (Vacuette® Tube 9 ml K2EDTA). According to the standard operating procedure (SOP) of the CHOCA biomarker protocol, the tubes were stored at room temperature for 30 to 120 min before they were centrifuged at 2300 g at 4 °C for 10 min. Serum and EDTA plasma samples were then aliquoted into Greiner tubes (Cryo.s™ Freezing tubes, 2 ml, GR-121280, Greiner Bio-One GmbH) and subsequently stored at − 80 °C until analysis. For the analysis we only selected samples without visible hemolysis. The Regional Danish Ethics Committee had approved the study (H-3–2014-055).

Study cohort and sample preparation for sub-study of temporal reproducibility and effect of storage

The bridging serum samples were collected from 8 patients with advanced pancreatic cancer included between March 2017 and July 2018 in the “BIOmarkers in Patients with Pancreatic Cancer (BIOPAC)” study—an ongoing prospective observational open-cohort biomarker study (Regional Danish Ethics Committee approval number: KA-20060113; ClinicalTrials.gov ID: NCT03311776; www.herlevhospital.dk/BIOPAC/). All samples were drawn prior to first line palliative chemotherapy. The 8 bridging samples were analyzed in 7 studies at 7 different time points between November 2018 and June 2021. A description of each study is available in Table 1.

Table 1 Description of original studies from where data was used

Serum samples were collected and processed according to the SOP of the BIOPAC protocol, identical to the processing in the CHOCA protocol (described above). To prepare the bridging samples, the frozen serum samples were thawed at room temperature and centrifuged at 3116 g at 4 °C for 10 min. Each serum was then aliquoted into 11 different Greiner tubes in aliquots of 200 ul and subsequently stored at − 80 °C until analysis of the Olink I-O assay at BioXpedia (Aarhus, Denmark).

Olink I-O assay

We used the 92-plex Olink I-O assay (Olink® Proteomics, Uppsala, Sweden). Data regarding precision, detection limit, measuring range and analytical specificity have been published by the manufacturer [13].

The analyses were performed at BioXpedia A/S, Aarhus, Denmark (www.bioxpedia.com). For the analyses, serum and EDTA plasma samples were thawed, mixed using a vortex mixer, and centrifuged at 400 g for 1 min. Then 1 μL of serum or EDTA plasma was manually transferred and mixed with the antibody-probe mix and analyzed according to the manufacturer’s instructions. Matched samples were analyzed side by side on the plate. All samples for the serum-plasma variability sub-study were analyzed on the same plate. As recommended by Olink, we also included two sample controls (pooled serumsamples from the study), two negative controls, and two inter-plate control (synthetic samples) on each plate. According to the manufacturer’s recommendations, all samples were normalized for any plate effects using the built-in inter-plate controls. Results were reported as NPX on a log 2 scale.

All samples for the serum-plasma variability sub-study were analyzed on the same plate using Olink Target 96 I-O assay v. 3112.

To investigate temporal reproducibility, we used results from six different studies where the same bridging samples (up to 8) had been analyzed. In four studies (BX0182, BX0214, BX0223), all 8 bridging samples were included on all plates. In BX0144, all 8 bridging samples were included on each of 14 of 19 plates. On the last five plates, the eight samples were distributed randomly, and for the analysis they were regarded as analyzed on the same plate. In BX0263, the eight bridging samples were only included on one plate. In BX188 and BX0202, only seven and six bridging samples were included and analyzed on each plate. During the study period, the manufacturer changed the versions and names of their I-O panel. In the first study, BX0144, we used the older version (v.) 3101 (Proseek® Multiplex Immuno-Oncology). In the five subsequent studies (BX0182, BX0188, BX0202, BX0214, BX0223), we used v. 3111 (Olink Target 96 I-O), and in the newest study (BX0263) we used v. 3112. All versions covered 92 proteins (protein list available in Table 2). However, four proteins were changed between v. 3101 and v. 3111; interleukin (IL) 21 (IL21), IL35, interferon (IFN) beta, vascular endothelial growth factor C (VEGFC) were only used in the older version. These were replaced by the following four in the newer version: lymphocyte activation gene 3 protein (LAG3), IL15, mucin-16 (MUC-16), and killer cell immunoglobulin-like receptor 3DL1 (KIR3DL1). Furthermore, the assays had been changed for tumor necrosis factor (TNF) and IFN-gamma between the older and newer versions. No changes were made between v. 3111 and v. 3112. Lastly, when comparing results between studies, proteins were removed if they did not pass quality control in at least one study. When comparing all six studies, five proteins were removed for this reason (IL1 alpha, IL2, IL13, IL33, and arginase-1 [ARG1]). Therefore, only 81 proteins were in common in all six studies.

Table 2 List of proteins included in the study

Statistical analysis

For the analysis of serum-plasma variation, proteins with 75% or more of the measurement below the limit of detection (LOD) in either serum or plasma were excluded from further analysis. Table 3 shows the LOD for each protein assay as reported by Olink [13]. For all 12 patients and all remaining proteins, we calculated the ratio on a linear NPX scale as 2^{(NPXserum − NPXplasma)}. Afterward, we estimated the geometric mean geometric standard deviation (SD), and geometric coefficient of variation (CV) of the serum-to-plasma ratio as well as median, 25% quartile (Q1), 75% quartile (Q3), and interquartile range (IQR).

Table 3 Protein serum and plasma levels and ratio stability.

To estimate variation between serum and EDTA plasma, we calculated the Spearman correlation coefficient plotting serum NPX vs. EDTA plasma NPX. To evaluate a potential association between concentration and ratio, a Passing-Bablok regression of EDTA plasma NPX vs. serum NPX was performed. For each protein, slope and intercept with 95% confidence interval (CI) were calculated and regression line plotted against an identity line. Likewise, modified Bland–Altman plots were generated for all proteins, plotting serum-plasma ratio against the mean plasma-serum concentration, and fitted linear regression using the lm function (stat package). Lastly, we tested the effect of bilirubin level on the ratio using linear regression. A 2-sided p-value of 0.05 was used as the level of significance for regression analyses.

For each protein, the reproducibility of the serum-plasma variation was evaluated using five criteria:

• 1) CV of the serum-plasma ratio was less than 20%. The threshold correspond to the highest inter and intraassay CV for the protein assays used in the panel [13]

• 2) The ratio reported in the Olink validation data document [13] was within range (Q1 − 1.5 × IQR, and Q3 + 1.5 IQR) observed in our study

• 3) The correlation coefficient was above 0.7

• 4) No association between concentration and ratio was identified. A protein ratio was defined as having an association if the slope of the Passing-Bablok regression was significantly different from 1 or a significant association was observed on Bland–Altman plots

To assess the temporal reproducibility and effect of storage, the inter-study, and the intra-study, inter-assay variation was quantified by calculating mean NPX, SD, and CV for each protein and bridging sample on a linear NPX (2^NPX) scale. Afterwards, an interstudy normalization was performed following manufactures recommendation using the bridging samples with plate from BX0150 as reference. The median linear NPX per study per target for all bridging samples from all plates (n = 36) combined was also calculated. Inter-study variation was also assessed using Pearson correlation between median NPX in all studies. The median was chosen because median NPX is recommended to be used for bridging. Inter-study variation was also compared using Bland–Altman plots, plotting the difference in median concentration between studies against the mean of the two median concentrations.

Statistical analyses were performed using R (R Core Team (2019. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria).

Results

The study investigated the serum-to-plasma ratio for 86 proteins using matched serum and plasma samples from 12 patients. Six proteins had more than 75% of the measurements below the LOD in serum or in both serum and EDTA plasma (IL1 alpha, IL2, IL5, IL13, IL33 and fibroblast growth factor 2 (FGF2)). The temporal reproducibility and the effect of storage were assessed using data from 6 studies. In total, we had data from 36 plates where the same serum bridging samples had been analyzed. On 31 plates, all 8 serum samples had been included and on the last 5 between 6 and 7 serum samples were used (Table 1). The temporal reproducibility was assessed for 81 proteins across all 6 studies after removing proteins where the assay was changed between versions, not included in both versions, or failed quality control in at least one study. For 88 proteins, it was possible to compare results across the five studies using the newer version of the Olink I-O. Table 2 lists all proteins included in each sub study.

Differences between protein levels in serum and plasma samples

For most proteins, the serum and EDTA plasma concentrations were similar (Additional file 1: Figure S1). Of the 86 proteins included in the sub-study, the mean serum-to-EDTA plasma ratio ranged from 0.41 to 3.01 (Table 3). Fifty-two proteins had a minor variation, with a mean ratio from 0.80 to 1.20, and the variation seemed to be randomly distributed between samples (Fig. 1). For the 12 samples, bilirubin ranged from 5 to 812 with two patients having a higher bilirubin than 200. There was no significant association between serum-to-EDTA plasma ratio and bilirubin after adjusting for multiple comparisons. Next, we evaluated the reproducibility of the ratio using the above-mentioned criteria. Figure 2 illustrates the consort diagram showing which proteins fulfilled each criterion.

Fig. 1

Boxplot of serum-to-plasma ratio on linear normalized protein expression (NPX) scale for the 86 proteins with more than 75% of samples over limit of detection. Proteins sorted according to median serum-to-plasma ratio. Samples from each patient connected with dashed lines

Fig. 2

Consort diagram

Criterion 1 (CV of the serum – EDTA plasma ratio < 20%): The geometric CV of the ratio ranged from 5.58% to 123.45%. Thirty proteins had a CV of more than 20%. Of note, most of these proteins had either a high or low median NPX in both serum and EDTA plasma (Additional file 1: Figure S2), and the highest CVs were observed for proteins with high or low ratios.

Criterion 2 (The ratio between serum and EDTA plasma reported by Olink Proteomics [13] is within range (Q1—1.5 × IQR and Q3 + 1.5 × IQR) of the observed in our study: Of the 56 proteins with a reasonable CV, there were four proteins where Olink reported a ratio that was markedly different from the range observed in our study [T-cell surface glycoprotein CD4 (CD4), CD40 ligand receptor (CD40), C–C motif chemokine 2 (MCP-1), matrix metalloproteinase-7 (MMP7), tumor necrosis factor receptor superfamily member 12A (TNFRSF12A)]. Seven proteins had both a high CV in our study and a ratio reported by Olink that was outside the range in study [pro-epidermal growth factor (EGF), CD40 ligand (CD40-L), C-X-C motif chemokine (CXCL) 5, CXCL11, C–C motif chemokine (CCL) 17, caspase-8 (CASP-8), latency-associated peptide transforming growth factor beta-1 (LAP TGF-beta-1)] (Fig. 3). The ratio was not reported for six proteins (KIR3DL1, MUC-16, TNF, IL15, LAG3, IFN-gamma) in the data provided by Olink, and it is therefore not possible to compare the results for these proteins. Of note, all six passed the other three criteria.

Fig. 3

Serum-to-plasma ratio in our study and as reported by Olink on linear normalized protein expression (NPX) scale. Red lines indicate boundaries of acceptance (25% quartile (Q) + 1.5 interquartile range (IQR) and 75% Q + 1.5IQR). Blue points indicate serum-to-plasma ratio reported by Olink. Data were not reported by Olink for 6 proteins: KIR3DL1, MUC-16, TNF, IL15, LAG3, IFN-gamma

Criterion 3 (The correlation coefficient found in regression analysis was above 0.7): Seventeen proteins had a poor correlation between results obtained in serum and EDTA plasma, with a correlation coefficient (r) ≤ 0.7. However, 16 of these also had a poor CV or Olink data were not within range, and therefore only one protein—decorin (DCN), was removed due to this criterion. Correlation diagrams for all proteins are shown in Additional file 1: Figure S3.

Criterion 4 (No association between concentration and ratio): For six proteins we identified an association between ratio and concentration. Five proteins had a slope of the Passing-Bablok regression which was significantly lower than 1 (platelet-derived growth factor subunit B (PDGF subunit B), angiopoietin-1 (ANGPT1), CXCL13, CCL23, MMP12), and for one protein the slope was significantly higher (CXCL1) (Additional file 1: Figure S3). A similar result was seen using Bland–Altman plots, where 5 of the 6 proteins (CXCL1, PDGF subunit B, ANGPT1, CXCL13, CCL23) had a significant association between ratio and protein concentration using linear regression (Additional file 1: Fig. S4). Five of the six proteins were already removed due to one of the prior criteria. Only CCL23 was removed due to an association between concentration and ratio.

Fig. 4

Normalized protein expression for four bridging samples in BX0223 and BX0182. Boxplots showing results of all plates (chips) for each protein and bridging sample as measured in BX0223 (number of assays = 5) and BX0182 (number of assays = 9). Each graph represents results from one bridging sample. Results from each assay connected with colored lines

Of the 92 proteins included in the Olink I-O panel, 80 proteins were possible to evaluate on all four criteria for serum-to-EDTA plasma variation reproducibility. Of these, 44 proteins fulfilled all four and 36 failed on one or more of the criteria (Table 3 and Fig. 2).

Temporal reproducibility and effect of storage

In general, we observed a similar intra-study inter-assay variation based on the serum bridging samples in the three studies that used three or more plates and in those that used the newest Olink panel (BX0182, BX0188, and BX0223). For these studies, 88 proteins were in common after removing four (IL1 alpha, IL13, IL33, and ARG1) that did not pass quality control. Very few proteins had a variation higher than 1 NPX, and only a few random outliers were observed. Figure 4 shows NPX values for all assays in BX0223 and BX0182 for 4 of the bridging samples. The three studies had a similar intra-study inter-assay variation. The mean CV was 11.3% in BX0188, 16.4% in BX0182, and 12.3% in BX0223. For the most recent of the three studies (BX0223), where bridging samples had been stored the longest, only seven proteins had a CV higher than 20% (IL2, IL5, CD40-L, endothelial nitric oxide synthase (NOS3), natural killer cells antigen CD94 (KIR3DL), pleiotrophin (PTN), IL4), but except for CD40-L, all were characterized by low NPX values, with several being below LOD. For the oldest study (BX0144) that used an earlier version of the Olink I-O panel, the inter-assay variation was higher. When comparing all 91 proteins used in the panel and passed quality control in the study, the interstudy CV was 26.0%, and when only the 81 proteins in common between all studies were included, the CV was 22.6%.

In all studies, pooled serum samples had been included on all plates as recommended by the manufacturer. For all studies the inter-assay intra-study variation based on the serum bridging samples were similar or better than the inter-assay variation based on pooled serum samples (36% for BX0144, 17% for BX0182, 10% for BX0188, and 17% for BX0223).

The mean inter-study CV across all studies based on 81 proteins measured using the serum bridging samples was 41.3% (range: 17.6–109.9%). After normalization between studies the mean inter-study CV decreased to 26.2% (range 11.3–108.5%). The inter-study variation was significantly lower after removing BX0144, which used the oldest version of the panel (pre-normalization mean CV = 26.9%, range 13.6–73.8%, post-normalization CV = 18.9%, range 7.3–66.1%). The pre-normalized CV was further reduced in the studies that only used v. 3111 (CV = 19.7%, range: 11.8–59.1%), but not the post-normalization CV (mean = 19.05%, range 7.5–67.2%). For the five studies using v. 3111, 3 proteins had a CV higher than 40% [IL4 (CV = 58.9%), MHC class I polypeptide-related sequence A/B (MIC A/B) (CV = 51.7%), and IL5 (CV = 41.6%)], and all three had low median NPX values and/or several samples below LOD. Pearson correlation analysis (Fig. 5) showed a high correlation (r ≥ 0.97) between results obtained in studies that used the newer version of the panel (v. 3111 and v. 3112) and highest for studies using the same version. The correlation decreased (r = 0.92–0.93) when results from BX0144 were compared with the six other studies, including the 81 proteins in common. One protein, LAP.TGF.beta.1, had a large difference between BX0144 (NPX = 2.99) and the other studies (NPX: 9.76–10.56). When LAP.TGF.beta.1 was removed, the correlation between the studies improved (r = 0.95–0.97). The Bland–Altman plots showed a similar pattern, with a smaller variation between studies using the same panel but a larger variation between the BX0144 and the other studies. Of note, no trends were observed in the plots (Additional file 1 Figure S5).

Fig. 5

Correlation diagram comparing bridging sample results between studies using median NPX for all proteins in common between BX0144 and BX0182 and the 5 other studies. In each plot, a dot denotes a given protein. Latency-associated peptide transforming growth factor beta-1 (LAP TGF-beta-1) highlighted as a significant outlier

Discussion

To avoid any effect on the results, it is generally recommended to use the same type of blood matrices (e.g. serum or EDTA plasma) in all phases of circulating protein biomarker studies [14, 15]. However, obtaining the same matrices from all patients in a study may not always be feasible or even possible. Limited data exist regarding serum-to-EDTA plasma variation for the PEA-based protein analyses, including the Olink® I-O panel [13]. In the present study, we evaluated 80 of 92 proteins based on four criteria. Six proteins were not evaluated due to a high number of samples below the LOD, and for 6 other proteins, we were only able to evaluate the serum-to-plasma ratio for three of four criteria. Of the 80 fully evaluated proteins, 36 (45%) had a high variation of the serum and EDTA plasma ratio, a poor correlation between serum and plasma values, or a possible effect of concentration on the ratio. Some of the variability might be explained by some proteins having a concentration at the lower or upper limit of quantification of the assay. A different explanation might be a difference in pre-centrifuge storage time, which might have led to degradation of proteins or cellular leak [16, 17].

Previously, Shen et al. investigated the effect of storage time and temperature on protein concentration in EDTA plasma using Olink® Cardiology and an inflammation panel including 50 of the proteins analyzed in our study. Compared with immediate centrifuging, they identified a significant change in protein concentration in 5% and 10% of the proteins stored at 22 °C for 1 and 3 h [17]. Since we stored the blood for up to two hours at room temperature before centrifuging the tubes, this might have impacted the protein results of our study. Interestingly, the proteins most significantly affected by pre-centrifuging conditions were CD40-L, EGF, PDGF-subunit B, CXCL5, and MMP7 [17]. All these proteins had a high variation or poor correlation between serum and plasma. It was not possible in our study to test whether we would obtain different ratios if pre-centrifuge conditions or storage times were changed because we used a SOP suitable for use in daily clinical practice.

Our study showed that bridging samples can be stored and used for over three years and with different versions of the same PEA panel and give compatible results. The inter-assay CV of the bridging samples was within the inter-assay variation based on sample controls, and for the studies using the newer Olink® I-O panel, our results are also compatible with the validation data document provided by Olink® [13]. The high inter-assay variation in the oldest study (BX0144), using the older version 3101 of the Olink® I-O panel), might be due to it using significantly more plates, which meant that BX0144 had to be run over several days. The inter-study variation was higher than the intra-study, especially when comparing data obtained using the old and new panel. Likewise, the correlation between results obtained using different panels was also poorer than the correlation between studies using the same panel. Some of these results might be explained by a higher inter-assay variation in the oldest study (BX0144), but they might also be due to version changes by Olink®. For the study (BX0263) using the newest panel (v. 3112), we only used one plate, and therefore it was not possible to assess intra-study variation, and comparisons between BX0263 and the other studies are too uncertain for firm conclusions.

Interestingly, most of the evaluated proteins with a high inter-assay CV were classified as having an inconsistent serum-to-EDTA plasma ratio, which points to these proteins being volatile and susceptible to changes due to pre-centrifuge preparation or preanalytical conditions. Likewise, our results might have been affected by the storage time of the samples for 3–51 months at − 80 °C. Enroth et al. investigated the effect of storage time on EDTA plasma concentration of proteins measured with PEA technology and identified 18 of 102 proteins where the protein level was affected by a storage time of up to 30 years, with 5–35% of the variation being explained by storage [18].

In summary, for 44 of 80 proteins included on the Olink® I-O panel and evaluated in the present study, the results obtained using serum and EDTA plasma were well correlated, and the serum-to-plasma ratio remained constant. Furthermore, we showed that bridging samples can be stored for several years and used on different versions of the same PEA panel with compatible results.