In the Accuracy study, we will:
- Set the acceptable working range
- Set the cutoffs for calling a sample positive or negative
- Assess how well our assay agrees with an alternative assay
Clinical samples can be FFPE or fresh frozen. FFPE is usually poorer quality than fresh frozen, so the two sample types are treated separately.
Section 1. Setting the Acceptable Working Range
The “acceptable working range” is based on the observed values from clinical samples. Once the acceptable working range is set, samples that have values outside the working range will be rejected.
To calculate the acceptable working range, we look at the observed values of Texas Red for the wild-type samples and calculate prediction intervals. In R, alpha = (1 – confidence_interval) and P is coverage.
To calculate prediction intervals, we can use either normtol.int
or nptol.int
:
normtol.int (df$Cq, alpha=alpha_num, P= coverage_level, side=2) # for normally distributed data
nptol.int (df$Cq, alpha=alpha_num, P= coverage_level, side=2) #for data that's not normally distributed
The Shapiro test for normality:
shapiro.test (df$Cq)
showed that the Ct values were non-normal, so the non-parametric nptol.int
was used to obtain the table below:
Sample | Alpha | Confidence | Coverage | 2-sided Lower | 2-sided Upper |
---|---|---|---|---|---|
FFPE | 0.01 | 0.99 | 0.95 | 23.32 | 35.13 |
FFPE | 0.01 | 0.99 | 0.99 | 23.27 | 35.51 |
FFPE | 0.05 | 0.95 | 0.9 | 24.99 | 32.36 |
FFPE | 0.05 | 0.95 | 0.95 | 24.55 | 34.84 |
FFPE | 0.05 | 0.95 | 0.99 | 23.27 | 35.51 |
FFPE | 0.1 | 0.9 | 0.9 | 25.04 | 32.18 |
FFPE | 0.1 | 0.9 | 0.95 | 24.55 | 34.84 |
FFPE | 0.1 | 0.9 | 0.99 | 23.27 | 35.51 |
Based on the table above, we decide on the acceptable working range. We usually go for the broadest range of values because we don’t want to reject too many samples.
Section 2. Set the cutoffs for calling a sample positive or negative
A sample will be called positive or negative based on its ΔCt value. Based on the plot of the ΔCt values with both positive and negative samples, we pick a cutoff that separates the negative samples from the positive samples.
Based on the figure above, we chose a ΔCt cutoff of 5. Samples with ΔCt <= 5 will be mutation-positive and samples with ΔCt > 5 will be mutation-negative.
Section 3. Setting the Acceptable Working Range
This is fairly easy and doesn’t need complicated statistics.
Original Method | Total | |||
Positive | Negative | |||
New Method | Positive | a | b | a+b |
Negative | c | d | c+d | |
Total | a+c | b+d | n |
No comment yet
2 pings
FDA Validation of Companion Diagnostic (PCR test) – Part 1 / n » Pauline PI - investigating science, math, and biology says:
December 21, 2018 at 11:36 am (UTC 8)
[…] takes a few hundred experiments to examine: Part 2. Pre-processing Part 3. Analytical Specificity Part 4. Accuracy Part 5. Run Control Specification Part 6. Reportable Range Part 7. Limit of Detection (LoD) Part 8. […]
FDA Validation of Companion Diagnostic (PCR test) » Pauline PI - investigating science, math, and biology says:
December 21, 2018 at 11:37 am (UTC 8)
[…] takes a few hundred experiments to examine: Part 2. Pre-processing Part 3. Analytical Specificity Part 4. Accuracy Part 5. Run Control Specification Part 6. Reportable Range Part 7. Limit of Detection (LoD) Part 8. […]