Objectives To determine the discrimination characteristics of a new algorithm and two existing symptom scoring systems for identification of patients with suspected colorectal cancer.

Design Derivation of algorithm by a case–control study and assessment of discrimination characteristics using receiver operating characteristics (ROC) curves. Three colorectal cancer scoring systems were investigated. The Bristol–Birmingham (BB) equation, which we derived from a large primary care dataset; the CAPER score, previously derived from a primary care case–control study and a symptom score derived from National Institute of Clinical Excellence (NICE) guidance for urgent referral of symptomatic patients. Their discrimination characteristics were investigated in two datasets: the BB derivation dataset and the CAPER score derivation dataset. The main analyses were ROC curves and the areas under them for all three algorithms in both datasets.

Setting Electronic primary care databases.

Main outcome measures Diagnosis of colorectal cancer.

Results In the BB dataset, areas under the curve were: BB equation 0.83 (95% CI 0.82 to 0.84); CAPER 0.79 (95% CI 0.79 to 0.80); the NICE guidelines 0.65 (95% CI 0.64 to 0.66). In the CAPER dataset, areas under the curve were: BB 0.92 (95% CI 0.91 to 0.94); CAPER 0.91 (95% CI 0.89 to 0.93); NICE guidelines 0.75 (95% CI 0.72 to 0.79). In subjects under 50 the discrimination characteristics of NICE referral guidelines were no better than chance.

Conclusions Both multivariable symptom scoring systems performed significantly better than NICE referral guidelines.