Interpreting SPI alongside AFP indicators
Source:vignettes/spi-interpretation.Rmd
spi-interpretation.RmdRead SPI alongside the conventional NPAFP rate, timeliness, and specimen quality. The examples below use the simple SPI, so every number can be checked by hand from cases, population and the national ratio. They are illustrative, not results from real data.
The model-based SPI has the same interpretation. It estimates the expected count with a statistical model and adds uncertainty intervals. The points specific to model results are at the end of this article.
What SPI compares
A district reported 50 NPAFP cases over the previous five years, among 1,250,000 child-years under 15. Its earlier rate is 4.0 per 100,000. This year it has 250,000 children under 15, so its expected count is 250,000 × 4.0 / 100,000 = 10 cases.
It reported 8 cases. Its observed-to-expected (O/E) ratio is 8 / 10 = 0.80. Across the country, reported cases equalled expected cases, so the national O/E is 1.00. The district’s SPI is 0.80 / 1.00 = 0.80.
Its NPAFP rate is 8 / 250,000 × 100,000 = 3.2 per 100,000, which meets a target of 3. The NPAFP rate says the district reported enough cases for its population. SPI says it reported 20% fewer than its own history suggests, relative to the country.
Four complementary views
| Dimension | Indicator | Question |
|---|---|---|
| Reporting against a fixed target | Conventional NPAFP rate | Does the number of reported non-polio AFP cases meet the target for the under-15 population? |
| Reporting compared with expectations | SPI | How does the district’s reported-to-expected ratio compare with the national ratio? |
| Timeliness | Notification, investigation, and specimen transport | Are cases notified and investigated, and specimens transported, quickly enough? |
| Specimen quality | Stool adequacy | Are specimens adequate for reliable virological testing? |
A district that previously reported many cases can report far fewer and still meet the NPAFP target. SPI can help identify this decline by comparing reporting with expectations based on earlier years. But years of low reporting can also lead to a low expected count. SPI may then be near 1 even while the NPAFP rate remains below the target.
Reporting against a fixed target. The NPAFP rate is the number of non-polio AFP cases per 100,000 under-15 population per year, compared with a fixed target. The examples use 3 per 100,000 under-15 person-years, matching the study definition; programmes should use the target that applies in their setting. The rate compares reporting with a fixed population-based target, regardless of the district’s history. Meeting it does not establish surveillance sensitivity.
Reporting compared with expectations. SPI compares reported counts with the district’s expected count, based on reporting history and population. The district’s observed-to-expected ratio is divided by the national ratio for the same period. SPI can show a decline in a district that still meets the absolute target.
For the simple SPI, NA means the index could not be
calculated. history_check identifies districts with no
earlier year or no earlier NPAFP case. Even with earlier history, SPI is
unavailable if the national ratio is zero. A missing SPI is not evidence
of adequate reporting; review the NPAFP rate and the available
history.
Timeliness. Timeliness indicators measure how quickly cases are notified and investigated, and how quickly specimens reach the laboratory. A district can meet a reporting-volume target while notifications, investigations, or specimen transport are delayed. The STEPS timeliness component uses specimen transport only.
Specimen quality. Stool adequacy is the indicator used for specimen quality. It summarises whether the required specimens met the programme’s adequacy criteria. Adequate specimens support reliable virological testing.
Absolute and relative reporting
The NPAFP rate and SPI can disagree. In this example each district has 250,000 children under 15 and the national O/E is 1.00, so each district’s SPI equals its own O/E:
| District | Observed | Expected | SPI | NPAFP rate | Interpretation |
|---|---|---|---|---|---|
| A | 20 | 36 | 0.56 | 8.0 | Target met, relative reporting shortfall |
| B | 6 | 5 | 1.20 | 2.4 | Below the NPAFP target; reported-to-expected ratio 20% above the national ratio |
| C | 22 | 20 | 1.10 | 8.8 | Target met; reported-to-expected ratio 10% above the national ratio |
District A meets the NPAFP target with room to spare, but it used to report far more: its earlier rate of 14.4 per 100,000 gives an expected count of 36. District B is below the NPAFP target even though it reported more than its low history suggests. District C meets the NPAFP target and has an SPI above 1. These values alone do not establish surveillance adequacy.
Taken together, the NPAFP target and SPI give four combinations.
spi_compare_npafp() assigns each district-year to one of
them, for either a simple or a model-based result:
| NPAFP target | SPI |
spi_compare_npafp() category |
Reading |
|---|---|---|---|
| Met | Below 1 | SPI below threshold only | Relative reporting shortfall although absolute volume meets the target |
| Not met | 1 or above | NPAFP below target only | The NPAFP rate is below target, although the district’s ratio is at or above the national ratio; its expected count may reflect years of low reporting |
| Not met | Below 1 | Both below | NPAFP rate below target and district ratio below the national ratio |
| Met | 1 or above | Neither below | NPAFP target met and district ratio at or above the national ratio |
National centring
SPI = 1 is the natural reference point: the district’s observed-to-expected ratio equals the national ratio for the same period. An SPI of 0.60 means the district’s ratio is 60% of the national ratio.
Centring changes what a value below 1 means. A worked example shows how the centred SPI and the district’s own ratio can differ:
| Quantity | Value |
|---|---|
| District observed-to-expected ratio | 0.80 |
| National observed-to-expected ratio | 0.50 |
| Centred SPI | 0.80 / 0.50 = 1.60 |
The district reported below its own expectation, but by less than the
country as a whole. Its centred SPI is therefore above 1. The district’s
own ratio, 0.80, is in the oe column of the simple SPI. For
the model-based SPI, centre = "none" in
spi_index() returns the uncentred ratio instead of the
centred SPI. Neither version establishes whether surveillance is
adequate.
Persistent low reporting
Persistently low reporting becomes part of the district’s history, and so part of its expected count. A district can then move towards an SPI of 1 without any increase in reported cases.
In this example the district has 250,000 children under 15 every year and reported 10 cases in each of the three years before year 1. From year 2 it reports 6 cases a year. The national O/E is 1.00. Each year’s expected count uses all earlier years, as the simple SPI does:
| Year | Earlier cases | Earlier child-years | Expected | Observed | SPI | NPAFP rate |
|---|---|---|---|---|---|---|
| 1 | 30 | 750,000 | 10.0 | 10 | 1.00 | 4.0 |
| 2 | 40 | 1,000,000 | 10.0 | 6 | 0.60 | 2.4 |
| 3 | 46 | 1,250,000 | 9.2 | 6 | 0.65 | 2.4 |
| 4 | 52 | 1,500,000 | 8.7 | 6 | 0.69 | 2.4 |
| 5 | 58 | 1,750,000 | 8.3 | 6 | 0.72 | 2.4 |
| 6 | 64 | 2,000,000 | 8.0 | 6 | 0.75 | 2.4 |
SPI identifies the fall in year 2. As the low years accumulate, the expected count drops and SPI rises towards 1, although reporting has not changed. The NPAFP rate falls below the target of 3 in year 2 and stays there. The model-based SPI also learns from earlier years, so the same drift applies to it. The NPAFP target therefore remains essential, and persistent failure to meet it is a reason for closer review.
National deterioration
Centred SPI can stay unchanged when district and national observed-to-expected ratios fall by the same proportion. In this example, both ratios fall by 30% between year 1 and year 2:
| Year | District O/E | National O/E | Centred SPI |
|---|---|---|---|
| 1 | 0.90 | 1.00 | 0.90 |
| 2 | 0.63 | 0.70 | 0.90 |
Centred SPI is unchanged, because the district’s position relative to
the country is unchanged. Review the national observed-to-expected ratio
separately: it is in $national for both the simple and the
model-based SPI. Also inspect absolute reporting rates and counts.
Check the data before interpreting a shortfall
The expected count is a reference based on reporting history, not a reporting target or an estimate of the true number of AFP cases. Interpret SPI within the country whose national ratio was used.
Review population estimates, district boundaries, reporting delays, data completeness, and the observed and expected counts. Small expected counts make ratios unstable. A rising SPI can result from more reports, a lower expectation, or both; it does not by itself show improvement. Differences in underlying NPAFP occurrence can also contribute to a shortfall.
Use the same SPI method and reporting period when comparing years. For model-based SPI, also keep the model specification and centring setting consistent.
What SPI is not
SPI compares reported cases with expected counts, despite the broader word “performance” in its name. It is not:
- an estimate of missed cases;
- a measure of poliovirus circulation or outbreak risk;
- a detection probability;
- a measure of overall surveillance quality.
Avoid these uses:
- comparing SPI values directly between countries calculated separately;
- reading an SPI below 1 as proof of surveillance failure;
- using SPI in place of the NPAFP target;
- mixing simple and model-based SPI, or SPI from different model settings, in one trend;
- reading a zero SPI interval as complete certainty.
Use it with the conventional NPAFP rate, timeliness, and stool adequacy. Using SPI for district review shows how to bring them together in the STEPS field guide.
Model-based results
The model-based SPI is read in the same way as the simple SPI. These points apply only to its additional outputs.
Uncertainty
Read the central estimate (the posterior median) together with its 90% credible interval. An interval entirely below 1 provides stronger evidence of a shortfall. If the interval includes 1, the model does not clearly distinguish the district’s ratio from the reference value.
The interval describes uncertainty in expected counts; reported counts are held fixed. It does not account for all sources of uncertainty, including random variation in reported counts, errors in population estimates, or the choice of model. When no cases are reported, SPI and both interval limits are 0: zero divided by any positive expected count is still zero. This arithmetic result does not establish surveillance failure or certainty about surveillance quality.
Values below 1 are descriptive, not validated action thresholds.
Programmes may choose lower values to focus review on larger shortfalls,
while continuing to consider the conventional NPAFP rate, timeliness,
and specimen quality. For the simple SPI, the field guide defaults to
spi_cut = 0.5. This cutoff selects larger relative
shortfalls; it does not measure uncertainty.
Annual and monthly SPI
Annual SPI is generally more stable for review because many districts
report few AFP cases per month. Monthly results help show how reporting
varies within a year, and year_end_month allows
non-calendar reporting years. The simple SPI uses calendar years.
Incomplete periods should normally be excluded from comparisons.