# PSK Reporter and WSPR spot archive

A continuous, deduplicated archive of the PSK Reporter spot stream and a
combined WSPR feed, published as Parquet you can query straight off the web
with no credentials and no download.

## Finding data

One series, one file per UTC day, keyed by the date of **transmission**:

```
spots/daily/day=2026-08-12/spots.parquet     PSK Reporter spots
wspr/daily/day=2026-08-30/spots.parquet      WSPR spots
integrity/pskr/day=2026-08-12/day.json       how complete that day is
manifest/pskr.json                           every sealed day
rollups/band_hour/pskr/day=<date>/           per hour, band and mode: counts, stations, SNR
rollups/station_day/pskr/day=<date>/         per callsign: when it was active, and on what
```

Keys are deterministic, so a URL can be built from a date. There is no special
prefix for notable events: the partial solar eclipse of **2026-08-12** is at
`day=2026-08-12` like any other day.

```sql
SELECT band, count(*), median(snr_db)
FROM read_parquet('https://archive.spots.radio/spots/daily/day=2026-08-12/spots.parquet')
WHERE band = '40m' GROUP BY 1;
```

For anything spanning many days, use the rollups. "20m spots per hour across
the whole archive" reads a few MB from `band_hour` instead of opening every
daily file, and returns the identical answer.

Files are sorted by `(band, sender_callsign, transmit_start)`, so a single-band
query reads only the row groups holding it. A one-band, one-day query over
1.6 million spots returns in about 0.2 seconds.

## Four things that will mislead you if you do not know them

**A spot is not a decode.** PSK Reporter reports a given transmitter, receiver
and band combination roughly once every five or six minutes, not once per FT8
cycle. Over a 55 minute sample the median gap between consecutive reports of
the same pair was 345 seconds, and 444,435 of 447,390 pairs appeared exactly
once. Any per-pair metric has to be aggregated over thousands of pairs before
it means anything. Aggregate volume is not the constraint: even a narrow
geographic zone yields thousands of spots per five-minute bin.

**Reports arrive late, with a long tail.** Median lag from transmission to
arrival is 69 seconds, p90 277, p99 459, worst case about 45 minutes. That is
why days are keyed on `transmit_start` and sealed two hours after midnight.
`received_at` is when this archive received the spot, and is not a property of
the radio path.

**`source_topic` is not always a partition.** Most days come from one capture.
On a day where two rigs overlapped, the same spot can arrive on both feeds;
deduplication keeps one row, and `source_topic` records whichever feed
delivered it first. Where that matters, `source_coverage` in the day's
integrity file gives what each rig contributed and how much both had.

**Not every day is the same feed.** Days before 2026-08-30 came from a rig
subscribed to `pskr/filter/v2`, which requires a valid Maidenhead locator at
both ends. From 2026-08-30 the archive takes `pskr/filter/v2raw`, a superset.
A spot count compared across that boundary is not comparing like with like, and
the `source_topic` column is how you see it.

## How complete it is, and how we know

Every stream is captured by at least two independent collectors that do not
coordinate, so a spot present in one and absent from the other is a measured
loss rather than an estimate. Each day publishes its own accounting:

- `missing_vs_union`, per collector: spots another collector had and this one
  did not. Exact.
- `gap_upper_bound`: derived from the contiguous `sq` sequence. A genuine upper
  bound, not a loss figure, because it also counts every spot the upstream
  ingester filtered before publishing. On the `v2` days that is around 12% and
  says almost nothing about capture quality. Use `missing_vs_union`.

Hourly files under `spots/live/` are marked `"provisional": true`, because an
hour compacted while it is still arriving shows shipping lag as apparent loss.
Only the sealed daily figures mean what they say.

## Columns

| Column | Meaning |
|---|---|
| `transmit_start` | when the transmission started, UTC. For FT8 a real correction: 15 seconds are subtracted when the receiver's software timestamps the transmission rather than the decode |
| `report_time` | the time PSK Reporter recorded |
| `received_at` | when this archive received the spot |
| `band`, `mode`, `frequency_hz` | 20m, FT8, Hz |
| `snr_db` | signal to noise as the decoder reported it |
| `sender_callsign`, `sender_locator`, `sender_adif_cc` | transmitting station |
| `receiver_callsign`, `receiver_locator`, `receiver_adif_cc` | reporting station |
| `sequence_number` | PSK Reporter's own row number, contiguous, which is what makes loss measurable |
| `source_topic` | which upstream feed delivered it |

## Notable dates

**2026-08-12**, partial solar eclipse. Umbra first touches Earth 17:02 UTC,
greatest eclipse 17:46 at 65.2N 25.2W, totality over Iceland about 17:48 and
northern Spain about 18:27, umbra leaves Earth 18:32. Partial phase over the UK
roughly 17:15 to 19:00. `reference/eclipse_track.csv` is the umbral centre line
at two-minute intervals, from NASA's published path table, which is what makes
distance-to-track a fixed property of a path rather than of a moment. Capture
that day begins 01:16 UTC, so there is a same-day baseline before the event.

## Provenance

Spot data originates from PSK Reporter (pskreporter.info), operated by Philip
Gladstone, and is published here with his agreement. Captured and packaged by
`pskr-archive` (github.com/m0lte/pskr-archive). Days before 2026-08-30 were
captured by `pskr-recorder` (github.com/m0lte/pskr-recorder) and imported.

The archive is public. Help yourself.