💾 Automatic backups

Every record the club holds, packed up on a schedule and sent somewhere that is not the club’s own website.

This exists because of how volunteer-run clubs actually lose their records: not a dramatic hack, but the person who set the website up moving away, the hosting quietly lapsing, and eight years of membership history going with it.

What is in it

Every Pavilion table as a CSV, plus the club’s settings, plus a README explaining what each file is and how to get it back into a fresh Pavilion. Passwords, keys and payment secrets are listed by name but not included — a backup that carries the club’s Stripe key is a liability sitting in somebody’s inbox.

How far back it reaches — grandfather, father, son

Keeping the last seven archives gives you a week. The failure that actually catches a club out is not the server dying last night — it is nobody noticing until the AGM that the membership records were wrecked in January. So three windows are kept instead: the last 7 whatever they are, a weekly one for the last 8 weeks, and a monthly one for the last 24 months. About twenty archives, covering two years.

Which archive is a weekly or a monthly is worked out afterwards, not booked in advance. Nothing says “the Sunday one is the weekly”, because WordPress can only run a scheduled job when somebody visits the site — a club whose website is quiet for a week would produce no Sunday backup at all and silently never make a weekly again. Instead Pavilion looks at the archives that exist and takes the newest in each week and each month. A quiet fortnight costs you those two slots and nothing else.

The weekly and monthly ones are always complete copies, never changes-only. A restore point from eighteen months ago that needs another file to make sense of it is exactly the fragility this is meant to avoid.

Changes-only backups

Optional, and off unless you turn it on. A complete copy is taken on a schedule you choose (weekly by default) and the runs in between carry only what has changed since that complete copy — not since the run before. Restoring is therefore two steps and never more: the complete copy, then one changes file. There is no chain to keep intact.

It saves space, not time. Say so to anybody who asks why the nightly run is no quicker: Pavilion still has to read every row to work out which ones changed, because most of its tables carry no “last edited” date. What gets smaller is the archive and the email, not the work.

The changes file also lists every surviving record, not just the altered ones — that is how a restore knows what was deleted, which is the thing a naive changes-only backup gets wrong. A table with no single id column cannot be compared row by row and goes in whole every time; the screen names any that apply to you.

If the complete copy it should be measured against has gone missing, the run promotes itself to a complete copy and says so. It will never write a changes file with nothing to apply it to.

Google Drive

Archives can be copied to the club’s own Google Drive, on a schedule of their own — back up nightly and copy up weekly if that suits. Set it up on the Google Drive tab.

The club makes its own Google credentials. It is the fiddly part and there is no way round it: Pavilion is asking for access to your Google account, and the only honest way to do that is with keys that belong to you and that you can withdraw. The screen walks through it, and the two things clubs get wrong are these.

• Press “Publish app” in the Google console. An app left in Testing has its access expired by Google after seven days — so the transfers work for a week and then stop. This is the single most common failure, and Pavilion names it in the error when it happens.
• Your site must be on https, and the address you register must match exactly, including whether it has www in it. The screen prints the exact address to paste.

Pavilion asks Google only for permission to touch files it created itself. It cannot see the rest of your Drive, and that is by construction rather than by good manners — Google does not return anything else.

Removing the copy from the server

Optional, and only ever after the transfer has been verified: Pavilion reads back the fingerprint Google holds for the uploaded file and compares it with the one on the server. Anything short of a match counts as a failure, the local copy is kept, and it tries again next time. A transfer is never called done on the strength of Google having answered politely.

Three things are never deleted, whatever the setting says: the most recent archive (so a club whose Google access is later withdrawn still has something), the fingerprint file the changes-only backups are measured against, and any archive another kept archive depends on.

If somebody deletes the Pavilion folder in Google Drive, anything already removed from the server is gone. Pavilion notices, stops deleting anything further, and says so loudly — but it cannot get those files back. It is the one place in this feature where a wrong click is unrecoverable, which is why the folder is Pavilion’s own and not one you use for anything else.

Where it goes

Kept on the server (with a retention count), emailed to a nominated address, or both. On the server it lives outside the web root where the host allows it, and otherwise in a protected folder with unguessable filenames — it is downloadable only by an administrator, signed in, because one file holds every member’s date of birth and every payment.

Restoring is a written procedure, not a button

Deliberately. A one-click restore that half-works on a live club is worse than a page of instructions, because it fails in the middle and leaves nobody able to say what state the data is in. The procedure is on the Restore tab and inside every archive.

Silence is not good news

Every run is recorded — when, how long, how big, and why it failed if it did. Repeated failures raise an alert, a run that stops moving is written up as failed rather than showing “in progress” for ever, and the screen warns when the last good backup is stale. A backup that fails quietly is worse than no backup, because the club believes it has one.

The manual export is still free to every club, on every plan, under Settings → Data & GDPR. A club that cannot get its own data out is a club being held hostage. What is paid for here is that it happens without anybody remembering.

Common questions

Will this slow the website down?

It works in chunks of a few hundred rows and picks up where it left off, so a shared host is never asked to do the lot in one go. It runs when the site is being visited anyway, because that is the only time WordPress can run anything.

Will the nightly backup get quicker if I turn on changes-only?

No, and it is worth knowing before you switch it on. Pavilion has to read every row to work out which ones changed, because most of its tables carry no “last edited” date. What gets smaller is the file and the email — often by a great deal — not the time it takes or the load on the database.

How far back can we actually go?

On the standard settings, about two years: the last seven archives, then one a week for eight weeks, then one a month for twenty-four months. The screen tells you the real figure for your club, because it depends on how long backups have been running.

One of these says complete copy when I asked for changes only.

Either it was due — a complete copy is taken on your chosen interval and the changes files hang off it — or the complete copy it would have been measured against had gone missing, in which case Pavilion takes a full one rather than write a changes file that could never be applied. The reason is recorded against the run.

What happens if we delete the complete copy by mistake?

Pavilion will not let you: an archive that another kept archive depends on is held back, and the tidy-up will never remove one. If you delete one by hand from the screen it warns you first and asks you to confirm.

Can it back up to Dropbox or S3?

Google Drive is built in. Dropbox and S3 are not, but the seam a developer would use is documented and Drive is written against it — they were left unbuilt rather than half-built, because a destination that looks like it uploaded and did not is the exact failure this feature exists to prevent.

The Drive transfers worked for a week and then stopped.

Your Google app is still in Testing. Google expires access for a testing app after seven days, every time. Open the Google Cloud console, go to the OAuth consent screen, and press Publish app. Then reconnect on the Google Drive tab. Requesting only the narrow permission Pavilion uses, publishing does not require Google to review you.

Is it safe to let Pavilion into our Google account?

It asks for one narrow permission — files it created itself — so it cannot read anything else in your Drive even if it wanted to. The keys are yours, made in your own Google account, and you can withdraw them at any time from your Google security settings without touching Pavilion. They are stored encrypted, kept out of the backup archives, and never written to a log.

What happens if somebody deletes the folder in Google Drive?

Anything that had already been removed from the server is gone, and nothing can bring it back. Pavilion detects it, immediately stops removing anything else, and shows the loudest warning in the feature. Keep the folder Pavilion made and do not tidy it — and if you are nervous, leave remove from the server switched off, which costs disk space and nothing else.