Skip to content
Lock & Mercer

The cutover runbook nobody writes

The build is finished, the redirects are verified, and the thing that takes the company down at midnight is a certificate nobody was watching.

3 min read

In short

What breaks during a website DNS cutover?

Usually mail, not the website. A domain’s web hosting, DNS and email are often held by three different parties, and a cutover planned around the website alone can move records that mail depends on. The site looks perfect and the company cannot receive invoices, which nobody notices for hours because nothing appears broken.

The website is the least fragile part

By cutover day the site has been built, reviewed and tested. It is the known quantity. What has not been tested is the set of things attached to the same domain: mail routing, certificate issuance, subdomains pointing at services somebody set up years ago, and whatever verification records other platforms rely on.

Those are owned by different people, often at different companies, and the person running the migration frequently does not know all of them exist. A registrar account, a hosting control panel, a mail provider and a CDN can be four separate relationships with four separate logins and no shared view.

Inventory the zone before you touch it

Export the current DNS zone in full and read every record. For each one, answer two questions: what depends on this, and what happens if it stops resolving for an hour.

The records that catch people are rarely A records. They are the MX entries and their SPF, DKIM and DMARC companions, the TXT records verifying ownership for other services, and the CNAMEs pointing at things like a status page or a legacy webmail host that somebody will notice only when it dies.

A certificate is the other trap. Automated issuance frequently depends on the hostname resolving to the host doing the issuing. Move the record and the renewal can quietly stop working, which produces nothing visible until the existing certificate expires weeks later. If mail services share that certificate, the failure lands on mail, and it lands without a deployment to correlate it with.

Sequence it in writing

A runbook is a numbered list where each step names the change, the person making it, how to confirm it worked, and how to undo it. It is written before the day, not during it.

Lower TTLs at least twenty-four hours ahead, because a record with a day-long TTL cannot be rolled back quickly and rollback speed is the entire safety margin. Move web traffic before touching anything mail depends on, verify, and only then proceed. Confirm certificate issuance on the new host before the old one stops answering.

Then check mail explicitly, in both directions, from an external address. Not by assuming. Send one and reply to it.

Not on a Friday

Cut over early in the week and early in the day, when the people who hold the other logins are reachable. The reason is not superstition. Propagation, certificate issuance and mail queues all have delays measured in hours, and a fault introduced on Friday afternoon is discovered on Monday by a customer.

Keep the old host running for at least a week. It costs very little and it is the difference between a rollback and an incident.

Watch the right thing afterwards

For the first fortnight the useful signals are not analytics. They are certificate expiry on every hostname including mail, deliverability from an external mailbox, redirect responses spot-checked against the map, and server logs for 404s on paths the inventory missed.

The last one is where the migration finds out what it forgot. Logs after cutover are the honest report on how complete the URL inventory was.

Questions

How long before a migration should I lower DNS TTLs?

At least twenty-four hours, and longer if the current TTL is high, because the old value has to expire from resolvers before the lower one takes effect. Lowering TTLs on the morning of a cutover achieves nothing.

Why did email stop working after a website migration?

Most often because MX or authentication records were replaced when the zone was rewritten for the new host, or because a certificate that mail services shared with the website stopped renewing once the hostname pointed elsewhere. The website is fine in both cases, which is why the fault takes so long to spot.

Should I keep the old hosting after migrating?

For at least a week, and ideally a month. It is inexpensive and it is what turns a bad cutover into a rollback rather than an incident.

Related

  • Migration and rescue

    The content is the easy part. The URLs are the asset.

  • Business Report

    An independent Kenyan business publication, rebuilt as a server-rendered newsroom its editors run themselves.

  • Khendo FM

    A radio station whose website is part of the transmission rather than a poster for it.

More notes