Said It Here

How to stop cache warming from crashing the server

Running scheduled curl scripts across hundreds of URLs every thirty minutes creates too much parallel traffic as a site grows. This heavy request load hammers the origin server and can easily kill site performance when the sitemap expands.

What people tried

Every workaround mentioned in the threads below. We haven’t tested any of them — and nobody here is claiming they worked.

  1. 1
    relying on commercial speed optimization plugins
  2. 2
    using edge caching services like Cloudflare APO
  3. 3
    triggering full purges only on specific events like updates rather than running warm repeatedly

In their words

Unedited, most upvoted first, each linked to the thread it came from.

“solid approach, but watch your server load if the sites grow. warming every 30 min is fine for smaller sitemaps but can hammer your origin if you're hitting hundreds of URLs in parallel.”source ↗

TopSydeWP · r/Wordpress · 1 upvotes

“Don’t curl your entire site every 30 minutes - that will kill your server with a decent number of pages.”source ↗

bluesix_v2 · r/Wordpress · 1 upvotes

Where this came up

r/Wordpressserver managementwordpress

People with this problem also raised