From ab976ea911403be2866b479ad8aa51d8b4b8c2d0 Mon Sep 17 00:00:00 2001 From: herbygitea Date: Mon, 20 Apr 2026 16:36:41 +0000 Subject: [PATCH] Create cloudflare-cache-warmer.md via n8n --- Tech/Projects/cloudflare-cache-warmer.md | 80 ++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 Tech/Projects/cloudflare-cache-warmer.md diff --git a/Tech/Projects/cloudflare-cache-warmer.md b/Tech/Projects/cloudflare-cache-warmer.md new file mode 100644 index 0000000..c8dddd7 --- /dev/null +++ b/Tech/Projects/cloudflare-cache-warmer.md @@ -0,0 +1,80 @@ +--- +project: cloudflare-cache-warmer +type: project-plan +status: active +path: Tech/Projects +tags: + - pbs + - website + - n8n + - cloudflare +created: 2026-04-20 +updated: 2026-04-20 +--- +# Cloudflare Cache Warmer — n8n Workflow + +## Problem + +Recipe pages fall out of Cloudflare's edge cache between human visits. When +Googlebot crawls a cold page, it hits WordPress directly — full PHP/plugin +execution, slow response. Google deprioritizes slow pages and marks them +"Crawled - currently not indexed." + +## Solution + +Scheduled n8n workflow that fetches the sitemap daily and hits every recipe +URL to keep Cloudflare's cache warm. Googlebot always gets a fast cached +response. + +## Architecture + +1. **Schedule Trigger** — runs once daily (off-peak, e.g. 4:00 AM ET) +2. **HTTP Request** — fetch ` +https://plantbasedsoutherner.com/sitemap_index.xml` +3. **XML Parse** — extract child sitemap URLs from the index +4. **Loop** — for each child sitemap: + - **HTTP Request** — fetch the child sitemap XML + - **XML Parse** — extract all `` URLs +5. **Loop** — for each URL: + - **HTTP Request (GET)** — hit the URL (this warms the cache) + - **Wait** — small delay (1-2 seconds) between requests to avoid +hammering the server on cold hits +6. **Google Chat Notification** — post summary to `webadmin` space: total +URLs warmed, any errors + +## Design Notes + +- Use the n8n Loop node (not Split In Batches) to serialize requests and +avoid overloading Apache workers +- The HTTP Request to warm cache only needs a GET — no need to process the +response body +- Consider a 1-2 second delay between requests so cold misses don't stack +up concurrent PHP workers +- Edge TTL is currently set to 8 days, so daily warming keeps everything +well within window +- As recipes grow, workflow scales automatically since it reads from the +sitemap + +## Validation + +- After first run, spot-check 3-4 recipe URLs in browser DevTools → Network +→ `cf-cache-status` should be `HIT` +- Monitor Google Search Console "Crawled - currently not indexed" count +over the following 2-4 weeks — should decrease + +## Open Questions + +- [ ] Confirm best time of day for the schedule (4 AM ET?) +- [ ] Decide whether to warm ALL sitemap URLs or filter to just recipe/post +URLs +- [ ] Should errors (timeouts, 5xx) trigger a separate alert or just log in +the summary? + +## Future Enhancements + +- Add a check: if `cf-cache-status` is already `HIT`, skip the URL (reduces +unnecessary requests as traffic grows) +- Track cache hit/miss ratio over time via a simple counter in pbs-hub + + +...sent from Jenny & Travis \ No newline at end of file