Disabling 'stale-while-revalidate' behavior
Unanswered
Palomino posted this in #help-forum
PalominoOP
I have API requests that I'd like to cache with
{ revalidate: 3600} , but after that time period, it needs to be refreshed. Returning a stale response while revalidating is not the right behavior because the cached response will contain expired image URLs, resulting in broken images for the user. How do you change Next's cache behavior to automatically revalidate expired cache entries before returning them?29 Replies
@Palomino I have API requests that I'd like to cache with `{ revalidate: 3600}` , but after that time period, it needs to be refreshed. Returning a stale response while revalidating is not the right behavior because the cached response will contain expired image URLs, resulting in broken images for the user. How do you change Next's cache behavior to automatically revalidate expired cache entries before returning them?
Not really possible with native feature right now. You need a cron job to trigger background revalidation at 3601st second to refetch new data. This is probably will be expensive for you
because the cached response will contain expired image URLsI suggest posting the image URLs in a separate db/storage first instead of using it directly. Or find a way to get the permanent static link.
PalominoOP
I feel like there should be some option in the fetch call to change that. None of those are an option except disabling cache entirely.
It can't be an option if it is really not designed to be like that.
What you suggests is a feature that requires active monitoring to see if the timer had run out or not, which requires a serverful architecture. That means this wont work if you are deploying on vercel
Next only knows if the cache is valid or not. It doesnt know the exact point when it has become invalid
Thats why using background revalidation is only used if you are okay with slightly stale data
PalominoOP
I'm not sure I fully follow. If Next knows that a cache is invalid (regardless of when that happened), it should be able to allow the original request to proceed as planned, bypassing the stale cache, returning a new value and caching it.
Ah you're right, theres should be an option for that. But for now, the reason why background revalidation is like that is to ensure that your page loads fast even when its being revalidated in the background. Hence only the second request after x time will show fresh data
PalominoOP
Thanks for the clarification. I really wish they would let you opt out of this behavior. It basically forces me to disable the cache for my entire application and slows down all page loads
This shouldnt be a problem if you have users that constantly checks your sites.
PalominoOP
It's pretty common for 3rd party API's to return certain data that expires ofter a period, like imageURLs or paging parameters. Not being able to ensure data is less than x minutes old means cache has to be disabled
Its also pretty common for those API to cache the result in a separate host such that those websites wont get affected by the expiry date, hence why i suggest storing it in a separate db/storage
@aardani This shouldnt be a problem if you have users that constantly checks your sites.
PalominoOP
Right - but this means users have to hit every single page every hour. In an app powered by a CMS with thousands of pages, that's usually not the case
@aardani Its also pretty common for those API to cache the result in a separate host such that those websites wont get affected by the expiry date, hence why i suggest storing it in a separate db/storage
PalominoOP
I agree, that solution would work, but it adds another layer of complexity to the architecture that's probably not worth it for me
Furthermore, try usnig the <Image> component? iirc Next.js creates a cached link under
/_next/PalominoOP
I can probably implement some logic using <Image onError> to check if the link is broken and refetch all data if it is
i meant using the <Image> component from
next/imagePalominoOP
I am using that component. I'm not sure how it helps though
PalominoOP
Hmm... does this work with dynamic images hosted on a CDN? The API request returns a unique url for the image hosted on the CMS. I set the src of the <Image> tag to that url. Will subsequent renders of <Image> tag with that same off-site image url render a cached image?
should be, i havent checked
want to get your attention to the minimumTTL prop too
PalominoOP
Hmm... ok, I can look into that. Looking at the code, I'm using the image url as a background url and not with the <Image> component
its been a hassle if you want to use image as background url yeah
PalominoOP
Also, would my API fetch requests return
x-nextjs-cache: STALE or x-vercel-cache: STALE? If so, I can also force a second request to get fresh dataIm not sure
PalominoOP
Ok - something to look into on a slow day. Thanks so much for your help. I appreciate your taking the time
@Palomino Ok - something to look into on a slow day. Thanks so much for your help. I appreciate your taking the time
Let us know what worked for you, so that i can close the post