Hello, I am not sure where to ask this questions, please let me know if I am asking in wrong place.

I'm currently using next 14.1 in the standalone mode. Application is self-hosted and has various dynamic routes that are built with ISR.

And there is custom cache-handler which is using redis.

Bug description:

The trouble we are having is that all non-existing pages are cached with 404 and written into the redis or in fetch-cache folder (if no redis is used) which brings us to a situations where we have a cache misses for every requested URL that we don't have in a sitemap.yaml

Example:

Requests to:
example.com/i-dont-have-this-page-1
example.com/i-dont-have-this-page-2
example.com/i-dont-have-this-page-3

Will render 404 page not found and will set 3 different keys in Redis with TTL of existing page (for example 10 min).

Looking for solution:

The solution I am looking for is to completely disable 404 pages cache or to have only one "404" entry in cache that would be a cache hit on every non existing URL requests. We had a Memory Pressure with our infra team that was caused by a lot of 404 hits and had to manually clear the cache to free up the memory as we have long TTLs for our created pages which brings us to the issue where if you send a number of garbage URLs requests the cache would fill the available memory and the app would crash with Memory Pressure.

Expected outcome:

404 page not cached or better cached but only one cache instance is created.

PS: I've come to https://github.com/vercel/next.js/discussions/35218 issue but it's no longer an available API to use in Next.js 14's versions.
PS2: Also opened a github discussion as I don't know where to ask https://github.com/vercel/next.js/discussions/61370