Squid not caching from npm, but is from curl
I'm hoping I'm being blind here, but I'm trying to cache registry.npmjs.org very aggressively using squid, but really can't get it working.
I've compiled squid with --enable-http-violations, and I'm getting the warnings when it starts, and I'm using the following cache line:
refresh_pattern -i registry.npmjs.org/.* 900 20% 4320 ignore-no-store ignore-auth ignore-private ignore-no-cache ignore-reload override-expire override-lastmod
Now it seems to cache as I expect, when using curl, but not when using the npm client. I've put the two logs below but I can't see what header/difference would be causing one to cache but the other not to
Using curl (caches as expected):
1488230087.926 0 127.0.0.1 TCP_MEM_HIT/200 2710 GET http://registry.npmjs.org/should-util - HIER_NONE/- application/json [User-Agent: curl/7.51.0\r\nAccept: */*\r\nProxy-Connection: Keep-Alive\r\nHost: registry.npmjs.org\r\n] [HTTP/1.1 200 OK\r\nServer: nginx/1.10.1\r\nContent-Type: application/json\r\nLast-Modified: Mon, 08 Aug 2016 15:17:14 GMT\r\nETag: "57a8a27a-871"\r\nCache-Control: max-age=300\r\nContent-Length: 2161\r\nAccept-Ranges: bytes\r\nDate: Mon, 27 Feb 2017 20:44:44 GMT\r\nVia: 1.1 varnish\r\nAge: 0\r\nX-Served-By: cache-lcy1134-LCY\r\nX-Cache: MISS\r\nX-Cache-Hits: 0\r\nX-Timer: S1488228284.071646,VS0,VE18\r\nVary: Accept-Encoding\r\nConnection: keep-alive\r\n\r]
Using npm client (does not cache):
1488230089.133 89 127.0.0.1 TCP_MISS/304 368 GET http://registry.npmjs.org/should-util - HIER_DIRECT/151.101.16.162 - [Accept-Encoding: gzip\r\nversion: 4.1.2\r\nAccept: application/json\r\nReferer: view should-util\r\nnpm-session: 169f8528ea8fd926\r\nnpm-in-ci: false\r\nUser-Agent: npm/4.1.2 node/v7.6.0 linux x64\r\nIf-None-Match: W/"57a8a27a-871"\r\nIf-Modified-Since: Mon, 08 Aug 2016 15:17:14 GMT\r\nConnection: keep-alive\r\nHost: registry.npmjs.org\r\n] [HTTP/1.1 304 Not Modified\r\nDate: Mon, 27 Feb 2017 21:14:49 GMT\r\nVia: 1.1 varnish\r\nCache-Control: max-age=300\r\nETag: W/"57a8a27a-871"\r\nAge: 83\r\nX-Served-By: cache-lcy1132-LCY\r\nX-Cache: HIT\r\nX-Cache-Hits: 2\r\nX-Timer: S1488230089.111498,VS0,VE0\r\nVary: Accept-Encoding\r\nConnection: keep-alive\r\n\r]
Мы столкнулись с аналогичной проблемой с использованием nginx proxy_cache для кэширования прокси-объектов для NPM - на speedtest.net мы остановились на этом проекте: https://github.com/mixu/npm_lazy для кэширования запросов к NPM. Он отлично работает и с приватными модулями. Мы использовали его на более чем 40 000 сборках без проблем, и он без проблем справлялся с npm-выходами из строя. При этом мы столкнулись с проблемами с ним и с Yarn, хотя, возможно, это был Yarn по ошибке.