Большой размер виртуальной памяти ElasticSearch JVM
Как Вы размещаете приложения WSGI? Принятие Вас использует mod_wsgi, следующие комментарии применяются.
Вы используете nginx для статических файлов и как прокси до Apache/mod_wsgi? Это - хорошее начало, поскольку оно дает различные преимущества вне просто лучшего статического обслуживания файлов. А именно, nginx изолирует Apache от медленных клиентов HTTP и позволяет Apache только входить в изображение, когда полный запрос от клиента готов. Дополнительная буферизация в расположении проксирования позволяет Apache разгружать ответ, более быстрый также. Все, позволяет Apache работать с меньшим количеством процессов/потоков и работать более эффективно.
Кроме этого, для mod_wsgi, необходимо действительно объяснить конфигурацию. Вы используете встроенный режим или режим демона? Какой Apache MPM Вы используете? Вы выполняете PHP в том же Apache с помощью mod_php? Вы сделали, гарантируют, что Вы все еще не загружаете mod_python в Apache?
I'm pretty sure that the answer you got from the ElasticSearch community relates to the ZFS ARC (Adaptive Replacement Cache). This of course assumes that your file system is ZFS?
On ZFS the ARC will potentially take up all of the available RAM on the host less 1 Gb. So on a ZFS host tools like top will sometimes show that your physical RAM is close to the limit even if it isn't. This is by design. The ARC will automatically release memory to processes that need memory. What memory the ARC uses counts as kernel memory so you can't really see it in a process output.
On most of the Solaris systems that I look at daily the physical RAM consumption is about 90%. This is not because they are very utilized, it is the ZFS that grabs unused RAM for its own purpose. Don't be alarmed by this. As the ARC is part of the kernel it can release memory to processes that need it at the speed of the light so to speak. Hence - although you can - I typically do not see a point in limiting the size of the ZFS ARC. Better to let ZFS do its job.
So, if we're talking about ZFS, then yes, file system caching doesn't show up as memory consumption on an individual process. You'll need to execute something like:
echo "::memstat" | mdb -k
to reveal how your memory is actually used. The line "Anon" covers all the user land processes that you see in e.g. prstat output.
The other thing for you to know is how the JVM works in terms of memory allocation and release of memory. The JVM grabs memory from the OS as it needs it only restricted by the JVM -Xmx command line parameter. The open question is how (if ever) the JVM will release memory back to the OS if it no longer needs it ? You'll find that is is very difficult to find information on this subject. It seems to depend on which garbage collector is used. As it is difficult to get precise information on this subject (don't know why really) your best option is to assume that the JVM is extremely reluctant to release memory back to the OS. In other words : if you allow a JVM process to grab say 50 GB memory then you better be in a position where you can afford this permanently rather than assuming that this is just a burst.
So if you want to limit how much memory the ElasticSearch process can consume then you need to look into the JVM command line parameters, in particular the -Xmx option.