Переключатель мог бы быть проблемой здесь. Если управляемый коммутатор, вот некоторые возможности:
when I will need to connect to IPMI, I will just make an SSH tunel like this
ONLY if you can make sure that this works - i.e. you need to have this possibility on pretty much every server. Because if the server that terminates the SSH tunnel goes down - there goes the IPMI ;)
I Personally would nt put a switch there but a small router (Mikrotik) with a switch chip ;) Then use that router to terminate a VPN.
I would really also recommend you to get a solid firewall appliance and use it as a VPN endpoint from the internet. This makes your ILO access independent from a certain server which has to be running, which ILO is actually for and gives you also the necessary protection from bad internet guys.