How do you handle firewall rules from multiple sources that each need dozens of ports opened?

We have multiple locations that all need to reach the same Azure vnet. I have a network security group for our private DMZ that needs to allow ports 22, 80, 443, 445, 3306, 3389, 8080, 8443, 8843, 8880, and several others.

I'm finding that the 4096 rule limit in Azure is filling up fast.

Since the source IP will always be our network, what are the implications of just allowing ALL from each source and reducing the 35-40 rules per site down to just 1 each. Our Linux hosts use iptables as well and have a more indepth policy that limits the port with source IP.