SREcon16 Europe

How do you secure your internal network when your servers are located on different continents with different providers and you don't trust your network?

IPSec is a great way to secure a network but it's usually deployed as a way of connecting a small group of trusted networks, and both tools and existing documentation reflect this. This is not really an option in some environments where you don't really control the network and want to interoperate across different providers, so you find yourself sailing through uncharted waters at times when trying to build a fully meshed network with IPSec, where each server can establish a secure connection to any other server in its cluster.

We wanted any of our servers around the world to be able to communicate securely with any other. We were using a peer to peer VPN, but it broke down badly at scale and we chose to go with IPSec. It wasn't a smooth transition; the tools were terrible, the documentation was vague and incomplete and we found some horrible bugs, but we survived and want to share with you some of the lessons we learned, what you definitely shouldn't do, and why you might want to do this.