Ordering two redundant internet connections can be done quickly, but getting your network to failover between the two internet lines, or even load balance traffic between them can be a real challenge! In a moment, I will show you how it can be done using policy based forwarding (PBF). If this is your first time here, I’m Lars from Consigas. We call ourselves the Palo Alto Networks experts because the next generation firewall is our passion! It’s what we do all day, every day: migrating firewalls, providing managed services, and most important implementing security best practices. When I started to work with this box, in 2010 nearly anyone knew about Palo Alto Networks But as an engineer I felt that this solution will change the world of cyber security, and yes, today We know it did big time, because it’s one of the few security solutions that can truly secure your network However, there’s a caveat. You need to set it up in the right way in order to be effective because while it’s awesome, it’s not a magic box! So over the years we became Professional Services Partners for Palo Alto Networks, as well as one of the few Elite Authorized Training Centers (eATC), after working in the field for so many years, & being a trainer I would like to share my experience with you! so over the next couple of weeks and months we’ll release new videos and core concepts, explaining the fundamental workings of the NG Firewall, starting with the Threat Landscape, deployment methods, NAT, App-ID, SSL decryption, VPNs and many more! So follow us on LinkedIn, YouTube or Twitter to stay up to date. But now let’s get started with policy based forwarding (PBF)! First of all, the main idea behind policy based forwarding is overriding the routing function, or the routing decision, made by the routing table, for instance, Here we have a default route pointing out to ISP number 1 and now if we use policy based forwarding, we can for instance say “Now traffic from a specific subnet actually goes out of ISP 2”, okay? Now the use case of having two independent ISPs , is actually one of the most common usages for PBF, so let’s have a look at this a little more detail! Let’s have a case where we have two independent ISPs and what we want to achieve is get Redundancy, if one ISP fails, all the traffic should fall over to the other one, and then ideally actually get a bit of load balancing. Meaning using both ISPs at the same time So that we are basically fully utilizing all the resources which are available, okay? How we can do this? First of all on our firewall we have a layer 3 interface on the inside, on the outside a dedicated one per ISP, and then of course we have our Virtual Router, Another thing we could simply do without any policy based forwarding Is to define a default route for each ISP. So let’s say here, we have the route of the ISP which is our next hop! And then what we can do is define a default route which points to this ISP. So the first one here, and then another default route also pointing into the second ISP. With two default routes on the same virtual route… now we can have a problem, because you know the VR has to make a decision where to send it, ISP1 or ISP2. and so we need something based on which we can decide the precedence. For this we can use a metric So we can assign to different routes different metric. Let’s say here we use a metric of 10, which is the default, and then for the second we actually use a higher metric of 20. With this now, by default all traffic arriving here to the virtual router would be sent out of ISP1 and then only if this route goes down everything would then route over ISP2. Now, what’s the problem? The problem is that for this route to go down the only reason would be that this interface goes down, and that’s not really a good test for availability of the internet because in most environments you would probably have here an internet switch, and only if this internet switch goes down then this interface goes down. However, if the ISPs route or maybe even the circuit is down, then the interface is still, the route is still active, the firewall would send out traffic there and nothing fails over, so that’s not really a good implementation, okay? From here what we can do however, is use a new feature called Route Path Monitoring, and that’s a new feature which was introduced in version 8. Route path monitoring is very powerful, so what we can do is we can apply a monitoring profile to the route itself where we’re saying “Ping or send out pings out to the internet from a specific source IP, so four instance our ISP interface, and then ping various IP addresses on the internet” and see if they are available, and if they’re not then cause a failure condition, which then causes the route to be deactivated, and then the next route would kick in, which becomes the default route. The route path monitoring obviously has nothing to do with policy based routing. We didn’t touch yet really on policy based routing, but for the scenario if you want to have very simple fail-over of the ISPs, the route path monitoring is the solution to go with because it’s very simple and very effective. If we have a use case which is a little bit more complex, like for instance, we would like to use actually both ISP at the same time, so send a bit of traffic out of ISP 1 and other bit of traffic out of ISP 2, to effectively use resources, in this case now we need a bigger “gun” which is actually PBF. So in order to apply our policy based routing to this, what do we need to do? First of all we’re gonna remove this route. Okay, so this default route to ISP 1 we’re gonna remove, remove all of this Including obviously the path monitoring, we’re not gonna need this now, and this default route, that’s gonna be on our primary default route. This primary default route points to the secondary, ISP 2. Okay, so we can change the metric here as well to the default metric of 10. And now we use policy based forwarding to define a policy which basically says that all traffic which should go out to the Internet, should actually be sent to the next hop ISP 1. Okay? What is the big difference? Well, first of all, with policy based forwarding we can apply and monitor as well. Similar to the path monitoring, we can all say right, try to reach certain IP address on the internet and only if they are reachable, only then this rule is active, okay? With this we have here our policy based forwarding, Basically, overwriting the routing decision and redirecting all traffic out, and then if the monitor of the PBF, is failing then the default rule will kick in and route all of the traffic out to ISP 2. Obviously this set up would be exactly the same like our path monitoring, so there is not really much change here, however What we can now do is, we can also define more granular rules to basically say certain traffic should go over ISP 2, so for instance let’s say we have a Guest Wi-Fi, and in this case the Wi-Fi is gonna have a dedicated network, which is 192.168.60.0/24, so this network is now connected to our firewall as well, nicely separated of course, and now here connects to our Virtual Router So now what we can do is obviously PBF, as the name says, it’s a policy which we define, with match criteria where we can basically say traffic from certain source IPS, for instance like this guest Wi-Fi, we can send out here, meaning we would now define a dedicated policy based forwarding rule just for the the guest Wi-Fi. This is now a PBF rule just for the guest Wi-Fi. So this is very granular and gives us a lot of use cases that we can do for traffic engineering. In this set up however, we have to be careful because what we have to consider is that policy based forwarding does not apply to traffic originating from the firewall! What does this mean? Let’s say here on the internet we have another firewall from a partner organization, and to this firewall we establish a site-to-site VPN. So a site-to-site VPN and we decide now, ISP 1, let’s say that’s our most stable internet connection, so we want that this VPN goes to this interface, to this public IP address. What is the problem? The traffic from this VPN arrives on this interface, and that’s okay, but now the return traffic, originates from the firewall because the IP address of this VPN is the firewall itself, okay? But the return traffic will not match against PBF, means it just makes a decision of the normal routing table, meaning that return traffic would now go down go out of ISP 2. So now we have asynchronous routing, so the traffic which we sent out of ISP 2, the source IP would actually be the IP address of ISP 1, okay? so effectively from a communication flow point of view it would look like this. So inbound traffic comes that way, and then return traffic Is actually sent this way! Depending on your ISP this might even work, that’s kind of the crazy thing about this… because this ISP, you have an IP address of this ISP on this interface, and when the traffic is sent out of the firewall, the source IP will be the IP address of this layer 3 interface, and it will be sent out the ISP 2 and now the destination IP address is the FW on this side, so it reached this FW, and it’s gonna go back, so unless this ISP is doing source IP checking, which they should do, and most ISPs, by the way, today are doing this, Then the traffic would actually go through, and this would be even worse because with this the VPN tunnel would work, but it wouldn’t work properly, because you have asynchronous routing and have delays between packets, so you definitely gonna run into trouble with traffic traversing on this VPN, and not directly will you actually see that this problem is related to the VPN, So this is why we have to be very careful How do we solve this problem? The way we’re gonna fix this is by setting up dedicated Virtual Routers for each ISP! What we’re gonna do is, we’re gonna get rid of here over our default Virtual Router, and instead we’re gonna set up a dedicated Virtual Router per ISP, so we have here our Virtual Router for ISP 1, we have another Virtual Router four ISP 2, and we’re gonna set up a Virtual Router as well for our internal network. Okay, so on these virtual routers, we can always define a default route pointing outbound. Okay, so here, we have a default route pointing to ISP 1’s VR, and then here obviously we still have our default route which you can actually leave in to point into ISP 2, and now on our VR over here, our internal VR, what we do is. we now have a default route pointing over here to this VR, so we do inter-VR routing. Here we have now our default route, and then on these VRs, we just put a route back pointing to the internal network, so we’re basically on each of these, just add the route saying that the internal networks, are reachable via at the internal VR, okay? For simplicity sake what you can do as well is actually just point all private IP addresses, all of these private IP addresses you can just point them down here, so that they are always reachable via the internal Virtual Router. So with this we fix the problem, because now if traffic arrives from this VPN on our firewall, then it will just follow the default route which points it back to ISP 1 and everything goes back and forth. With this we don’t have this problem, and having such a clean separation with multiple ISPs I think it’s very important, site-to-site VPNs, are just one example, another example would be Global Protect remote access VPN, for instance if you want to have fail-over for remote access VPNs you want to use kind of both interfaces for this, and then this is really the only solution for it. Obviously you can see this is a little bit more complex, but having two ISPs and taking full resource usage out of them Always is a little bit more complex, from a routing point of view, and this would be an appropriate solution for it. Technically when you think about this, in theory, you could also just use two Virtual Routers, so, one VR on the inside, and one for is ISP 2 and this one, so this would work as well however, from a best practice point of view, I will always prefer to have 3 setup because it’s a cleaner separation. It’s one per ISP, and one Internal, and unless you have limitations and running out of VRs on your firewall, I would always recommend you to set up the scenario using 3 Virtual Routers. By the way if you’re interested in security best practices for Palo Alto Networks, Then check out the blog on our webpage , in the best practice section you can download this worksheet with over 120 best practices for the next-generation firewall and Very soon we will also launch the security best practice training With a lot of videos explaining all of these security best practices in detail, so if you’re interested then sign up to our mailing list, and we will let you know as soon as this free training is available!