Rulebase Updates

Why are you deprecating the FTP access to updates?

Technically, ftp is a challenge because it requires two pipes instead of one. In the case of active ftp (old school I know, but still out there), the server has to actually create a connection back to the client -- if there is a client firewall in place that often won't (and probably shouldn't) work.

The "shouldn't" part has to do with security-- there's no good reason to allow incoming connections to anything other than a server (most of the time).

If the inbound connection is to a server it is a good rule of thumb that the inbound connection should ONLY be allowed for some service that server is itself providing. Other ports should be strictly off-limits.

Some of this can be simplified for the client side of things with passive FTP... but what about on our end? With FTP of any kind we have to have a lot more "holes" in the firewall with FTP because that second pipe has to come through somewhere -- and unless we're going to serve only one client at a time that means lots of inbound ports left open. (I know I'm oversimplifying).

Anyway - the advantages to HTTP is the way we are using it are:

• HTTP is stateless and transaction oriented - that matches exactly what we want in this case --- The request is simple (give me the file if it's newer) and the response is just as simple (here's the file, you don't want it, or I don't have it.) Stateless translates directly into reliability and scalability -- If a server goes down in the middle of a transaction - (or more likely between transactions) - the next exchange of bytes simply goes to a different server. There is no "session" to keep track of in this case.

Load-balancing is a snap to understand and deploy because there is always a single, simple TCP connection and a short exchange - once it's over it's over. Since we're only serving files with this (not applications) we can strip off anything that might execute a command on the HTTP server. No commands ever go to the OS - only to the HTTP software which is only capable (in this case) of reading a file and sending it to the client.

Although FTP can be used this way - under the covers it is much more complex because it is designed as a sesson-based protocol. You log in, use a wide range of commands to browse and otherwise do what you want, and then you log out... and if something happens during that session you have a problem to resolve. Did the server go away? Did the client go away? Did some error occur and if so how do you want to handle that? Lots of options for every case, as long as the session is still active, the client can do the unpredictable. If you restrict the client's options then folks have trouble because there's no single "correct" way to use an FTP session.

Since not all FTP clients are created equally, and not all FTP scritps are likely to be equal - the possibility for problems or security hassles to creep in is much bigger. Even now we have a constant, low level of problems with log file uploads due to the security measures we have in place. To a lesser extent the same thing is true of rulebase downloads via FTP...

For security reasons we strictly limit the commands that are accepted on our FTP server. It never fails that someone will try to use a command we don't allow and as a result the system is broken from their perspective. A little coaching and debugging is generally required in order to figure out what they or the script or FTP client is trying to do that isn't allowed, or whether the firewall is the problem (blocking the data link is a common recurring problem that is often reported incorrectly or simply causes an ftp client to "hang")...

In contrast, with HTTP - if you have a connection then you have the connection you need. There is no session to break --- you make your request and you get your response. Even there - the options are pretty strictly limited and there is a single correct way-- GET. There's no need to POST anything so it's not correct. You don't need to navigate anywhere - your URL _IS_ where you are going.

FTP = more complex than HTTP all the way around.

• Just about every firewall setup will allow statefull outbound connections from the client side to us ( that's pretty secure). If some stone-age router (there's always one out there) won't make a statefull connection - then at least there is only a single port to open and it is so ubiquitous that almost nobody will be confused about it.
• Everybody (with very few exceptions) has a handy debugging tool for http on their computer -- their ordinary web browser. In general, no matter what web browser you use, if you can download a rulebase file with it - your command line tool should be able to do the same thing; and if you can't for some reason you are VERY likely to get a clear picture of why not.
• On our side of the net it is a very easy, off-the-shelf, and conventional thing to scale up http services. Not so w/ FTP. Also, from a security point of view there is always only one hole to open in the firewall.