Are proxy servers a load of crap?

[Virchanza]

I'm trying to understand why an administrator would choose to use a proxy server on a network instead of a plain old router.

People say that proxy servers are good for caching... but tell me why a computer acting as a router (i.e. doing IP forwarding) can't do exactly the same thing?

Also they say proxy servers are good for security because the internet can only communicate with the proxy server... but how is that any different from what can be achieved with Network Address Translation?

I like to keep things simple in a network. I'd definitely choose to have a router instead of a proxy server.

And here's something else I don't understand. Either there's something I need to learn, or the network administrator was a half-monkey-half-man: When I was in college, there was a massive network, something like a thousand computers (if not a couple of thousand). Every machine in the college had a public IP address (yes, the college actually went to the bother of registering a C class address). You might think that the public IP addresses would have been beneficial because you wouldn't need to perform NAT on such a massive amount of machines... but then someone came along and decided to put in a proxy server. So basically you have a thousand machines with public IP addresses, and they all go through a proxy server to the internet, so the internet never sees any of their public IP addresses. Now I hope I'm missing something, because that sounds ridiculously stupid to me. The reason they have the proxy server is to do caching and also filtering, but they could have just gotten a router to do that, and also they wouldn't need to perform NAT because the machines have public IP's. I really hope I'm missing something.

[streaker69]

I played around with Squid for a while at the office. Mostly because for a time we had severely limited bandwidth and by setting up a Caching webproxy I was able to provide an appearance of higher bandwidth to the end users.

Many of the users would check the same few websites every day, news, sports and such. The first user there would establish the cache for that site and the others would then feed from it.

By using Squid, I was also able to limit the types of sites users could visit by the tools built into Squid. I was also get a daily report of exactly what sites were visited, by whom, how long they browsed and how much bandwidth each computer was using through the proxy server.

This is of course very helpful on networks where they're very strict on Internet Usage Policy.

As for your questions between using a Proxy and just using a Router, I had my proxy box sitting behind a router which was Nat'ed. You do not need to have your proxy server be your exit to the internet, you just need to tell your machine it is on the network and as long as it has a path to the internet, all other machines will pass through it. Hell, you don't even need two nics in the machine.

[Virchanza]

It seems that Squid did a great job for you, streaker69. In order to use Squid, I presume you had to go to every computer on the network and set its proxy server. For instance, you might have had to go into Mozilla on every machine and set the proxy server to "192.168.1.250:8080".

My problem with this however... is that it's a bit... well... unnatural. For instance, if you were to go to the commandline on one of these machines and do "ping google.ie", it would fail because the ping program hasn't got a clue about proxy servers.

Think about all the great stuff that Squid can do. Now, instead of Squid forwarding packets as a proxy server, imagine it just forwarded packets as a router. It could still do all the same stuff internally, (e.g. caching, filtering), but there wouldn't be the need for going to every machine and setting the proxy server. Also, when you ping at the command line, it will actually work. You'll have all the benefits of Squid, except the network will stay simple and natural.

I have no doubt that there's some great proxy server programs out there that can do all sorts of fancy stuff... I'm just wondering why they accept proxy connections instead of just acting as a plain ol' standard router the forwards packets. In fact maybe even some of these programs have the option of acting just like a router; if such is the case, I don't know why any administrator would choose to accept proxy connections instead of just acting like a router. Proxies are slower too because they have to establish a TCP connection on the LAN before they go out to the WAN. I hate proxies.

[purehate]

Thats not exactly true. I have a firewall box which also handles routing and dhcp and I just recently added squid to it. Squid can be run in transparent proxy mode so that all port 80 requests from the LAN are automatically redirected to the proxy port regardles of browser or IPs. This eliminates the need to set up each browser.

[streaker69]

It seems that Squid did a great job for you, streaker69. In order to use Squid, I presume you had to go to every computer on the network and set its proxy server. For instance, you might have had to go into Mozilla on every machine and set the proxy server to "192.168.1.250:8080".

Uh, No, I don't run to every machine for any configuration. That's what GPO's are for. I make 1 change at my desk and within the next hour, all the machines are up to date with that change. Work smart, not hard.

My problem with this however... is that it's a bit... well... unnatural. For instance, if you were to go to the commandline on one of these machines and do "ping google.ie", it would fail because the ping program hasn't got a clue about proxy servers.

Wrong. ICMP traffic would still flow regardless of the proxy server in the configuration that I used. Since the Proxy tab has no setting for ICMP, it wouldn't be routed through it, generally only 80, 443, and such goes through the proxy server.

Think about all the great stuff that Squid can do. Now, instead of Squid forwarding packets as a proxy server, imagine it just forwarded packets as a router. It could still do all the same stuff internally, (e.g. caching, filtering), but there wouldn't be the need for going to every machine and setting the proxy server. Also, when you ping at the command line, it will actually work. You'll have all the benefits of Squid, except the network will stay simple and natural.

You can configure a machine to do this. But it's unnecessary.

I have no doubt that there's some great proxy server programs out there that can do all sorts of fancy stuff... I'm just wondering why they accept proxy connections instead of just acting as a plain ol' standard router the forwards packets. In fact maybe even some of these programs have the option of acting just like a router; if such is the case, I don't know why any administrator would choose to accept proxy connections instead of just acting like a router. Proxies are slower too because they have to establish a TCP connection on the LAN before they go out to the WAN. I hate proxies.

A correctly configured proxy on a fast machine with a fast harddrive is not slow. Plus as Pureh@te mentioned, with Squid running in transparent mode, no one even knows it's there. I made the switch at my office and no one even noticed. No one even noticed when I switched it off a month later.

[Virchanza]

Thats not exactly true. I have a firewall box which also handles routing and dhcp and I just recently added squid to it. Squid can be run in transparent proxy mode so that all port 80 requests from the LAN are automatically redirected to the proxy port regardles of browser or IPs.

So when a machine on the LAN wants to send a HTTP packet to google.com, it sends it to the IP address of google.com (as opposed to the IP address of the proxy server), is that right? If so, then the IP address of the "proxy server" would be LAN machine's default gateway? That right? If so, then that's the ideal setup, i.e. Squid is acting as a router instead of as a proxy server. If this software is already readily available, then I don't know why any administrator in their right mind would choose a "non-transparent" proxy server, especially if the LAN machines have public IP addresses (as was the case in my college)!

I think I'll download Squid and give it a go, it really sounds good. I might be the administrator of a network real soon so I'm gonna have to get my tools together. First things first, I'm banning ARP. Every machine's gonna have MAC addresses hardcoded into it, and nice big loud alarm's gonna go off if an ARP packet of any kind is detected . For convenience, I'm gonna change the default gateway's MAC to 00:01:02:03:04:05.

By the way, does anyone know where I can get info about the HTTP proxy protocol? I've done no end of Googling and I can't get info about the protocol header (...which kind of leads me to believe that it might not have a protocol header). I need to have intimate knowledge of it for my Inp program. Or is there no protocol at all, does the original packet just get sent to the proxy's IP instead of the destination web server's IP? Normally I'd just open up Wireshark at a time like this but unfortunately I don't have a guinea pig proxy server to play around with!

[hhmatt]

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

[Virchanza]

Wrong. ICMP traffic would still flow regardless of the proxy server in the configuration that I used. Since the Proxy tab has no setting for ICMP, it wouldn't be routed through it, generally only 80, 443, and such goes through the proxy server.

So you're saying there's a default gateway on the network that leads to the internet? You have a router and a proxy server? If such was the case in my old college, I would have just blanked out my proxy settings and went straight through the router to get to the internet

In my old college, there was no router out to the internet, fullstop. There were a few separate LAN's in the college and there were routers between them, but there was no way of actually getting a packet out to the internet. The only way of accesing the internet was via the proxy server, and of course the proxy server performed filtering.

I wonder, when Squid operates in "transparent proxy mode", what kind of packets does it forward to the internet? Does it substitute the source IP address for the IP address of the proxy server, or does it maintain the original source IP address? It would be great if it maintained the original source address because then you could take full advantage of having a network of public IP addresses (e.g. each machine would be able to open a ridiculous amount of sockets and you wouldn't have to worry about NAT running out of port numbers).

[streaker69]

So when a machine on the LAN wants to send a HTTP packet to google.com, it sends it to the IP address of google.com (as opposed to the IP address of the proxy server), is that right? If so, then the IP address of the "proxy server" would be LAN machine's default gateway? That right? If so, then that's the ideal setup, i.e. Squid is acting as a router instead of as a proxy server. If this software is already readily available, then I don't know why any administrator in their right mind would choose a "non-transparent" proxy server, especially if the LAN machines have public IP addresses (as was the case in my college)!

No, that is not correct. The default gateway of the machines does not HAVE to be the IP address of your Squid machine. In my case, it absolutely was not. It was just another machine on the network, the default gateway was the LAN side of my Cisco PIX firewall. But the way it works is when the browser sends an HTTP request, it goes to the proxy server as per the proxy server settings in the browser options. But ICMP traffic would not go through the proxy server, but directly to whatever the default gateway is of the machine.

I think I'll download Squid and give it a go, it really sounds good. I might be the administrator of a network real soon so I'm gonna have to get my tools together. First things first, I'm banning ARP. Every machine's gonna have MAC addresses hardcoded into it, and nice big loud alarm's gonna go off if an ARP packet of any kind is detected . For convenience, I'm gonna change the default gateway's MAC to 00:01:02:03:04:05.

I'm curious as to what method you're going to remove Arp. Please elaborate. It sounds as though you're going to attempt to whitelist the MAC's that are allowed to communicate on your network? I hope you don't have more than 10 devices otherwise you're going to go nuts with administration tasks. There are other methods for doing what you want without going through the hassle of disabling arp, I'm not even sure that's possible.

By the way, does anyone know where I can get info about the HTTP proxy protocol? I've done no end of Googling and I can't get info about the protocol header (...which kind of leads me to believe that it might not have a protocol header). I need to have intimate knowledge of it for my Inp program. Or is there no protocol at all, does the original packet just get sent to the proxy's IP instead of the destination web server's IP? Normally I'd just open up Wireshark at a time like this but unfortunately I don't have a guinea pig proxy server to play around with!

The way I understand it, is the clients webbrowser sends a request to the proxy server. The proxy server then establishes the connection to the remote site and downloads/caches the data in it's local store, then serves the data to the client. If another client requests the same page, then instead of retrieving it again, it is served from cache based upon the TTL of the cache. The client never has direct communication with the remote site when passing through the proxy. The proxy does all the work, thus it makes it very easy to limit and monitor what goes on.

[Virchanza]

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

That's the HTTP protocol. Are you saying that a proxy server just accepts the HTTP protocol without any changes? So for instance, if your web browser wants to go through a proxy server, all it has to do is send the HTTP packet to the proxy server's IP address instead of to the destination web server's IP address? That right? So it doesn't prepend any sort of "proxy header" to it, no? If that's the case, then that seems very restrictive (or then again, maybe it was designed to be restrictive). I say restrictive because you can't do stuff like specify port number (e.g. www.google.com:81) unless of course the proxy server is listening on all ports and forwards HTTP packets on to the appropriate port, but I doubt that! Also you'd always need to use a URL from which an IP address can be determined, for instance you couldn't do a GET command for "\" from a particular web server.