Hello all! Been a while. I recently did a write up of an overview on WCCP for work as a side project. There is no confidential information contained in this document, so I thought I would share it with all of you who are looking to deploy WCCP to transparently redirect traffic to a proxy, such as Websense’s Content Gateway. Below it is basically copy/pasted in the blog article, so it is missing pictures and formatting. You can get the document in it’s original PDF format including pictures, HERE.
This document is presented as-is with no warranty involved in its accuracy of information and content. While I have been able to test and verify some of the content, it is mostly based on what “should” happen according to the sources listed at the end of the document, and theory. Ultimately, it is Cisco’s protocol and equipment, and given the WCG’s mysterious behavior at times, results may vary. Also, I am not claiming this document as my own work as that would be plagiarizing considering a large portion of content has been re-used from the sources listed at the end of this document. Now that that’s out of the way, enjoy!
Introduction to Web Cache Communication Protocol (WCCP)
WCCP enables routers (which of course include multilayer switches and firewalls) to transparently redirect selected traffic to a WCG or cluster of WCGs. It allows one or more routers enabled for transparent redirection to discover, verify, and advertise connectivity to one or more WCGs.
Once connectivity is established, the routers and WCGs form Service Groups to handle redirection of traffic whose characteristics are part of the Service Group definition.
The protocol provides the means to negotiate the specific method used for load distribution among WCGs and also the method used to transport traffic between the router and WCG.
A single WCG within a Service Group is elected as the designated web-cache whose responsibility is to provide the routers with the data which determines how redirected traffic is distributed between the WCGs in the Service Group.
Enhancements Provided by v2
WCCP v2 provides several enhancements over v1:
· Multi-Router Support
· Multicast Support
· Improved Security
· Support for redirection of non-HTTP traffic
· Packet return
· Alternative Hashing
· Multiple Forwarding Methods
· Multiple Assignment Methods
· Command and Status Information
Terminology
Assignment Method – The method by which redirected packets are distributed between WCGs in a cluster.
Designated Web-Cache – The WCG in a cluster that dictates to the router or routers how redirected traffic should be distributed among the WCGs in the Service Group.[1]
Forwarding Method – How redirected packets are transported from the router to the WCG.
Packet Return Method – How packets will be returned to a router for normal forwarding by the WCG.
Service Group – A group of one or more routers plus one or more WCGs working together to perform transparent redirection for a particular service such as HTTP or HTTPS.
Joining a Service Group
A Service Group is identified by Service Type and Service ID. There are two types of Service Groups; Well Known and Dynamic. The WCG uses Dynamic Service IDs, so in order for the Router to use the Dynamic Service ID by Service ID alone and no other information, the WCG must describe the characteristics of the Dynamic Service Group.
Think of yourself as a traffic cop at a busy intersection, and I’ve told you, “I want you to let regular traffic coming from this direction continue on its path, but I want to you to direct Websense traffic to turn right and go down this road instead.” You might ask yourself, “Well how do I identify the Websense traffic among the rest?” This is where I tell you that the Websense traffic will consist of SUVs, painted blue, whose license plates say “Websense”. In other words, I’m describing the characteristics of that traffic to you.
This required information is contained within the “Here I am” message sent by the WCG. This message contains the Protocol, Service Flags, and Port Fields that describe the Dynamic Service to the router.[2]
The Web Cache Identity Info component of the message identifies the WCG by IP address.
The Service Info component identifies and describes the Service Group that the WCG wishes to participate in.
A router responds to the Here I Am message with an “I See You” message. If the Here I Am message was unicast , the router responds with its own unicast message. If the Here I Am message was multicast however, the router will respond via a scheduled multicast message every 9 seconds.
The Router Identity component in an I See You message includes a list of the WCGs to which the packet is addressed.
Lastly, if a router receives a Here I Am message describing a Dynamic Service that it has not been configured for, it will discard the message.
Security
WCCP v2.0 provides simple authentication via an MD5 password. This requires that each router and WCG that wants to join a Service Group be configured with the Service Group password. This means that this password must be configured on the WCG AND the router. Each WCG and router in the Service Group will communicate the password and MD5 checksum within each packet in the Security Info component. After validating the WCCP message header, the Security Info portion is checked. Packets failing authentication are discarded.
It is normal for the checksum values to not appear to match when viewing the Here I Am and I See You messages in a packet capture.
Maintaining Connectivity
In order for the router to consider the WCG “usable”, and to maintain two-way connectivity, the router will send an “I See You” message with a “Receive ID”. This is maintained separately for each Service Group, and is incremented each time the message is sent.
This message must be reflected back by the WCG in its next Here I Am message. If the receive ID in the next Here I Am message does not match, the router discards the message. This results in the WCG not being considered useable and will prevent the WCG from joining the Service Group. In the event this begins occurring where the WCG is a member of the Service Group, this condition will eventually cause the WCG to be removed from the Service Group.
The router uses the WCG IP information to create a cluster view, or a list of WCGs in the Service Group (If you have multiple). This view is sent via an I See You message to each WCG in the Service Group, making the WCGs aware of each other. In the example below, one WCG is in the cluster view[3].
Electing the Master of the Universe (aka The Designated Web Cache)
When utilizing multiple WCGs within a Service Group, and once the Service Group has stabilized after a membership change, the WCG with the lowest IP address in the Service Group is elected as the designated web cache. The designated web-cache must be receiving I See You messages from every router in the Service Group.
WCCP Negotiation
A WCG and router will negotiate the method which packets are forwarded to a single WCG by the router, how packets are distributed between multiple WCGs in a Service Group, and how packets are returned to a router for normal forwarding.
There are two methods used forwarding and returning traffic: L2 and GRE. There are also two methods of distributing traffic among multiple WCGs: HASH and MASK.
The router advertises what methods it supports using the optional Capabilities Info component of the I See You message. The WCG then inspects this, selects a supported method based on what it has been configured for, and reports this back to the router via the optional Capabilities Info component of its next Here I Am message. The router inspects this and if the selected method is supported, the router finally considers the WCG useable and adds it to the Service Group.
Packet Forward/Redirect Method
This negotiation is per WCG, per Service Group, so WCGs participating in the same Service Group may negotiate different forwarding methods with the Service Group Routers. The absence of the optional Capabilities Info component in the WCG’s Here I Am message implies that it is requesting the default GRE encapsulation method. The absence of this in the router’s I See You message indicates that the router supports the default GRE encapsulation method only.
Assignment Method
This negotiation is per Service Group, so WCGs participating in several Service Groups may negotiate a different assignment method for each Service Group. The absence of the optional Capabilities Info component in the WCG’s Here I Am message implies that it is requesting the default Hash assignment method. The absence of this in the router’s I See You message indicates that the router supports the default Hash assignment method only.
Packet Return Method
This negotiation is per Service Group as well, so again, WCGs participating in several Service Groups many negotiate a different packet return method for each Service Group. The absence of the optional Capabilities Info component in the WCG’s Here I Am message implies that it is requesting the default GRE packet return method. The absence of this in the router’s I See You message indicates the router supports the default GRE encapsulation method only.
L2 vs. Generic Routing Encapsulation (GRE)
“L2” is translated to Layer 2, the Data Link layer of the OSI model. Utilizing the L2 method simply means that that when forwarding or returning traffic, the destination MAC address is rewritten to the appropriate device, be it the WCG or the router. The problem with L2 is that MAC addresses cannot be routed, so it requires all participating devices to be directly connected at L2, or “L2 Adjacent”.
This is where GRE comes in. GRE encapsulates packets in a second L3 header. This allows the router to forward traffic to a WCG that can be one or more “hops” away on a separate subnet.
Pictured below is a packet capture of a GET request that has been forwarded from an IOS Router to a WCG utilizing GRE. Toward the bottom, you can see the first/original L3 header with the source IP of the client, 10.212.8.8, and the destination IP of the web server (In this case, www.dyepaintball.com). Near the top you can see the second L3 header added by GRE containing the source IP address of the router, 10.212.0.3, and the destination IP address of the WCG, 10.212.8.14.
When the WCG performs a packet return to a router for normal forwarding, as would happen when the WCG receives traffic for a website that has a static bypass for it configured in ARM, the destination and source IP addresses in the second L3 header would be reversed and the first L3 header obviously stays the same. This encapsulation allows the WCG to return traffic to a router for normal forwarding that has redirection applied to its internal interface, preventing a looping condition that could exist utilizing the L2 method due to the source IP of that packet matching the interesting traffic specified in a “redirect list”[4].
Traffic Distribution
Traffic distribution is performed in one of two ways; Hash Tables and Mask/Value sets. It is the responsibility of the designated web-cache to assign the Hash Tables or Mask/Value sets to each of the routers in the Service Group. This information is contained in a “Redirect Assignment” message, which is generated whenever there is a change in Service Group membership, and is sent to the same set of routers that it sends Here I Am messages. The designated web-cache will wait 15 seconds after a change in order to allow the Service Group membership to stabilize before sending the message. The designated web-cache uses either the Assignment Info or Alternate Assignment Info Component to list the WCGs to which traffic should be distributed.
Hash Tables
When using hash assignment, each router uses a 256-bucket Redirection Hash Table to distribute traffic for a Service Group among the WCGs. The designated web-cache uses the Redirect Assignment message to assign the routers’ Redirection Hash tables.
The Redirection Hash Tables are conveyed in the Assignment Info Component of the Redirect Assignment message. This contains an Assignment Key that will be reflected back to the designated web-cache in the router’s next I See You message.
The Redirect Assignment method may be repeated after 10 seconds if the I See You message returned by the router indicates that the router has not received an assignment.
A router will flush its Redirection Hash Table if a Redirect Assignment message is not received within 50 seconds of a Service Group membership change. A router will also flush its Redirection Hash Table if it receives a Redirect Assignment message in which it is not listed.
Redirection with Hash assignment is a two-step process. In the first step, a primary key is formed from the packet, which is defined in the Service Group description within the Service Info component. This hash key is the destination IP address by default[5]. Because the hash itself is not configurable, and is deterministic in nature, all routers within the same Service Group will make the same load-distribution decision given the same hash key.
In the second step, a bitwise hash is performed on the key identified as part of the Service Group, creating an index. This index is mapped to a bucket. The packet will then ultimately be sent to the WCG that has that specific bucket assigned. If the index matches an unassigned bucket, the packet is forwarded normally.
Mask/Value Sets
When using mask assignment each router uses masks and a table of values to distribute traffic for a Service Group across the WCGs. The designated web-cache uses the Alternate Assignment Info component in a Redirect Assignment to assign the routers’ mask/value set.[6] Just like Hash Tables, this message is reflected back to the designated web-cache, and the message may be repeated after 10 seconds if the router hasn’t indicated it received an assignment. Also just like Hash Tables, a router will flush its mask/value set if a Redirect Assignment is not received within 50 seconds of a Service Group membership change, or if it receives a Redirect Assignment in which it is not listed.
Like Hash Tables, redirection with Mask/Value Sets is a two-step process. The first step involves performing a bitwise AND operation between the mask from the first mask/value set in the Service Group definition and the contents of the packet. The 96-bit output of this operation is then compared to a list of mask/value pairs. Each mask/value pair is associated with a bucket, and each bucket is assigned to a WCG. If absolutely no match is found, the packet is forwarded normally.
Sources:
Catalyst 4500 Series Switch Cisco IOS Software Configuration Guide, 12.2(31)SG – Configuring WCCPv2 Services
Configuring Web Cache Services Using WCCP
Daren Matthews >> WCCP Load Distribution (Hash and Mask)
Internet Draft for WCCP v2
http://www.wrec.org/Drafts/draft-wilson-wrec-wccp-v2-00.txt
[1] This is called the “Leader” in v7.6
[2] This is transmitted every 10 seconds over UDP port 2048. This communication can be configured as unicast to each router, or multicast.
[3] This communication can also be unicast or multicast.
[4] This is not always true. There are some special circumstances and devices that do not so simply abide by these rules; more on that in a future module.
[5] This can be and often is changed to utilize the source IP instead. An example of where this is needed is when a single site request pulls parts of the web page from multiple destination IP addresses, which end up being serviced by multiple WCGs in the Service Group and causes the site to fail to load properly.
[6] Unfortunately, I haven’t been able to get a capture of Mask assignment, so there are very little pictures in this section. This section is also for the most part word-for-word out of the RFC. Sorry!
