If you want to get up and running with Clickwatch on an ASP.NET server you should grab the download from the right column and skip down to the section that walks you through the installation process. If you're interested in the inner workings you should read on - it's probably simpler than you'd think.
To implement browser-based visualization of real-time HTTP traffic on a particular server, you need to solve the following problems:
- 1) Gather event data in a web application
- 2) Massage this data in your application so it is transformed into meaningful information - this step involves filtering the data gathered, translating IP addresses to geographic coordinates, etc.
- 3) Deliver this data to a visualization client
- 4) Display the data in an engaging fashion
Since we're talking about ASP.NET, the trivial way to intercept information about traffic as it's happening is to hook into IIS with an HttpModule. Ultimately we want to have the following information about every pageview:
- Page being requested
- Visitor IP address
- Amount of time it took to process the request
- The size of the server's response
An HttpModule provides a very versatile interface for post-processing (or pre-processing if you wish) requests that reach ASP.NET. There's a lot of documentation on the subject so I won't go into generic details; suffice to say that HttpModules are usually implemented in managed code (ours is a C# class library) and are registered with a simple entry in your web.config file. If you need more information on HttpModules, Google will help you.
The first two items (page URL and remote IP address) are readily available in the
OnEndRequestevent. To figure out the other two bits of information we need to hook into
OnBeginRequestas well. In our
OnBeginRequesthandler, we store the current timestamp (so we can calculate the time taken once we get to
OnEndRequest), and we also attach a simple passthrough filter to the
Responseobject so we can count the number of bytes as they're being sent to the client. Unfortunately there's no way to simply query the size of the response in an OnEndRequest handler, or anywhere else in an HttpModule for that matter. Why Microsoft thought it wasn't important to implement
Response.OutputStream.Lengthis beyond me - but the workaround is simple even though it involves about a hundred lines of mindless boilerplate code for the passthrough filter.
OnBeginRequestwork done, once
OnEndRequestis called we have almost all the information we need.
To translate the client's IP address to geographic information I chose to use a free database available from MaxMind. They also publish an open source library that can be used to query their binary databases, and it was easy to implement a COM wrapper around their C API, so the entire thing lives in
IpAddressToGeoInfoGeoIp.dll. There's not much that's worth mentioning about the work done here: 99% of the code is boilerplate COM and the MaxMind library source.
It must be said that I am somewhat disappointed by the convoluted and less-than-optimal code that MaxMind makes available. I would have preferred to work with the database and API published by Ip2Location - their C library code isn't optimal either, but it's very simple and easy to change so it uses memory mapped files instead of fseek-ing a hojillion times per lookup. Ip2Location's DB is not free - but even if it were, their placenames are all capitalized which feels so 1980s, which gets us back to preferring MaxMind after all. But I digest.
The aforementioned COM object is instantiated and called upon by the
OnEndRequesthandler to translate the IP address to geographic information. Once we have this final piece, we package the information into an object and store it in an array that's also accessible from code running in a regular ASP.NET page.
This is the final piece to the backend. It simply parses the aforementioned array, and returns a simple textual representation of every object not yet sent to the calling client. This is an ASP.NET page with a whopping 30 lines of code inside. The visualization client needs to poll this script every few seconds to get a list of new hits to the site.
The client page is by far the most complex piece, over 1000 lines, but the good news is that it is completely portable. It is an .aspx page but the server-side script is limited to a couple of global variables that define the default size of the map.
Installing on an ASP.NET platform
It should be a matter of minutes to be up and running with Clickwatch if your server already runs ASP.NET. Grab the zipfile from the right-hand column, extract it to a temporary location and follow these instructions.
The IP address mapper COM module
geolookup.x64folder to its permanent location - there are no special requirements about where you put this. You need to choose the right platform for your server though. If your server is running 32-bit Windows you need the .x86 version, otherwise you'll want the .x64 one - unless you're running a 32-bit ASP.NET runtime on an x64 computer, which is very common, and works fine. So - know your ASP.NET platform and choose the appropriate COM object.
Once you've settled on a location for the geolookup COM module, go to MaxMind.com and download their free binary database. Once extracted you'll have a file called
GeoLiteCity.dat. Move this file right next to the
IpAddressToGeoInfoGeoIp.dllCOM object, and rename it to
Now open a command prompt, navigate to the directory where the COM object was placed, and type "
regsvr32 IpAddressToGeoInfoGeoIp.dll". You should get a message indiating success. To test if everything works properly, double-click the
testme.jsfile in the
src\IpAddressToGeoInfoGeoIpdirectory of the zip file that you downloaded. If the COM object and the database have been installed properly, you'll get a message indicating the location of a certain IP address.
This was the hard part.
The web components
Copy everything in the
wwwrootdirectory to your own web root, including the subdirectories. The contents of the
bindirectory must be in your own
binfolder, etc. The only exception is
web.config- if you already have a web.config file you should not overwrite your old one, but rather merge the contents of the two files. This should be straightforward.
If you now type
http://your-server/clickwatch-refresh.aspxinto a browser you should get a nice blank page, or a page with one or more lines of text. If you keep refreshing the page and it remains blank, it means that the HttpModule is not seeing any traffic from the Internet. If Clickwatch cannot resolve an IP address to a geographic location (as is the case with LAN addresses used on intranets) it will silently ignore it. In any case, if you got no error messages then everything seems to be fine - you can now try
http://your-server/clickwatch.aspx. If you're not on the same LAN as your server, you should start seeing requests flying across the screen.
Why am I seeing just .aspx files?
Clickwatch will only see what's being handled by the ASP.NET engine. Plain files such as zip archives don't go through our HttpModule so they're invisible to us. You can fix this if you want - just map the extensions you're interested in to ASP.NET. For example, to see PDFs being downloaded in addition to your .aspx pages, right-click your website in IIS manager, select Properties, click the Configuration button near the bottom of the Home Directory tab, and add a mapping for the .pdf extension to the ASP.NET runtime. (Copy the "Executable Path" value from the .aspx mapping for best results.) You need to do this for every file type that you want to see in Clickwatch. And, obviously, you cannot remap other scripts such as legacy .asp or Perl pages to be handled by ASP.NET, so those will remain invisible.
And this, mapping plain files to ASP.NET, is where we may start having some problems. On the Windows 2003 servers that I've tested this with, a single hit on such a newly mapped file will appear exactly 101 times in Clickwatch. This appears to be a bug in the ASP.NET runtime, and the HttpHandler is being called a hundred times more than necessary. (There are a few mentions of this occurring out on the Internets, but there are no resolutions.) If you encounter this problem you will need to run your server in IIS 5.0 isolation mode. (Right-click the "Web Sites" root in IIS Manager, select the "Service" tab and choose "Run WWW service in IIS 5.0 isolation mode".) This fixes things for 32-bit servers, and 64-bit servers with the 64-bit ASP.NET runtime, but in a mixed environment (64-bit server, 32-bit ASP.NET) your website will most likely fail to start up with all kinds of nasty errors in the System log. It appears that IIS 5.0 isolation mode is not compatible with running the 32-bit ASP.NET runtime on 64-bit servers. To remedy this, switch to full 64-bit mode:
C:\>cscript.exe adsutil.vbs set W3SVC/AppPools/Enable32BitAppOnWin64 falseThen:
C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727>aspnet_regiis -i -enable Start installing ASP.NET (2.0.50727). ...................... Finished installing ASP.NET (2.0.50727). C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727>iisreset Attempting stop... Internet services successfully stopped Attempting start... Internet services successfully restarted C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727>Note that above I've ran
aspnet_regiisin the Framework64 directory as opposed to the plain 32-bit Framework one.
Questions, comments? Drop me a line at .
Please note that I don't have the time to do tech support for Clickwatch - but if you have ideas for improvements or ports to other platforms, I'd be very interested to know.