In-Depth

Security with UrlScan

UrlScan is an invaluable security tool for Web sites. Although IIS 6.0 incorporates some of its functionality, it’s still useful for IIS 6.0 sites—and critical for older versions of IIS.

• By Martin C. Brown
• December 01, 2003

You can take a lot of steps to help protect your Internet-facing machines from malicious attacks. The most fundamental is to keep on top of your Web server configuration and make sure you’re not accidentally opening up a Web site area to potential abuse. Utilities like the IIS Lockdown tool and many of the built-in security features of IIS 6.0 will certainly help in the effort.

The other chief method is through implementation of UrlScan. UrlScan does exactly what its name suggests: It scans incoming URL requests and its associated data. It uses a series of rules to determine whether the information in that request is potentially dangerous (i.e. requesting an application or accessing a folder with periods in its name), or contains information not normally expected (such as high characters—UTF8 or Unicode—or excessive amounts of information). To help you diagnose any potential problems and any attempts to upset your server, UrlScan can also log requests—including the offending request data—up to a 128KB for each request.

Because UrlScan works as a filter before the information is passed on to the script or application that handles the request, it can act as a buffer, so you don’t have to modify your existing code. Therefore, if a request is identified as being a potential risk, the script can immediately return a 404 error (its own or the IIS custom error message) to the client, without the information ever reaching the script. This protects the script, your Web site and your server.

UrlScan isn’t new. Version 1.0 was released by Microsoft in 2001. Version 2.0 came out soon after to fix some early bugs and problems in the first version that actually broke many Web sites. Version 2.5 was released by Microsoft to coincide with the release of IIS 6.0, which is part of Windows Server 2003.

IIS 6.0 contains some significant security improvements of its own, including providing some of the functionality of UrlScan, but that doesn’t mean that IIS 6.0 replaces the tool. In general, I recommend installing UrlScan on all IIS servers that provide some sort of dynamic content or host any type of application. Even if your site only contains static data, installing UrlScan can help to track potential attempts to break into your servers, which helps prevent future attacks and provides useful information on other areas of the network that were potentially infiltrated. And, of course, there are still many Web sites out there running previous versions of IIS. Those sites should consider UrlScan a mandatory tool. (Microsoft recommends only using UrlScan 2.5 with IIS 6.0; users of previous IIS versions should use 2.0.)

Installation
Download the UrlScan installer from the Microsoft Web site. You can get it from UrlScan homepage at www.microsoft.com/technet/security/urlscan.asp.

The installer adds the UrlScan.dll and UrlScan.ini configuration file to the %systemroot%system32\inetsrc\urlscan directory. If you’re upgrading to UrlScan 2.5 from a previous version, your existing configuration information won’t be overwritten, but it will have been updated to configure the default settings for the new features.

UrlScan will be added to IIS as a global ISAPI filter, applied to all of your Web sites. If you only want it to apply to some of your Web sites—perhaps because it’s broken some of your sites until you can resolve the issues—remove it from the global filters list and add it to each site individually. If you do this, make sure to promote UrlScan to be the first ISAPI filter, to ensure the maximum protection.

Within IIS 6.0, UrlScan also makes some configuration changes to ensure that the logging and execution of the script work correctly with the new process-based model of the IIS 6.0 worker process isolation mode.

Configuration and Operation
The UrlScan extension uses the parameters in the urlscan.ini file installed in the %systemroot%\system32\inetsrv\urlscan folder to determine which requests to let through, and which to reject. There are a number of different options; as with any security configuration, it’s a question of balance between protecting your servers and network and providing enough access to allow the application and system to provide necessary services.

The configuration file takes the form of a standard .ini file and is logically split into two sections. The primary section, Options, specifies a combination of global options with references to specific elements within the .ini file to process within the logical second part of the file. For example, the UseAllowVerbs option specifies whether UrlScan should process the [Allow Verbs] section or the [DenyVerbs] section. Figure 1 shows a small sample of the configuration file.

Figure 1. The UrlScan configuration file is a basic .ini file, specifying all the different base options and main sections within the rest of the file to be processed. (Click image to view larger version.)

UrlScan intercepts requests (by acting as the primary ISAPI filter). It uses the options within the configuration file to determine whether the request is valid. If the request passes the qualification, the ISAPI filter lets the file go on to the next ISAPI filter in the list. If rejected, UrlScan redirects the request to an error page.

You can control the specific response behavior in the event of a rejection through the configuration file. The UseFastPathReject option, if set, forces the request to be rejected immediately, in which case the user gets a generic 404 error message. The request itself isn’t logged in the UrlScan log, which is useful if performance is critical. If that option isn’t set, UrlScan redirects the user to the URL pointed to by the RejectResponseUrl option.

Rather than looking at the rest of the options in detail, we’re going to concentrate on how UrlScan integrates with, or has been replaced by, IIS 6.0.

Augmenting IIS 6.0
There are two ways in which UrlScan augments the security within IIS 6.0. The first is by monitoring the size of incoming requests (URL length, HTTP header length and query string data) and rejecting them before the information is ever sent to the script or application. The second augmentation is in the filtering of the URL requested from the server. Configurable parameters for size monitoring include:

The [RequestLimits] section of the configuration file specifies the limits contained in a request. Any request exceeding these limits is automatically rejected.

MaxAllowedContentLength is the maximum size of the Content-Length header field, which stores the length of any additional data (such as that from a form) or files uploaded through WebDAV. The default is 30,000,000 bytes, which—let’s face it—is quite high. Even working with fairly hefty WebDAV requirements, it’s unlikely you’d be posting documents that size. If your sites use relatively light forms and don’t support WebDAV, you might want to set this as low as 1,000 bytes. Even with WebDAV, I’d recommend reducing this figure to 100,000.

MaxUrl is the maximum size of a URL request. The default is 26.00 (the same length as the maximum size of a path) and should stop clients from trying to access excessively long URLs. Generally, you can leave this figure at its standard level, but be aware that heavily organized and directory-led sites might find this limitation restrictive.

MaxQueryString is the maximum length of a query string appended to a URL. The default is 2,048 bytes, which should be more than enough to handle a simple form. You can also limit specific HTTP headers by prefixing the header with “Max-”. Using these requires in-depth knowledge of the HTTP headers sent in a typical request. If you want to make use of this, I’d set the following limits:

Max-Content-Type=100

Max-Accept-Charset=100

Max-Accept-Encoding=200

Max-Authorization=50

Max-Referer=260

Max-User-Agent=100

The URL filtering system rejects requests based on URL content, including some you’d never have thought were allowed in the first place. The primary configuration options here are:

AllowHighBitCharacters. If set to 1, this allows characters with an ASCII or Unicode number above 128. On a strictly English-based site it’s probably best to enable this. On a site with foreign language material, you might find that this blocks perfectly legal pages.

AllowDotInPath. If set, this allows one or more periods in a URL (outside of a file extension). Enabling this should stop users from trying to refer to parent directories by using the “..” notation in a directory.

[DenyUrlSequences]. This specifies a list of characters to be rejected in a URL. The default options here are “..”, “./”, “\”, “:”, “%” and “&”. You might also want to add “,” , “$”, “^”, “!”, “@”, “~”, “<>”.

IIS 6.0 has certainly gotten better at working with URLs and components it doesn’t recognize—now, it just ignores them—but an extra level of filtering will help address issues you might not have considered yet. Of course, the flip side to adding these filters is to ensure that your Web team doesn’t develop sites whose URLs include those banned characters.

Using IIS 6.0 instead of UrlScan
The main focus for Microsoft when developing IIS 6.0 was to improve the overall security of its Web server. This starts with IIS no longer being installed by default. It continues through all areas of the system, including the mechanisms for restricting files, extensions and applications, as well as worker process isolation.

Does it therefore make UrlScan obsolete? No. But there are quite a few areas where UrlScan previously provided security that IIS 6.0 is now capable of handling. A few of the key areas IIS 6.0 addresses that supercede the functionality offered by UrlScan include:

Extensions. In past versions of IIS you could access any file, providing you knew its full name and extension. UrlScan and the DenyExtensions feature could be used to allow access only to files with recognized extensions. In IIS 6.0, files are only returned if IIS can identify the file’s MIME type from the MIME type/extension mapping built into the server. If an extension isn’t in the map, there’s no access to the file. I always recommend switching off all MIME types within IIS, except those required for the sites you’re serving. For example, in a basic, static site you might limit the MIME types to support only .htm, .gif and .jpg files. You can see a sample of the MIME type properties in Figure 2.

Figure 2. You can set MIME types on a server- or Web site-level. Unlike previous IIS versions, IIS 6.0 will only honor requests for files it can identify through the MIME type map—any other file/extension not listed generates a 404-class error. (Click image to view larger version.)

WebDAV. In the past, UrlScan could help control WebDAV access; now you can control WebDAV access directly through IIS. To enable or disable WebDAV for your server, use the Web Service Extensions Manager, available within the IIS tree of IIS Manager.

URL filtering. IIS 6.0 provides basic URL filtering by changing the way it determines whether or not to execute an application or script. This makes the NormalizeUrlBeforeScan option of UrlScan irrelevant, as the URL is no longer the sole determiner of whether to execute a script. Also, the AllowDotInPath option has been superceded by the same execution mechanism and the improvements to the MIME map that limits which files are returned.

Request lengths. Although not heavily promoted, you can limit some elements of a request directly within IIS 6.0. There are two areas where you can set limits: Through the MaxRequestEntityAllowed and ASP MaxRequestEntityAllowed Metabase properties, and a set of Registry properties within the HKEY_LOCAL_ MACHINE\System\CurrentControlSet\ Services\HTTP\Parameters node.

Even with these enhancements, UrlScan has a vital role to play in the security of your server. If you don’t want to use UrlScan, at least make sure you’ve made the changes suggested above. For extra protection, use UrlScan to help IIS 6.0 make more informed decisions about what URLs and pages it should be serving.

Other Threats
Although UrlScan and IIS 6.0 combined are an effective way of protecting yourself from a large number of attacks, they shouldn’t be your only means of protection. You should also install and consider at least two or more of the following:

Firewalls still play a vital role in the protection of your site and servers. The primary purpose and advantage of a firewall is to protect and limit access to your network to and from specific IPs and through specific ports. A firewall is, in fact, the only effective way to restrict traffic through your site to the types you need to support. A good firewall will also monitor for denial of service (DoS) attacks and port scans and will include the ability to block off connectivity from hosts or sites you suspect may be launching attacks.

IPSec/VPN technology is best applied within your company network, to protect the security of the machines using your intranet, and for companies and organizations using applications remotely. For example, an extranet or remote office should use IPSec and/or a traditional VPN solution to ensure that communication and access between that office and your machines is encrypted and secured.

Bad code is always a worry. There is, of course, no way to guarantee or protect yourself from bad code, but UrlScan certainly goes a long way toward addressing some of the problems that bad code can highlight. You should always test your code, including stressing it by over- and under-supplying information and additional data not normally required or accepted by the script.

Keeping up to date should really be top of the list. Don’t underestimate the significance of that security announcement from Microsoft or the latest security patch. You should always install these patches on your test machines once they’ve been released and ensure it doesn’t break any of your existing infrastructure.

Finally, don’t ignore common sense. All the technical security in the world, including UrlScan, won’t save you if you forget to use decent passwords, close off ports and features not really needed, or leave the door to your network operations center propped open with a fire extinguisher.