Use IPSec for Filtering Ports and Authentication

How To: Use IPSec for Filtering Ports and Authentication

 

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

Improving Web Application Security: Threats and Countermeasures J.D. Meier, Alex Mackman, Michael Dunner, Srinath Vasireddy, Ray Escamilla and Anandha Murukan
Microsoft Corporation
Published: June 2003
Last Revised: January 2006

Applies to:

• Microsoft Windows® 2000 operating system

See the "patterns & practices Security Guidance for Applications Index" for links to additional security resources.
See the Landing Page for the starting point and a complete overview of Improving Web Application Security: Threats and Countermeasures.
Summary: Internet Protocol security (IPSec) is a protocol, not a service, that provides encryption, integrity, and authentication services for IP-based network traffic. Because IPSec provides server-to-server protection, you can use IPSec to counter internal threats to the network, including eavesdropping, tampering, man in the middle attacks, IP spoofing, and other password-based attacks. IPSec is completely transparent to applications because encryption, integrity, and authentication services are implemented at the transport level. Applications continue to communicate normally with one another using TCP and UDP ports.

Note   This How to is applicable to the Microsoft Windows 2000 operating system. For more information on Internet Protocol security (IPSec) support in Windows Server 2003, see "Internet Protocol Security for Microsoft Windows Server 2003."

Contents

This How To contains the following sections:
What you must know Restricting Web server communication Restricting database server communication Restricting server-to-server communication Using IPSec tools Additional Resources

What You Must Know

Before you start to configure IPSec, you should be aware of the following.

Identify Your Protocol and Port Requirements

Before you create and apply IPSec policies to block ports and protocols, make sure you know which communication you need to secure including the ports and protocols used by day-to-day operations. Consider the protocol and port requirements for remote administration, application communication, and authentication.

IPSec Does Not Secure All Communication

• Several types of IP traffic are exempt from filtering. For more information, see Microsoft Knowledge Base article 253169, Traffic That Can and Cannot Be Secured by IPSec.

Firewalls and IPSec

If a firewall separates two hosts that use IPSec to secure the communication channel, the firewall must open the following ports:

• IP Protocol ID 50 for IPSec Encapsulating Security Protocol (ESP) traffic
• IP Protocol ID 51 for IPSec Authentication Header (AH) traffic
• UDP port 500 for Internet Key Exchange (IKE/ISAKMP) negotiation traffic

Filters, Filter Actions, and Rules

An IPSec policy consists of a set of filters, filter actions, and rules.

• Filters A filter is used to match traffic. It consists of:
  
  • A source IP address or range of addresses
  • A destination IP address or range of addresses
  • An IP protocol, such as TCP, UDP, or "any"
  • Source and destination ports (for TCP or UDP only)

  Note   An IP filter list is used to group multiple filters together so that multiple IP addresses and protocols can be combined into a single filter.

• Filter Actions A filter action specifies which actions to take when a given filter is invoked. It can be one of the following:
  
  • Permit. The traffic is not secured; it is allowed to be sent and received without intervention.
  • Block. The traffic is not permitted.
  • Negotiate security. The endpoints must agree on and then use a secure method to communicate. If they cannot agree on a method, the communication does not take place. If negotiation fails, you can specify whether to allow unsecured communication or to whether all communication should be blocked.
• Rules A rule associates a filter with a filter action and is defined by the IPSec policy.

Restricting Web Server Communication

The following example shows you how to use IPSec to limit communication with a Web server to port 80 (for HTTP traffic) and port 443 (for HTTPS traffic that uses SSL.) This is a common requirement for Internet-facing Web servers.

Note   After applying the steps below, communication will be limited to port 80 and 443. In a real world environment, you will require additional communication such as that required for remote administration, database access and authentication. A complete IPSec policy, in a production environment, will include all authorized communication.

Create filter actions

1. Start the Local Security Policy Microsoft Management Console (MMC) snap-in.
2. Right-click IPSec Security Policies on Local Machine, and then click Manage IP filter lists and filter actions.
3. Click the Manage Filter Actions tab.
4. Click Add to create a new filter action, and then click Next to move past the introductory Wizard dialog box.
5. Type MyPermit as the name for the new filter action. This filter action is used to permit traffic.
6. Click Next.
7. Select Permit, click Next, and then click Finish.
8. Create a second filter action called "MyBlock" by repeating steps 4 to 8. This time, select Block when you are prompted by the Filter Action dialog box.
9. Click Close to close the Manage IP filter lists and filter actions dialog box.

Create IP filters and filter lists

1. Right-click IPSec Security Policies on Local Machine, and then click Manage IP filter lists and filter actions.
2. Click Add to add a new IP filter list., and then type MatchAllTraffic for the filter list name.
3. Click Add to create a new filter and proceed through the IP Filter Wizard dialogs boxes by selecting the default options. This creates a filter that matches all traffic.
4. Click Close to close the IP Filter List dialog box.
5. Click Add to create a new IP filter list, and then type MatchHTTPAndHTTPS for the filter list name.
6. Click Add, and then click Next to move past the introductory Wizard dialog box.
7. Select Any IP Address from the Source address drop-down list, and then click Next.
8. Select My IP Address from the Destination address drop-down list, and then click Next.
9. Select TCP from the Select a protocol type drop-down list, and then click Next.
10. Select To this port and then specify port 80.
11. Click Next and then Finish.
12. Click Add, and then repeat steps 9 to 14 to create another filter that allows traffic through port 443. Use the following values to create a filter that allows TCP over port 443:
    
    • Source Address: Any IP address
    • Destination Address: My IP Address
    • Protocol: TCP
    • From Port: Any
    • To Port: 443

After finishing these steps, your IP Filter List should look like the one that Figure 5 shows.

Figure 5

IP Filter List dialog box
After creating the filter actions and filter lists, you need to create a policy and two rules to associate the filters with the filter actions.

Create and apply IPSec policy

1. In the main window of the Local Security Policy snap-in, right-click IPSec Security policies on Local Machine, and then click Create IPSecurity Policy.
2. Click Next to move past the initial Wizard dialog box.
3. Type MyPolicy for the IPSec policy name and IPSec policy for a Web server that accepts traffic to TCP/80 and TCP/443 from anyone for the description, and then click Next.
4. Clear the Activate the default response rule check box, click Next, and then click Finish. The MyPolicy Properties dialog box is displayed so that you can edit the policy properties.
5. Click Add to start the Security Rule Wizard, and then click Next to move past the introductory dialog box.
6. Select This rule does not specify a tunnel, and then click Next.
7. Select All network connections, and then click Next.
8. Select Windows 2000 default (Kerberos V5 protocol), and then click Next.
9. Select the MatchHTTPAndHTTPS filter list, and then click Next.
10. Select the MyPermit filter action, click Next, and then click Finish.
11. Create a second rule by repeating steps 5 to 10. Instead of selecting MatchHTTPAndHTTPS and MyPermit, select MatchAllTraffic and MyBlock.

After creating the second rule, the MyPolicy Properties dialog box should look like the one in Figure 6.

Figure 6

MyPolicy Properties dialog box
Your IPSec policy is now ready to use. To activate the policy, right-click MyPolicy and then click Assign.

Summary of What You Just Did

In the previous three procedures, you performed these actions:

• You started by creating two filter actions: one to allow traffic and one to block traffic.
• Next, you created two IP filter lists. The one called MatchAllTraffic matches on all traffic, regardless of port. The one called MatchHTTPAndHTTPS contains two filters that match TCP traffic from any source address to TCP ports 80 and 443.
• Then you created an IPSec policy by creating a rule that associated the MyBlock filter action with the MatchAllTraffic filter list and the MyPermit filter action with the MatchHTTPAndHTTPS filter list. The result of this is that the Web server only allows TCP traffic destined for port 80 or 443. All other traffic is rejected.

Restricting Database Server Communication

On a dedicated SQL Server database server, you often want to restrict communication to a specific SQL Server port over a particular protocol. By default, SQL Server listens on TCP port 1433, and UDP port 1434 is used for negotiation purposes.
The following steps restrict a database server so that it only accepts incoming connections on TCP port 1433 and UDP port 1434:

• Create two filter actions: one to permit traffic and the other to block traffic. For details, see the Create filter actions procedure under "Restricting Web Server Communication" earlier in this How To.
• Create two filter lists: one that matches all traffic and one that contains two filters that match TCP traffic destined for port 1433 and UDP traffic destined for port 1433. For details, see "Create IP filter lists and filters" under "Restricting Web Server Communication" earlier in this How To. The required filters are summarized below.
  • Enter the following values to create a filter that allows TCP over port 1433:
  • Source Address: Any IP address
  • Destination Address: My IP Address
  • Protocol: TCP
  • From Port: Any
  • To Port: 1433
  • Enter the following values to create a filter that allows UDP over port 1434:
  • Source Address: Any IP address
  • Destination Address: My IP Address
  • Protocol: UDP
  • From Port: Any
  • To Port: 1434
• Create and apply IPSec policy by repeating the procedure under "Restricting Web Server Communication" earlier in this How To.

Restricting Server-to-Server Communication

You can also use IPSec to provide server authentication. This is useful when restricting the range of computers that can connect to middle-tier application servers or database servers. IPSec provides three authentication options:

• Kerberos To use Kerberos, the computers must:
  
  • Be part of the same domain and forest
  • Be within a specific source address range
  • Be within the same subnet
  • Use static IP addresses
• Pre-shared secret key To use pre-shared secret-key-based authentication, the two computers must share an encryption key.
• Certificate-based authentication To use certificate authentication, the two computers must trust a common certificate authority (CA), and the server that performs the authentication must request and install a certificate from the CA.

In this section, you set up IPSec authentication between two servers by using a pre-shared secret key.

To perform server-to-server authentication

1. Start the Local Security Policy MMC snap-in.
2. Right-click IPSec Security policies on the local machine, and then click Create IP Security Policy.
3. Type "MyAuthPolicy" for the name, and then click Next.
4. Clear the Activate the default response rule check box.
5. Click Next and then Finish. The MyAuthPolicy Properties dialog box is displayed so that you can edit the policy properties.
6. Click Add, and then click Next three times.
7. In the Authentication Method dialog box, select Use this string to protect the key exchange (preshared key).
8. Enter a long, random set of characters in the text box, and then click Next. You should copy the key to a floppy disk or CD. You need it to configure the communicating server.
9. In the IP Filter List dialog box, select All IP Traffic, and then click Next.
10. In the Filter Action dialog box, select Request Security (Optional), and then click Next.
11. Click Finish.
12. Test your application to verify the configured policy.

Using IPSec Tools

This section describes two useful IPSec diagnostic tools that are available as part of the Windows 2000 resource kit:

• Netdiag.exe
• IPSecpol.exe

Netdiag.exe

Before creating a new policy, determine if your system already has an existing policy. You can do this by performing the following steps:

To check for existing IPSec policy

1. To install Netdiag.exe, run the Setup.msi program from the \Support\Tools folder on the Windows 2000 Server CD. The tools are installed in C:\Program Files\Resource kit.
2. Run the following command from the command line:

   netdiag /test:ipsec
   

   If there are no existing filters, then the output looks like the following:
   

   IP Security test . . . . . . . . . : Passed IPSec policy service is active, but no policy is assigned.
   

IPSecpol.exe

The Internet Protocol Security Policies tool helps you automate the creation of policies in local and remote registries. The tool supports the same settings that you can configure by using the MMC snap-in.
Download the tool from the Microsoft Windows 2000 Web site at http://technet.microsoft.com/en-us/windowsserver/2000/default.aspx.
For detailed examples of using Ipsecpol.exe to create and manipulate IPSec rules, see Microsoft Knowledge Base article 813878, "How to Block Specific Network Protocols and Ports by Using IPSec."

Additional Resources

For more information, see the following resources:

• Step-by-Step guide to IPSec at http://technet.microsoft.com/en-us/windowsserver/2000/default.aspx.
• IP Security for Windows 2000 Server at http://technet.microsoft.com/en-us/windowsserver/2000/default.aspx.
• "How To: Use IPSec to Provide Secure Communication Between Two Servers" in the How To section of "Building Secure ASP.NET Applications" on MSDN.
• Article 313190, "How To: Use IPSec IP Filter Lists in Windows 2000" in the Microsoft Knowledge Base.
• Article 813878, "How to Block Specific Network Protocols and Ports by Using IPSec" in the Microsoft Knowledge Base.
• Article 313195, "How To: Use IPSec Monitor in Windows 2000" in the Microsoft Knowledge Base.
• IPSec considerations at http://www.microsoft.com/technet/prodtechnol/winxppro/proddocs/sag_IPSECsec_con.asp.
• IPSec Operations on Windows Server 2003 at http://technet.microsoft.com/en-us/library/cc706993(WS.10).aspx.