Using Active Directory Group Policy to Protect Against SYN Flood DoS Attack

I have been seeing waves of SYN Flood denial of service attack against our applications every so often. During the attack, hugh volume of TCP SYN packets was being sent to our web application until they exhausted all the resources on the corresponding server. It is pretty annoying because it affects the experience of our other legitimate users using our applications.

Our firewalls could keep track of the number of connections from a single source, and if that exceeds a certain threshold, they could deny further connection attempts. This does a pretty good job so far. But I am a little concerned about the growing number of those attacks, and eventually a distributed denial of service one.

Turns out on the Windows 2000/ 2003 server, there are several registry settings that one can use to harden the TCP/IP stack. e.g. SynAttackProtect. By setting this registry value to 1, the server will time out all half open connections more quickly during a SYN attack (determined by several thresholds), and be able to recover the resource sooner to serve legitimate users.

I want to enable it on all our servers. Can Active Directory Group Policy help? Yes! Here is how:

• Created a custom Administrative Template as follows:
  
  CLASS MACHINE
  CATEGORY "Network"
  CATEGORY "TCP/IP Hardening"
  POLICY "SynAttackProtect"
  KEYNAME "SYSTEM\CurrentControlSet\Services\Tcpip\Parameters"
  EXPLAIN "This registry value causes Transmission Control Protocol (TCP) to adjust retransmission of SYN-ACKS. When you configure this value, the connection responses time out more quickly during a SYN attack (a type of denial of service attack). Set SynAttackProtect to Disabled (0 - default) for typical protection against SYN attacks. Set SynAttackProtect to Enabled (1) for better protection against SYN attacks. This parameter causes TCP to adjust the retransmission of SYN-ACKS. When you set SynAttackProtect to 1, connection responses time out more quickly if the system detects that a SYN attack is in progress. Refers to Microsoft KB 324270 for more information."
  VALUENAME "SynAttackProtect"
  VALUEON NUMERIC 1
  VALUEOFF NUMERIC 0
  END POLICY
  END CATEGORY
  END CATEGORY
  
  
• Save the file as SynAttackProtect.adm.
• In the Group Policy Editor, go to View menu | Filtering, uncheck the "Only show policy settings that can be fully managed" checkbox.
• Then go to Computer Configuration | Administrative Templates, and use Add/Remove Template to add this SynAttackProtect.adm custom administrative template.
• Now, under Administrative Template | Network, a TCP/IP Hardening category will appear.
• Under this TCP/IP Hardening category, double-click on SynAttackProtection and change the status to Enable.
• Close the Group Policy Editor to finish.
• Link this group policy to the right OU to apply onto all the servers under it. (always do it on test servers first!)
• Within 90 minutes (default time for Active Directory Group Policy refresh), the servers will have the new registry setting.
• You still have to reboot the servers to make the registry change effective.

One thing you have to know is that this kind of group policy is a tattoo type policy. The registry setting would not be changed back to the original value if you later on remove this group policy. To undo the change, you will have to explicitly change this group policy SynAttackProtect setting to Disable.

Refer to Microsoft KB 324270 article for complete detail on how to harden the TCP/IP stack against denial of service attacks in Windows Server 2003.