DROP vs REJECT

This article is translated (with some omission) from my own Japanese blog article DROP vs REJECT論争、そしてWindowsとLinuxのファイアウォールの動作の違いについて — turgenev’s blog.

Overview

In this article, I explain how DROP and REJECT, two methods of handling unwanted packets in IP communication, are used in Windows and Linux firewalls. I also introduce a specific method to make Linux behave more like Windows, which is desirable for some cases but probably not well known.

Today, the entire globe is connected via the internet and we never know when, from where, or what kind of packets might arrive at our homes. When an “uninvited packet” from an unknown source reaches us, we have two options: one is to simply ignore (discard) the packet, and the other is to notify the sender that “this cannot be accepted” (and then discard it).

In Linux, packet handling settings such as iptables (and its successor, nftables) use the terms DROP for the former (ignoring) and REJECT for the latter (explicit rejection response). Therefore, I will use these two terms in my explanation. Some softwares are using different terms; for example, a firewall for Linux, ufw (explained later), uses the word “deny” instead of DROP.

Classification of REJECT Methods, TCP, and UDP

DROP is straightforward in that it just ignores incoming packets, but with REJECT, there are different ways to convey the rejection response.

The protocol used for rejection responses is ICMP, a Layer 3 protocol like IP and used in ping, though I don’t fully understand it.

The following are some types of ICMP rejection messages:

icmp-net-unreachable
icmp-host-unreachable
icmp-port-unreachable
icmp-proto-unreachable
icmp-net-prohibited
icmp-host-prohibited
icmp-admin-prohibited

For example, icmp-host-unreachable is returned (from the computer itself) when pinging a non-existent IP address in the same subnet. When trying to access a closed (no application listening) UDP port, icmp-port-unreachable is returned. This is the same in both Windows and Linux.

I’m not sure about other messages, but they probably have their uses somewhere.

In the case of TCP, one can return any of the above or choose to send a TCP RST (reset) packet. ICMP is an entirely different protocol from TCP (for example, it does not have the concept of port numbers), but a TCP RST is more straightforward as it works within TCP. When accessing a closed TCP port in Windows or Linux, a TCP RST is returned.

Other than these messages, there seems to be an echo-reply used in some responses to pings (https://www.asahi-net.or.jp/~aa4t-nngk/ipttut/output/rejecttarget.html), but generally, the options for REJECT are as mentioned above.

Testing DROP and REJECT

You can test the behavior of DROP and REJECT on your browser.

For example, if you type localhost:8000 (8000 is randomly chosen) into your browser, you should get a response saying “connection refused” immediately (or after about ~2 seconds on Windows). This is REJECT behavior.

If you type google.com:8000 (8000 is randomly chosen) , it will keep loading indefinitely without any response, and after tens of seconds, the browser finally gives up and displays some message like “the connection has timed out” or “ERR_CONNECTION_TIMED_OUT.” This is DROP behavior, where there is no response at all, so the browser can’t tell whether it’s just taking time due to network delay or if it’s being ignored.

Also, you can test DROP/REJECT behavior by command-line tools like “nmap”. nmap typically shows “closed” for REJECT and “filtered” for DROP. I omit the detail on how to use such tools, but they are more useful for testing.

Differences in TCP and UDP

Relating to DROP vs REJECT, there are differences in the behavior of TCP and UDP.

TCP is a connection-based protocol, so an initial exchange of SYN and SYN/ACK packets is necessary before any successful TCP communication. On the other hand, UDP is a connectionless protocol, so applications only have to send contents of interest. This means that applications like VPN don’t have to send any reply to unwanted communications, such as a message with invalid credentials. That’s why nmap shows “open|filtered” in UDP port scanning (because it can’t judge if the packet was DROPped by OSes or received but ignored by applications) and so-called “port checkers” can only check TCP port availability.

In other words, UDP is slightly better than TCP in security because it can make “stealth” open port.

Windows Firewall

Having become familiar with the concepts of DROP and REJECT, let’s now discuss the operation of firewalls in actual network devices, starting with Windows.

The Windows firewall primarily employs DROP (the firewall in Windows is divided into ‘Inbound’ and ‘Outbound’ rules, but the discussion here is mainly about ‘Inbound’). A distinctive feature is the ability to set allow/block rules for each application, allowing only packets from applications that meet the criteria to pass through the firewall. In other words, ports are opened only when the application is running; otherwise, everything is dropped, which is typical behavior in Windows.

If you have two PCs, one of which is Windows, you can test this with nmap. Ports with no active applications will appear to outside as ‘filtered’ for TCP and ‘open|filtered’ for UDP. The TCP status changes to ‘open’ as soon as an application starts, while UDP remains the same (or becomes ‘open’ if the application responds).

I don’t know the specifics of how this ‘instant switch when an allowed application starts’ behavior is implemented, but that’s how Windows works.

Linux Firewall

In Linux, the firewall is often disabled by default. If you have two devices, including one with a default state Linux (Android works too), you can test with nmap. Likely, ports with no applications running will be shown as ‘closed’ for both tcp and udp. Upon activation, they change to ‘open’ and ‘open|filtered’ (or just ‘open’), respectively.

Of course, you can enable the firewall in Linux. A widely used one is ufw, which internally uses iptables or nftables (command name is nft), so you can configure it with either. By default, ufw also uses DROP (or “deny” in ufw terminology). Users can change this, setting various responses like --reject-with icmp-port-unreachable or --reject-with tcp-reset(TCP only).

Unlike Windows, ufw doesn’t have application-specific rule settings but allows or denies traffic based only on ports. Software like GitHub — matrix-ac/LAF: Linux Application Firewall seems to allow application-specific settings, but it’s not widely used, so I won’t discuss it here.

To allow traffic on some port, you would use a command like ufw allow 8080/tcp .

Differences Between Windows and Linux

Based on this explanation, you might think the operation of Windows and Linux firewalls (using ufw’s deny) is not much different, except for application versus port-based decisions. However, a significant difference lies in the behavior when applications are not running.

The Windows firewall allows packets based on running applications. So, when no application is using a port, communication to that port is simply dropped as usual.

On the other hand, in Linux, the firewall is set to allow traffic to a specific port, such as ‘allow tcp traffic to port 8080’. This means the port is always open, regardless of whether an application is using it or not. When an application is running, it behaves like Windows, but when not, it does not drop packets but sends a message indicating the port is unavailable. For UDP, this would be an icmp port-unreachable message, and for TCP, a RST packet. This behavior is a default response by the Linux kernel and there seems to be no direct method to change this.

For reference, the following pages also suggest this is not possible:

This is not ideal for a firewall that primarily uses DROP. When some ports are open for external services and the service is not running, those open ports behave differently from other ports, observable from the outside. For TCP, it’s less of an issue since external detection happens anyway when the application is running. However, for UDP, the advantage of ‘open ports being indistinguishable from filtered ones’ is lost when the application stops.

For TCP, setting --reject-with tcp-reset can align the behavior of ‘open but unlistened ports’ with ‘unopened ports’, but for UDP, setting --reject-with icmp-port-unreachable changes in turn the response when the application is running.

Another approach might be scripting to dynamically open ports only when the application is running, but compared to Windows firewall’s straightforward ‘ignore if no application is listening’ behavior, this seems like a less elegant solution.

So, one of the points I wanted to make in this article is: Isn’t the Linux firewall fundamentally flawed? Even if we overlook the lack of application-specific rules, shouldn’t a firewall at least offer the ability to set DROP/REJECT responses for packets to ‘open but unlistened’ ports?

Addendum: Windows Stealth Mode

I learned later that Windows has a feature called ‘stealth mode’, where packets to closed ports are dropped without response, which causes the behavior described above. This can be disabled via the registry.

(However, I’m not clear on the behavior when stealth mode is disabled in Windows. Perhaps all ports start sending reject responses, and the firewall’s role becomes ‘only blocking communications to unauthorized applications’? In that case, would it be DROP or REJECT??)

Routers

Alongside general-purpose computers like Windows and Linux, I’ll also discuss the behavior of routers, an essential component of networks.

However, many routers, including low-cost commercial products, often run on a Linux-based OS, so there’s not much point in strictly differentiating them from PCs. Conversely, a Linux machine used as a general-purpose PC can also function as a router (though it would be challenging without two ethernet ports). In such cases, the basic settings available in Linux should be applicable.

Speaking only about the operation of typical home routers, however, I expect almost all ones use “DROP” behavior and that is not changeable. Routers do have “packet filtering” settings, but these are primarily for handling packets arriving at explicitly configured ports and going inside, like by DMZ or port forwarding, and do not allow changes to the response to packets arriving at unrelated ports.

If you’re considering exposing ports through a router to the internet, it’s advisable to align the behavior of the internal ports with that of the router. If the router uses DROP, it’s better for the exposed PC ports to also use DROP to avoid standing out, and the same goes if the router uses REJECT.

DROP vs REJECT, Which is Better?

After reviewing the information so far, let’s return to the question: which is better, DROP or REJECT? Here are some existing discussions on the topic:

Drop versus Reject

IPtables: TCP-Reject vs DROP : linux

iptables REJECT vs DROP | todisco.de

[ale] iptables: DROP vs. REJECT — reject-with tcp-reset

Closed Ports Vs Stealth Ports, Drop Rules Vs Reject Rules — Which is better | Ron’s Tech Tips

linux — REJECT vs DROP when using iptables — Server Fault

ufw Linux firewall difference between reject and deny — Stack Overflow

ip — Is it better to set -j REJECT or -j DROP in iptables? — Unix & Linux Stack Exchange

SANS — Internet Storm Center — Cooperative Cyber Threat Monitor And Alert System

Revisit REJECT vs DROP in #507 · Issue #2217 · fail2ban/fail2ban · GitHub

Is it better a CLOSED port or a FILTERED port to for DoS attack protection? : AskNetsec

linux — Why does iptables accept packets on a given port, when it is closed? — Unix & Linux Stack Exchange

[ubuntu] Is “Stealth” more secure than “closed” ports?

Summarization of these articles and our discussion so far (including my own opinion) is as follows:

Arguments in favor of DROP:

Forces time consumption during attacker’s port scanning due to timeouts (a few seconds).
Does not send response packets, reducing PC and network load and enhancing resilience to certain DDoS attacks.
For UDP, open and DROP ports behave similarly, concealing the presence of running applications.
Most commercial routers default to DROP (thus, internal configurations must follow suit).
Generally safer as it’s commonly used (using REJECT might attract attention from attackers who assume you’re an advanced network user).
google.com uses DROP

Arguments in favor of REJECT:

Immediate response, avoiding unnecessary waiting for non-malicious users.
For TCP, using tcp-reset for REJECT can mimic Linux’s state of ‘port open but no application running’.

The most crucial point is that the behavior of the outermost device (exposed to the internet) must be consistent. Since attackers usually come from outside the LAN, the behavior of PCs exposing services through the router should follow the router’s operation. In most home networks, where routers operate on DROP, the overall setup should be DROP.

Also, DROP has clear advantage to make open UDP port “stealth”, while REJECT has no clear advantage in security.

“REJECT with tcp-reset can simulate a closed port” is correct, but, as previously discussed, this is just a workaround for Linux’s behavior, where tcp-reset is sent when an application isn’t running on an opened port. In Windows, if no application is running, the port drop any packets, seamlessly aligning the behavior of all ports to DROP policy. It is not that tcp-reset is superior, but rather that Linux, which forces the use of tcp-reset, is inferior.

In conclusion, my recommendation is DROP. The problematic behavior of unlistened port is Linux’s responsibility. I will later introduce a method to address this.

Finally, I’ll repeat this: more important than choosing between DROP and REJECT is not exposing vulnerable applications to the internet.

Making Linux Behave Like Windows

We’ve decided to adopt DROP. Then, in using Linux, we need some method to make it conform to DROP policy.

The current issue is that when accessing a port that is ‘open but not listening,’ Linux sends a TCP RST packet for TCP and an ICMP port-unreachable for UDP. If we can prevent these packets from being sent, it would be functionally same as ignoring them (i.e., DROP).

As mentioned before, we cannot stop the Linux kernel from generating these packets, but we can prevent them from leaving the Linux machine at the moment of sending.

For this, we use iptables or nftables (for more recent Linux versions). I deal with ntfables here because my Ubuntu use nftables, but things are essentially same in iptables.

For TCP, we need to eliminate TCP RST packets coming from a specific port. For UDP can eliminate outgoing ICMP port-unreachable packets for specific ports (by checking ICMP payload).

The concern here is whether these packets are used for purposes other than responding to access on closed ports. Stopping all these packets might cause unintended side effects.

However, first, the ICMP port-unreachable is used exclusively as an error message when a port is unreachable (as its name indicates). Since it’s part of an ICMP protocol, stopping it won’t affect legitimate UDP communications. TCP RST, although sent by some applications, is used in abnormal situations where a connection must be forcibly terminated. In normal connection terminations (like finishing loading a webpage via HTTP or being kicked from a game server), FIN packets are used, so there shouldn’t be substantial issues if RST can’t be used. Additionally, RST packets sent when a port is closed have a sequence number of 0 (though some packets with a sequence number of 0 might be sent by applications, but I haven’t encountered such a case). This can be used to more specifically target RST packets sent from closed ports.

Still, we want to minimize the impact on the traditional network environment. So, I’ll try a method to set up a dedicated IP address for exposed web services and prohibit outgoing RST and ICMP port-unreachable packets only from the IP.

Adding an IP Address and Blocking Various Packets

Let’s assume the original IP address of the PC is 192.168.1.2, and the additional IP address is 192.168.1.3. The NIC name is eno1. There is a router with DROP policy outside the PC.

First, add the IP address with the following command:

sudo ip addr add 192.168.1.3/24 dev eno1 label eno1:1

This settings will be reset after rebooting, so write it in /etc/network/if-up.d or similar for persistence. For more details on adding IPs, see other sites.

Next, prohibit outgoing ICMP port-unreachable and TCP reset from 192.168.1.3. I’ll use TCP port 8000, but it’s possible to target port ranges.

First, create a new table for IP(v4)-related rule additions (I’m not exactly sure of the best practices for nft; adding to an existing table is also fine).

sudo nft add table ip my_ip_table

Then, add a chain for setting a postrouting filter to this table.

sudo nft add chain ip my_ip_table postrouting_filter “{type filter hook postrouting priority 700; }”

The priority setting here is somewhat important, as nftables rules are applied in order of ascending priority value. According to Chapter 58. Getting started with nftables Red Hat Enterprise Linux 8 | Red Hat Customer Portal, the range of values used in default settings is about -300 to 300, so by specifying a larger value (700 was chosen arbitrarily), this filter will be applied last.

Here, I’m using the postrouting hook. You could do something similar with output, but output applies a bit earlier when the packet is generated, so if you want to target packets after applying SNAT, you need to use postrouting, which applies just before sending.

Now, set the rules. First for TCP:

sudo nft add rule ip my_ip_table postrouting_filter ip saddr 192.168.1.3 tcp sport 8000 “tcp flags & rst == rst tcp sequence == 0 drop”

We drop packets from TCP port 8000 that have the rst flag (flags & rstis a bitwise operation) and a sequence of 0. There are several syntaxes, like tcp flags & rst == rst @th,32,32 {0} drop, which works the same. There seem to be also syntaxes like [payload load 4b @ transport header + 4 => reg 1] [cmp eq reg 1 0x00000000], but I couldn’t get this work (maybe old syntax).

Next for UDP:

sudo nft add rule ip my_ip_table postrouting_filter ip saddr 192.168.1.3 icmp type destination-unreachable icmp code port-unreachable @th,240,16 {8000, 8080} drop

This way, you can drop only port-unreachable packet for port 8000 and 8080. Refer to external sources (e.g. https://flylib.com/books/en/3.223.1.80/1/) for ICMP payload structure.

Since I prohibited RST and port-unreachable uniformly without limiting the destination, the effect can be confirmed even from the computer itself.

$ nmap -p 7999-8001 -sT 192.168.1.3 -Pn
PORT     STATE  SERVICE
7999/tcp closed irdmi2
8000/tcp filtered http-alt
8001/tcp closed vcom-tunnel

As you can see, only port 8000 of 192.168.1.3 appear as filtered. Similar behavior can be confirmed in UDP.

Basically, this makes Linux behave almost like Windows.

Making Tailscale Work Properly

In fact, the initial purpose of writing this article was to prevent ports opened for Tailscale VPN appearing closed when Tailscale is not running. The settings so far have achieved this, but now Tailscale’s direct connections via UDP do not work. This is because Tailscale attempts to use the main IP, 192.168.1.2, as the outgoing IP, which doesn’t match well with external communications which go through 192.168.1.3.

This issue could arise with any software that actively initiates outbound connections, not just Tailscale.

As a solution, I apply DNAT/SNAT between 192.168.1.2 and 192.168.1.3, routing all external communications via 192.168.1.3. In the previously created my_ip_table, I add nat type chains for postrouting and prerouting and add rules for each. Here, I use port 41641, the default for Tailscale.

sudo nft add chain ip my_ip_table postrouting_nat "{type nat hook postrouting priority srcnat;}"
sudo nft add chain ip my_ip_table prerouting_nat "{type nat hook prerouting priority dstnat;}"
sudo nft add rule ip my_ip_table postrouting_nat ip saddr 192.168.1.2 udp sport 41641 snat to 192.168.1.3:41641
sudo nft add rule ip my_ip_table prerouting_nat ip daddr 192.168.1.3 udp dport 41641 dnat to 192.168.1.2:41641

In addition to this, map port 41641 on the router to 192.168.1.3:41641, and Tailscale’s port becomes publicly available. Whether it’s running or stopped, the port number cannot be guessed from outside.

Note that 192.168.1.2:41641 becomes unusable, but 192.168.1.3:41641 can be used for LAN-internal devices. If there are problems, you might exclude LAN (private IP) communications from the snat rule.

Checking and Persisting Settings

To summarize and check the settings made with the nft command, use sudo nft list ruleset. The contents of my_ip_table should look like this:

table ip my_ip_table {
        chain postrouting_filter {
                type filter hook postrouting priority 700; policy accept;
                ip saddr 192.168.1.3 icmp code port-unreachable drop
                ip saddr 192.168.1.3 tcp sport 8000 tcp flags & rst == rst tcp sequence 0 drop
        }
        chain postrouting_nat {
                type nat hook postrouting priority srcnat; policy accept;
                ip saddr 192.168.1.2 udp sport 41641 snat to 192.168.1.3:41641
        }
        chain prerouting_nat {
                type nat hook prerouting priority dstnat; policy accept;
                ip daddr 192.168.1.3 udp dport 41641 dnat to 192.168.1.2:41641
        }
}

If it’s not working as intended, compare the results of sudo nft list rulesetwith the above.

Like adding an IP address, the nft settings will also be reset after rebooting, so they need to be persisted.

First, enable nftables service (sudo systemctl enable nftables). This is a OneShot type service that loads the nftables configuration file (e.g., /etc/nftables.conffor Ubuntu) at system startup (it’s not a constantly running daemon). Then, add the contents of my_ip_tableto the nftables.conf. Alternatively, you can use includeto split it into separate files. Many sources suggest copying the output of sudo nft list rulesetdirectly into nftables.conf, but this might inadvertently involve dynamically inserted rules like those from Tailscale. I personally prefer to edit the original file directly.

Conclusion

As for whether DROP or REJECT is better as a firewall behavior, DROP should be chosen due to the majority’s preference and its ability to conceal UDP services.

Linux’s firewall has a flaw not present in Windows, where the behavior of closed ports does not default to DROP. However, this can be resolved without much impact on normal use by correctly setting up iptables/nftables rules.