quietlife.net

Securing a Linux box in the Great Wide Open

18 Dec 2003

Ideally, any GNU/Linux server should be behind a firewall. However, sometimes you don’t have a choice. My nameservers are great examples of this – they just aren’t in places where a firewall is practical, and Cisco IOS ACLs are an unsavory alternative because they are easily (inadvertently) blown away.

So, it’s important to know how to secure GNU/Linux so that you can confidently put it on the Internet without fear of compromise. It’s not as difficult as it may seem.

No matter what distribution you choose, it’s inevitable that it will probably install with more services than you actually need running. The first step, then, is to portscan the server from another machine. (At this point, you should be on a private, protected network behind a firewall – if you already have the server up and running, open on the internet, it’s already too late.) Here’s an example, using a newly installed Debian 3.0 server at 192.168.0.6:

root@scanbox:~# nmap -sS -p 1-65535 192.168.0.6

Starting nmap 3.48 ( http://www.insecure.org/nmap/ ) at 2003-11-06 11:25 CST
Interesting ports on host06.quietlife.net (192.168.0.6):
(The 65523 ports scanned but not shown below are in state: closed)
PORT     STATE SERVICE
9/tcp    open  discard
13/tcp   open  daytime
21/tcp   open  ftp
22/tcp   open  ssh
23/tcp   open  telnet
25/tcp   open  smtp
37/tcp   open  time
79/tcp   open  finger
111/tcp  open  rpcbind
113/tcp  open  auth
515/tcp  open  printer
1026/tcp open  LSA-or-nterm

Nmap run completed -- 1 IP address (1 host up) scanned in 7.759 seconds

As you can see, there are quite a few ports open. All these services, without exception, are controlled by daemons running that are spawned from scripts in /etc/init.d/ during startup. The solution then is to isolate the services we don’t require and turn them off (and remove the software if it is unnecessary). Let’s say we are setting up a DNS server. In the end, we will want nothing but port 22/tcp and 53/udp open.

First, services like “9/tcp open discard” and “13/tcp open daytime” are services run by inetd. inetd is a daemon itself that encompasses many different services, all of which we don’t need. Let’s turn it off:

testbox:~# /etc/init.d/inetd stop
Stopping internet superserver: inetd.

Now, let’s see what nmap looks like:

root@scanbox:~# nmap -sS -p 1-65535 192.168.0.6

Starting nmap 3.48 ( http://www.insecure.org/nmap/ ) at 2003-11-06 11:31 CST
Interesting ports on host06.quietlife.net (192.168.0.6):
(The 65532 ports scanned but not shown below are in state: closed)
PORT     STATE SERVICE
22/tcp   open  ssh
111/tcp  open  rpcbind
1026/tcp open  LSA-or-nterm

Nmap run completed -- 1 IP address (1 host up) scanned in 7.602 seconds

Pretty impressive – already we have gotten rid of 8 open services, none of which we require. To be thorough, let’s go ahead and remove inetd, since we won’t be needing it:

testbox:~# dpkg -l | grep inetd
ii  netkit-inetd   0.10-9         The Internet Superserver
testbox:~# apt-get remove netkit-inetd
Reading Package Lists... Done
Building Dependency Tree... Done
The following packages will be REMOVED:
  fingerd ftpd lpr netbase netkit-inetd pidentd ppp pppconfig pppoe pppoeconf
  talkd telnetd ytalk
0 packages upgraded, 0 newly installed, 13 to remove and 0  not upgraded.
Need to get 0B of archives. After unpacking 2704kB will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 19451 files and directories currently installed.)
Removing fingerd ...
Removing ftpd ...
Removing lpr ...
Stopping printer spooler: lpd.
Removing pppoeconf ...
Removing pppoe ...
Removing pppconfig ...
Removing ppp ...
Stopping all PPP connections...No /usr/sbin/pppd found running; none killed.
done.
dpkg - warning: while removing ppp, directory `/etc/chatscripts' not empty so not removed.
dpkg - warning: while removing ppp, directory `/etc/ppp/ip-down.d' not empty so not removed.
dpkg - warning: while removing ppp, directory `/etc/ppp/peers' not empty so not removed.
Removing pidentd ...
Removing ytalk ...
Removing talkd ...
Removing telnetd ...
Removing netbase ...
Removing netkit-inetd ...
Stopping internet superserver: inetd.
testbox:/etc/init.d# dpkg --purge netkit-inetd
(Reading database ... 19266 files and directories currently installed.)
Removing netkit-inetd ...
Purging configuration files for netkit-inetd ...

Now, let’s return to our nmap. We still have two services running that we don’t require: “111/rpcbind”, and “1026/tcp LSA-or-nterm”. How do we figure out what daemons are holding those ports open? Enter “lsof”. lsof is a program that lists detailed information about files and sockets held open by processes. The output of lsof for open ports will have a line with “LISTEN” in it, which is what we will grep for with “lsof -n” (-n turns off name resolution so it won’t hang trying to reverse DNS query IP addresses):

testbox:~# lsof -n | grep LISTEN
portmap   2642  root    4u  IPv4      19541               TCP *:sunrpc (LISTEN)
rpc.statd 2649  root    5u  IPv4      19625               TCP *:1026 (LISTEN)
sshd      2738  root    4u  IPv4      19782               TCP *:ssh (LISTEN)

Now we can see the two daemons responsible for those ports: portmap and rpc.statd. But how do we turn them off? First, we need to find out what init.d script starts them:

testbox:~# cd /etc/init.d/
testbox:/etc/init.d# egrep -l 'rpc.statd|portmap' *
mountnfs.sh
nfs-common
portmap

There you go. It appears that these are both NFS-related services. If you need NFS, naturally, you should leave these alone, but since we don’t for our nameserver, we will stop them:

testbox:/etc/init.d# /etc/init.d/portmap stop
Stopping portmap daemon: portmap.
testbox:/etc/init.d# /etc/init.d/nfs-common stop
Stopping NFS common utilities: statd.

You never know when you might need NFS in the future, so I am not going to uninstall it entirely, but we don’t want it starting up on reboots in the future, either, so here is how you can remove it from startup:

testbox:/etc/init.d# update-rc.d -f portmap remove
update-rc.d: /etc/init.d/portmap exists during rc.d purge (continuing)
 Removing any system startup links for /etc/init.d/portmap ...
   /etc/rc0.d/S10portmap
   /etc/rc6.d/S10portmap
   /etc/rcS.d/S41portmap
testbox:/etc/init.d# update-rc.d -f nfs-common remove
update-rc.d: /etc/init.d/nfs-common exists during rc.d purge (continuing)
 Removing any system startup links for /etc/init.d/nfs-common ...
   /etc/rc0.d/K81nfs-common
   /etc/rc1.d/K81nfs-common
   /etc/rc2.d/S19nfs-common
   /etc/rc3.d/S19nfs-common
   /etc/rc4.d/S19nfs-common
   /etc/rc5.d/S19nfs-common
   /etc/rc6.d/K81nfs-common

Now these services are stopped, and won’t start up on boot. Let’s check out “lsof -n” again:

testbox:/etc/init.d# lsof -n | grep LISTEN
portmap   2642  root    4u  IPv4      19541               TCP *:sunrpc (LISTEN)
sshd      2738  root    4u  IPv4      19782               TCP *:ssh (LISTEN)

Hm, that’s odd. “portmap” is still running, even though we shut it down. Sometimes, you have to be a little more rude. The second column lists the PID, 2642, so let’s kill it:

testbox:/etc/init.d# kill -9 2642
testbox:/etc/init.d# lsof -n | grep LISTEN
sshd      2738  root    4u  IPv4      19782               TCP *:ssh (LISTEN)

Now we can see that only one port is being held open for SSH. We’re looking good. Let’s doublecheck the portscan to confirm:

root@scanbox:~# nmap -sS -p 1-65535 192.168.0.6

Starting nmap 3.48 ( http://www.insecure.org/nmap/ ) at 2003-11-06 11:55 CST
Interesting ports on host06.quietlife.net (192.168.0.6):
(The 65534 ports scanned but not shown below are in state: closed)
PORT   STATE SERVICE
22/tcp open  ssh

Nmap run completed -- 1 IP address (1 host up) scanned in 7.376 seconds

Looks good! Now let’s check “lsof -n” one more time, but this time, we’ll look for UDP services, to make sure they are shut down too. We could use nmap to scan for UDP with the “-sU” flag instead of “-sS” but UDP scans are extremely slow. This is because RFC 1812 suggests that ICMP error messages be rate-limited, and most operating systems comply. So, we’ll just check directly:

testbox:/etc/init.d# lsof -n | grep UDP
dhclient-  124  root    6u  IPv4         79               UDP *:bootpc

Hm, bootpc is listening for DHCP broadcasts. That’s okay – I used DHCP to configure this box on my LAN, but once I put it into production, it will have a static IP and dhclient will not be running.

Now for the last step. Sometimes, even this level of lockdown is sufficient, but there is a lot of information to be gleaned even from normal TCP connection refusals (such as OS fingerprinting and TCP sequence vulnerability). Further, one of the most common tactics in compromising a server is to find a vulnerable service being run (sshd, for example), and get a root shell. The second step is usually to install a root shell listening on a higher, discreet port. By doing this, the attacker gains unfettered access to the box on this port. This is done because often, compromising something like sshd is a one-shot deal, and the daemon will subsequently crash.

To combat this potential, we can go a second step and configure our server to drop packets except destined for services we determine. In GNU/Linux this is done with “iptables” (ipchains is an older form of the same thing in 2.2.x kernels which is highly deprecated). I’ll provide a good skeleton script for doing this in an forthcoming entry.