Hardening a Linux Server

What is Server hardening?

It’s making an electronic device more secure(yes, hardening is not something that you just do on big corporate servers). To be more precise it is reducing the attack surface of that device.

Then again, what’s that? The attack surface is all the options that an attacker has to gain access to that device, through a service like apache if you are hosting a web, or an open wifi spot to your phone. There are lot’s of ways to infect a device and today, almost every device is a computer. The most important thing when handling security is to be austere and simple, you won’t give a user access to a folder that he/she doesn’t need, or have installed nginx in a server that host msysql for the intranet, and you could be even more specific and turn off all the php modules you have enabled by default and you are not using.

But hardening is not only something you need to do on the device, it is something you need to do on yourself. For example, you need to be careful with the data you transmit over which networks; like, if you got on a trip, and do remote stuff over the hotel network, or having a coffee and using that wifi. I’m not saying not to trust any network that it’s not your own, but, take precautions even in your own networks. If you want to know a little bit more about this, you could read other articles on our web about anonymizing your web traffic or how to encrypt your mail content. You could also stop by and join the discussion on or community forums.

Why you should consider doing it?

Now that we know something of what we’re talking, let’s talk about something even more important, why would you do this?

Hardening your device is about protecting not only your information, but also your name/reputation and money. Your phone usually handles critical information about yourself (banking accounts, health care, social media, etc,) and usually you plug it anywhere, even at work. So with all this information at hand, you could get your id stolen, lose access to some important account (even your whole computer), or maybe someone hacks you to gain access to your company network.

In the end it doesn’t matter what you lose, it’s yours, and you surely don’t want to lose it.

How can you do it?

This is the long part, but we will try to cover the basics here:

Read: one of the most important things you need to do to stay secure is read. Try to keep up with tech news about the things you use, read the links you click, or the wifi’s you connect to. It’s not the same loot than |oot and it may be easy to spot with this typography, but it will not be this easy always.Read the warnings from browser and OS, because usually, malware use our custom to close system/browser warnings to infect. Be curious, do some googlefu when you don’t know something and check it against at least two sources.

Know what you are using and what you need: you don’t need to be an expert, but you need to know the capabilities of your devices so you can choose one that fits your needs. Knowing that you are more capable to perceive what things you need to look out for and you also are able to reduce the attack surface avoiding devices that have things you don’t need or want. It’s not the same having an android than an ios phone, or a linux/windows machine. Knowing your environment let’s you know the current threats, the attack surface of the device/platform, and how to prevent it or minimize. And more important, knowing this will make you buy better devices, because you’ll be aware of known issues, if there’s a community backing it up, and in the end make an informed decision.

Be careful: you don’t need to be paranoid, you don’t need to leave civilization or technology behind. You need to know what you are doing, what you are willing to share and which risks you are willing to take. It is like when you were a kid and your mum told you not to take candy from strangers. Here is basically the same, you don’t click a link that you don’t know where it came from, you don’t give personal information over the phone or insecure connections, etc.

Default: by default you should always change the default configuration of apps, services and devices, and mostly passwords. This configuration usually tends to be beneficial for the company providing the service, or to have the easiest usability for the user, but security it’s not a concern. For default passwords it is the same. If you don’t change it, you are prone to be victim of automated attacks because the default password is a known password. The most important thing that you need to take care here are privacy settings, most of the hacks are based on information gathering, and your privacy settings will take an important role in what someone could find about you on the web.

Make backups: it doesn’t matter if we are talking about the photos of a family trip or a code repository, you should always make backups. On the cloud, external server/hdd or anything that could suit your needs.

Use VM’s, test what you need: VM’s are great tools that will let you break things without the need to be worrying about the fact that you could break it. Experimenting with your stuff is always fun, but you need to know if it’s useful or usable, or better than what you got.

Last but not least, never be afraid to ask or to share what you know, even if you think that you know little, that little could help someone, even if it’s someone that has far more experience than you. There is always a chance to learn something new, and having more information will always be a great tool. Ask google if you don’t believe me.

How to: Hardening a Linux Server

So far we’ve talked about why should you harden your server and your devices, now we will talk about how to do harden on a linux server. Please note: All this has been tested on ubuntu 16.04, but should work on 14.04.

Before getting into the technical stuff, one piece of advice, use this as a guideline, but add things you know that I haven’t mention, you could also add it in the comments, search other guides or more information elsewhere. Cybersecurity is always evolving, so the best you can do is read. In that way you may find things I haven’t, or that did not exist at the time I wrote this guide.

We will be tweaking several configuration files, so you should be comfortable using command line. You could use the text editor you like, I’m a vim guy so all the examples will be with it.

Starting with sysctl.conf

This file handles changes in the kernel behaviour during runtime to improve the handling of data packages associated with network traffic, avoiding certain attacks.

$ sudo vim /etc/sysctl.conf

The configurations that we will modify are the following:
Note: they will include comments on the use of each group of settings:

These options will check that the ip address used to send the packets is valid and not spoofed.
Spoofing is when an attacker changes the real address for a fake one, with evil intent.

The options are:

0 = default, do nothing
1 = reject only evident spoofing
2 = make an exhaustive check for spoofing

With this, the option to send info to the broadcast address is disabled, to avoid the mapping of the network. You may not have a network to map, but it’s good to have it configured to deny the access to the information of the server/network configuration as a good practice.
Source route is an option of the IP protocol that allows to record the hops of the packet and dictate the route the packet will take, allowing it to avoid the routing tables of devices that may intercept it and/o drop it. Disabling this will avoid the possibility of skipping some traffic check by changing the route in the packet to avoid the device.
This two are used to avoid attacks on the routing table, that could divert your traffic to where the attackers want. The first want will block the sending of this messages from the server, the second one will disable accepting this kind of packages.
This will avoid DDOS based on SYN flood, this is an attack sending several SYN packages (to initiate the handshake for a connection with the server) and never respond to the ack from the server, in order to leave connections open and rendering the server unresponsive in the end.
Securing host.conf

This file will handle how the service that resolves hosts work. We will add or review the following lines:

The result of this is:

The order will tell to resolve the query first with a nameserver (bind) and then with the host (host file in the server)
Nospoof on will activate the option to check the host address and the hostname, and if they don’t match, the query will fail.

Protect the shared memory.

In order to do this you need to change the configuration of /etc/fstab

You have the option to leave it only readable:

Or leave it read-write but disable the option to run a process, change a UID of a running process, create a block or a character device in the namespace. This will depend on your needs.

Hardening of php.

Things marked with “*” were changed to be more secure, differing from the original file. Things marked with + were added to at the end of it. A brief explanations goes with the configurations made.

Securing su

We secure su to restrict who can use it.

Create a group to handle su.

Add the users to the group.

Set su to only be executed by the users of the group with root privileges.

Securing ssh

Change the default port to one of your choice (over 1000 to avoid using a know port):

Uncomment Port, and the one you choose before.

Change the port in ufw:

Then type:

With this the key is created with rsa encryption using 4096 bits (instead of the default 2048). When the key is created, it offers the possibility to add a passphrase, if you create one, ssh will ask for the passphrase each time that the connection is established.

This will prompt for password of the remote server, to store the key. If everything went well and you try to login again, there should log in directly.

Automation and Countermeasures

To keep working in the hardening of the server we will automate the task of reviewing logs and taking countermeasures.

Executing the comand:

To install denyhosts, this software is a Python script that analyzes the sshd server log messages to determine what hosts are attempting to hack into your system. It also determines what user accounts are being targeted. It keeps track of the frequency of attempts from each host. Additionally, upon discovering a repeated attack host, the /etc/hosts.deny file is updated to prevent future break-in attempts from that host. Once installed, to configure it we will do:

The default options for everything are good, the few changes that need to be made are:

In this options you are also able to choose to relay your emails through a service like zoho or gmail. And then you could change the configuration for the synch daemon. This will allow denyhost to post and rethrive data from a denyhost centralized server, to host already banned by others. To do so, uncomment the following line:

Installing Fail2ban

Fail2ban is similar to denyhosts, but has more service available to monitor. Both will complement each other.

Make a copy of fail2ban.conf and jail.conf, to be able to personalize it. The files to work with are fail2ban.local and jail.local. to edit them:

In here we are able to configure what services we are going to monitor, the default action configured for an event is to ban and report via mail with the whois and relevant log lines (action_mwl)

Configuring a service is like this:

It is possible to add only one filter, and it’s also possible to create your own filters. This is in python, and there is documentation on how to create a proper filter.

Note: fail2ban allows to configure an api to talk with cloudflare.

Once that all the filters that could be useful to you were activated, I restarted the fail2ban service:

This two services will help to prevent brute force attacks, delaying the connections attempts and creating rules to block the traffic that is generating trouble.

Installing psad as an IDS service

In order to do so, you need to install some dependencies like make, to be able to install psad:

  • Make is a program used to manually install software.
  • Whois, to look for dns registers for ip.
  • GCC a c compiler.
  • And installed libdate-calc-perl, to resolve some perl dependencies.
Note: Usually I recommend to install dependencies one by one, in order to find dependencies problems.

To install psad you need to do:

Added iptables rules to be able to log activity:

And configured the service:

In here I updated the emails to get notifications, hostname, enabled_auto_ids .

Once everything was finished, I reload the service:

And then checked the status:

Auditing with Logwatch

It’s necessary to automate the process of auditing the server in search of rootkits and other harmful stuff. In order to do so, I installed:

And created a weekly script that sends a mail with what happened.

And then installed the following rootkits detection software:

For rkhunter post install configurations do:

Then auditing with everything.

I installed more than one rootkit detection software, not for overkill, but to reassure that there are no false positives. A common one in chkrootkit is this one:

https://www.cert-bund.de/ebury-faq

Setup reports by Email

Create a task in crontab that sends reports by mail:

In order to send mails with the server, you will need to configure postfix. If you don’t know or don’t remember the configuration, execute the following command:

This command will guide you through the same configuration process as when postfix was installed. The chosen options were:

  • Type of configuration to use (Internet site)
  • Wich domain will you use [domainname.com (fqdn)] Aliases configuration (To wich user will you route root and postmaster mail)
  • List of other destinations to accept mails from.
  • Force synchronism update (no)
  • Configuration the blocks to relay mail (default)
  • Mailbox limit (0 to unlimited mailbox)
  • Character to define local address extension
  • Select wich internet protocol to use

You could leave it like this and it will send mails, or you could configure it as a relay and use another services like zoho to do it. To do so add a file with the data of the relay:

Then type:

the last thing is to get the outbound mail you are going to use, you will need this only if you will relay the mail:

With this you will generate the db that postfix will use to consult.

restarting the service to take the changes.

And in the end check that is working with:

If you get an email, it is working. If you are relaying your mail, check that the mail is working and sending with the intended user.

Hope you find this useful.