Dec 162014

This weekend, the NGINX PPAs were updated.

Stable PPA: Packaging resynced with Debian 1.6.2-5 to get some fixes and version updates for the third-party modules into the package.

Mainline PPA:

  • Updated verison to 1.7.8.
  • Module updates:
    • Lua module updated to 0.9.13 full from upstream. (Update needed to fix a Fail To Build issue)
    • Cache purge module updated to 2.2 from upstream. (Updated to fix a segmentation fault issue)

NGINX and Ubuntu: Using Flavors Other than `nginx-core`

 nginx, Server Packages, Ubuntu  Comments Off on NGINX and Ubuntu: Using Flavors Other than `nginx-core`
Dec 132014

Today, a bug was filed reporting that nginx-full and nginx-common were different versions.  Naturally I went hunting, and found this not to be the case.  (The bug is here)

It looks like unless you enable Universe, you won’t be able to see the updated versions for nginx-common and nginx-full, and there will be version mismatch errors. You’ll see the older version(s) in Main, though, because Main ships with nginx-core and nginx-common specifically, and not the other flavors

So, here’s a little tidbit for NGINX users on Ubuntu: Make sure you have the following repositories enabled if you want to use a specific flavor of nginx:

  • nginx-core: Main
  • nginx-full: Universe
  • nginx-light: Universe
  • nginx-extras: Universe
  • nginx-naxsi (Obsolete, removed with Ubuntu 15.04): Universe

NGINX in Ubuntu, PPAs, and Debian: naxsi packages dropped as of September 26, 2014 (repost)

 nginx, NGINX Mainline PPA, NGINX Stable PPA  Comments Off on NGINX in Ubuntu, PPAs, and Debian: naxsi packages dropped as of September 26, 2014 (repost)
Dec 092014

Back in April, I upstreamed (that is, reported a bug to Debian) regarding the `nginx-naxsi` packages. The initial bug I upstreamed was about the outdated naxsi version in the naxsi packages. (see this bug in Ubuntu and the related bug in Debian)

The last update on the Debian bug is on September 10, 2014. That update says the following, and was made by Christos Trochalakis:

After discussing it with the fellow maintainers we have decided that it is
better to remove the nginx-naxsi package before jessie is freezed.

Packaging naxsi is not trivial and, unfortunately, none of the maintainers uses
it. That’s the reason nginx-naxsi is not in a good shape and we are not feeling
comfortable to release and support it.

We are sorry for any inconvenience caused.

I asked what the expected timeline was for the packages being dropped. In a response from Christos today, September 15, 2014, it was said:

It ‘ll get merged and released (1.6.1-3) by the end of the month.

Update (December 9, 2014): In Ubuntu, these changes will not make it into 14.10, but are now included in 15.04 as part of nginx 1.6.2-4ubuntu1.  As such, the naxsi variant of nginx is no longer included starting with 15.04.  Community support for the naxsi variant is also limited as well.

In the PPAs, the naxsi packages will be dropped with stable 1.6.2-2+precise0 +trusty0 +utopic0 and mainline 1.7.5-2+precise0 +trusty0 +utopic0.

In Debian, these changes have been applied as 1.6.2-2.


(Originally published on September 26, 2014 @ 12:34)

NGINX Webserver Admins: Don’t Use SSLv3 in Your SSL-Enabled Sites!

 nginx, NGINX PPA, Server Packages, Ubuntu, Uncategorized  Comments Off on NGINX Webserver Admins: Don’t Use SSLv3 in Your SSL-Enabled Sites!
Oct 242014

The SSLv3 “POODLE” Vulnerability.

Most of us are aware of the recent protocol flaw vulnerability in SSLv3. Officially designated CVE-2014-3566, it is more commonly referred to as the “POODLE” (Padding Oracle On Downgraded Legacy Encryption) vulnerability.

The vulnerability is a result of a flaw in the way that the (now old) SSLv3 protocol behaves and operates. There is a Ubuntu-specific question on the POODLE vulnerability on Ask Ubuntu (link) which answers common questions on it. There is also a more general question on the POODLE vulnerability on the Information Security Stack Exchange site (link) with more general details on the POODLE vulnerability. If you would like more details, you should refer to those sites, or read the OpenSSL Whitepaper on the POODLE vulnerability (link).

As this is a protocol flaw in SSLv3, ALL implementations of SSLv3 are affected, so the only way to truly protect against POODLE is to disable SSLv3 protocol support in your web application, whether it be software you write, or hosted by a web server.

Disable SSLv3 in nginx:

Since the recommendation is to no longer use SSLv3, the simplest thing to do is disable SSLv3 for your site. In nginx, this is very simple to achieve.

Typically, one would have SSL enabled on their site with the following protocols line or similar if using the example in the default-shipped configuration files (in latest Debian or the NGINX PPAs, prior to the latest updates that happened in the past week or so):
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;

To resolve this issue and disable SSLv3 support, we merely need to use the following instead to use only TLS:
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

Note that on really old implementations of OpenSSL, you won’t be able to get TLSv1.1 and TLSv1.2, so at the very least you can just have TLSv1 on the ssl_protocols line. You should probably consider updating to a more recent version of OpenSSL, though, because of other risks/issues in OpenSSL.

Update OpenSSL to get TLS_FALLBACK_SCSV Support:

More importantly than just disabling SSLv3, you should definitely update your OpenSSL, or whatever SSL implementation you use, to receive support for TLS_FALLBACK_SCSV. There is an attack vector that would make you vulnerable to POODLE by starting a TLS session, but then falling back to SSLv3, and then open you to the POODLE vulnerability. By updating, and then having the use of TLS_FALLBACK_SCSV, you will be protecting yourself from protocol downgrading attacks which would also make you vulnerable to POODLE.

Ubuntu Users:


Fortunately for all users of Ubuntu, the OpenSSL packages were updated to protect against SSL downgrade attacks. This is detailed in “USN-2385-1: OpenSSL vulnerabilities” (link). Simply running sudo apt-get update with the security repositories enabled should get you the OpenSSL update to address this.

nginx from the Ubuntu Repositories:

Due to the vulnerability, and Debian already having these changes done, I was able to get in a last-minute update (courtesy of the Ubuntu Security Team and the Ubuntu Release Team), into the nginx package for the Utopic (14.10) release, which happened officially yesterday (October 23, 2014). In Utopic, the nginx package’s default config does NOT have SSLv3 on the ssl_protocols line. All other supported versions of Ubuntu do not have this change (this means that Precise and Trusty are both affected).

PPA Users:

Of course, many users of Ubuntu and nginx like the newer features of the latest nginx Stable or Mainline releases. This is why the nginx PPAs exist. Originally maintained by some of the Debian maintainers of the nginx package, I’ve taken primary responsibility of updating the nginx packages, and keeping them in sync (as close as I can) to the Debian nginx packaging.

As of today (October 24, 2014), both the Stable and Mainline PPAs have been updated to be in sync with the latest Debian packaging of the nginx package. This includes the removal of SSLv3 from the default ssl_protocols line.

Debian Users:


Fortunately, like Ubuntu, Debian has also updated the OpenSSL packages to protect against SSL downgrade attacks. This is detailed in “DSA-3053-1 openssl — security update” (link). Like in Ubuntu, this can be fixed by running sudo apt-get update or similar to update your packages.

nginx in the Debian Repositories:

If you are on Debian Unstable, you are in luck. The Debian package in Unstable has this change in it already.

If you are on Debian Testing or Debian Stable or Debian Old Stable, you’re unfortunately out of luck, this change isn’t in those versions of the package yet. You can easily do the aforementioned changes, though, and fix your configs to disable SSLv3.

NGINX Stable with Yubikey Auth

 Personal NGINX Builds, Stable NGINX + Yubikey  Comments Off on NGINX Stable with Yubikey Auth
Oct 022014

So, I was bored of using authbasic because there’s a million ways to intercept the passcodes. However, there’s some OTP methods that are always capable of working. To that end, there is a Yubikey auth module which works on NGINX Stable.

This kind of helps deal with the need for auth_basic, and actually helps with me, because I can secure my sites with a still-basic auth mechanism that uses a Yubikey for authentication.

Currently, the working code for 1.6.x is in the ‘compile-fix`’ branch on Github, but it works as intended. As well, I’ve made a separate PPA containing the nginx-stable builds from the NGINX PPAs, plus the Yubikey auth module for all binary variants. It makes nginx-light a little less light, but it still adds what I consider the brilliance of Yubikey OTP authentication.

NOTE: Your Yubikeys that you configure must all be on the YubiCloud system for OTP authentication to work. This is because the module uses the Yubico OTP verification/cloud system for code verification.

Ubuntu: Get notified of package changes during `apt-get upgrade` and `apt-get dist-upgrade`

 Ubuntu  Comments Off on Ubuntu: Get notified of package changes during `apt-get upgrade` and `apt-get dist-upgrade`
Sep 112014

Unlike Debian, Ubuntu seems to not show the changes and not trigger massive notifications when a NEWS article is created in a Debian package. This sometimes lets massive changes slip by, such as the change in the nginx PPAs and Debian prompting this other blog post.

This sometimes prevents users and admins from knowing that major changes that can break functionality are going through. To that end, there is the `apt-listchanges` package. During `upgrade` / `dist-upgrade` updates, it will provide you a full list of the changes in the packages being updated/upgraded. You may install this software with this command:
sudo apt-get update
sudo apt-get install apt-listchanges

Sep 092014

Due to changes in NGINX, there are now changes done to the default configurations which will break fastcgi sites.

NGINX in Debian, and as such, the PPAs, were previously shipping different configuration files which differed from NGINX itself. The Debian package has now synced with the nginx configurations upstream, and as such, certain very different changes have happened.

This is the massively-detailed NEWS entry in Debian for these changes:

nginx-common (1.6.1-2) unstable; urgency=medium

As of nginx-1.6.1-2 we have synced all configuration files with upstream and
we plan to keep them in sync from now on.

Unfortunately that might break existing configuration for some users. Please
check the matrix below for more information:

File Changes
koi-win whitespace
koi-utf whitespace
mime-types whitespace, changed js/rss mime type,
minor other changes & additions
scgi_params whitespace, added HTTPS
uwsgi_params whitespace, added HTTPS, removed UWSGI_SCHEME
fastcgi_params whitespace, removed SCRIPT_FILENAME
fastcgi.conf new upstream configuration file

Fastcgi configuration issues

nginx shipped a modified `fastcgi_params`, which declared `SCRIPT_FILENAME`
fastcgi_param. This line has now been removed. From now on we are also
shipping fastcgi.conf from the upstream repository, which includes a sane
`SCRIPT_FILENAME` parameter value.

So, if you are using fastcgi_params, you can try switching to fastcgi.conf
or manually set the relevant params.

You might also want to read the documentation section before proceeding.
section: $fastcgi_script_name variable.

You will need to change fastcgi_params to the fastcgi.conf file, or manually set the relevant parameters in your site configs, in order to make things work again.

(This is introduced in the PPAs as 1.6.1-2+precise0 +trusty0 and +utopic0. This is also introduced in the PPAs as 1.7.4-1+precise0 +trusty0 and +utopic0. THis will also be in Ubuntu in versions after 1.6.1-2)

nginx-core is now in Ubuntu Trusty 14.04 Main!

 Uncategorized  Comments Off on nginx-core is now in Ubuntu Trusty 14.04 Main!
Mar 132014

Thanks to the efforts of myself and others, we have been able to get NGINX into the Ubuntu Main repositories for Trusty 14.04!

Having said this, none of the already-established flavors of nginx are included in Ubuntu Main (nginx-light, nginx-full, nginx-extras, and nginx-naxsi). The Ubuntu Security Team has said that the third-party modules are wildly different in coding and therefore cannot be supported.

To that end, we created a package called nginx-core which has been included in the Main repository. This package contains only the modules that ship with the stock nginx tarball. We do not include any third-party modules with this package, just the modules that come from NGINX upstream.

Thanks to everyone on the MIR an Security teams for all their help in getting nginx into Main!

A Minimal IPTables Setup: WebServer, MySQL, PHP. (LAMP, for example)

 iptables, Ubuntu, Uncategorized  Comments Off on A Minimal IPTables Setup: WebServer, MySQL, PHP. (LAMP, for example)
Aug 032012

A few weeks ago, someone posted on Ask Ubuntu asking for a minimal IPtables setup for LAMP servers.

As you can guess by the only answer there, I posted the following iptables commands for this:
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p udp --dport 80 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-unreachable

The above is extraordinarily minimal. Time to go through this, and figure out what I should have included, and what I intentionally left out.

First, we’ll look at the first command, iptables -A INPUT -i lo -j ACCEPT. This adds a rule which states that “Any traffic originating on the loopback interface, or localhost, is allowed.”.

The next rule, iptables -A INPUT -m conntrack –ctstate ESTABLISHED,RELATED -j ACCEPT, uses a specialized module, called ‘conntrack’, which tracks connection states. This rule states that pre-established connections, and related connections, are accepted and not blocked.

The next rule, iptables -A INPUT -p tcp –dport 22 -j ACCEPT, accepts all traffic on port 22 (for default SSH installations and setups).

The next two rules, iptables -A INPUT -p tcp –dport 80 -j ACCEPT and iptables -A INPUT -p udp –dport 80 -j ACCEPT, accept HTTP traffic on Port 80. Since HTTP traffic is both TCP and UDP, you need to have both.

The last rule in the above set, iptables -A INPUT -j REJECT –reject-with icmp-host-unreachable, uses a specialized target chain called “REJECT” which rejects the packets. The specific ICMP packet that will be used to reject the packet is “host unreachable” which terminates the networking connection to the server.

Now apparently, there are a few things I intentionally left out. Unless your server specifically hosts MySQL databases for external servers, you don’t need to allow MySQL traffic. Therefore, I did not add a rule to allow traffic from outside of the server itself related to MySQL (local mysql traffic is handled via the loopback rule that we first added). PHP has no reason to listen externally for requests, so it listens only on (in PHP 5.3.x, on Ubuntu Precise), or on the UNIX socket /var/run/php-fpm.sock (or similar, in PHP 5.4.x, on Ubuntu Quantal and later). Therefore, since PHP traffic will only be local, it’ll also be covered by that first rule, so no rule is needed for the PHP traffic.

I intentionally left out filtering of ICMP packets, such as pings. For an absolutely minimal setup of IPTables, you probably won’t want people to send you large pings (Pings of Death), or other ICMP packets outside of local traffic. I did not add ICMP filtering rules because I generally to not accept ICMP packets such as pings from external networks. I have a specialized program that establishes a TCP connection via HTTP and Curl automatically from my servers to one central server on an Amazon EC2 instance. The nginx access.log therefore will actually list those connections, and since I run that connection script every two minutes, its a semi-effective way of determining if the server infrastructure is online or not. If it’s not, i’ve got a nag-script that emails me if one misses a scheduled check-in several times, or if it hasnt responded in the past hour.