NGINX: Mixing and Matching Ubuntu Repositories (and NGINX team PPAs on Launchpad) or Debian Repositories with Upstream Repositories will result in problems.

 Debian, nginx, NGINX, NGINX PPA, Server Packages, Ubuntu  Comments Off on NGINX: Mixing and Matching Ubuntu Repositories (and NGINX team PPAs on Launchpad) or Debian Repositories with Upstream Repositories will result in problems.
Apr 042015

We’ve seen this before, but we see it too frequently. People want the latest NGINX version. So they use the upstream repository to get it. They try and install, and you immediately get conflicts.

In Ubuntu, people then file bugs on this thinking it’s an Ubuntu issue (such as this bug here). Problem is, it’s not an Ubuntu bug. Nor is it a bug in the PPAs I maintain. Nor is it a bug in Debian. It’s a problem that arises when you mix the upstream repositories and either Ubuntu, Debian, or the Ubuntu PPAs, and assuming you can ‘upgrade’ cleanly with the upstream repositories.

Basically, this is what it comes down to:

For Debian, if you’re depending on third party modules, you should stick to Debian’s packaging and wait. For Ubuntu, you should use the PPAs which I maintain (under the nginx team on Launchpad) if you want latest software and features, based off of Debian’s packaging decisions.

If you want/depend on NAXSI though, you have no choice but to recompile NGINX with NAXSI yourself, in order to get it working in a sane way. Neither Debian, nor Ubuntu, nor the PPAs have naxsi in the builds anymore.

If none of those apply to you, you will have to purge all nginx binaries from your computer, and all nginx packages and configuration files with this command before installing from the upstream repository: sudo apt-get purge nginx nginx-doc nginx-common (This should also purge the other dependent packages as well)

But, if you’re curious why you can’t mix the repositories, this post explains it from my perspective. Here’s a breakdown of why you cannot mix repositories such as this, and the problems you run into.

Problem 1: Debian / Ubuntu / NGINX PPAs (maintained by yours truly) have flavors; nginx upstream does not.

And by flavors, I mean nginx-light, nginx-full, nginx-extras, nginx-naxsi (up until 1.6.2-2), and nginx-core (Ubuntu only, since Ubuntu 14.04). Each of these flavors contains a different set of modules, based on demand originating in Debian or the community (and ultimately implemented in Debian). I will not go into the differences here, however you can go to here and read my answer to the question for more details.

As a result of how NGINX modules are currently incorporated into the program’s binaries, it is absolutely critical to separate out the configuration files and default sample files and locations so that you can switch between flavors (and upgrade between versions between Ubuntu/Debian/PPAs) without issues and conflict between configuration files. This requires the introduction of a package called nginx-common – a package which contains files and other items that are common to all versions of the nginx flavors in those versions of the source package. This nginx-common is wholly the brainchild of Debian’s work, and inherited in Ubuntu and the PPAs I maintain.

The problem is: NGINX upstream does not ship ‘flavors’. They enable all the default modules that are shipped in the nginx upstream ‘core’ code, and do not include any third party modules, nor do they discriminate the modules to enable (to make ‘light’ builds, or ‘full’ builds). As such, the nginx upstream package is a single solitary ‘nginx’ package. It does not separate out configuration files, nor does it separate out the binaries.

Problem 2: NGINX upstream doesn’t have third party modules.

Now, I know what you’re thinking: “Why would Debian/Ubuntu include third-party modules in the packaging?” Turns out, in Debian, the demand for such ‘third party extensions’ was so high, that they decided to include the modules. Such modules include the nginx-lua module, the nginx-perl module, and even the NAXSI modules for NGINX (which were in nginx-naxsi up until 1.6.2-2).

The problem: These are third party modules, maintained separately from the NGINX code base itself. They’re shipped as part of some of Debian’s packages due to demand, but they in and of themselves can cause packaging issues and conflicts, to the point where it requires updating the modules’ code with each new release to fix issues in those modules. This in and of itself lends to ‘maintainability’ problems. This is why the nginx-naxsi flavor was dropped from NGINX in Debian and Ubuntu as of package version/revision 1.6.2-2. To fix even simple bugs in the nginx-naxsi flavor (and its related packages), the entire NAXSI module needed to be removed and replaced with the latest upstream revisions of the code and plugin. This means that to even fix bugs in how the NAXSI rules were handled (in order to match the actual rule formats that needed to be in place for whitelists and such), you’d have to do a replace of the entire NAXSI module in the nginx-naxsi flavor. In Ubuntu, this would break the ‘Stable Release Update’ in that new features would be added to the package that could break things, old features could be removed, and it would go beyond the ‘nitpick fix’ that’d be needed for a Stable Release Update, such that the system would never be fixed.

As a result, third party modules have to be maintained and updated with almost every code update from NGINX upstream. For NGINX Mainline, the Lua module needed to be updated three times for build failure fixes in the PPAs. Since a lot of these third party modules (such as NAXSI or Lua) are in demand by the community, but not available in the NGINX Upstream repository (or in the case of the NAXSI release, even the PPAs nowadays), you should not mix repositories, as you will lose those modules, or lose some of the modules and gain others.

NGINX in Ubuntu Vivid: If upgrading to Vivid on a 32bit i386 platform, consider upgrading to 64bit amd64 platform in the process!

 nginx, NGINX Mainline PPA, NGINX PPA, NGINX Stable PPA, Server Packages, Ubuntu  Comments Off on NGINX in Ubuntu Vivid: If upgrading to Vivid on a 32bit i386 platform, consider upgrading to 64bit amd64 platform in the process!
Apr 042015

The latest in updates done to Debian and Ubuntu’s nginx packaging has changed slightly the compilation of the nginx package, namely that two new hardening features have been enabled in the compiling: making the executables Position Independent, and activating immediate binding.

There’s a small problem, here, however. In amd64 (64-bit), Position Independent Executables work fine. However, there is a performance impact that will be noticeable in higher-performance-requiring uses of the nginx executables in 32-bit i386 platforms.

As such, it is highly recommended that if you are planning on upgrading a 32-bit i386 server running nginx with Ubuntu Utopic to Ubuntu Vivid, and your applications that are running via nginx require much higher performance demands (small, static sites don’t necessarily count), then you should strongly consider upgrading to a 64bit amd64 platform, rather than sticking with a 32bit i386 platform due to the performance hit that will be caused as a result of the Position Independent Executable compilation option.

This will affect Ubuntu Vivid (all nginx flavors) and will in future also affect the nginx PPAs. (It has not yet been implemented in the PPAs as of yet, however it will likely end up there in the future.).