Tuesday, December 2, 2014

How to build nginx from sources with some extra modules

Here's a short tutorial on how to build nginx from sources on Ubuntu. I've tested it on 12.04 and 14.04.

Pay attention to things like version numbers or dates and everything should work as a charm.

Thursday, November 27, 2014

The life of a software developer

I've stumbled upon this: Working as a Software Developer but then I've realized that there's a couple of things missing from it:

- the number of programming languages one has to know and the way to solve problems in each of them. It's like speaking a foreign language for each programming language you know. And while you can learn the abstracts and apply them based on the language you need to write the code in, you still have to know the caveats and how to optimize things for the specific language

- the sheer size of the information in the industry one needs to keep up with every day / week / month so that you can always make the best decisions when implementing changes

Saturday, October 4, 2014

How to cache responses in nginx and memcached

I've faced an interesting issue this week at work, how to improve our response times while not touching our software in the process.

Obviously, Varnish would be the answer for it but since our infrastructure architecture is already complex, there was a need to do it without introducing yet another layer just yet.

So, looking over what we can do, I've found out how nginx can cache things directly to memcached. Of course, it's always easy when you read about them but it turns out that there's no clear tutorial for how to actually do it.

As such, I've headed on the Internet trying to find a response to my question and quickly enough, there was a post about the same problem with no answer http://stackoverflow.com/questions/25639445/nginx-caching-response-from-remote-server-with-memcached/

Long story short, here's the solution I've came up with, maybe anyone else has a better one?


Add this line under nginx.conf (it adds support for Lua, see below why)

    lua_package_path '/usr/local/lib/lua/?.lua';

site config (in my case default):

    upstream memcached {
    server 127.0.0.1:11211;
    keepalive 32;
    }
    
    server {
    listen 8080 default_server;
    
    root /usr/share/nginx/html;
    index index.fhtml index.fhtm;
    
    # Make site accessible from http://localhost/
    server_name localhost;
    
    location = /memc {
    internal;
    
    memc_connect_timeout 100ms;
    memc_send_timeout 100ms;
    memc_read_timeout 100ms;
    memc_ignore_client_abort on;
    
    set $memc_key $arg_key;
    set $memc_exptime 300;
    
    memc_pass memcached;
    }
    
    location /memc-stats {
    add_header Content-Type text/plain;
    set $memc_cmd stats;
    memc_pass memcached;
    }
    
    location / {
    set_by_lua $key 'return ngx.md5(ngx.arg[1])' $request_uri;
     
    srcache_fetch GET /memc key=$key;
    srcache_methods GET;
    srcache_store_statuses 200 301 302;
    
    error_page 403 404 502 504 = @fallback;
    }
    
    location @fallback {
    proxy_pass http://127.0.0.1:80$request_uri;
    set_by_lua $key 'return ngx.md5(ngx.arg[1])' $request_uri;
    srcache_request_cache_control off;
    srcache_store PUT /memc key=$key;
    }
    
    }


My setup is like on Ubuntu 14.04, nginx running on port 8080 and Apache on 80 (just to test this) with nginx 1.7.5 compiled with the following arguments in "debian/rules" under "full_configure_flags"

    full_configure_flags := \
    $(common_configure_flags) \
    --with-http_addition_module \
    --with-http_dav_module \
    --with-http_geoip_module \
    --with-http_gzip_static_module \
    --with-http_image_filter_module \
    --with-http_secure_link_module \
    --with-http_spdy_module \
    --with-http_sub_module \
    --with-http_xslt_module \
    --with-mail \
    --with-mail_ssl_module \
    --with-http_ssl_module \
    --with-http_stub_status_module \
    --add-module=/opt/nginx/modules/ngx_devel_kit-0.2.19 \
    --add-module=/opt/nginx/modules/set-misc-nginx-module-0.26 \
    --add-module=/opt/nginx/modules/memc-nginx-module-0.15 \
    --add-module=/opt/nginx/modules/srcache-nginx-module-0.28 \
    --add-module=$(MODULESDIR)/headers-more-nginx-module \
    --add-module=$(MODULESDIR)/nginx-auth-pam \
    --add-module=$(MODULESDIR)/nginx-cache-purge \
    --add-module=$(MODULESDIR)/nginx-dav-ext-module \
    --add-module=$(MODULESDIR)/nginx-echo \
    --add-module=$(MODULESDIR)/nginx-http-push \
    --add-module=$(MODULESDIR)/nginx-lua \
    --add-module=$(MODULESDIR)/nginx-upload-progress \
    --add-module=$(MODULESDIR)/nginx-upstream-fair \
    --add-module=$(MODULESDIR)/ngx_http_substitutions_filter_module

I've compiled Lua and other modules, as you can see. The need for Lua was because I wanted to have a consistent way to hash the values for the memcached keys without having to worry about what will happen if someone will send some unexpected values as well as be able to hash it in the same way from the backend.


EDIT:
You can get the modules that I've added from here:
- Nginx Development Kit https://github.com/simpl/ngx_devel_kit
- ngx_set_misc https://github.com/openresty/set-misc-nginx-module
- ngx_memc https://github.com/openresty/memc-nginx-module
- ngx_srcache https://github.com/openresty/srcache-nginx-module

Sunday, July 20, 2014

It's alive! (again)

The new blog is alive! (again). Now powered by Blogger.
Lets see how this goes.
I'll post more stuff #throughglass and Google Glass related soon.