Sunday, March 29, 2015

Handling SSL certificates with Go

Recently I needed to implement an SSL certificate to a Go server I was working on.
The certificate provider I had to use is Wildcard SSL from Comodo.
And of course things are never as easy as they should be and it was a bit more pain that I expected.
First of all, they don't know how to export the certificates for a Go server but fortunately they do it properly for nginx so I didn't had to copy paste certificates in the correct order or something.

What I've ended up with was something along the lines:

There are a few things that still don't work but I do get an A rating with almost perfect scores in the SSL Test here: https://www.ssllabs.com/ssltest/analyze.html

Tuesday, February 24, 2015

Go plugin for IntelliJ gets the "Christmas tree lights" update and what is alpha about

UPDATE:
Seems I was a bit too trigger happy to announce the release but since then, we had to revert it. The new one should be up again soon I hope. The things below still apply (minus the links ofc).

UPDATE 2:
All good!

I'm happy to announce that the latest release of the Go plugin for IntelliJ IDEA 1.0.0.alpha#134 supports a high degree of customization for highlighting (among many other things).

See the screenshots below on how it can look like:

Settings page

 
Possible configuration (don't worry, the default setup is much better ;) )


The release is available on the Github releases section (please read the docs before installing and install the zip file).

Also, I'd like to address some questions about the "alpha state" of the plugin.

Q: When will we have a stable, official release?
A: When it's good enough for us and our users.

Q: Should I install this at all? It has an alpha in it's name.
A: See below for the lengthy detail on why it's named alpha

Q: Should I install the version from the plugin manager? (0.9.15.3)
A: No, at this point 1.0.0-alpha releases are in a much better state than the 0.9.x releases.

Q: Is the plugin written in Go? Or can it be written in Go?
A: No. IntelliJ Platform is written in Java and can run a great deal of plugins that compile into a JVM compatible output (I believe, I've seen plugins written in Groovy, Scala and Kotlin) but Go is a no go for it

Q: Can you include oracle, golint, gofmt or other tools?
A: It depends on the tool itself. For example, for gofmt, while the plugin will have support for it initially, the functionality should be ported for the IntelliJ Platform and not rely on the tool itself. In case of oracle, that's one of the main features the Platform is providing for us native ;)

Q: Can you include X / Y / Z in the release? Or change A / B / C feature?
A: It all depends on the issue or feature you are requesting. It also depends if we can find people to help with the implementation. Or if you want / can implement it yourself even better.

Currently there are a great deal of efforts to bring the plugin in line with what the users of IntelliJ platform expect from a professional plugin. There's a bunch of work to be done to get there but some very talented people help us, the community, to get there.

The releases are currently labeled as alpha as they don't have all the functionality we'd like to have but it is already used by a lot of people in their day to day work.

You can view the list of 1.0 issues as well as a list of possible issues to be included in the 1.0 release.  

A big Thank you! goes to Sergey (@ignatov) and Alexander (@zolotov) who are helping us quite a lot to bring this together as almost all you can see is their work in play, and also to our early adopters and people contributing to the plugin itself.

Wednesday, February 4, 2015

The state of Go plugin for IntelliJ IDEA plugin

I've started to write this after someone asked for some details on what's happening with the plugin and where is it going.

To start with, there have been some major developments in 2014.

We've stopped using my dropbox link and started using an automated build process which builds the plugin for us and releases it at: https://github.com/go-lang-plugin-org/go-lang-idea-plugin/releases

Towards the end of the year, the master branch, aka 0.9.x received some major refactorings for the internals in order to fix some underlying issues that otherwise would have been next to impossible to fix.

We've also got a major contribution in terms of a completely rewritten plugin for some friends of ours, which is now known as 1.0.0-alpha0 branch.

Meanwhile, Mihai and me, both got stuck with availability issues, as we contribute to the plugin from our spare time. As you can imagine, that's very hard to do when there are many things in your personal life. So this is a major problem (maybe the biggest).

The second major problem we have is that despite the huge interest in the plugin, and people willing to use it, we've only had 4 other guys helping us out with active code contributions during 2014. And we had them for at most 1-2 months.

Maybe a big reason for people contributing to the plugin is that it's written in Java... Maybe it's because despite their best efforts, the documentation on how to do / solve different issues with the plugin framework provided by JetBrains, writing a language plugin is still a really really hard task.

Fortunately, one way or the other, my availability issues will finish after 1st of March 2015 but still, there are many things I don't know how to do. Fortunately, both Sergey and Alexander will be there to help me out but when I get stuck, but working 3-4 hours / day at best won't come near to working 8+ hours / day on it.

I'll start working on the 1.0.0-alpha0 branch, and port things from 0.9.x while working on the documentation for the steps I'm doing. I hope this would help others learn things faster.

Definitely in 2015 we'll have a better Go plugin for IntelliJ IDEA and other IDEs while building the way to the best experience possible for go.

If you are reading this and made it this far, and want to help, you can always start here: https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/v1.0.0-alpha0/CONTRIBUTING.md It should provide all the information that you need to work on the plugin.

You can also join our Gitter channel here https://gitter.im/go-lang-plugin-org/go-lang-idea-plugin


Thank you for reading this, I hope I'll see your PR soon ;)

Friday, December 26, 2014

Contributing to Open Source is hard

I realized after trying to maintain the Go plugin for IntelliJ IDEA, how hard it is to actually contribute and maintain something for the open-source community.

Not only there's a problem with the time you have, my case after 10-12 hours of being away from home, getting at 8-9 PM to do something is hard. Then, there's the problem of switching from different context. Again, in my case, I'm switching from doing servers / automation or Go to Java and a whole bunch of different problems to solve, worlds apart would be a proper definition of it. Then you have to deal with issues coming in from your users, which contrary to what you'd expect, them being developers as well, 95% suck at reporting issues. And ofc they get impatient when their pretty little bug / feature request doesn't get attention from the developers, I guess it's easy to request things done for free by others in their spare time, no?

Don't get me wrong, occasionally, once a month or so, you get an e-mail from a random user saying: Thank you. (and not demanding anything else in turn) and that helps you remember why you are doing it as it feels so nice but more often then not people just don't understand how hard it is to do it.

As such, Thank you all of you out there who are doing such a nice work and share things with others, for free, in your spare time. And I don't mean those who get paid to do it from their job / as a living. If you open-source and maintain something that you did in your company, that's nice, but you got payed to do it. The real heroes are the ones doing it for free and demanding nothing in return.

Tuesday, December 2, 2014

How to build nginx from sources with some extra modules

Here's a short tutorial on how to build nginx from sources on Ubuntu. I've tested it on 12.04 and 14.04.

Pay attention to things like version numbers or dates and everything should work as a charm.

Thursday, November 27, 2014

The life of a software developer

I've stumbled upon this: Working as a Software Developer but then I've realized that there's a couple of things missing from it:

- the number of programming languages one has to know and the way to solve problems in each of them. It's like speaking a foreign language for each programming language you know. And while you can learn the abstracts and apply them based on the language you need to write the code in, you still have to know the caveats and how to optimize things for the specific language

- the sheer size of the information in the industry one needs to keep up with every day / week / month so that you can always make the best decisions when implementing changes

Saturday, October 4, 2014

How to cache responses in nginx and memcached

I've faced an interesting issue this week at work, how to improve our response times while not touching our software in the process.

Obviously, Varnish would be the answer for it but since our infrastructure architecture is already complex, there was a need to do it without introducing yet another layer just yet.

So, looking over what we can do, I've found out how nginx can cache things directly to memcached. Of course, it's always easy when you read about them but it turns out that there's no clear tutorial for how to actually do it.

As such, I've headed on the Internet trying to find a response to my question and quickly enough, there was a post about the same problem with no answer http://stackoverflow.com/questions/25639445/nginx-caching-response-from-remote-server-with-memcached/

Long story short, here's the solution I've came up with, maybe anyone else has a better one?


Add this line under nginx.conf (it adds support for Lua, see below why)

    lua_package_path '/usr/local/lib/lua/?.lua';

site config (in my case default):

    upstream memcached {
    server 127.0.0.1:11211;
    keepalive 32;
    }
    
    server {
    listen 8080 default_server;
    
    root /usr/share/nginx/html;
    index index.fhtml index.fhtm;
    
    # Make site accessible from http://localhost/
    server_name localhost;
    
    location = /memc {
    internal;
    
    memc_connect_timeout 100ms;
    memc_send_timeout 100ms;
    memc_read_timeout 100ms;
    memc_ignore_client_abort on;
    
    set $memc_key $arg_key;
    set $memc_exptime 300;
    
    memc_pass memcached;
    }
    
    location /memc-stats {
    add_header Content-Type text/plain;
    set $memc_cmd stats;
    memc_pass memcached;
    }
    
    location / {
    set_by_lua $key 'return ngx.md5(ngx.arg[1])' $request_uri;
     
    srcache_fetch GET /memc key=$key;
    srcache_methods GET;
    srcache_store_statuses 200 301 302;
    
    error_page 403 404 502 504 = @fallback;
    }
    
    location @fallback {
    proxy_pass http://127.0.0.1:80$request_uri;
    set_by_lua $key 'return ngx.md5(ngx.arg[1])' $request_uri;
    srcache_request_cache_control off;
    srcache_store PUT /memc key=$key;
    }
    
    }


My setup is like on Ubuntu 14.04, nginx running on port 8080 and Apache on 80 (just to test this) with nginx 1.7.5 compiled with the following arguments in "debian/rules" under "full_configure_flags"

    full_configure_flags := \
    $(common_configure_flags) \
    --with-http_addition_module \
    --with-http_dav_module \
    --with-http_geoip_module \
    --with-http_gzip_static_module \
    --with-http_image_filter_module \
    --with-http_secure_link_module \
    --with-http_spdy_module \
    --with-http_sub_module \
    --with-http_xslt_module \
    --with-mail \
    --with-mail_ssl_module \
    --with-http_ssl_module \
    --with-http_stub_status_module \
    --add-module=/opt/nginx/modules/ngx_devel_kit-0.2.19 \
    --add-module=/opt/nginx/modules/set-misc-nginx-module-0.26 \
    --add-module=/opt/nginx/modules/memc-nginx-module-0.15 \
    --add-module=/opt/nginx/modules/srcache-nginx-module-0.28 \
    --add-module=$(MODULESDIR)/headers-more-nginx-module \
    --add-module=$(MODULESDIR)/nginx-auth-pam \
    --add-module=$(MODULESDIR)/nginx-cache-purge \
    --add-module=$(MODULESDIR)/nginx-dav-ext-module \
    --add-module=$(MODULESDIR)/nginx-echo \
    --add-module=$(MODULESDIR)/nginx-http-push \
    --add-module=$(MODULESDIR)/nginx-lua \
    --add-module=$(MODULESDIR)/nginx-upload-progress \
    --add-module=$(MODULESDIR)/nginx-upstream-fair \
    --add-module=$(MODULESDIR)/ngx_http_substitutions_filter_module

I've compiled Lua and other modules, as you can see. The need for Lua was because I wanted to have a consistent way to hash the values for the memcached keys without having to worry about what will happen if someone will send some unexpected values as well as be able to hash it in the same way from the backend.


EDIT:
You can get the modules that I've added from here:
- Nginx Development Kit https://github.com/simpl/ngx_devel_kit
- ngx_set_misc https://github.com/openresty/set-misc-nginx-module
- ngx_memc https://github.com/openresty/memc-nginx-module
- ngx_srcache https://github.com/openresty/srcache-nginx-module

Sunday, July 20, 2014

It's alive! (again)

The new blog is alive! (again). Now powered by Blogger.
Lets see how this goes.
I'll post more stuff #throughglass and Google Glass related soon.