First class functions over channels


Inspired by reading go-cache and reading about core.async I was interested how well sending functions over a channel can work, here is the quick hacky result.


Clojure with its access to a very concise way to construct functions lends itself very nicely to the idea of using CSP with first class functions as messages passed over a channel.

(let [c (chan)]
  (>!! c #(zero? %)))

Given that a channel is only consumed by one thread, this provides an interesting way to sync on a data structure without the need for explicit locking as the channel with by itself serialize the operations. This could be quite performant depending on the consumer as multiple operations could be pipelined for example by polling the channel for a batch of operations to be dispatched to different workers.

Why not?

This is mainly an idea taken from playing with go-lang and implemented in clojure, for access to a basic data structure like a map, even so it can now be a transient map, the built in transactions will provide much better performance.

Installing golang-1.7 on ubuntu 16.10

Ubuntu, and in fact any debian as far as I’m aware has the the

$ update-alternatives

command to manage different versions of installed software. Currently when installing golang ubuntu will default to 1.6, but the repository contains 1.7 as well as package golang-1.7. To allow switching between the versions with the builtin system it can be setup via

$ update-alternatives --install /usr/bin/go go /usr/lib/go-1.7/bin/go 1

Where 1 is the priority, when using auto setup.

$ update-alternatives --config go

Now allows switching between the favorite go version.

GoGuru scope in emacs

GoGuru is an amazing tool and can be really useful when exploring a project, setting up the scope can be a bit of a pain so as the default function doesn’t even support autocompleting. After some quick hacking here is the first attempt to make at least allow for some easier setup, taking the current file and assuming it is the main package, this little function will setup the scope relative to this file.

Building hack-lang on Ubuntu 16.10

Having hacked with PHP every now and then, and running my blog on wordpress I wanted to give hack-lang a spin. Obviously wanting to be bleeding edge I decided to build hhvm from source on my Ubuntu 16.10 machine. It almost worked straight out the box but some poking was needed as I first encountered an error on the build following the instructions on the page. Here is what got me past the initial errors.

Install the packages documented on the page, but using gcc-4.9

$ sudo apt install autoconf automake binutils-dev build-essential cmake g++-4.9 gcc-4.9 gawk git 
  libboost-dev libboost-filesystem-dev libboost-program-options-dev libboost-regex-dev 
  libboost-system-dev libboost-thread-dev libboost-context-dev libbz2-dev libc-client-dev libldap2-dev 
  libc-client2007e-dev libcap-dev libcurl4-openssl-dev libdwarf-dev libelf-dev 
  libexpat-dev libgd2-xpm-dev libgoogle-glog-dev libgoogle-perftools-dev libicu-dev 
  libjemalloc-dev libmcrypt-dev libmemcached-dev libmysqlclient-dev libncurses-dev 
  libonig-dev libpcre3-dev libreadline-dev libtbb-dev libtool libxml2-dev zlib1g-dev 
  libevent-dev libmagickwand-dev libinotifytools0-dev libiconv-hook-dev libedit-dev 
  libiberty-dev libxslt1-dev ocaml-native-compilers libsqlite3-dev libyaml-dev libgmp3-dev 
  gperf libkrb5-dev libnotify-dev libpq-dev

Checkout the hhvm source from github

$ git clone git:// --depth=1
$ cd hhvm
$ git submodule update --init --recursive

I updated the submodules and they are now at the following versions for me

$ git submodule status --recursive
 af252b08f715a0e348175d0754d533949f2e390c third-party (af252b0)
 98ed7a23a83d64133b0a36a884e489bffb0eb864 third-party/brotli/src (v0.3.0)
 8bb673f4410819df06920fdcfd24e18d235d84f7 third-party/brotli/src/terryfy (third-15-g8bb673f)
 f1b955eee1cb94c51074878264aecdd6ee1350dd third-party/double-conversion (v1.1-86-gf1b955e)
 f1217348a868bdb9ee0730244475aee05ab329c5 third-party/fastlz/src (heads/master)
 e21df038e0b58855ef9344d1a3a4389e03ba2cc7 third-party/fatal (0.2-573-ge21df03)
 1d2d4f326acc0825690c151c38ac92d146b78146 third-party/folly/src (v2016.11.07.00-21-g1d2d4f3)
 eb021a100e761616b35ad62c910d6c336e0711d2 third-party/libafdt/src (heads/master)
 1d8b1ac4d20b8ef8d3f5d496dabebaa0ff9019ff third-party/libzip/src (rel-1-1-3-3-g1d8b1ac)
 d86dc916771c126afb797637dda9f6421c0cb998 third-party/lz4/src (r131)
 dbf8878c0136e5f94e4bb0aef6f931d5aecfa74d third-party/mcrouter/src (v0.34.0-2-gdbf8878)
 931a3231375201ee58fb495f4fad0d8fdfc36251 third-party/proxygen/src (v0.32.0-410-g931a323)
 718df09610fee584c9038d8d519697e507e09c9b third-party/re2/src (2016-07-01-6-g718df09)
 8468f07fcc8d96b4af69f1c8b75d404458f8e6b4 third-party/squangle/src (8468f07)
 5e5773be567eeb53a75565a911ece7de2c89ee2b third-party/thrift/src (v2016.11.07.00-8-g5e5773b)
 34a04f01668ae88f0dd7ca18aedf50e41689e8f9 third-party/wangle/src (v2016.11.21.00)
 a9e580b5a0baa768210ef10544c8fab52003ec0b third-party/webscalesqlclient/mysql-5.6 (a9e580b)
 dcc898b0215cee3b1baa88149c1f39e37e9bfd09 third-party/webscalesqlclient/mysql-5.6/rocksdb (rocksdb-3.13-431-gdcc898b)

with those versions building with gcc-4.9 seems to work fine

$ cmake -DCMAKE_C_COMPILER=/usr/bin/gcc-4.9 
        -DMYSQL_UNIX_SOCK_ADDR=/var/run/mysqld/mysqld.sock .
$ make -j <number of cores for me 4>

Happy hack-ing!

Zeus console in Emacs

Zeus is quite a useful tool when working with larger rails applications, or other larger ruby applications for that matter. Especially as it reduces the time to open a interactive environment for the project and makes the whole development process much more fluid. Combining it with Emacs power to send pieces of code directly from a buffer to be executed in a ruby process this is quite a nice way to explore and work. To quickly get access to a Zeus console simply drop the following in your Emacs configuration.

(add-to-list 'inf-ruby-implementations
             `("zeus console" . "zeus console"))

Maybe my suggestion to add this to inf-ruby will be added, but even if not it’s simple enough to get a console with this via C-u M-x inf-ruby.

Hacking on rails using emacs

Rails contributor?

Apparently I committed to rails before, I’m pretty sure it was a doc update, but none the less this is something I actually like to do more. So first step get a working setup for running rails tests locally and getting a reasonably productive environment setup. Since I’m mostly using emacs these days it would be nice if everything would actually work out of this editor/operating system.

rake test

The easiest way to run the tests after cloning down rails and bundling is to use rake.

$ bundle exec rake test

in the root rails folder will run all the available tests, this is great but when working on something also not what I want most of the time. But running it in a subfolder like activesupport will actually do the right thing and run only this subset of tests

$ cd activesupport 
$ bundle exec rake test

To make this easier I’m using textmate.el to execute most commands while inside a project from the determined project-root. Meaning if I switch to a file I set the working directory to the current root automatically.

(defun coder/default-project-dir ()
  (if (textmate-project-root)
      (setq default-directory (textmate-project-root))))

(add-hook 'find-file-hook 'coder/default-project-dir)

Adding .emacs-project files to all the subdirectories will now make sure that when I run a rake command it will be run in the correct directory. Combining this with rake.el allows to run test via a simple M-x rake and selecting test, M-x rake-rerun will now rerun the given tests with a simple command.

This works great while working in quick running test suites but running the whole suite on every change can become inconvenient quickly, this is solved by 2 more enhancements to the flow

running a single test file

Rake allows for passing a file name to limit the run to a single test file

$ bundle exec rake test TEST="test/abstract/callbacks_test.rb"

for example will run only this file of the actionpack testsuite for example. Make sure to run with bundle exec, rake.el does not detect this when in a subfolder, but C-u M-x rake will let you modify the command as needed.

running a single test

Should a file have a lot of tests it can be a good idea to only run a single one, this is possible by passing an option to the minitest testrunner.

$ bundle exec rake test TEST="test/abstract/callbacks_test.rb" TESTOPTS="--name=test_around_action_works"

TESTOPTS will be passed through to minitest which will only run this one test.

integration in emacs

All this works on the command line, as well as using rake.el, personally I use rake.el and have rake rerun bound to C-c r to allow me to quickly run the tests I’m currently working on, this executes in compilation-mode, which allows emacs to parse out file urls and makes them clickable in case of errors.

Next up to make it even better would be to figure out the test name I’m currently in, this works quite simply for files which use the

class TestClass < Minitest::Testcase
  def test_foo
    assert_true doing_the_foo

syntax, but is much harder with files

class TestClass < Minitest::Testcase
  test "doing the foo" do
    assert_true doing_the_foo

So for now I stick with doing it by hand.

Some F# fun

I’m making my way through the The book of F# and been playing with implementing the RPN calculator

personally I found it quite interesting to see my choices during this small exercise, especially the use of Option types to handle error cases comes quite naturally. Another interesting thing I found was that, similar to how I would write the same thing in Lisp, it is very obvious to define functions inside other functions to hide implementation details I would like to give a name to but don’t want to share with the rest of the program.

How to remember circuit breaker configuration with Emacs

Some things are just not easy to remember and looking them up, saving a bookmark to wherever, is just tedious. So how about saving a bookmark and at the same time automating the calculation? Well this is why I started a file which hopefully grows with all those little formulas, using the lisp docstrong to provide a note and a link to why this is relevant.

First thing, when configuring a circuit breaker we need to set a size for how many parallel requests we want to accept, luckily Netflix has a nice formula for this, and even more lucky it is documented on the yammer tenacity repository.

(defun hysterix/thread-pool-size (p99 reqs)
  "calculate and insert the thread pool size for a hysterix
circuit based on the p99 and the requests per secound see for
more details"
  (interactive "nP99 in ms:
nRequests per sec: ")
  (let ((thread-pool-size (ceiling (* (/ (float p99) 1000.0) reqs))))
    (insert (format "%d" thread-pool-size))))

All I need to do now when configuring a circuit, hit M-x hysterix/thread-pool-size and insert the 99th percentile for request latency I can easily get from the service monitoring, as well as the requests per second. If ever in doubt I can use C-h f hysterix/thread-pool-size to get the documentation.

Creating Pull Requests quickly from Emacs

A lot of code I work on lives in GitHub, both work and personal project normally make their way there quickly so maintaining it means I work with the GitHub interface quite a lot as well. To make my live a little easier I decided to create a quick Emacs function to drop me in the matching interface and create or visit the pull request I’m currently working on. Luckily most of the pieces where already out there. All that was needed was grabbing the correct URL and branch from the current repository.

Switching git users in Emacs based on repository

Using Github internally for work is really nice, but especially when working on multiple projects, some on public and some on internal Github it can also be quite hard to always remember to use the correct username and email for the commits. Because I tend to forget such things I decided to create a little elisp function to make sure the correct user is set when I start magit. Right now I only have to switch between 2 users so it is managable, but I think this is easily extended to more if the need arises (god I hope not).