Daemons and output in ruby

Backgrounding a process in ruby is easy, just add

Process.daemon

This works great but by doing so you will actually lose any output since daemon
will redirect stdout and stderr to /dev/null by default. Depending on how you
handle logging this might actually not be a great idea, I personally prefer
redirecting the output of a daemon process myself so I can add logging easily.

This is easily solve so by passing both parameters for daemon, first to either
do or don’t change the working dir to / and the second to not redirect stdout
and stderr.

Process.daemon true, true

Just a nice thing which is quickly overlooked.

Using zeus without rails

At the last rubyshift the question came up if
zeus could be used without rails, and how to do
it. At the time I kind of expected it to be but wasn’t sure if there are any
complexities which I didn’t see at the moment because I never tried it. So
trying it out it actually works allmost as expected, the only thing needed is a
modification to the custom_plan.rb file which defines how the preloading
actually works. So just as a quick example I
setup a repository which
defines 2 tasks

$ zeus rake
$ zeus foo

This should get everybody started, as the tasks are pretty obvious and easily
modified for your needs.

Working with CSV in SQLite

Recently I came across a csv with some reports I needed to work with, and since
it was quite a lot of data I wanted some sane way to query it, so I remembered
a neat little trick I discovered together with
@leif time ago.

Without much hassle you can make sqlite import csv

get the name for the columns, this can be done in most cases simply via

    $ head -n1 mycsv.csv
    > foo,bar,baz

create a table in sqlite to import into, the column names are just copy and
paste (for persistence pass a db file name, but this is optional)

    $ sqlite3 mydb.db
    >> create table mycsv(foo,bar,baz);

Still in sqlite import the csv

    >> .separator ',' 
    >> .import mycsv.csv mycsv

That’s it, since sqlite actually does not care about the types unless you want
it to this works for about any csv. Now you can use the power of SQL to work
with the data from the csv.

Just a nice little trick to make working with csv data just a little bit nicer

After emulation march (inspired by @coreyhaines)

Sometime has gone by since
I decided to join in on
emulation march
so I guess now it’s time for some review on it.

First of all, I really think for me it was worth it. Even though I guess I
didn’t do much ‘emulation’ by its core meaning, I got way more involved in OSS,
and I feel good about it. A lot of things changed on how I see my own software
in general, and how I see OSS projects overall. It just got me out of the
feeling that others are ‘responsible’ for the project, and more into the habit
of just fixing what I think needs fixing. I now know how much more I could/need
to do to become a more well-rounded developer but knowing is the first, and I
like to think sometimes most important step.

So after this (about 2 actually) month, I have to say I will go on and try to
improve on it, I am hooked now! Oh yeah people say contributing a PR is the
greatest form of admiration, well getting pull requests accepted by people you
look up to is the greatest from of giving back.

Splitting server implementation from API design via canned responses

… or what’s the idea behind canned.

Let’s face it designing an API is really hard.
The interface should be easy to use from the outside, as well as simple
from the inside. In the web world this normally means that the backend
should not have to go through extensive trouble to build up the response,
and the consuming code should not need to munge through huge amounts of data, just
to get it into a usable form. So this is what API design is all about.

The backend

When building a backend we normally try to be reasonable about the amount
of data we query from the database, and also try to minimize conversion of it, because this takes time. If it would take hundreds of queries while
sifting through JSON building up just the right response for this exact
case, our app will probably not be very responsive. So
the way to go is build up a REST (or at least somewhat REST-like) API
which serves the object backed by the database in a simple form making it easy to handle a lot of
requests.

But those guys on the other side…

Normally when the first (alpha whatever) version of the API is ready to go the
consumer side starts building, this might be Javascript frontend code,
or any other app consuming our maybe JSON / XML. Now we can either be really
lucky and the app is just so simple (or we are just so awesome at API design)
that everything is great and we can keep our simple API just how we liked it.
Actually more likely is that there are new requirements based on object
interaction in on the cosumer side we did not think about, or even willingly
ignored as it seemed not important and would have cluttered our backend.

… can’t we just add X?

Since we were kind of unsure what the API consumer really needed, and they
couldn’t really tell us as well without trying it out our API is build around
what we can serve there is a natural bias to build what is easy for the backend.
So when the frontend finally can start implementing they realize all those
little things which just don’t work with the current API output quite as they
should, and so we are in the classic ‘Can’t we just add X?’ or ‘I can just build
this in the frontend!’ mode where we start to not really work with our API but
against it. On the backend we drag to add features to our so nice and simple
code, while on the frontend we hate to just munge another JSON object into
another JSON object just to combine field X and Y which should have been
together in the first place. But didn’t TDD / BDD teach us to design outside in
from a user / consumer perspektive?

So why can’t we do better?

Even though designing code outside in (BDD style) has become really popular,
it’s not really happening when building an API at least for most apps. In most
cases the API is backend focused when it comes to design, it just serves what
the backend can serve easily. So how about the frontend and backend developers
working together? Why not designing the API from a ‘consumer perspective’ and
implementing the real code later? Isn’t this how it should be? And by doing so we can
really provide a nice API which at least works great for our current front-end,
and if it works great for one chances are it will work for something else as
well.

Enabling outside in API development: A workflow with canned responses

So far nice theory, but now let’s get some actual code going! I have to say I am
just trying things but this works currently, and to me makes a nice workflow:

Mock the API consumer side first

Writeup the responses the API should serve first, and try to use this exact data
in the front-end code, see how it works, and what needs change. Would providing
additional field X provide a significant improvement to the consumer code? You
can just do this inside the consumer itself via a mock.

Responses are just data, treat them as such

Now after an initial version is kind of working, move it out of a mock and into
it’s own file, this is where canned, a little api mock server, comes
into play. It let’s you serve those responses just via normal files instead of
having to create a big backend. By using the underlying folder structure and
some special keywords (any, index) we can easily make it act like a basic REST
API.

Your acceptance test are right there!

Backend development can now start it’s own outside in
development process, since it can verify the responses against the files
extracted to be served via canned, they can easily check what works and what
doesn’t. What makes the responses complex? Can we remove that? By having a common basis we can propose changes early, while
checking their impact immediatly on both sides, and evaluate.

Conclusion

Splitting the response data from the server implementation opens up a great way
to independently develop server side and client side code without integration
troubles. It can speed up development by facilitating parallel work on consumer
and client API side, as well as provide documentation as a by product.

I would by no means call myself an expert on this but to me it seems like a nice way to build, and so far it seems to really work great. Further refinement is probably needed for better
automation of the verification process. Also serving responses from files has
it’s limits, but actually those limit seems to work towards a nice API not
against it. Currently canned is just
a quick extraction of a concept I kept implementing again and again, so feedback is
greatly appriciated.

Any furthur ideas? hit me up on twitter and checkout canned in
npm or on github.

Have an awesome day!

Github pages with stasis (or other static side generators)

Ok first things first, github pages is awesome!
It’s so easy to just get a page up and running, and it greatly integrates with
jekyll to provide an nice blogging solution. Sadly nothing besides jekyll is
really supported well as far as static side generators go, and personally I
prefer stasis to build a side with just a view pages, which is not a blog. But
luckily with a little help from Rake this can be fixed easily (btw thanks to
@k_merz for reminding me at the last
munich-rubyshift how awesome
Rake is!).

Welcome stasis on github

So here we go, welcome
github-stasis which makes the
whole process really easy. It is just a Rakefile, which handles the setup and
managment of a repository using github pages, mainly setting up the needed
branches and a first set of files using stasis for setting up a static page.
After initialization it is just a matter of

rake publish

to create a new version of the page, ready to be pushed live using

rake live

Ok, ok so how do I get started?

Actually it is 4 steps…

  1. create a new folder where your page is gonna live


$ mkdir USERNAME.github.com
$ cd USERNAME.github.com

  1. download the Rakefile and run it


$ curl https://raw.github.com/sideshowcoder/github-stasis/master/Rakefile > Rakefile
$ rake setup_repository_for_github_user_page

  1. create a repostitory for your user or organization page on github

  2. set the origin and push


$ git add origin MYREMOTEREPOSITORY
$ git push --all origin

Yeah that’s it, and yeah it is not done and mainly a quick hack to get
going fast. Any contributions are welcome, just fork on github and create a pull
request, or tell me what you need.

Enjoy

Emulation March inspired @coreyhaines

Last week I read about emulation march and the
idea really stuck with me. I was thinking about all the people who inspired me
in the last couple years. But since I have to pick one, I was thinking about one
quality I admire and I try to emulate.

Since recently I got more into github as
a platform for getting involved in open source projects, I decided that for the
month of march I will make at least 1 hour each day to try to help out on a
project. And since this is emulation march: the person who inspires me, is Steve
Klabnik, @steveklabnik, who it seems like spends 2 days each day helping people and projects in the friendliest way possible, with all kind things picking up stuff left and right.

So here we go, let’s see how march goes :).