Using CouchbaseLite from RubyMotion

Recently Couchbase Lite 1.0 got released, and you can find anything about it at
the Couchbase Mobile Developer Portal.
As I’ve been quite interested in RubyMotion since some time, I decided to check
how easy it would be to take CouchbaseLite for a spin on RubyMotion, and I have
to say it is quite easy. Besides some small problems with RubyMotion handling
of blocks everything worked as expected. The full project is up on
github
for you to checkout,
but I’d like to walk through some of the parts worth noting here.

Getting the database ready

First things first, opening a database file for CouchbaseLite works as expected
simply porting over what is presented in the examples.

https://gist.github.com/sideshowcoder/48ef31df4bad66c1b152.js

This database object can now be passed down to wherever it is needed. So now is
the time to get onto one of the big features of CouchbaseLite, which is sync. To
get this working we authenticate against Facebook using ACAccountStore

https://gist.github.com/sideshowcoder/8d9a3200a97e0364f9eb.js

and feed the credentials back to CouchbaseLite to authenticate any replications,
which are setup.

https://gist.github.com/sideshowcoder/b48433984955b0bb99cb.js

And with those little pieces of code we actually have a working sync setup.

Data in…

A syncing database is nice, but without data in it also kind of useless ;), so
let’s get something in there. CouchbaseLite actually provides a nice layer on
top here, letting you define models for your data which you can then read and
write like you would with any ORM. And it’s more than that, you can actually use
those as well to directly bind to a view, but more about that later. First let’s
get some data in there, by first creating a model to represent this.

https://gist.github.com/sideshowcoder/a714de4e703ef1b9feaf.js

So all there is left to do is creating a new object of this kind and save it.

https://gist.github.com/sideshowcoder/cb9049a7d4d56fcfa72f.js

And that’s it, we can now store items in our database fairly simple.

… and data out

With this we can take advantage of one of the other big features of
CouchbasLite, at least to me, which is using CouchbaseLite as an
TableViewDataSource. This cuts out huge amounts of boilerplate code, so

https://gist.github.com/sideshowcoder/a18925f3f59b878bdd91.js

will get everything setup, and make sure that the display is always up to date.

One more (sad) thing…

Sadly right now, due to a bug in the handling of Procs in RubyMotion, it does
not allow you to setup map and reduce blocks via Ruby. This will probably be
resolved by Rubymotion soon, but for now it means you have to setup the blocks
from Objective-C. But don’t worry it’s not hard, basically it’s just the code
for the block itself which is pretty easy to grasp.

https://gist.github.com/sideshowcoder/399a22d8544a92de27c3.js

In TodoLite-Motion I set it
up to be a vendored Xcode project, so it can easily be compiled and changed when
needed.

The end

And that’s it, a basic project up and running with everything needed to have a
synced and shared database in RubyMotion.

Building Couchbase from Source

Thanks to @trondn it is easier than ever to build
couchbase from source. So I decided to take the new
build system for a spin on my Macbook based on the
documentation
provided.

Getting all the dependencies

First of all if you are not yet using homebrew already, start
doing so it will make your live a lot easier throughout the world of open source
tool chains. Ok let’s get started, by installing all the needed dependencies

Installing dependencies

Ok now let’s see what all this is

  • repo is a tool which ties together
    multiple git repositories so we can use them as one big project. It it also
    reponsible for the whole code review process when used in conjuction with
    gerrit
  • v8 is googles JavaScript engine, know for
    being amazingly fast. It is used in Couchbase server to execute JavaScript.
  • snappy is a compression library optimized
    for speed. This is used internally to compress the data written to disk.
  • icu4c unicode library to handle all
    the unicode needs.
  • google-perftools provides
    TCMalloc a faster
    malloc implementation which is used whereever malloc is needed.
  • erlang a programming language build for highly
    concurrent systems. In Couchbase this is used in multiple places but
    especially the cluster managing.
  • libevent event library used internally in couchbase.
  • cmake cross platform build system to make it easy to
    build on multiple platforms native build systems. Couchbase uses this now to
    generate the build files needed on either Windows / Mac and Linux.
  • git Version control system.

This covers the basics now it’s time to get the source and start building!

Getting the source Couchbase

As metioned before Couchbase relies on repo to manage all the interdepended
repositories. So to get the source in the right versions you need to get the
manifest for the version of Couchbase you want to build so repo can get all the
right source for you. This is easy enough, simply create a folder for the source
and setup repo in there in this case with the master branch.

Initializing the repository

And finally we get the source by telling repo to sync our local copy

Download the source

That’s it time to build

Building Couchbase

Building the software itself is now as simple as running make

Build with make

Afterwards couchbase should be ready to run

Running Couchbase server

and you can visit the webinterface for all your
configuration needs.

Couchbase web GUI

The End

And there you have it, that’s all you need todo to get your locally build
version of couchbase up and Running, to maybe tryout all the new features
currently in development.

Setting up Couchbase PHP-SDK with XAMPP on Windows

Getting up and running with PHP on Windows has always been quite easy due to
XAMPP. It provides everything needed
to get started quickly and build something using a default stack available
everywhere. A lot of people get (or got) started this way, including me. But now
that MySQL, PHP, and Apache are setup maybe it’s time to try something new. For
me it’s been a while since I ran a setup like this and so I thought it’s worth
documenting the process.

Getting all the pieces

First of all you’ll need:

  • XAMPP for all your PHP and
    Apache needs, so install at least PHP and Apache
  • Couchbase, remember that you can try
    out the Enterprise version for development for free.
  • Couchbase-PHP-SDK
    XAMPP uses thread safe PHP so make sure you download the correct version
    (either x86 or x64 Thread Safe) for your environment.

Getting everything installed

For Couchbase and XAMPP you basically follow the instructions on the screen to
get it up and running, and make sure you configure Couchbase as well at the end
by visiting the web interface. Now all there is left is
moving the PHP stuff into place, which is probably the most complicated part of
a really easy setup. The zip downloaded from the windows PECL
page
contains 2 dll files,
libcouchbase.dll and php_couchbase.dll which need to be moved in the right
place. First to make php find libcouchbase it’s needs to be present c:apachebinlibcouchbase.dll and to make sure you can also use
php from the command line also copy it to c:xamppphplibcouchbase.dll and
add the XAMPP PHP to your PATH by going to Control Panel -> System and Security
-> Advanced System Settings -> Environment
Variables and edit the PATH
variable by appending c:YOUR XAMPP DIRECTORYphp. And that’s it you are
now ready to use Couchbase in XAMPP, by adding the extension to php.ini, so just add

https://gist.github.com/sideshowcoder/df131b67c173066291c3.js

Try it out!

Create a new PHP file at c:YOUR XAMPP DIRECTORYhtdocscb-test.php containing

https://gist.github.com/sideshowcoder/047d7989ad77e2b955e1.js

and open it in your browser to make apache run
it
you should now see the result in your browser.

cb-test-result

you can also use the command line of course.

cb-test-cmd

The end!

OK so you are now ready to use Couchbase from PHP, and if you need any help and
are in London this week stop by the Couchbase Developer Day as part of Big Data
Week
,
or even if you don’t still you should still stop by.

EDIT

You should also checkout Trond Norbye’s Blog basically describing the same setup.

Setting up Couchbase views with Rake

If you are like me, you like your code and at best everything else about your
app to be versioned. This makes setting up a new development environment quick,
and even more important reproducible. So one thing which always bothered me was
setting up external code outside of the application to generate Couchbase views,
since those views are build using JavaScript, and by default live in the
web interface, far away from my application code. If you want to know more about
them watch the Couchbase 103 – Views and
Map-Reduce
by the way.

Since those JavaScript functions are code which is closely related to my app, I
like to keep it inside as well, and provide a task to actually set them up for
the given environments. The following quick gist provides a rake task to do
this, adding all views in config/couchbase_views to Couchbase.

https://gist.github.com/sideshowcoder/10465780.js

This works great together with dotenv to provide the connection details via
environment variables, but can also depend on any other way to
set up the database connection easily.

Installing Couchbase sync_gateway via Homebrew

Homebrew is my favorite package manager on the Mac, sadly
there is no package to setup and install the Couchbase
sync_gateway
which is used for
interaction between a Couchbase
Server
and a Couchbase Lite
mobile database
. Since
this is easily fixed, I created a formula for sync_gateway. Currently the
Pull-Request is still open so
if you like to have this in homebrew make sure to let the homebrew maintainers
know. For the time being you can install it by referencing my fork of homebrew
directly in the brew install.


brew install https://raw.github.com/sideshowcoder/homebrew/sync-gateway-add-formula/Library/Formula/sync_gateway.rb

Quick Erlang Snippet: Run EUnit Tests from the shell

While working through the code samples for Learn you some Erlang I found it helpful to quickly define and run some EUnit test cases. So far all the Erlang I did involved using rebar, but that didn’t make sense for code snippets so here is a simple command line for recompiling and running the unit tests for a module.

https://gist.github.com/sideshowcoder/9170939.js

Safari Readinglist to Readability

Ok so I got a Kindle Fire and since there is no way to sync my Safari
Readinglist over I decided it is time to move to something “portable”. Luckily
it is quite easy to get a list of all the current Safari Bookmarks, which
includes everything on the Safari Readinglist, the basics are in a Gist by andphe

https://gist.github.com/andphe/3232343.js

So all there is left to do is use the
readit gem to get everything into
Readability, using oauth to authenticate first. Overall this went pretty well
and quickly, so for anybody interested you can find the scripts on
github.

Enjoy

Exploring the Erlang NIF with Murmur3: Part 1

Yesterday I read about calling C libraries in Haskell, especially how the
Haskell foreign function interface
works
.
Never had to do this in Erlang I decided it would be a nice exercise, so here we
go wrapping Peter Scotts Murmur3. The
end result is available on github
for anybody interested. Also I am not an Erlang so take everything with a grain
of salt.

Looking for FFI in Erlang

There are actually multiple ways to call out from Erlang, you can use Ports and
actually communicate with an external process via passing messages which is nice
and save, or you can implement wrappers in C and use them as NIFs. NIF stands
for native implemented function, so the library is actually dynamically linked
to Erlang which can be dangerous since an error in the library can crash the
Erlang VM, but it is also the fastest way to call out to C land. Since I was
interested in the closest thing to FFI really I decided to create a NIF wrapper.

Exploring Murmur3

The C library for the Murmur3 hash function actually consists of 3 functions, 2
optimized for x86 and one for x64, but since they all work on any architecture
it made sense to wrap all of them. All functions look really similar.

void MurmurHash3_x86_32 (const void *key, int len, uint32_t seed, void *out);
void MurmurHash3_x86_128 (const void *key, int len, uint32_t seed, void *out);
void MurmurHash3_x64_128 (const void *key, int len, uint32_t seed, void *out);

The first argument is the thing to be hashed, being of length len. The hash is
also seeded so passing in the seed value is mandatory. After it is done the
result will be placed in out, which is either 32 Bit or 128 Bit in size.

Setting up the project

Using rebar has proved to be the easiest way to
setup a project.

$ mkdir murmerl
$ cd murmerl
$ wget http://cloud.github.com/downloads/basho/rebar/rebar && chmod u+x rebar
$ mkdir src
$ touch src/murmerl.app.src
$ touch src/murmerl.erl

murmerl.app.src is really just a really basic app file containing only the
following.

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/src/murmerl.app.src?footer=0

Now all we need is a place for storing the C code, and put the Sources in.

$ mkdir c_src
$ cp <Download>/murmur3.c c_src/
$ cp <Download>/murmur3.h c_src/
$ touch c_src/murmur3_nif.c

After telling rebar about the C sources the project can be compiled and run.

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/rebar.conf?footer=0

$ rebar compile

Which will create priv/murmerl_drv.so as well as the beam and app files in
ebin.

NIF in the Erlang world

NIFs behave like any other function from a callers perspective, so we need to
define a module for them which is in charge of loading the library as well as
provide some fallback if loading fails. Providing fallbacks is done via
providing alternative function implementations.

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/src/murmerl.erl?footer=0&slice=32:35

To make it now work we need to load the library via the on_load for the Erlang
module and make sure all the functions are exported as well.

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/src/murmerl.erl?footer=0&slice=0:30

Thats it for the Erlang side of things, now for C.

NIF in the C world

Everything needed to bind everything together is present in erl_nif.h which
needs to be imported. Each of our functions looks really similar, they return an
Erlang Term, and take the current environment, argument count and arguments
array. Pulling the arguments apart needs to be done inside the function so this
is the first thing.

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/c_src/murmur3_nif.c?footer=0&slice=20:42

Erlang provides a lot of functions
here
we expect something string
like as the first argument so we pull the first argument inside the in struct
and extract the second as an int. The actual return value will be a binary so we
need to allocate one of the right size, 4 to get 4 Bytes = 32 Bit. Since this
is just a piece of memory, it is best to zero it out before using. Now all
there is to do is let MurmurHash3_x86_32 write to the allocated chunk and return
as a Erlang Binary for everybody to use.

To make the functions available to Erlang we need to tell the VM about them,
this is done here

http://gist-it.appspot.com/github/sideshowcoder/murmerl/blob/6eee6fb4a1f26890a86dbdbbfc6fef72c34d5a4a/c_src/murmur3_nif.c?footer=0&slice=88:1000

Which exports the nif_funcs array containing a structure with

{ name_in_erlang, arity, c_function }

for each of the exported functions. And thats it the functions can now be called
from Erlang like any normal Erlang Function in the module. For the full code
see the github repository
.

Where to go from here?

Currently all the module does is forward the calls as is to the C library, which
does not feel very Erlang like, so building a nicer API is up next.

Resources

More examples can be found in davisp github
repository
, also the Erlang NIF
page
can be of great help but you
need to know what to look for.