Setting up Riak with Vagrant and Puppet

Setting up Riak is pretty easy and straight forward,
but depending on my needs there is the need to change configuration
depending on the current use case. This means that having one instance of it on
my development machine just doesn’t cut it. Luckily there is
Vagrant which provides a quick way to setup VMs as
needed for a given project, along with Puppet this allows me to bundle all the
configuration for a given project.

Basho provides a nice module to setup Riak with puppet, but is pretty complex
to handle all the different use cases. Since I just need it for development I
decided to make it easier for me and other devs by providing an easy to setup and
a quickly understandable module for my needs. It just installs the most recent
version of Riak from the debian-pkg provided for Ubuntu 12.04 LTS 64Bits. So here we go

Vagrant configuration is short and simple, puppet folder setup as well.

https://gist.github.com/sideshowcoder/6439548.js

while for riak_precise64 I just add a submodule via git

$ git submodule add
https://github.com/sideshowcoder/riak_precise64.git
./puppet/modules/riak_precise64

and here we go a simple

$ vagrant up

will give us a nice VM with riak ready to use, and any config nicely contained in app.config.

Get links from twitter to improve RSS Reader experience

For some time now I’m using Fever to get more out of my RSS feeds. It’s nice to have a rating on articles which are important to me. There are some high volume feeds I just don’t have the time to scan, but still if something important comes up I don’t want to miss it. This is also where a great service comes in a guy I know from College build, it’s called Chillitweets. Chillitweets scans your twitter stream to extract links and presents them to the user. Since, by definition, I follow users I’m interested in they will as well post links I’m interested in which get added via Chillitweets RSS feed to my Reader. Now they can be used to rate the feeds I’m presented with. I’ve been a user of Chillitweets since the early days and I gotta say Mario really build a great service there. And to see NodeJS in action is always nice :).

Xen Long Distance Live Migration

Reading about VMWares Long Distance VMotion, which they offer in connection with Ciscos Data Center Interconnect, I was wondering if the same thing wouldn’t be possible using Citrix XenServer as well as OpenVPN. My current test setup is running a XenServer instance in Tubingen at my University, as well as an instance at the TU Munich. The distance between both locations is about 220KM, and a ping reveals a latency of about 8ms. Both locations provide a Bandwidth of about 100MBit through the Universities Internet connection, while throughput is limited by OpenVPN further. I got a Server running OpenVPN as well as NFS for shared storage at the TU Munich as well, while the OpenVPN Client in Tubingen is running as a VM inside XenServer itself to provide the VPN Tunnel.
The purpose of the setup is to provide an environment to demonstrate the possibility to migrate a VM between different subnets while maintaining reachability throughout the migration, which is the topic of my thesis. Even though this is not implemented yet the migration over the distance, while just using a pretty standard internet connection and a VPN Tunnel, is. I decided to create a screencast to demonstrate this, in it’s first form, which is working since yesterday.
I plan to have a demo of a working migration with constant reachability when my thesis is done.

http://vimeo.com/moogaloop.swf?clip_id=12429809&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=ffffff&fullscreen=1

Kind of hard to see in the current video quality, the top server is located at Tubingen in a 10.1.1.0/24 Subnet, with a TestVM running, while the other Server is located at the TU Munich with a 10.0.0.0/24 Subnet, both running XenServer 5.5. The TestVM runs a little script printing the current date every second, to demonstrate there is no real downtime during the Live Migration.

My Setup Part 2: Vim and Coding

Working on my diploma thesis, like seen in the last post, I’m currently writing a lot. Especially since I’m writing lot of different things like configs, code, as well as the thesis itself I got used to really relying on a small amount of tools I try to master instead of using different tools for each task. I will try to outline this, as well as the hardware in use.

The main tool in my workflow currently is Vim. Even though it is a very old editor, in my opinion it probably is at least one of the most powerful, especially when it comes to customizing it. I use it to edit pretty much everything, and since syntax highlighting for a lot of stuff is build in I don’t need to rely on a lot of outside themes, modes, and plugins but there are some essential ones. First of all there is snipMate, which brings the awesome power of snippets, known to Mac users from TextMate, to Vim. It’s just so much easier to Code if you just have to type def TAB to get a whole new method setup when editing ruby, instead of writing it out yourself.. and Thats just the beginning defining your own snippets is easy, and the included ones are powerful as well.

Other very nice plugins are surround, to quickly change tags in HTML or LaTeX, as well as endwise to make sure you don’t forget those brackets at the end in C or the end in Ruby. To manage plugins, modes, as well as themes I use autoload provided by pathogen which enables automatic loading of every plugin you put in a special folder. For everybody interested all my configs are available at Bitbucket, because there is so much more about Vim configuration I’m not going to describe it all here. Also check out vimcast I pretty much get to know a new great feature every time an episode is out.

On the Terminal side of things I got used to use screen to manage a whole bunch of ssh sessions, as well as to get an IRB running every time I open a Terminal, which is extremely handy since most of the code I write is ruby, and explorative programming makes a lot of things a lot easier. Of course I use Vim as an interactive editor for IRB which makes editing those code snippets I try out a breeze, and wirble to give me autocompletion in IRB.

I guess it’s kind of obvious that I’m a big Mac/Unix fan so to round it up I use a first generation Macbook Pro Unibody as my main machine, running VMWare Fusion to administer those XenServers under XP, as well as to give myself a Linux environment when needed. Currently the Macbook is equipped with a 500GB HDD as well as 4GB of RAM to keep those VMs running smoothly. Until today I was using a 500GB FW800 Drive for Backup, which sadly died today and will be replaced with something RAID1 and about 1TB shortly.

Thats it for this Post, I will be continue this series hopefully soon, and maybe include some things about my home Windows 7 Nettop Box as well.

Stay tuned, also for more XenServer Stuff which is on it’s way as soon as seamless migration is working as I want it to between those mentioned locations. Until then I’ll be hacking away happily using the old and trusty Vim ;).

My Setup Part 1: Web use basics

Recently I’ve been reading up on a Series called the setup where a bunch of really interesting people describe how and especially what they use for their computing needs. I got a whole bunch of new and useful tools out of it and also got in a habit to really think about what I’m using, why, and how. Since I can’t keep this all in my head I decided to make al little series out of it, and to start with I’m going to write about all the tools I use daily on and with the Web.

Thinking about it got me to the realization that I don’t need a mail client anymore, but using GMail on the web works even better with my workflow, especially due to the great search features, since I’m not a big fan of folders. Combine this with a notifier to keep track of new emails as they come in and it will stack up to pretty much any Desktop Mail client, especially after enabling shortcuts, so that archiving email is just a ‘e’ away.

With the use of webmail comes the need for a fast and stable browser, since Firefox is becoming more and more the opposite of fast and stable lately I switched to Chrome. Of course since I’m on a Mac Safari would have been the obvious choice, but Safari is just way to slow when handling multiple tabs, which especially if you are using a bunch on online tools are essential. Chrome now really is the basis of all my web use, and since the extensions are there it replaces more and more of my daily desktop tools with online equivalents, like Chromed Bird  for Twitter. Even though I tried to replace NetNewsWire as well with Google Reader, to me Reader is just not there especially when using a lot of feeds because of missing options like Snippets, Open in Background, Open in new Browser Window, even though all this is possible it’s really rough around the edges.

So to me EMail an Browsing is my basic web use, there are a lot of tools I use on the web, and with the web especially when it comes to sync, and I will write about those in a later post. Also to come is the probably biggest part of my daily computer usage which is Terminal, MacVim, and IRB to which I posted the configs to Bitbucket, but will also describe how to make everything work together to beat every IDE and make Eclipse hide weeping in the corner.

The art of unix tool configs

While unix , in my case MacOSX, is the source of many great tools, most of them only get really great if they are configured correctly. For example looking at a “out of the box” Vim, it seems like an almost unusable editor, but configured right its probably one of the best, well it is the best for me at least.

While Ruby, especially Rails, preaches what is called “sane defaults”, which means that the default configuration is what most people should need, this is not true for most of the command line Unix tools, I’m sad to say. Or maybe it’s just me I don’t know. Since reconfiguring all the tools like I need them every time I’m at a new machine is a real hassle, so a long time ago I setup a svn repository with all of my configs so I can simple pull them down whenever I need them. Since svn is getting a little old fashioned, and also I think there is much to learn about configuration from reading others, I decided to just push everything to Bitbucket. I also wrote a really simple bash script to setup the configs needed simply by issuing

./setup_config.sh vim

to setup vim. Or use hg, screen, xemacs, bash to set them up.

I got to say so my vim configuration is rather large and is also spread over two directories, so maybe take a look there first, and check out the comments in the files since I at least try to keep them documented throughout their building process.

iPhone 3.1.3 on O2 Germany with costum IPCC Carrier Bundle

Recently Apple released Version 3.1.3 of the iPhone OS, reenabling tethering for Carriers which do not originally have the iPhone. I myself got an iPhone from italy completely unlocked running on the O2 Network in Germany, until Version 3.1.3 I was using a costum IPCC Carrier Bundle file to get a nice O2 logo and also to have my APNs and MMS preconfigured so I didn’t have to do that everytime I change SIMs. Sadly this IPCC stopped working with 3.1.3, well it did not stop working technically but tethering was not enabled while using it, also the MMS settings didn’t get applied properly.

<

p>Googling around for some time last night I was able to setup a carrier bundle for O2 which enables tethering, pre populates all the APN and MMS settings, while getting rid of those nasty “Call forwarding” messages when calling somebody, so for anybody interested you can download it.

To apply it simple run

defaults write com.apple.iTunes carrier-testing -bool TRUE

in the Terminal with iTunes closed to enable custom carrier bundles. Afterwards you can apply the IPCC simply by Option+Click on Check for Updates in iTunes and selecting the downloaded IPCC.

Setup Mercurial over SSL with Apache

Since some of my SVN Repositories did not survive the last Update of the Database, and were really hard to recover, or some even needed to be recovered from older Backup, I decided to switch to Mercurial for all me Repository needs.

Looking around on the Web there a probably 100s of guides explaining the basics and setup, but actually None of them worked as a whole for me, so I decided to write down my experiences maybe they apply to somebody, and if not, at least it will be easier for me next time.

First of all thanks to the following Guides I got everything working:

Second I’m running on a Debian Lenny Server with all Updates until 5/5/2009, Mercurial is 1.0.1, Apache is Apache/2.2.9 (Debian).

Now a couple hints, make sure cgi is enabled

sudo a2enmod cgi
sudo a2enmod cgid

Now Setup a new Location for Mercurial like this one

ScriptAliasMatch ^/merc(.*) /MYREPOPATH/hgwebdir.cgi$1
   <Location /merc>
   Allow from all
   Options ExecCGI
   AuthType Digest
   AuthName “REPOSNAME”
   AuthDigestProvider file
   AuthUserFile PATHTOPASSWORDFILE
   Require valid-user
   </Location>

Now copy the file hgwebdir.cgi found in /usr/share/hg to the Repository Path and Setup a hgweb.config file in the same location to point to your repositories

[collections]
MYREPOPATH = MYREPOPATH

Now just the repositories need to be initilized in this Path

mkdir testrepo
cd testrepo && hg init

and the Password file has to be created

mkdir testrepo
htdigest -c PASSWORDFILE ‘REPOSNAME’ USERNAME

If you now add allow_push = USERNAME to the hgrc file for the repository you can push your changes to the newly initilized repository after cloning it.

Hope those hints will help, if not comments are welcome.

Wifi Speed Draft-N: 2.4GHz vs 5GHz

Just a quick comparison, after using my Airport Extreme with Draft-N Wifi in the 2.4GHz b/g compatibility mode for some time I realized how much more speed you can get using the N only 5GHz mode, just for comparison:

Using 2.4GHz B/G/N mode: 140MBit

Using 5GHz N only mode: 270MBit

Since I have 2 Wifi routers at home using 1 for B/G and the Airport Extreme for N only really pays of, suddenly copying large files via Wireless makes sense and is useable. Knowing that the addition of the second Wifi Set in the latest Airport Extremes make a lot of sense, especially when sharing Media over Wifi.

Using S/MIME in GMail

I’m a big fan of EMail encryption and Authentification, because I use EMail like Letters, and would like to assure the recipient that it’s really me who’s sending it. Also if it’s Privat or if I might be sending the occasional Password, I would like to encrypt.

Since S/MIME is build into pretty much every EMail Programm on the Planet (Apple Mail, Outlook, Thunderbird, Evolution etc.) I prefer it over PGP/GPG. Well I got my Certificat from www.thawte.com which worked really flawlessly, and they also tell you how to Integrate it in about ever Mail Programm there is, the only Problem I have is:

What do I do If I’m on a Computer without my Mail Programm?

But since I use GMail there is a nice solution, GMail S/MIME I finally no longer have to run Thunderbird on my Netbook but can stick to Firefox, and thats all I need. Simply Import your Certificate, and the Thawte Root CA under Certificates (Preferences -> Advanced -> View Certificates ), and your good to go, no other Configuration needed.

It works great for me, so I guess finally I can tell more People to use S/MIME.