Par: awesome text reformatting

Since I’m constantly working with text reformatting it in a useful way that is quick and easy is essenttial to keep working. Previously I used to rely on my editors reformatting, but since this kept constantly screwing up especially working with Python where whitespace is important and I was looking for a better way, and discovered the nice little utility par. So far pretty much all the reformatting happend as expected, it works perfectly with nested comments and quoted email.

Adding a quick line to the vimrc file makes Vim use par for reformatting which is really useful to never really have to leave my editor.

set formatprg=par -w80q

This will make reformat with a width of 80 characters (-w) and keep quoting even if line is empty (-q).

Xen Long Distance Live Migration

Reading about VMWares Long Distance VMotion, which they offer in connection with Ciscos Data Center Interconnect, I was wondering if the same thing wouldn’t be possible using Citrix XenServer as well as OpenVPN. My current test setup is running a XenServer instance in Tubingen at my University, as well as an instance at the TU Munich. The distance between both locations is about 220KM, and a ping reveals a latency of about 8ms. Both locations provide a Bandwidth of about 100MBit through the Universities Internet connection, while throughput is limited by OpenVPN further. I got a Server running OpenVPN as well as NFS for shared storage at the TU Munich as well, while the OpenVPN Client in Tubingen is running as a VM inside XenServer itself to provide the VPN Tunnel.
The purpose of the setup is to provide an environment to demonstrate the possibility to migrate a VM between different subnets while maintaining reachability throughout the migration, which is the topic of my thesis. Even though this is not implemented yet the migration over the distance, while just using a pretty standard internet connection and a VPN Tunnel, is. I decided to create a screencast to demonstrate this, in it’s first form, which is working since yesterday.
I plan to have a demo of a working migration with constant reachability when my thesis is done.

http://vimeo.com/moogaloop.swf?clip_id=12429809&server=vimeo.com&show_title=1&show_byline=1&show_portrait=1&color=ffffff&fullscreen=1

Kind of hard to see in the current video quality, the top server is located at Tubingen in a 10.1.1.0/24 Subnet, with a TestVM running, while the other Server is located at the TU Munich with a 10.0.0.0/24 Subnet, both running XenServer 5.5. The TestVM runs a little script printing the current date every second, to demonstrate there is no real downtime during the Live Migration.

My Setup Part 2: Vim and Coding

Working on my diploma thesis, like seen in the last post, I’m currently writing a lot. Especially since I’m writing lot of different things like configs, code, as well as the thesis itself I got used to really relying on a small amount of tools I try to master instead of using different tools for each task. I will try to outline this, as well as the hardware in use.

The main tool in my workflow currently is Vim. Even though it is a very old editor, in my opinion it probably is at least one of the most powerful, especially when it comes to customizing it. I use it to edit pretty much everything, and since syntax highlighting for a lot of stuff is build in I don’t need to rely on a lot of outside themes, modes, and plugins but there are some essential ones. First of all there is snipMate, which brings the awesome power of snippets, known to Mac users from TextMate, to Vim. It’s just so much easier to Code if you just have to type def TAB to get a whole new method setup when editing ruby, instead of writing it out yourself.. and Thats just the beginning defining your own snippets is easy, and the included ones are powerful as well.

Other very nice plugins are surround, to quickly change tags in HTML or LaTeX, as well as endwise to make sure you don’t forget those brackets at the end in C or the end in Ruby. To manage plugins, modes, as well as themes I use autoload provided by pathogen which enables automatic loading of every plugin you put in a special folder. For everybody interested all my configs are available at Bitbucket, because there is so much more about Vim configuration I’m not going to describe it all here. Also check out vimcast I pretty much get to know a new great feature every time an episode is out.

On the Terminal side of things I got used to use screen to manage a whole bunch of ssh sessions, as well as to get an IRB running every time I open a Terminal, which is extremely handy since most of the code I write is ruby, and explorative programming makes a lot of things a lot easier. Of course I use Vim as an interactive editor for IRB which makes editing those code snippets I try out a breeze, and wirble to give me autocompletion in IRB.

I guess it’s kind of obvious that I’m a big Mac/Unix fan so to round it up I use a first generation Macbook Pro Unibody as my main machine, running VMWare Fusion to administer those XenServers under XP, as well as to give myself a Linux environment when needed. Currently the Macbook is equipped with a 500GB HDD as well as 4GB of RAM to keep those VMs running smoothly. Until today I was using a 500GB FW800 Drive for Backup, which sadly died today and will be replaced with something RAID1 and about 1TB shortly.

Thats it for this Post, I will be continue this series hopefully soon, and maybe include some things about my home Windows 7 Nettop Box as well.

Stay tuned, also for more XenServer Stuff which is on it’s way as soon as seamless migration is working as I want it to between those mentioned locations. Until then I’ll be hacking away happily using the old and trusty Vim ;).

My Setup Part 1: Web use basics

Recently I’ve been reading up on a Series called the setup where a bunch of really interesting people describe how and especially what they use for their computing needs. I got a whole bunch of new and useful tools out of it and also got in a habit to really think about what I’m using, why, and how. Since I can’t keep this all in my head I decided to make al little series out of it, and to start with I’m going to write about all the tools I use daily on and with the Web.

Thinking about it got me to the realization that I don’t need a mail client anymore, but using GMail on the web works even better with my workflow, especially due to the great search features, since I’m not a big fan of folders. Combine this with a notifier to keep track of new emails as they come in and it will stack up to pretty much any Desktop Mail client, especially after enabling shortcuts, so that archiving email is just a ‘e’ away.

With the use of webmail comes the need for a fast and stable browser, since Firefox is becoming more and more the opposite of fast and stable lately I switched to Chrome. Of course since I’m on a Mac Safari would have been the obvious choice, but Safari is just way to slow when handling multiple tabs, which especially if you are using a bunch on online tools are essential. Chrome now really is the basis of all my web use, and since the extensions are there it replaces more and more of my daily desktop tools with online equivalents, like Chromed Bird for Twitter. Even though I tried to replace NetNewsWire as well with Google Reader, to me Reader is just not there especially when using a lot of feeds because of missing options like Snippets, Open in Background, Open in new Browser Window, even though all this is possible it’s really rough around the edges.

So to me EMail an Browsing is my basic web use, there are a lot of tools I use on the web, and with the web especially when it comes to sync, and I will write about those in a later post. Also to come is the probably biggest part of my daily computer usage which is Terminal, MacVim, and IRB to which I posted the configs to Bitbucket, but will also describe how to make everything work together to beat every IDE and make Eclipse hide weeping in the corner.

The art of unix tool configs

While unix , in my case MacOSX, is the source of many great tools, most of them only get really great if they are configured correctly. For example looking at a “out of the box” Vim, it seems like an almost unusable editor, but configured right its probably one of the best, well it is the best for me at least.

While Ruby, especially Rails, preaches what is called “sane defaults”, which means that the default configuration is what most people should need, this is not true for most of the command line Unix tools, I’m sad to say. Or maybe it’s just me I don’t know. Since reconfiguring all the tools like I need them every time I’m at a new machine is a real hassle, so a long time ago I setup a svn repository with all of my configs so I can simple pull them down whenever I need them. Since svn is getting a little old fashioned, and also I think there is much to learn about configuration from reading others, I decided to just push everything to Bitbucket. I also wrote a really simple bash script to setup the configs needed simply by issuing

./setup_config.sh vim

to setup vim. Or use hg, screen, xemacs, bash to set them up.

I got to say so my vim configuration is rather large and is also spread over two directories, so maybe take a look there first, and check out the comments in the files since I at least try to keep them documented throughout their building process.

Howto package a ruby gem

Today, to get used to the Ruby HTTP api I wrote a little gem to talk to the hostip.info service, which handles things like getting current IP, and location services like country and geo location. Since I also need to get used to packaging Software for my Thesis which will be written mostly in Ruby I decided to work my way through setting up a Ruby gem to be installed via the gem tool.

Coming from a background of C/ObjC and Python packaging software and libraries is not new to be and Ruby makes this really easy, too. I decided to do a quick walkthrough since I had to piece everything together from about 20 sources, most of which I will probably never find again anyway. The Process is documented on the Ruby-gems Webpage, but is not completely obvious. Over all its a 3 step Process:

Setup the directory structure for the gem, and copy the code in the correct directories
Setup a Rakefile which also functions as the spec file (Manifest) for the gem to build
Build the gem

Part 1 is really easy, just setup the directories as follows

The lib directory is where all your code is going to live the pkg directory is the place where the gem will be located after the build. Also quite important are the tests which in my opinion should be included in every gem since they provide a nice way to check the gem against your current ruby install, and will also detect missing dependencies and so on. 2 more files are needed to complete the gem, the Rakefile and the README. I choose rdoc as the format for the README since it can be used by Ruby directly for documentation, and will also be parsed and displayed by SCMs like Github.

The last part to complete the whole setup is the Rakefile, it includes all the specifications, like author, webpage, version, included files, tests and so on. The Rakefile will later be used to build the gem. Since I build a real simple one, it might be a good idea to just look at my source to figure out the basic format.

Lastly the gem can be build by issuing rake in the gem directory which will put the finished gem in pkg to be installed via gem install, afterwards it can be tested via gem check -t GEMNAME.

Thats pretty much it! Of course I build a really simple gem to get started, gems can also include things like native code, which would be located in ext, also the build instructions can be as complex as needed because rake is as powerful or maybe even more powerful then unix make. All in all to Ruby gems to me seem like a really nice and easy way to package and publish code comparable to CPAN for perl or easy_install eggs for Python.

Check out the code at Bitbucket and feel free to contact me about things I’m doing wrong or right.

Hostip ruby gem update

Since I’ve been trying out the ruby way to package Software I was also interested in how easy it is to include a command line accessible bin file with the gem, and since the hostip ruby gem actually works really well I decided to include a script to access the functions from the command line. All that is to do to make that work is adding a bin directory to your gem and adding

bindir = ‘bin’

executables = [‘BINNAME’]

to your Gem specification.

If you update the gem to Version 0.2.0 it will also install a tool called hostip in your system path, which is /usr/bin on MacOSX by default for ruby gems, to get your current Internet routable IP as well as the city, country, and geo-location for that or any specified IP.

I actually use it quite often now to check if my proxy works etc. which is way faster from the command line than doing it by visiting a website like hostip or those millions other that offer that service.

Again for more info how it is made to work visit the gem on BitBucket.

Updated Dropbox Dropzone to handle folders as well

I just pushed the code to handle folders in my Dropbox Dropzone to GitHub. If a folder is dragged on the Dropzone it is now archived and uploaded to Dropbox as an archive to be shared.

Enjoy :)

PS: Love those guys at Aptonic my changes got pulled minutes after I sent the request.

Edit: Updated again to also handle multiple files dragged on, they are also zipped now instead of producing an error. Waiting for Aptonic to pull, but I guess it will be up shortly

iPhone 3.1.3 on O2 Germany with costum IPCC Carrier Bundle

Recently Apple released Version 3.1.3 of the iPhone OS, reenabling tethering for Carriers which do not originally have the iPhone. I myself got an iPhone from italy completely unlocked running on the O2 Network in Germany, until Version 3.1.3 I was using a costum IPCC Carrier Bundle file to get a nice O2 logo and also to have my APNs and MMS preconfigured so I didn’t have to do that everytime I change SIMs. Sadly this IPCC stopped working with 3.1.3, well it did not stop working technically but tethering was not enabled while using it, also the MMS settings didn’t get applied properly.

p>Googling around for some time last night I was able to setup a carrier bundle for O2 which enables tethering, pre populates all the APN and MMS settings, while getting rid of those nasty “Call forwarding” messages when calling somebody, so for anybody interested you can download it.

To apply it simple run

defaults write com.apple.iTunes carrier-testing -bool TRUE

in the Terminal with iTunes closed to enable custom carrier bundles. Afterwards you can apply the IPCC simply by Option+Click on Check for Updates in iTunes and selecting the downloaded IPCC.

Growling …

Since the latest Version Cryptr can Growl when a job is finished and also directly open the file when the message is clicked. Working with Growl I realized how great the integration of this notification service really is. It’s amazingly simple to just post a message from an Application in just a few lines of Code, even though the basic tutorial is rather extensive looking at some example Code on Bitbucket and Github really helps to see that using Growl is really simple.

Since I’m using just the really basic features, maybe the Cryptr Source might offer some example Code to get started using Growl, it to me seems like one of those frameworks almost no App on the Mac should life without, another being Sparkle.

Enjoy the new Version of Cryptr.