Hostip ruby gem update

Since I’ve been trying out the ruby way to package Software I was also interested in how easy it is to include a command line accessible bin file with the gem, and since the hostip ruby gem actually works really well I decided to include a script to access the functions from the command line. All that is to do to make that work is adding a bin directory to your gem and adding

bindir = ‘bin’

executables = [‘BINNAME’]

to your Gem specification.

If you update the gem to Version 0.2.0 it will also install a tool called hostip in your system path, which is /usr/bin on MacOSX by default for ruby gems, to get your current Internet routable IP as well as the city, country, and geo-location for that or any specified IP.

I actually use it quite often now to check if my proxy works etc. which is way faster from the command line than doing it by visiting a website like hostip or those millions other that offer that service.

Again for more info how it is made to work visit the gem on BitBucket.

Howto package a ruby gem

Today, to get used to the Ruby HTTP api I wrote a little gem to talk to the hostip.info service, which handles things like getting current IP, and location services like country and geo location. Since I also need to get used to packaging Software for my Thesis which will be written mostly in Ruby I decided to work my way through setting up a Ruby gem to be installed via the gem tool.

Coming from a background of C/ObjC and Python packaging software and libraries is not new to be and Ruby makes this really easy, too. I decided to do a quick walkthrough since I had to piece everything together from about 20 sources, most of which I will probably never find again anyway. The Process is documented on the Ruby-gems Webpage, but is not completely obvious. Over all its a 3 step Process:

  1. Setup the directory structure for the gem, and copy the code in the correct directories
  2. Setup a Rakefile which also functions as the spec file (Manifest) for the gem to build
  3. Build the gem

Part 1 is really easy, just setup the directories as follows

The lib directory is where all your code is going to live the pkg directory is the place where the gem will be located after the build. Also quite important are the tests which in my opinion should be included in every gem since they provide a nice way to check the gem against your current ruby install, and will also detect missing dependencies and so on. 2 more files are needed to complete the gem, the Rakefile and the README. I choose rdoc as the format for the README since it can be used by Ruby directly for documentation, and will also be parsed and displayed by SCMs like Github.

The last part to complete the whole setup is the Rakefile, it includes all the specifications, like author, webpage, version, included files, tests and so on. The Rakefile will later be used to build the gem. Since I build a real simple one, it might be a good idea to just look at my source to figure out the basic format.

Lastly the gem can be build by issuing rake in the gem directory which will put the finished gem in pkg to be installed via gem install, afterwards it can be tested via gem check -t GEMNAME.

Thats pretty much it! Of course I build a really simple gem to get started, gems can also include things like native code, which would be located in ext, also the build instructions can be as complex as needed because rake is as powerful or maybe even more powerful then unix make. All in all to Ruby gems to me seem like a really nice and easy way to package and publish code comparable to CPAN for perl or easy_install eggs for Python.

Check out the code at Bitbucket and feel free to contact me about things I’m doing wrong or right.

Updated Dropbox Dropzone to handle folders as well

I just pushed the code to handle folders in my Dropbox Dropzone to GitHub. If a folder is dragged on the Dropzone it is now archived and uploaded to Dropbox as an archive to be shared.

Enjoy :)

PS: Love those guys at Aptonic my changes got pulled minutes after I sent the request.

Edit: Updated again to also handle multiple files dragged on, they are also zipped now instead of producing an error. Waiting for  Aptonic to pull, but I guess it will be up shortly

Growling …

Since the latest Version Cryptr can Growl when a job is finished and also directly open the file when the message is clicked. Working with Growl I realized how great the integration of this notification service really is. It’s amazingly simple to just post a message from an Application in just a few lines of Code, even though the basic tutorial is rather extensive looking at some example Code on Bitbucket and Github really helps to see that using Growl is really simple.

Since I’m using just the really basic features, maybe the Cryptr Source might offer some example Code to get started using Growl, it to me seems like one of those frameworks almost no App on the Mac should life without, another being Sparkle.

Enjoy the new Version of Cryptr.

Using Python, SSH and Threads to quickly configure Servers

Configuring a bunch of Servers can be a pain, recently I had to setup tc priority queues for limiting the bandwidth and response time on 10 Servers (going in an out), that would mean 100 command line calls, and since they are all different of course doing it via parallel-ssh would not help. Being a Python guy it immediately sprang to mind that there is a good CSV lib, and since 10×10 Servers is a Matrix it can easily be represented in a CSV. I wrote about handling CSV files earlier, so check out Handling CSV Files. The tricky part is the handling of SSH commands in the script, but lucky me, theres a lib for that, it’s called Paramiko. With a little abstraction done by Commandline.org it is easily useable to setup connections pass commands and get results. The Tutorial on Commandline.org is actually quite nice to get started on SSH in Python.

The setup so far:

  • CSV with 10×10 Rows/Columns including the hostnames and ports to connect to
  • A script parsing the CSV and passing the needed commands via ssh.py to the host

The problem:

  • It takes forever!

Since the whole script runs sequentially, and connection via SSH is not the fastest thing in the world, the script is painfully slow to run. The solution: Threading! Since all connections are obviously independent they can easily be run in parallel without worrying about race conditions. This is where the nice Python API comes in. A Thread is just a class derived from Thread.

class Controller(Thread):
def init(self, var1, var2):
Thread.init(self)
self.var1 = var1
self.var2 = var2

All there is to do now, is implementing the run method and doing whatever is needed there

def run(self):
do something cool

Now to integrate it in the Main Script

controllers = []
for r in rows:
for c in columns:
current = Controller(r, c)
controllers.append(current)

for i in controllers:
i.join()

All that does is create a separate thread per item, and append to a array of threads. By running join it makes sure that every thread is done before the scripts exists. And thats it, you now got a threaded script running ssh commands on a bunch of hosts.

Handling CSV files with Python

Today I got my Cellphone bill and since I’ve been using Tethering a lot lately I wanted to find out how much data I actually used during the last month. Looking at the Bill I got I realized they just keep telling you the data for every single connection but never the whole amount, and since adding up which feels like hundreds of numbers, is not really my favorite past-time I decided to try my luck with the .csv file they offer, too. Sadly the MacOS Numbers.app failed completely importing it, so from the 8 rows only 2 showed up. Realizing that a spreadsheet seemed not an option I decided to crank out a small script to do it for me and here we go.

Since I python is my preferred scripting language, and I knew there must be a module to handle .csv, I settled for it and after reading the documentation I found it to be really easy. Opening the file just works as always

csvfile = open(“YourFile.csv”)

Now comes the fun part since, of course, the file is not really comma separated (like you would think seeing .csv) but | separated, and has like a way to many rows and columns setting it up by hand might take a while, and will probably break on my next bill, I was looking for a more generic way. This is where the great CSV module shines because there is actually a piece of code to determine the, so called, dialect which is the delimiter, and row / column setup.

csvsniffer = csv.Sniffer()
dialect = csvsniffer.sniff(csvfile.read(4096))

The only tricky part is to get a large enough sample (the read(4096) part) to be sure the Sniffer gets the right setup. In the Python documentation they always take like 1024, which if the file has a lot of columns will probably fail. 4096 is a pretty big sample but, well I won’t run into performance issues anyway. Finnally there is actually just one more thing to do to really get a hold of all the data in the file, parse it with the now determined options.

reader = csv.DictReader(csvfile, dialect=dialect)

Using the DictReader instead of the normal reader simplifies everything a lot since the result is a collection of Python Dictionaries in which all items are tagged with the column they came from. Now to get the data you want is just a matter of running a loop over the collection, and filtering the right tags. In my case it looked like this

data_used = 0
for item in csv_dict_reader:
if item[‘TARIFGRUPPE’] == ‘WAP / Internet’:
data_used += int(item[‘DATENVOL’].split(’ ‘)[0])

Thats it, if you like check out the source for this script.

Safari Full Screen Bookmarklet

sometimes really arbitrary, window size. Playing around with Javascript for about 5 min got me a solution, which isn’t perfect jet, but works at least

Resize to full screen

Just drag this Link to your Bookmarksbar, and when you click it the Window will resize to your full Screen Size. The only problem is that this stops working a soon as there is more than 1 Tab open. If anybody got a solution please comment.

Palm Pre Development …

Today I finally got around to download and setup my development environment for WebOS (Palm Pre). Since the Pre is supposed to hit Germany via O2 in October it seems like a great Idea to look at it now to see if it’s the device for me.

So far I gotta say the whole experience feels kind of clunky, I’m sad to say. The simulator does not work out of the box but requires some permission setting to work with the Palm Eclipse Plugin, I got it to work quite easy googleing the error, but it still leaves a bad feeling.

Looking around in the documentation I have to say I will miss Interface builder, but oh well I guess I get used to it (and JavaScript) sooner or later. So for now I guess I gotta check out the SDK before the Pre hits Germany, so I can decide what to buy Palm Pre or iPhone 3GS (from Italy).