Configuring a bunch of Servers can be a pain, recently I had to setup tc priority queues for limiting the bandwidth and response time on 10 Servers (going in an out), that would mean 100 command line calls, and since they are all different of course doing it via parallel-ssh would not help. Being a Python guy it immediately sprang to mind that there is a good CSV lib, and since 10×10 Servers is a Matrix it can easily be represented in a CSV. I wrote about handling CSV files earlier, so check out Handling CSV Files. The tricky part is the handling of SSH commands in the script, but lucky me, theres a lib for that, it’s called Paramiko. With a little abstraction done by Commandline.org it is easily useable to setup connections pass commands and get results. The Tutorial on Commandline.org is actually quite nice to get started on SSH in Python.

The setup so far:

• CSV with 10×10 Rows/Columns including the hostnames and ports to connect to
• A script parsing the CSV and passing the needed commands via ssh.py to the host

The problem:

• It takes forever!

Since the whole script runs sequentially, and connection via SSH is not the fastest thing in the world, the script is painfully slow to run. The solution: Threading! Since all connections are obviously independent they can easily be run in parallel without worrying about race conditions. This is where the nice Python API comes in. A Thread is just a class derived from Thread.

class Controller(Thread):
def init(self, var1, var2):
Thread.init(self)
self.var1 = var1
self.var2 = var2

All there is to do now, is implementing the run method and doing whatever is needed there

def run(self):
do something cool

Now to integrate it in the Main Script

controllers = []
for r in rows:
for c in columns:
current = Controller(r, c)
controllers.append(current)

for i in controllers:
i.join()

All that does is create a separate thread per item, and append to a array of threads. By running join it makes sure that every thread is done before the scripts exists. And thats it, you now got a threaded script running ssh commands on a bunch of hosts.

Today I got my Cellphone bill and since I’ve been using Tethering a lot lately I wanted to find out how much data I actually used during the last month. Looking at the Bill I got I realized they just keep telling you the data for every single connection but never the whole amount, and since adding up which feels like hundreds of numbers, is not really my favorite past-time I decided to try my luck with the .csv file they offer, too. Sadly the MacOS Numbers.app failed completely importing it, so from the 8 rows only 2 showed up. Realizing that a spreadsheet seemed not an option I decided to crank out a small script to do it for me and here we go.

Since I python is my preferred scripting language, and I knew there must be a module to handle .csv, I settled for it and after reading the documentation I found it to be really easy. Opening the file just works as always

csvfile = open(“YourFile.csv”)

Now comes the fun part since, of course, the file is not really comma separated (like you would think seeing .csv) but | separated, and has like a way to many rows and columns setting it up by hand might take a while, and will probably break on my next bill, I was looking for a more generic way. This is where the great CSV module shines because there is actually a piece of code to determine the, so called, dialect which is the delimiter, and row / column setup.

csvsniffer = csv.Sniffer()
dialect = csvsniffer.sniff(csvfile.read(4096))

The only tricky part is to get a large enough sample (the read(4096) part) to be sure the Sniffer gets the right setup. In the Python documentation they always take like 1024, which if the file has a lot of columns will probably fail. 4096 is a pretty big sample but, well I won’t run into performance issues anyway. Finnally there is actually just one more thing to do to really get a hold of all the data in the file, parse it with the now determined options.

reader = csv.DictReader(csvfile, dialect=dialect)

Using the DictReader instead of the normal reader simplifies everything a lot since the result is a collection of Python Dictionaries in which all items are tagged with the column they came from. Now to get the data you want is just a matter of running a loop over the collection, and filtering the right tags. In my case it looked like this

data_used = 0
for item in csv_dict_reader:
if item[‘TARIFGRUPPE’] == ‘WAP / Internet’:
data_used += int(item[‘DATENVOL’].split(’ ‘)[0])

Thats it, if you like check out the source for this script.

sometimes really arbitrary, window size. Playing around with Javascript for about 5 min got me a solution, which isn’t perfect jet, but works at least

Resize to full screen

Just drag this Link to your Bookmarksbar, and when you click it the Window will resize to your full Screen Size. The only problem is that this stops working a soon as there is more than 1 Tab open. If anybody got a solution please comment.

Today I finally got around to download and setup my development environment for WebOS (Palm Pre). Since the Pre is supposed to hit Germany via O2 in October it seems like a great Idea to look at it now to see if it’s the device for me.

So far I gotta say the whole experience feels kind of clunky, I’m sad to say. The simulator does not work out of the box but requires some permission setting to work with the Palm Eclipse Plugin, I got it to work quite easy googleing the error, but it still leaves a bad feeling.

Looking around in the documentation I have to say I will miss Interface builder, but oh well I guess I get used to it (and JavaScript) sooner or later. So for now I guess I gotta check out the SDK before the Pre hits Germany, so I can decide what to buy Palm Pre or iPhone 3GS (from Italy).

I finally got around to implementing a nice way to enable switching between Menubar an Dock Mode for Cryptr. Even though this was actually harder than I expected it’s working fine now and you are able to switch between Menubar and Dock Mode via the Preferences (restart is required so).

I guess I’ll be making a new Video soon about Cryptr because the other one is looking and sounding horrible after all.

Enjoy

Edit: We are up to Version 1.0.5 now due to some bug I found right after uploading Version 1.0 so be sure to Update once more sorry