random walk through fields of musings

Wednesday, February 25, 2009

Dada this

I'm not easily impressed by spam, but I liked this one:

20090225175246-obama_spam

I am glad that Google warned me about the impersonation -- I might have thought I got it straight from the man's Blackberry following his address to the joint session of Congress.

Tuesday, February 3, 2009

scooped

At the beginning of my love affair with Google Earth, I proudly wrote some KML-spitting-out pages that allowed me to record or playback a tour in it. It was written using the generic reporting framework I built using the once-upon-a-time-and-still-somewhat-spiffy-but-totally-unsupported Zapatec framework in a few hours. It worked and still works well. Too bad that Google Earth 5 now has support for Touring (recording and playback native in KML without using network links) built-in.

I suppose I can rewrite my KML to do either view-based refresh or the new way based on the user-agent. Wait. Can I do capability detection in KML?

on calendars and things

Google calendar has become my preferred way of keeping track and coordinating events with my wife, family, cohousing community, friends, travel and almost anything else that is temporal it seems. The hard exception is work, which till recently used "Meeting Maker", a piece of bloatware that had no way to exchange calendars in any known, open format, but now is based on Microsoft Exchange.

Exchange doesn't have an ical feed (that Google Calendar can easily consume, and that I've set almost everything else up to produce for its' consumption), however Exchange sends event data as iCalendar attachments in emails. Thankfully, Google Mail (Gmail and mail for apps) recognizes the text/calendar attachments, parses the iCalendar info, and offers to add it to your Google Calendar.

Google Mail recognises .ics attachments by djfiander.

Amazingly, Exchange also will parse and allow Google Calendar invites (which are also iCalendar format) to be added (or refused) to my work calendar (I just had to get the correct email address to send it to for it to work!), there is a bit of broken-ness in dealing with replies to invitations from Google Calendar, but that I can bear to kludge by hand.

An well-documented, open, hardly novel way of exchanging calendar information by email is fantastic and I've added such attachments to the GO online meal signup and shift system, here's an example (part of a multipart/alternative message):


------=MIME_BOUNDRY_altpparts
Content-Type: text/calendar; charset="utf-8"; method=REQUEST
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename=invite.ics

BEGIN:VCALENDAR
PRODID:-//Grot Org//Meal tracker//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:REQUEST
BEGIN:VEVENT
DTSTART:20090203T231500Z
DTEND:20090204T000000Z
DTSTAMP:20090129T154454Z
ORGANIZER;CN=go-meal:mailto:foo@plzs.org
UID:1680@meals.plzs.org
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=FALSE;CN=Fitzgerald/Aditya;X-NUM-GUESTS=4:mailto:nobody@grot.org
CLASS:PRIVATE
CREATED:20090203T130300Z
DESCRIPTION:Wings Over Great Oak by tim details at https://example.com/meal-signup/show?m=1680
LAST-MODIFIED:20090129T154454Z
LOCATION:go-meal
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Wings Over Great Oak (go-meal)\, 4 diners
TRANSP:OPAQUE
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:REMINDER
TRIGGER;RELATED=START:-PT15M
END:VALARM
END:VEVENT
END:VCALENDAR
------=MIME_BOUNDRY_altpparts--

Netapp OnTap per-volume statistics

I'm a fan of Netapps, and I've grown used to using the OnTap SNMP agent to collect statistics on various filer functions, stuff them into RRDs and troubleshoot using trend graphs (I really like drraw for that). I can then make cool graphs like:

drraw by you.

However, those ops numbers are for the filer as a whole making it hard to tell *what* is the target for all that activity -- we need per volume and even per disk stats to find hot spots. The "stats" command in OnTap can expose that data, however not via SNMP and only via the cli (maybe via the SDK, but that's a whole level of complication I'd like to avoid). The whole list of supported stats counters is exposed using the command stats list counters. So using ssh with public keys (and using stats start/stop every 5 minutes), I was able to "plug-in" the output from periodically querying the stats command into the existant set of RRDs that drraw draws from. Here's the per-volume read/write/other ops for the same filer:

drraw by you.

and one showing total time spent serving along with amount of data served in bytes:

drraw by you.

and bandwidth by volume:

drraw by you.
the relevant code fragment is quite simple and follows the SNMP_util::snmpmaptable semantics, each "row" of instance data is sent to a callback function which writes it into the appropriate RRD (with an example invocation):

my($getStatCols);

$getStatCols->{'volume'}->{'type'} = "volStats";
@{$getStatCols->{'volume'}->{'cols'}} = ("read_data", "read_latency", "read_ops",  "write_data", "write_latency", "write_ops", "other_latency", "other_ops");

for my $stat (keys %{$getStatCols}){
my($stattxt) = "${stat}:*:" . join(" ${stat}:*:", @{$getStatCols->{$stat}->{'cols'}});
getStatTable($username, $netappHostname, $getStatCols->{$stat}->{'type'}, $stattxt,  \&printfun);
}

sub getStatTable {
my($user, $host, $id, $statcmd, $callback) = @_;

my(@ret) = `ssh ${user}\@${host} "stats stop -I ${id} -O print_zero_values=off -c -d |"`;
my(@start) = `ssh ${user}\@${host} "stats start -I ${id} ${statcmd}"`;
my($startcmd) = join(' ', @start);
if ($startcmd){
   $startcmd =~ s/\s+//g;
   chomp($startcmd);
   if ($startcmd ne ""){
     print STDERR "ERROR starting stats collection for ${user}\@${host} $statcmd: " . $startcmd . "\n";
   }
}

if ($#ret <= 2){
   print STDERR "ERROR retrieving stats from ${user}\@${host} ${statcmd}: " . join(" ", @ret) . "\n";
} else {
   shift(@ret);
   shift(@ret);
   shift(@ret);
   for (my $i=0;$i<=$#ret;$i++){
     my($l) = $ret[$i];
     chomp($l);
     $l =~ s/^\s+//g;
     $l =~ s/\s+$//g;
     if ($l !~ /|$/){
       $l .= "|";
     }
     my(@cols) = ($id, $i, split(/\|/, $l));
     &$callback(@cols);
   }
 }
}

sub printfun {
#shift(@_);
my(@vals) = @_;
for (my $i=0;$i <= $#vals; $i++){
if (! $vals[$i] || $vals[$i] eq ''){
$vals[$i] = 0;
}
}
#next if (! @vals);
#next if (! $vals[0] || ! $vals[1]);
print join('|', @vals) . "|\n";
}