random walk through fields of musings

Wednesday, March 24, 2010

it's old-fashioned to swap to disk...

or not only is swapping to disk harmful, it's downright silly on webservers (conditions apply and exceptions are many).

Back when the OS on your webserver was 32-bit and 512MB was a lot, arguments like this might have been germane, but I'll just claim to have been a decade ahead of my time when I wanted my FreeBSD 2.1 webservers (behind a load-balancer with shared nothing) configured without swap so that they would just panic and reboot if they ran out of memory rather than suffer the slow death of context switches and disk IO (we'd end up rebooting them anyway).

Now with 64-bit OSes and less-than-expensive multi Gigabyte RAM, plus effective ways to cap memory usage of JVM and VMs, there is little use for disk swap on clusters of machines which are identical and data-less. Even on large DB servers with 30+ GB of RAM it is silly to consider swapping to be okay unless it's for long-running queries that can tolerate large latency. OLTP type DBs should just have the DB process restarted if it gets too big.

I'm sure there are lots of cases where you don't want to lose data and can tolerate the latency, but it certainly seems for all the low-latency "social web" sorts of applications, a load-balancer that sends new sessions to real servers based on least-response-time plus a quick reboot are going to be better remedies than the slow-death of disk swapping.

So unless you can come up with a compelling argument other than "it's always been done that way", I'm going to practice not recommending or building "clustered", web servers for interactive applications with disk swap until experience or overwhelming evidence suggests otherwise. So there.

Sunday, March 21, 2010

implementing high-volume queues cheaply

Implementing write-heavy queues is hard to do in a technically "inexpensive" way -- RDBMS' are typically read-optimized and too heavy for the task of a simple queue, dedicated queuing packages require frameworks to run in etc., so the most simple queue that is "web-writable" I could come up with, which is chronologically ordered is to simply use the webserver logs as a queue. Serving static files, as simple as a text file that contains a simple string such as "ok" and named "ok.txt", is efficient in most webservers, and a small file is easily cached, so physical disk IO would be limited to writing the webserver access logs. To add items to the queue you can just use query parameters that will get recorded in the weblogs and then can be parsed out, ie.:


and the query params won't be "interpreted" by the static file, just show up in the weblogs as long as GET query params are set to be logged.

Most webservers allow writing the timestamp in a format that is easily machine readable (milliseconds since the epoch in UTC is probably a good choice). Using spread to write the logfile in realtime to the network would reduce the disk IO on the local webserver but a spread listener would have to write it somewhere, though the ability to add multiple listeners could spread the load.

Wednesday, March 10, 2010

album of my family (ala rattlesnake)

Courtesy of an IR camera, you can now "see" what some snakes can without such a camera.
IR 0126  IR 0202 Divya IR 0205 Jai IR 1462
kid hiding under a blanket:
IR 0220 Jai under a  blanket
a family pet in a few poses:
 IR 0127 IR 0193 bunny IR 1464 IR 1465
and the kids asleep:
IR 0835 Divya sleeping IR 0836 Jai sleeping

seeing things in a different light

Not being able to justify purchasing an IR ("Thermal imaging") camera and not finding one nearby for as good a price, I rented a FLIR b50 from flir.com for $425 per week ($25 sales tax plus I pay overnight shipping both ways) for the past week. The main purpose of the rental was to take followup pictures of our community's common house to see if the added insulation in the attic really made a difference but comparing the before and after IR pictures was complicated by the fact that the before pictures, taken by an "Energy Audit" firm, didn't have the temperature to color legend embedded.

In any case, I took pictures in my own unit, and the results mostly confirmed what I already knew.
  IR 0124 lens cap
don't take pictures with the lens cap on!

IR 0125 skylight
the skylight leaks heat like crazy -- 15 degree F difference between inside and skylight is more than the 10 degrees F a regular window typically has:

IR 0128 front door
blinds and most types of shades really do work well to keep the heat from leaking out windows -- here's the gap between shade and window and another one looking at an upstairs window which also has a thin curtain in front:
IR 0194 leakage around a honeycomb blind IR 0200 MBR upstairs window with shade
one of our upstairs windows was broken by a small rock a few years ago and replaced, however the replacement definitely isn't as well sealed as the original:
IR 0833 MBR S windows no drapes, right is Diamond glass replacement IR 0834 MBR S windows, right is Diamond glass replacement
(hint: replacement is on the right in both pictures -- I took two just to show that reflection works in all spectra) - that shoddy replacement is almost 10 degress F worse it seems...

Finally, our upstairs Master bedroom is always colder in winter than either the downstairs (where the thermostat is) or even the other bedrooms on the same floor. Originally we thought it was a lack of insulation in the attic above the master bedroom (which wouldn't account for it retaining heat well when heated by sunlight, but anything else seemed remote and unlikely), so we added insulation to the attic to no avail. An HVAC person confirmed that the room seemed pretty tight, but noted that the two ducts leading to that room were each less than half the size of the ducts to any other part of the house and so he posited that sufficient warm air, especially on the long run to that room, was just not making it to that room. To fix that, he'd have to replace the ducts which would have involved tearing open the walls etc., so might just be cheaper to put in booster vents in that room. With the IR camera I have proof that he was right:
 IR 0195 vent while furnace is on IR 0196 cold air return
that's a downstairs vent while the furnace is running -- around 114 degress F coming out (even if the numbers are uncalibrated, we hope that in relative terms they are pretty accurate) and a vent in the bathroom on the same floor, at an oblique angle and farther from the furnace. Next, we look upstairs:

IR 0201 guest room vent (compare with MBR vents)
in the guest room upstairs, it's about 104 degress, pretty good and that bedroom gets sufficiently warn by our experience. Now the kicker:

IR 0198 upstairs bedroom E vent (about 8 degress cooler than downstairs vent) IR 0199 upstairs MBR SE vent (12 degress cooler than downstairs vent)
Those are the two vents in the master bedroom, anywhere from 8 to 12 degrees cooler than the vent in the guest room. Clearly, not enough warm air is making it up into that room, and part of what does make it into that larger room probably escapes into the landing and up via the skylight due to a chimmney effect.

All these photos were taken at night to minimize sun glare/reflection effects.

Thursday, March 4, 2010

(mostly) machine readable event info

Being a geek means I cringe when easily parseable data isn't. The AADL is a fine public library and offers pretty cool online tools, but they only give us a RSS feed of checkouts with the due dates in the "summary" stanza. Although they send us an email reminder 2 days before the item is due, that doesn't fit into my iCal feed grokking calendar. There are a few other such RSS feeds that hide the event data within and being sure that I'm not the only one facing this problem I looked around for a generic solution and finally hit upon a few others using Yahoo Pipes!. It took far too long to get it just right (and even then timezone parsing is still off) but I made my first publically somewhat useful pipe, described as part of a thread on the AADL forums. If you want to see the source for the pipe or use it, read the forum post and then click on http://pipes.yahoo.com/rpaditya/aadlduedates.