An aspect of testing Java programs that seems to me fairly neglected is testing conformance to interfaces, and (nearly identically) testing that functionality of abstract classes works properly in all concrete subclasses. Certainly this has been mentioned before, generally under the name Abstract Tests or Contract Tests. Also, the idea seems to me very much in keeping with Behaviour Driven Development (BDD) and the Design by Contract (DbC) philosophy.

TestNG does not explicitly support Contract Tests, as far as I can tell, but it’s fairly easy to make it work using the little trick I describe below.

Read the rest of this entry »

The problem

In searching for a way to back up one of my Linux boxes to Amazon S3, I was surprised to find that none of the many backup methods and scripts I found on the net did what I wanted, so I wrote yet another one.

Read the rest of this entry »

The Symptoms

I have two 3ware 8506-4 controllers with three disks each connected to them. I’ve been getting high iowait on my box whenever there’s an even slightly disk-intensive task going on. The weird thing is that even processes that don’t use the high-usage disk get stalled in iowait. As a result, load skyrockets and the system becomes unresponsive.

Inducing disk load with bonnie++ and watching things with “iostat -x” revealed that even disks with extremely low bandwidth usage and very few i/o requests end up with 100% utilization. However, only those disks on the same controller as the high-usage disk are affected. The observed total throughput is around 50 Mb/sec, e.g., the limit of the high-usage disk. It’s nowhere near the bandwidth limit of the controller, the SATA channel, or the PCI bus. So, there’s something wrong with the controller that effectively limits it to control one disk at a time with reasonable performance.

The kernel is the current CentOS 3.6 one, 2.4.21-37.0.1.ELsmp.

The write cache is enabled on the controllers.

This seems to be related to at least some of the posts on the infamous RHEL vs. 3ware performance bug.

The Diagnosis

Apparently, what is happening is that the four SATA channels on the controller share a single request queue, with a depth of 256 requests. When requests to one disk arrive at a high rate, they fill the queue, thereby starving the other disks.

Controllers from other manufacturers typically limit the number of requests to a single disk to some number smaller than the total queue depth for the controller. Thus, the queue for a single disk can get full without blocking access to the other disks. Why 3ware made the default per-disk queue depth the same as the depth for the whole controller is beyond me. Fortunately, this is correctable.

Read the rest of this entry »

My Powerbook G4 has been running way slow lately, and I finally have a little breathing room to address the problem. My impression is that a) my disk is really full, b) my disk is really fragmented, and c) this especially impacts virtual memory.

So my solution is:

1. Confirm that my latest daily Retrospect backup worked.
2. Delete old & large files; DiskInventoryX helps a lot to identify them.
3. Quick Defragment in iDefrag (oops, superfluous given later steps). Holy crap, it crashed in the middle of defragmenting! How nervewracking is that!? OK, no more iDefrag. Sheesh.
4. Optimize/repair in Disk Utility
5. Carbon Copy Cloner the disk to an image on another drive (using Target Disk Mode so the drive is not the boot disk)
6. Repartition to make a 2Gb Swap partition at the beginning of the disk (Maybe I could have done this without the cloning step using iPartition, but that’s expensive).
7. Carbon Copy Cloner from the image back to the (now slightly smaller) main partition. This should automatically defrag everything too.
8. Actually move the swap files to that partition using Xupport
9. Quit Mail.app, Delete ~/Mail/Envelope Index, start it up again and rebuild
10. Full Spotlight reindex of the disk (automatic, since CCC deletes the index)
11. Disable unneeded services… e.g., I had MySQL running but I don’t really use it
12. Mail.app prefs: delete messages immediately; etc.