23-Nov-2005 (Wed)
Wherein photos are presented.

In the last month, we have had no less than three drives die, some the day we brought them home. We came frighteningly close to losing three years worth of data. This confirms my opinion that RAID is pretty much worthless: the software will never tell you when one of the drives has failed (even though it's supposed to) and it will never be easy to replace one of the drives (even though it's supposed to). The only reliable way to do backups is to rsync to a different machine, preferably one in a different building.

Oh, and also, Soundweb died. Soundweb is the computer/appliance that runs out sound system, so for the first half of that night, the only sound in the club was what was coming out of the dj's monitors.

Sunspots! Or one of our customers is testing out their HERF gun on us.

Photos are now up of Cabaret Verdalet (a great show as always) and of Android Lust + Equilibrium. I really liked Android Lust a lot! I was also pleasantly surprised that they were a real band, not just laptops.

Check out the "Stained" video on their web site, it's very cool.

Upcoming events worth attending:

      Thu, Nov 25:   Meat: If ever there was a night to go to a club called "Meat", it's Thanksgiving, right?
      Sat, Nov 26:   Pop Roxx.
      Sun, Dec 11:   Ayria: They played here in July 2004. They weren't bad.
Wed, Dec 14:   Conjure One: Rhys Fulber of Front Line Assembly and Delerium.
      Sun, Dec 25:   Death Guild Christmas Night.
Thu, Jan 05:   Specimen: Yes, that Specimen, of "Sharp Teeth Pretty Teeth" and "Kiss Kiss Bang Bang"! Is noted psytrance hippie Olli Wisdom still in this version of the band? I haven't been able to figure that out yet.

12 Responses:

  1. jp4r says:

    Did you contact Creation Audio Labs to rescussitate your soundweb? They're pretty much the go-to guys in North America when it comes to dead soundwebs (and preventive maintenance to avoid them).

    • jwz says:

      Since we couldn't wait, we bought a new one right away. The plan is to get the old one fixed and then try to sell it, since these fuckers cost way too much money to just have a spare sitting around.

      • jp4r says:

        Have you considered buying a cheap DSP like a Behringer DCX2496 (less than $300) that would enable you to get by (barely, though -- in mono most probably, so no go if you have riders to fill) if a soundweb goes down?

  2. cabrius says:

    RAID isn't necessarily worthless, but it does unfortunately have a popular image as being a panacea for all possible data integrity problems. There's still no replacement for a good backup scheme as long as there's still the possibility of environmental disaster ("ide0: controller is on fire"), buggy filesystems, random corruption, accidental deletions, bad luck...

    • jwz says:

      My point is this: any problem that RAID reliably fixes isn't a problem I give a shit about.

      • cacepi says:

        Just out of curiousity, what's the DNA backup regimen? My brother runs a bar, and he asked for my thoughts on it (payroll, receipts, etc.), but I wasn't really sure how to respond. I thought someone with some real experience working in a downright computer-unfriendly environment like a bar might have more insight than just "do it early and often."

        • jwz says:

          Yeah, I really don't have a better answer for you than that one right there.

          Oh, and, "if it all burns down, we've got bigger problems than backups."

  3. edouardp says:

    I was having the RAID vs Backup discussion with one of my friends at (my recently old) work. He was relying on mirroring to save his data, except that raid mirroring is a block level thing, and if the file system decides to corrupt your bits on one disk, the raid system will faithfully replicate that failure to the other disk. Not a good situation to find yourself in. My story to him was about cats, vases, resulting pools of water and my Palm. That wasn't a good situtation either.

    RAID is there to provide uptime, and that's it. It's not a backup solution, because there are a range of failure situations that it will not save you from. That only thing it saves you from is having your system go down *now* because a disk failed. If your running a live webcast, for example, that might be a good thing.

    Having run some proper RAID systems (back in my sysadmin days), a good one with hot spares standing by, hot-swap caddies and proper monitoring software is both reliable and fast. This was for the big commercial UNIX boxes however (the "charges like a wounded bull" end of the market). I've never actually seen a real RAID system for the home PC market - it's all the usual poorly made, sort-of works, half-arsed imitations of the real stuff that passes for "solutions" in that space.

    But, as I'm starting to ramble, even with a real RAID system like that we still did full rotating daily backups, because no matter how much you spent on a RAID system, it wasn't a backup solution.

    The only other (blindingly obvious) information I have to share is that hard drives are incredibly unreliable. A lot of people don't tend to notice when you just have one or two in a PC ("I've never had a disk crash! It's got a 5 year warrenty!") but when you run arrays of them, you know the failure rate is very high.

    And the other thing a lot of sysadmins can tell you from experience, if you buy 5 identical drives on the same day from a vendor (and you often will get sequential serial numbers doing that), if one of those disks fails, the chance of another disk from the same batch failing within the next couple of days is something close to 50%.

    (Esp in a RAID situation - not only where the drives made under exactly identical conditions, they will have had exactly identical disk access for their entire life as well.)

    Backups are good. Of course I was a sysadmin back in the days when a tape was actually cheaper and bigger than a harddrive. I have no idea what the kids do these days...

  4. fantasygoat says:

    We've lost drives on our Xserve X-RAID units a couple of times now. They put up a nice "uh, dude, bad drive" notice and you just swap the drive. Works brilliant, you don't even have to power down - it just repopulates the drive.

    The three-drive Xserve with RAID works just as well but with only three drives, you don't really get the same level of redundancy as with the 14-drive unit, but it's a slightly cheaper no-hassle solution.

    Of course that setup costs like $10,000 so there you go.

  5. I do backups (like, to tape) for a living, and advocate (and rely upon) hardware RAID.

    These things lead me to disagree with you here about zero.

    • bifrosty2k says:

      HW Raid > *
      IMHO software raid is useless and pointless, and is more likely to introduce problems than solve them. I have HW raid everywhere I can, and it really doesn't add that much more cost to a system if you choose wisely.
      For permanent storage of stuff, Tape or Optical media is hard to beat, although tapes are spendy for regular joe-backup usage.

  6. defenestr8r says:

    specimen?!
    i feel REALLY old now.
    well, at least they're not dead.