Feed on
Posts
Comments

I frequently eat weekend/late night breakfast in diners alone, with iPad in tow to catch up on books I’m reading or browse Wikipedia over few cups of coffee. Despite iPads being out for a couple of years now and practically all over the media, strangers sitting next to me frequently stare inquisitively and ask what it is as if they’ve never seen one. It ALWAYS starts with “What is that?” or “Is that an iPad?” I’ve noticed business and tech people have their own little twist on this, they know what it is but always ask “Is that the newest iPad?”

For whatever reason, sometimes these little conversations derail into batshitcrazy wonderland about random conspiracies and incorrect facts, that are so out there that I don’t even know how to begin to respond. I just have to go back to eating my french toast. It can’t be geography, this even happens in places a mere stone’s throw away from the heart of Silicon Valley.

Two recent examples:

Today somebody asked me if buying ebooks was cheaper than buying real books. We started chatting about how books were a little cheaper and about the convenience of having them delivered electronically. He goes off on how today’s youth expects everything instantly instead of having patience for things. He hates how cell phones are being used for location tracking too. He goes off into the deep end by saying he used to work for BoA and the government installs tracking codes into the mag stripes of credit cards. “Anything you buy, these other corporations can see it. If you go into Wal-Mart and buy a big bottle of drugs, they’re [they == The Man/The System?] going to deny you life insurance forever. That’s why I don’t use credit cards and only pay cash.” I can’t imagine what he’d have to say if he knew I worked at Facebook.

A few Saturdays ago the guy next to me asks, “Is that an iPad?” He starts of talking about how he’d like to get one, but says he won’t because he used to work at Apple on the first Mac, Apple used slave labor now, and they screwed a bunch of people out of jobs by moving the company to Sacramento.

I got nothing.

It also reminds me of a time long ago in 1997 when I was searching for a location to co-locate some equipment for my ISP. I would approach local business owners, explain how much space I needed and how I’d give their business free high-speed Internet access in return (woo 56k frame relay!). One woman after hearing my pitch told me no, because she used to get calls from the internet. “The phone would ring, I’d pick it up and it’d say internet and hang up”. What?

Farewell, Firefox

Firefox is dead to me. In the early days, Firefox was leaps and bounds better than anything else out there, and that was fantastic. Now it feels incredibly slow and every time I turn around I’m being nagged about wanting to apply an upgrade. We’re jumping major version numbers like some sort of Microsoft Office product. Somewhere we went from version 3 to version 9 to version 13. I have an old MacBook Pro maxed out with 2 GB RAM and an SSD, and I basically have to restart FF each day to be able to watch a video or load flash. Even then, it eats all my RAM and works the CPU enough to kick on the fans.

I’ve moved on to Chrome. It’s not a favorite, but I have yet to restart it after two weeks. I can (so far) reliably kill off out of control rendering processes if they do happen to leak a ton of memory. It’s helping stave off upgrading to that new shiny Retina display MacBook Pro, at least for a few weeks!

Ever since the mini-ITX bug bit me this spring, I’ve been intrigued by the little Linux clusters people have been building with them. I absolutely despise loud and bulky home servers (not to mention their energy consumption) and thought this was a great idea to get a lot of computers into a small footprint with little to no noise. My favorite one was Glen Gardner’s mini-cluster as described on mini-itx.com. In my daytime profession I’m constantly dealing with the automation of lots of Linux systems and running distributed apps, so I thought it’d be cool to have my own little setup at home to wrench on. One doesn’t get much experience with load balancing or fault tolerance with a couple of systems. I could’ve just installed a bunch of VMs on a CentOS box (which I’ve already done) or use EC2 instances, but I like getting my hands dirty and felt like building something.

In my setup I wanted it to be more of a general purpose lab, optimize for a greater number of individual systems and not necessarily raw computing power. I wanted local storage for each system so I could someday play with things like HBase/Hadoop. Most of all I wanted it to be silent and consume only a modest amount of energy since it’s likely to be idle a lot. CPU fans may be okay, but so far I’m sticking to fan-less Atom boards.

I discovered picoPSUs, which are DC-DC power supplies, very small and eliminate the rats nest of ATX cabling. Being a ham operator I already had the token bigass 12V Samlex linear power supply laying around so I thought I had power solved. I also already had a stack of SATA hard drives, so this sparked pulling the trigger and buying the rest of the gear. At first I bought a few Shuttle XPCs from Weird Stuff Warehouse for $50 to salvage the motherboards, but turns out they were dead and/or very broken. I also discovered my linear PSU had shorted out and was horribly inefficient (50% ?!) anyways. By now I was sort of committed to the project with half the gear needed. I wound up going with Intel Atom D2500 motherboards and a new 192 W power adapter.

The first idea for mounting was to use threaded bolts between wooden plates to stack motherboards, but this wound up being a horrible idea. If the holes were off the slightest bit, they caused the rods to bind through the motherboard mounting holes. Plus, I still wanted hard drives. I sketched up a design for a little blade enclosure, so each system of motherboard and hard drive(s) could be mounted on a metal blade to be slid into a rack. I decided this was too much work and abandoned the idea.

Fast forward to a couple of weeks ago when I was randomly googling and discovered the “Helmer” linux cluster. Somebody had taken an IKEA HELMER cabinet, cut it up, and stuffed it full of micro-ATX boards. Looking at the dimensions of the drawers I realized they were almost exactly the dimensions of the blades I wanted to build. Then I found another design, and another. It also has the added benefit of being sized to slide underneath a desk to get out of the way. IKEA had solved the chassis problem I had and that weekend I went out to buy one to get hacking.

My design differs from others in that I made doors out of expanded metal to put on the front+back of the case, and went with system “blades” consisting of a mini-ITX board and dual 3.5″ hard drives. My requirements for fanless and quiet dictated that I couldn’t use a bunch of ATX power supplies and needed the airflow for passive cooling. Plus I didn’t want cables spilling out all over the place, both for aesthetic purposes and the fact that my cat loves playing with dangling SATA cables. In the end it really does look like a cabinet you’d find in a datacenter.

The cabinet was designed to hold six drawers (4″ tall) and thus six systems. I wanted to try doubling the density, putting 10-12 systems in the cabinet. This involves putting aluminum angle stock in between the existing rails, providing for 2″ tall systems. For six systems I’ll likely cut down the metal from the drawers and use them for blades, for the remainder I’m using sheets of acrylic. I have no idea at this point how hot things will get and how much the plastic will sag or even melt. I figure this can easily be solved with aluminum cross-members.

Right now I don’t have a real way to turn the boards on/off without shorting the power connector or pulling the plug. I configured the BIOS of each one to always power-on after power loss, and discovered wake-on-LAN works. Each motherboard is also configured to PXE boot first, so if I want to fiddle with a system (e.g. kickstart, memtest) without dragging out a monitor and keyboard I can drop in a PXE config on my tftp server or just let it boot locally.

Overall the project went much smoother than I expected. The HELMER assembled with only a screwdriver and I wasn’t left with bloody fingers from sharp edges like I would a normal 42U cabinet in a datacenter. The only power tool I needed was a drill, I was able to cut all of the expanded metal and aluminum stock with shears and tin snips. (silent construction + apartment = win!) It’s not a cheap project, the cost of the retrofit was basically the same cost as the HELMER and going the picoPSU route was more expensive than an army of ATX PSUs.

So far I only have three blades and an ethernet switch installed and heat doesn’t seem to be an issue yet. It’s virtually silent, although I’ve noticed if the hard drives seek just the right way it causes the door to rattle a bit. Each blade draws 30 W at idle, peaking at 50 W during hard drive spinup or heavy CPU usage, so this limits about 3-4 systems per 192 W PSU. Eventually I may have to give in a add a large (but low rpm!) fan or two on the back door, we’ll see.

If you’re looking for kit, I recommend mini-box.com. They stock several mini-ITX boards, invented the picoPSU, and have some nice simple compact cases. Particularly I like them because they’re local; I can either drive over to pick up parts if I want or USPS delivery is basically next day.

Photos on flickr

Time flies

Looking through old photos I often wonder how I had so much time to do and see things in Texas. Then I realized that I lived there six years and I’ve barely been here a few months. The weekends fly by so fast now.

Hot weather finally arrived and it feels fantastic! Completely clear and sunny, not a speck of cloud in the sky. Yesterday was the first time I wore shorts, and I went to the botanical garden in Berkeley to wander about in the sun all afternoon. It makes me feel really glad I finally moved. I’m told it’ll get cool again soon, but I’m enjoying this now. My jasmine is blooming and between that and the petunias, it makes the balcony smell fresh and outdoorsy. It’s awesome to finally be able to sit outside on a warm night.

Oakland hills firestorm

A couple of weekends ago I happened to catch Richard Misrach’s Oakland-Berkeley firestorm exhibit at the Oakland Museum of California. I was only vaguely aware of the firestorm since it happened in ’91, long before I met anyone here, so it was interesting to learn some local history. Basically something started a fire, and intense, dry hot winds turned it into an extreme wildfire that completely scorched the hills. A curator explained some of the stories he had collected, how some of the staff lost their homes, how people dumped valuable belongings in swimming pools as a last ditch effort to save them. (TIL Wil Wright of SimCity fame lost his house in the fires and used the experience of rebuilding as an inspiration to make The Sims.)

It was amazing to see the level of destruction of the area. There were no smoldering charred ruins left, it was all gone. If it wasn’t brick or metal, it was completely turned to ash and blew away. Cars and bicycles stripped of all rubber, melted tricycles, staircases and decks leading nowhere, trees turned to ashy stumps. In the exhibit hall was a book where people were free to write their stories. One entry written by a child said it looked like “another planet, like the moon”; that was an accurate assessment.

What really made me sad was that among the photographs was one of food+water dishes that somebody had left out presumably for their pets. There were several “missing cat” photos posted and it made me realize that lots of beloved pets lost their lives too. I like to think people were able to evacuate with their pets, but I know there were many people who were unable to return home. There would have been nowhere for animals to run, and I imagine even if the front door were left wide open many pets would be too scared to flee. :(

Shit * Says

@lindvall: Feeling a lot of déjà vu from Shit Silicon Valley Says at Seattle Beta.

This has certainly been an adventurous move, mainly because of how open-ended it was, which then turned into “oh shit, what” when the snow storm hit Seattle. Despite the panic about last minute obstacles, so far they’ve been overcome. I’m not complaining, just what a ride!

I initially put off moving a week because I needed more time to pack and take care of things. At first, because the leasing office at the new place was closed on Sunday, I pondered loading and leaving Friday evening. Then the big snow storm hit Wednesday and buried the PNW under several inches of snow. The roads weren’t looking good so I considered loading Saturday and spending two nights on the road.

Then I realized Oregon and northern California got hit with the storm too and started checking the weather+road conditions there. The Siskiyou Pass in southern Oregon became interesting because it was frequently snowing heavily. At least once I-5 was completely closed and traffic diverted. There was a severe weather hazard Friday with chains mandatory, which cleared off to a warning Saturday so it looked like it was getting better.

By now I was completely packed and was just waiting. I finally decided to pick up the U-Haul on Saturday. Here’s where allll the plans when out the window and I had to start improvising everything. Because I didn’t know when the roads would be clear I hadn’t scheduled movers to help load the truck. I hopped on Yelp and started down the list, asking who was available Sunday. Most were booked weeks out. I found a crew who booked me at 1 PM but said they’d come earlier if they could.

8 AM the next morning I get a call that they’re on their way! I snapped up off the couch and started getting ready. It took a bit longer than I expected, we finished loading the U-Haul after 1 PM. They did a superb job at wrapping my furniture and packing the truck very well. Everything fit into a 14′ truck with room to spare. The only things I left behind were my pile of Rackable servers and closetserver, mainly because I didn’t have sturdy packing materials for them. A few last minute meals and errands I left Redmond at 5 PM. By now there was a travel hazard on the mountain pass because of heavy snowfall. Because I left so late I’d have to spend the night in Medford, OR anyways.

Felix was well behaved on the trip for the most part. He would complain for several minutes after I put him in the carrier and would occasionally get restless and very vocal. All the times I let him loose in the cab, he hid behind the driver’s seat.

The drive took way longer than expected even with generous time estimates. I rolled into Medford at 4 AM, on a very cold morning. Signs all along I-5 warned of severe weather up ahead in Ashland. I holed up in a Motel 6 for a few hours and slept. My previous estimates said I needed to be there by 2 AM, leave at 7 AM, in order to have time to get the lease signed, get keys, and get unloaded all on Monday.

At 8 AM Monday I woke up and checked the pass conditions. The snow pack had broken up and chains weren’t required anymore, so off I went. The mountain looked amazing, it was blanketed by a layer of fog just thick enough to blot out the sun. All of the big evergreen trees were sagging with snow, everything was white. The sand on the pavement was practically the only color. Slow going but eventually made it over.

I passed by Mount Shasta on into Yreka. Every time I’ve been here, it looks different. I’ve been through at night, at winter sunset, summer sunset, and now covered in snow. Again, amazing scenery along Mt Shasta. Felix was being behaving much better today. Initially he refused to go in the carrier and complained for a while after, but eventually he just laid there snoozing in the sun not saying a thing the entire drive. Even on a bumpy road where he was clearly getting bounced around, he still slept like a champ.

On the drive down I was again on Yelp looking up movers and found a crew who could unload the truck at 1 PM. The drive was still taking forever and I didn’t roll into Fremont until 4 PM, so the movers rescheduled for Tuesday morning at 9 AM. I had the night in my empty apartment and it was kind of nice to sprawl out in the emptiness, I didn’t miss my stuff (but worried that somebody would break into the truck on the street).

Tuesday morning we almost got everything in the apartment — except the couch. It was just too big for the sharp turn to get into the doorway. Because the apartment is on the 2nd floor, going up over the balcony would require serious manpower; they estimated at least six men, yikes. Instead I just loaded it back on the truck to deal with later.

Today (Wednesday) I started calling around and found a storage unit, now worrying about who I’m going to get to help me unload this sucker. I had joked earlier about getting day laborers and now it was looking like reality. I decided I could at least get it off the truck by myself, so if I had a trolley I was gold. The storage place had a big elevator and carts where I could just wheel it around, perfect. Initially I started unloading it myself and got it on a cart to wheel it in, then the owner came out to give me a hand. I’m really grateful he did, as it made it much easier; I’m not sure if I could’ve stood it up through the door by myself.

Nowwwww the next chapter in the story is getting back to Redmond to pick up my truck. Did I mention a SECOND drive, this time non-stop?

Moving to California

So I’m moving to California to seek out new opportunities. This should be no surprise to anyone who’s known me, since I’ve rattled on about it incessantly in one form or another (not to mention basically driving/flying down every chance I get). I’ve wanted to live there my entire career since that’s where the action always was and it’s a beautiful place, but it was always a big jump from wherever I was. I almost moved there last summer but instead landed in Redmond; I couldn’t really afford to live there for my remote gig and I didn’t know anyone. Since then things have changed, and I’m in a better position to try it and I’m well motivated.

Finding a new place was tricky. I have no idea where I’d spend the majority of my time there, and it turns out the Bay Area is pretty big. I swear there’s time/distance dilation there. As a random example, traveling one way it only took five minutes to get to my destination, but going the other direction seemed to take 30 (even without traffic). I really love the hills and park system of the East Bay, but any job I work at would very likely be on the peninsula somewhere or in the South Bay. Over New Years vacation I drove around a lot, checking out areas of interests and neighborhoods. At first I thought San Mateo was a good compromise, but the rental prices still made me sick.

On a friend’s advice I looked at Fremont on the day I left and liked what I saw. Really nice area and rentals were relatively cheaaaaap, although nightlife is non-existent from what I can tell. I decided I probably wouldn’t be going into SF all the time so the extra $$$ to live closer wasn’t worth it. Fremont is also a bit sprawl-y, but it’s good since I have a large vehicle. I looked at the traffic maps and decided was acceptable. I flew back down the next weekend to look at more units and finally signed on one. Bonus: it faces Mission Peak. It remains to be seen if this where I want to stay long term, traffic/social/life/work may dictate otherwise, so at least I was able to get a shorter lease for when I want to uproot.

I thought about trying to move this weekend, but looking over what needed to be done and getting quotes from movers, it just wasn’t possible. Plus I wanted to hang out with my peeps more before I split town! Looks like I’ll be moving myself, by the time I rent a truck, drive down, fly back, drive my pickup down, I’m still saving 50% off of what using movers would cost me. It may all be a horrible idea to not use movers, we’ll see! The plan is to move this upcoming weekend, but the details are still in flux.

It’s all exciting and surreal and nerve wracking!

On a less serious note, I miss you Continental Airlines. You never angered me, not even once, in 90,000 miles of travel which is pretty amazing. Being from Austin made it super easy to catch your flights out of Houston. Your express customs lanes and priority baggage loading were so awesome! Even international coach was great and domestic is okay. I decided I’m going to have to make an effort to fly you more (but you’re still a cog in the whole air travel experience). [Well okay, there was that one time you made me run through EWR to catch my flight to AMS. And that one time you delayed me 6 hours going to IAD, but you gave me free first class.]

Alaskan Air, you’re a super easy way to get up and down the coast and have been a life saver at last minute flights but during descent on every flight you make me sit through a solid 3 minutes of scripted marketing crap about your Visa and miles program over the PA that I can’t drown out or ignore. I’ve already paid to get in your airplane, leave me alone. If my SEA->OAK flight is 30 minutes long, that means I spent 10% of it listening to you talk, right?

Delta, I use you because I have to whenever I have to go to Oklahoma. You make me watch a long ass seat-back video of all your employees thanking me, then an annoying smug woman with cheap humor doing the safety lecture, followed by a bunch of commercials that I can’t turn off.

TSA in Tulsa today, wow, you were the most frisky pat down I’ve ever received since you rolled out the millimeter wave machines. I still feel the “back surface” of your hands groping my crotch and running down my leg. shiver. I pondered how fake/real offended I was about it but I’d rather blog about it instead.

American, well, not so much to say good or bad about you. I think some of your planes are older than me.

It was really fantastic to go back home for Thanksgiving, but I’m remembering I can pretend to make a difference by voting with my wallet.

The past 24 hours have been surreal. I unexpectedly had to put Patch to sleep this morning.

Over the past month or two Patch had slowly been losing weight and being more reserved. I always attributed this to Felix bullying her, since he was getting big enough to push her out of the way at her food bowl and would play rough with her. Whatever she was doing, he had to be right there. I thought I was making progress, I gave her food in the bedroom and trained him to leave her alone while she ate. I sprayed him with a water bottle when he played rough with her. I separated them during the day while I was at work to see if that helped, and spent extra time with her alone. I was trying different wet foods and was succeeding in getting her to eat a little meat with the gravy she always loved to lap up. I generously gave her treats which she would always eat, hoping the extra calories would help her gain weight. I was so excited when one night when I was able to coax her into playing, she hadn’t been that playful since getting Felix!

Day-to-day she would always meow-nyaaaah at me as usual, hop up and drink from the sink while I was getting ready in the morning, go out on the balcony, hop in bed, sit on the couch with me, and come out to see my visitors, so it was hard to suspect that anything else was wrong. The only thing different was that she walked a little more cautiously and was in her cat bed more, which is where she usually retreated when the kitten was playing.

Then last week I couldn’t get her interested in treats. Her ears would perk up and she’d come to me when I’d say “Patch, treats!” but she didn’t do more than sniff them. I rarely saw her at the food dish, and I could barely get her to eat more than a tablespoon of gravy at a time from her food. Saturday afternoon I wanted to get to the bottom of it so I made an appointment at the DVM for Monday morning.

Yesterday (Monday) morning the vet agreed she was in rough shape, took some blood for testing, and gave her some antidepressants to help stimulate her appetite and administered some LRS for dehydration. We discussed the possibility of inserting a feeding tube as a last resort if the bloodwork was clean. As soon as I took her home and let her out, she started eating a good size dollop of the A/D pâté they gave me (and she has never showed interest in pâté food). That was very promising.

I had left her alone in the bedroom while I went to work so Felix wouldn’t bother her after the vet visit. When I came home she looked like she hadn’t moved all day long. She hadn’t eaten anything nor could I get her interested in food again. I sat on the bed and put her on my lap, she just laid there resting her head on my leg. This was unusual too since she very rarely laid on my lap. I stayed there with her for a couple of hours and in the meantime I had missed some frantic phone calls from the vet.

The bloodwork came back and it showed she was extremely anemic and there was no sign that she was generating any red blood cells. The vet referred me to another hospital and urged me to go right then, as she may need an emergency blood transfusion and would need the help of an internal medicine team to figure out what was wrong.

Around 9 PM I took Patch to Seattle Veterinary Specialists. After examining her and reviewing test results with an internist, the new diagnoses was very bleak. Something had stopped her bone marrow completely and something was destroying her red blood cells. There were four possible diagnosis for the loss: it could be FIP, a gastrointestinal ulcer, neoplasia, or an immune system deficiency. If it was one type of infection, it could be treated with antibiotics but she’d have it forever and it could flare up again. They said none of these were really a strong fit based on her symptoms, it would take more tests to confirm. Unfortunately, the tests would take several days, if not weeks, to confirm. She’d need a blood transfusion, hospital stay, ultrasounds, and a wide variety of treatments to make it that long. Worse yet, even if correctly identified three of those would be terminal and nothing would guarantee her marrow would start working again. On her own, she would only have a few days.

I couldn’t bear the thought of putting her through a bunch of work and stress just to lose her later. The vet said I didn’t have to do anything right then, I could take her home and decide what to do. I decided to bring her home to have one last night with her to say goodbye. The day started off so different, and now this is how it was ending. I curled up on the bed with her, talked to her, and cried all night long. At this point she was not moving beyond just rolling to lay on her other side. After a couple of hours she still seemed restless so I gave her some of the pain meds they gave me. She laid her head down and slept the rest of the night. I wondered when I’d be taking her back the next day.

This morning (Tuesday) I woke up and she was in even worse shape. She hadn’t moved, she was just sprawled out flat on the comforter not making any effort to remain upright. I was afraid she had already died since she didn’t initially respond to me saying her name or petting her. She finally perked up a bit, but it was clear she was exhausted and didn’t have long. Picking her up, she was completely limp and didn’t offer any resistance. I got dressed and took her back to SVS because I knew it would be a quiet environment and I could get in quickly. She said a few deep throaty “nyaaaaas” on the way to the truck but otherwise didn’t move much. She laid limp in my arms as the doctor administered the drugs. She went quickly and quietly.

It’s been really hard. She was my buddy that kept me company, always laying with me when I was working, sleeping next to me, exploring everything around her. Always well behaved, soft and delicate. When I left the hospital I broke down crying in the parking lot when I reached in my pocket for my keys and felt her collar, still warm from her wearing it ten minutes earlier.

It all feels so sudden. One day she’s pawing at a string, not showing any signs of distress, and now she’s gone. In retrospect, it may not have been behavioral issues leading to her weight loss, she may have been ill for quite some time, or maybe both. It’s possible the reason she was being bullied was because she didn’t have the energy to fight back. I feel guilty for not taking her in sooner, but I’m told it wouldn’t have really mattered; she’d still have a terminal affliction and the goal would be to keep her comfortable for however long she had. I guess at least I’d known it was coming. I hope I gave her all the attention she wanted. It feels weird walking into to the bedroom and not seeing her happily snoozing on the foot of the bed or on a pillow. When I scan the living room I expect her to be on her red cat bed in the corner watching me or on top of the cat tree, and she’s not there.

Rest in peace my sweet kitty, I love you and I’ll miss you.

« Newer Posts - Older Posts »