Feed on
Posts
Comments

Woot new new apartment

I have a new apartment at Riata in the works. I originally didn’t consider it because my coworkers had me believe it was super expensive, but after actually checking it out, it wasn’t bad at all.

I had put down an application fee at another property, and was quite excited about it, but eventually started having second thoughts. I’ve gotten quite used to taking my bike out into the neighborhood here for casual after work cycling, the new place is on top of a bigger hill with no neighborhood access. Riata is flat, has some neighborhood, easy access to Parmer, problem solved.

The new new place should work out pretty well for medium distance cycling too. Sunday afternoon I drove Parmer to see where it would take me. Turns out it takes me 25 miles north over gently rolling hills, with little traffic (once beyond 620) and wide shoulders to nearly Lake Georgetown. Taking it all the way up to RR 2338, then west takes me to a little community called Andice. There and back would be about 60 miles. I haven’t figured out a good route afterwards, since riding along 183 is out of the question, and Hwy 29 isn’t much better. Ideally I’d like to find a 120 mile route up north.

The mechanics finally gave up on my truck after two weeks in the shop. They couldn’t cause a leak and gave me blessing to put radiator sealant in. I picked up some sealant tablets (GM branded, too) from Champion Chevrolet and dropped them in. It’s interesting, because the main ingredients listed on the pack are crushed walnut shells and ginger root. It seems to be working, as I’ve driven almost 3,000 miles and the coolant level hasn’t budged at all. This is even with my 800 miles of 75+ MPH driving to Oklahoma a couple of weekends ago in 100+ F weather.

I finally decided on buying a travel trailer. I had toyed with the idea of buying one a few weeks ago when I needed to rent one for Burning Man. I went as far as going to Buda to a couple of lots and checking them out. I didn’t know what to do with it outside of Burning Man, then realized I could totally rent a spot for it in west Texas and have it as a weekend getaway.

I really like the idea of having a trailer to tinker on, such as lining the roof with solar panels and wiring it up for a wind turbine. I figure 360 W of photovoltaic + 100 W of wind production is clearly a solution looking for a problem. Unfortunately I don’t think it’s feasible to generate the 160 A at 12 VDC to run a 13,500 BTU air conditioner with this.

So settled on the Puma 19 FS, which is a 20′ trailer with a bunk sleeper in the rear. Brand new they seem to be going for $16k, but I found at least one of last year’s model in Corpus for $10k. I need to find out where I can get a loan for that. Interestingly, people totally finance these things for 10-20 years. I guess if you just bought a $100k coach, that might be feasible.

Of all the times I’ve mentioned my trailer idea, more than once somebody has asked me “why don’t you get a popup?” I don’t really know where this comes from. Pop-ups are a horrible idea for BM. They’re tiny, and you’ll be spending a good amount of time inside for a solid week. Not only that, they’re made of clear vinyl and plastic. That’s begging to be a sweltering solar oven in the desert sun. I experienced this firsthand while on the lot in Buda in the middle of a hot Texas afternoon. They were the hottest things on the lot! Oddly, the Airstream trailers were pretty toasty too compared to the traditional trailers and 5th wheels. I decided this would be a good test to see what’d they’d be like with no A/C and no vents open, since it doesn’t get much hotter than here.

Remembering Tulsa

I’ve been reading wiki about Tulsa and it seems to be such a distant time and place. I’ve been back several times since moving to Austin and I’m an outsider with no ties to it anymore. I can barely remember what it was like to live there for four years. I’m sitting here looking at a panoramic photo of Tulsa, and I can’t think of anything I ever did that was uniquely Tulsan. Did I just rot in front of the TV after work each day?

The notable things I do remember about living there:

  • My group of friends, which ran 20-30 people thick at any given time.
  • My eighty mile out-and-back bicycle rides to Keystone Dam.
  • Many hours spent cycling along Riverside.
  • Spending lots of time at Ryan’s house hanging out and/or helping remodel and/or destroy things.
  • Trying to balance running a business along with full time school at TCC and TU.
  • Living with Courtney, then CJ & Jess.
  • Hanging out at Eric’s house, and/or going out with Colleen to Sutures, 71st Depot, or Full Moon Cafe.

I did so, so much driving when I lived there. Outside the random roadtrips to South Carolina, Wisconsin, and then my practically regular commute to Kansas City every other weekend, a lot of driving was for work. Regular weekly trips to Stigler to check on the office. Frequent trips to POP sites to check on gear. I traded in truck #1 (a ’97) at 130,000 miles. Truck #2 (a ’00) got traded in at 80,000. Truck #3 (the ’03) had 50,000 when I moved to Austin. That comes out to 260,000 miles in seven years. Nearly 40,000 miles a year of driving. That’s down to 22,000 miles per year since I moved.

Frankly it’s all a bit depressing to think about. Did I just not do all that much on my own and whittle away my early 20s? I will have to think about what I’m doing now to make sure in five years I can answer “what the hell did I do in Austin” with some level of memory and accomplishment.

Work seems like it consumes so much of my life. When I ran the company, I was always plotting, putting out fires, worrying about this or that. Giganews was great in that I could stop thinking after 6 PM and walk away. The work I left could easily be picked up the next day. Now I’m back to plotting and scheming how to make things better at the current job. I can’t walk away after 7 PM, it’s a series of problems that keep eating at my mind.

It took right at nine months, but I finally finished reading The Snowball: Warren Buffett and the Business of Life. Why, that averages out to 100 pages a month!

Engine coolant drama

The truck has been in the shop all week and I’ve realized it’s a great way to save money and diet. All of a sudden, shopping and cheeseburgers are not convienient. Lots of bike riding in the meantime. This morning I feasted at Taco Shack and loaded my backpack full of groceries from Randalls. This afternoon I went for a cooker ride around the neighborhood. My weight is varying wildly, which I blame on the heat.

Things aren’t looking well for the truck. They still haven’t found the leak, but discovered cylinders on the same side of my busted O2 sensor is running hotter than the other. The concern now is that the initial lifter rattling is a sign that the engine has a sludge problem, due to coolant seeping in somewhere. Even if the leak is fixed, is it too late? We don’t know yet how bad it is, nor if it is recoverable.

I haven’t been able to decide what I want to do with it. I got a couple of quotes on new & used engines. A crate engine+install is around $4200, whereas a used engine with 41k miles is $2900 installed. I keep coming up with different numbers for replacing heads, could be anywhere from $1000 to $3000. At that point it’s worth the extra grand to put a new engine in.

Alternatively, I could just put that money toward buying a new one. I really, really do not want another truck payment. 2009 Silverados are $32k, wtf? 2007-2008s are still in the mid-$20s. I’ve pondered buying a 4WD Z71 if I did get another, since I’ve been in a few spots where that’d be handy. Ironically, I found a 2006 Z71 at a Mercedes/Lexus/Maserati dealer in Dallas that was a trade-in.

Right now I’m leaning toward a new engine, just to get it over with. If the shop comes back and says they can make it go away with a $1k head gasket replacement, that works for me!

I’m anti-vehicle this week. The bicycle is doing very well at getting me around. Unfortunately it takes careful effort to go someplace without being a sweaty, smelly mess. I have a horrible craving for a bacon cheeseburger at Roaring Fork or Trudy’s.

I learned two things this week. The first being the reason why my 2611 broke at home after a day. I had an ACL on my egress interface that blocked DHCP traffic so my router could never renew its lease.

The other was how to get back to my apartment from 360 on a bicycle. If I go to the top of 360 & 183 and take Jollyville south, this meets Mesa which is only 4-5 miles from home via the back way. A mile of gradual climing, then relatively flat on Mesa. It’s a longer distance, but it’s certainly easier than a 202 b.p.m. pulse, crank-bending frontal assault on the hill that is my driveway.

For lovers of storing IP addresses as integers in MySQL tables, there’s a snag when doing this for IPv6 addresses. A 128-bit number is too big to stuff into a BIGINT type. All Fs integer equivalent is 340,282,366,920,938,463,463,374,607,431,768,211,455. An unsigned BIGINT tops out at 18,446,744,073,709,551,615.

These are your options:

  • Store it as a VARCHAR(39) in full ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff form. Which of course, is a horrible idea since you can’t easily sort and it’s prone to garbage-in, garbage-out problems because of the various forms of valid shorthand. (Admittedly, I’ve done this in places with v4 addresses and have almost fully recanted in my code!)
  • Denial. “IPv6 is just a fad.” “I’ll have a new job before I have to deal with it.”
  • Wring your hands. File countless MySQL bug reports asking for native IPv6 and IPv6-capable INET_ATON, INET_NTOA functions. (The former still needs to be added in MySQL 5.x)
  • Fan the flames. Proclaim you never had this problem in PostgreSQL because of native data types that handle IPv6 and CIDR, and all MySQL ninnies should escape their new Oracle overlords and convert.
  • Split up a v6 address at the 64-bit boundary, stuff the two halves into dual BIGINT columns. You could even go as far as saying if one BIGINT column is null, then it must be a v4 address.
  • Store it in a DECIMAL(39). You’ll still need to invent a way to convert the address to integer and back. To do this in perl, Net::IP’s intip() method will at least give you IPv6 -> integer. To get it back, you’ll have to roll your own inet_ntoa6 function. Don’t forget Math::BigInt. Alternatively, here’s some MySQL stored functions for INET_ATON6 and INET_NTOA6. You can either use these directly, or use them as inspiration for your own inet_ntoa6().

Since I have apps that require storing v6 addresses now, I’ve chosen the latter.

5,000 mile 6to4 journey

So here’s a traceroute to the anycast 6to4 address from my parents’ place. It winds up in Frankfurt, Germany! Their upstream, MBO is multihomed to Level3 and Sprint. I assume they’re defaulting to Level3, since the path to the Sprint 6to4 is 120 ms.

traceroute to 192.88.99.1 (192.88.99.1), 64 hops max, 40 byte packets
 1  * * *
 2  192.168.1.1 (192.168.1.1)  2.534 ms  1.691 ms  1.416 ms
 3  invalid.mannford.ok.mbo.net (64.111.243.1)  52.790 ms  50.316 ms  38.353 ms
 4  10.255.2.1 (10.255.2.1)  43.310 ms  50.755 ms  48.458 ms
 5  10.255.0.1 (10.255.0.1)  46.152 ms  44.434 ms  46.582 ms
 6  core7200-1-ser-5-0.mannford.ok.mbo.net (216.150.101.13)  46.222 ms  45.412 ms  51.211 ms
 7  core7513-1.mbo.net (216.150.100.31)  50.435 ms  47.729 ms  50.265 ms
 8  gw-espire.mbo.net (216.150.100.45)  48.338 ms  58.902 ms  48.981 ms
 9  so-10-2.hsa1.Dallas1.Level3.net (209.246.148.209)  59.113 ms  62.695 ms  64.900 ms
10  vlan51.csw1.Dallas1.Level3.net (4.68.122.30)  79.931 ms  66.449 ms  76.519 ms
11  ae-63-63.ebr3.Dallas1.Level3.net (4.69.136.153)  55.804 ms  60.723 ms  71.074 ms
12  ae-7.ebr3.Atlanta2.Level3.net (4.69.134.22)  77.580 ms *  79.353 ms
13  ae-2.ebr1.Washington1.Level3.net (4.69.132.86)  95.636 ms  106.422 ms  98.889 ms
14  ae-91-91.csw4.Washington1.Level3.net (4.69.134.142)  102.153 ms  85.858 ms  92.550 ms
15  ae-92-92.ebr2.Washington1.Level3.net (4.69.134.157)  111.244 ms 113.997 ms  117.641 ms
16  ae-43-43.ebr2.Frankfurt1.Level3.net (4.69.137.57)  181.691 ms 
    ae-42-42.ebr2.Frankfurt1.Level3.net (4.69.137.53)  181.550 ms 
    ae-43-43.ebr2.Frankfurt1.Level3.net (4.69.137.57)  183.324 ms
17  ae-92-92.csw4.Frankfurt1.Level3.net (4.69.140.30)  187.870 ms 
    ae-62-62.csw1.Frankfurt1.Level3.net (4.69.140.18)  181.219 ms 
    ae-72-72.csw2.Frankfurt1.Level3.net (4.69.140.22)  186.367 ms
18  ae-2-79.edge3.Frankfurt1.Level3.net (4.68.23.75)  176.536 ms 
    ae-1-69.edge3.Frankfurt1.Level3.net (4.68.23.11)  179.577 ms  186.361 ms
19  tenge-1-3.cr2.NBG1.content-core.net (212.162.19.34)  193.185 ms 187.075 ms  188.304 ms
20  Tenge1-3-57.cr2.FRA3.content-core.net (212.123.123.202)  196.904 ms 211.622 ms *

A big thing that I see a lot in the content and service provider world, when some outage or negative event happens the line is always “we apologize for the inconvenience.” This is another saying that irritates me to no end. I say this because over the past 36 hours tifa’s host, The Planet, had a couple of outages. I noticed in all their posts they never once mentioned inconvenience.

Every hosting customer of every company I’ve worked for are almost all universally businesses, using our network connections or servers to run their business. When something happens to their infrastructure, it’s not a mere inconvenience. It’s a risk with a measurable negative effect, that is in essence, causing pain to the bottom line.

Going to the store and finding there’s no milk is an inconvienience. Making plans with a friend who later flakes is an inconvienience. Not being able to access your online banking or buy tickets online is an inconvienince to the consumer, but not for the company who’s job is to provide the online services. For some of them, their entire business is built on your mouse clicks, SMS messages, telephone calls and eyeballs.

I personally will not insult your intelligence by apologizing for an inconvienience to your business, if you won’t insult mine by claiming an event stopped you from earning part of your $500 billion annual revenue on your Windows 2000 server (true story).

The corollary to this is, if you really do have a healthy revenue stream that depends on your internet connection and servers, then for god’s sake, invest in it! Let’s be honest, things can and will happen. It’s basic risk management. People will make honest mistakes, automated systems will be mis-programed, power supplies fail. There’s only so much you can do. The technology and know how to help is certainly out there.

I’ve seen far too many solutions riddled with single-points-of-failures that could’ve survived with even basic things such as bonded/teamed NICs in servers or redundant firewalls. Hell, even doing backups and actually practicing restores is worthwhile.

Otherwise, I have zero sympathy when your only Windows 2000 server is down for patches or when human error disconnects something, and you claim you’re losing millions. (another true story)

Learning about IPv6

A few observations with IPv6 fiddling all day long:

IPv6 peering is going to be crazy important. Only a couple of our transit providers, Tiscali and Verizon (f/k/a UUnet) will give us native/dual-stack transit. To help ensure independent connectivity we’ll have to turn up v6 peering at our various peering points. With IPv4 the driving factor is not so much connectivity as reducing transit costs, as we have several different v4 paths to a destination. To use ‘Google over IPv6’, where they will serve up an AAAA record for www.google.com for your network, you had better have good v6 connectivity to Google to ensure it’s reachable at all times.

A /32 is a crazy amount of address space. The popular ISP allocation practice now is to give /48 or /56 to each customer. A /32 is 2^32 (4.2 billion) /64s, or an entire IPv4 internet worth of /64s. And there’s 2^32 /32s.

IPv6 PTR records is a lot of decimal places. Hurricane routed me a /64, for which I delegated reverse DNS to myself. When setting up the zone file, you can’t leave out the trailing zeros. This results in:

[bwann@tifa ~]$ host tifa.ipv6.wann.net
tifa.ipv6.wann.net has IPv6 address 2001:470:1f0f:6c4::2

[bwann@tifa ~]$ host 2001:470:1f0f:6c4::2
2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4.c.6.0.f.0.f.1.0.7.4.0.1.0.0.2.ip6.arpa 
domain name pointer tifa.ipv6.wann.net.

Fortunately since I’m delegated only a /64, that saves typing out 16 digits in the zone file:

2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0  IN     PTR     tifa.ipv6.wann.net.

Having lots of zeros in the address takes out a lot of grief, since you can leave off leading zeros and use :: shorthand.

Since an ASA doesn’t do plain GRE tunnels, I brought home my 2620 and will use that to run a v6 tunnel to HE for my entire home network. I still need to get my hands on an ASA for firewalling purposes, since it handles v6 directly. This should give me a feel for how auto configuration works.

Now I’m trying to find the biggest websites that serve up AAAA records. I keep hoping some nerd at Facebook, MySpace or CNN has a small setup somewhere. Haven’t found anything terribly exciting so far. This leads me to believe we have a lot of work in the content provider space to get content accessible via v6. It’s obviously a chicken/egg scenario, but I think solving the server side is a slightly easier problem.

Dave Meyer gave a somewhat frightening talk at NANOG 45 about how dual stack has failed us. The original model around 2000 was to start assigning IPv4 + IPv6 addresses to everything. Over time more and more things would run dual stack, eventually being the majority, just as IPv4 space was exhausted.

The point he was trying to make was that we should’ve made dual stack a high priority back in 2000-2003, but we didn’t. IPv6 uptake hasn’t happened anywhere near on the scale it needs to, now IPv4 is almost exhausted, which prevents the use of future dual stack!

Instead we’ll start to see “carrier grade NAT” devices come along. A large ISP/cableco/telco could come along, put all their customers behind it and give them all IPv6 addresses. The CGN will take care of IPv4 accessibility. Vendors will love it since they’ll get to sell big iron. But, it may be nearly impossible to rip it out later and restore the natural end-to-end connectivity we’ve come to enjoy and expect.

The vast majority of our customers have load balancers in one shape or another. The problem is, not many load balancers actually support v6; the Cisco CSM and CSS don’t, the ACE maybe. Citrix Netscalers do, but they’re very expensive boxes. A comment I read last night was “do you seriously expect you’ll do so much v6 traffic you need to load balance?” While likely very true, not many of our customers will go for it. I don’t fully understand IPv6 anycasting to know if it’s a possible solution or not.

IPv6 in the home

I don’t know where March and April went, but they’re long gone.

Since we finally put in for a IPv6 allocation at work, I decided it was high time to finally get it working at home. It’s still a bit of voodoo but I finally have a few things working.

My home network is on NAT behind a Cisco PIX 501. This prevents me from using 6to4 or a tunnel broker, because it won’t pass IP protocol 41. I wound up using a Teredo tunnel from my MacBook (using Miredo for OS X) and a Linux box using miredo as well. It feels sort of sketchy because there’s not many public Teredo tunnel servers out there.

Being the nerd I am, after hitting http://ipv6.google.com/, I tried to get on an IPv6 irc server. This was a totally annoying endeavour at least. BitchX on my Linux box didn’t have IPv6 support compiled in, so I needed to rebuild the RPM. Unfortunately the box is running Fedora Core 6 which was apparently taken down from the usual fedora-updates repositories, and I needed a few things like ncurses-devel. Eventually I rebuilt it, then figured out I couldn’t connect.

Giving up, I turned to my MacBook. The Darwin ports of BitchX builds without IPv6 either, and I didn’t want to learn how to manually rebuild the package. I turned to Xirc, which was horrible; it would crash on connection attempt to an IPv6 host. I finally had success with Colloquy!

Then I turned my attention to tifa. She’s not running behind NAT, so I could easily play with my HE tunnels (with the downside of breaking IPv4 and locking myself out). For whatever reason I couldn’t add a ::/0 default route with iproute2, but I could with net-tools. Now I have a nailed up tunnel providing full access to the various services (i.e. web!) running, and a jumping off point for testing more things. Hooray!

« Newer Posts - Older Posts »