Feed on
Posts
Comments

Through a long story of poor planning and choices, I recently found myself trying to get home in the middle of the night in the suburbs without my vehicle and with a dead iPhone. In the beginning the idea was to just summon an Uber or Lyft to take me home. I realized my battery was getting really low (thanks old iPhone 6) and I didn’t quite know exactly where I was, so I quickly fired up Google Maps to at least orient myself. Then my phone completely died and I was on my own. I thought “ok, fine, I’ll just walk down the street to find a gas station and call a cab.” After a very long walk out of the neighborhood and down a street I recognized toward home, I looked for a gas station or at least familiar territory. Boy was I wrong.

Nobody has pay phones nor yellow pages anymore. We all know this, but it sure would’ve been nice at the time. I found a station and their payphone was out so I asked the cashier to use their phone. I fumbled around with automated 411 saying “taxi”, “yellow taxi”, “checker taxi”, to try to find a taxi company. By now it was around 5 a.m.

I quickly realized not all of the taxi companies in the search results have 24/7 dispatch, as several calls went to voicemail. Of the two that did answer, one wouldn’t pick me up unless I was at the airport, and the other just didn’t care about picking me up that early. On top of this, I realized I had walked a terribly long ways in the wrong direction (I later figured out it was 4 miles), thinking I was going toward home but instead walking away from it. (Next time double check where Polaris is!)

Finally a very gracious random customer heard my taxi-calling plight and offered to summon an Uber ride on her own phone and dime to get me to BART. I had plenty of cash on me and offered to pay her for the trip, but surprisingly she declined and told me to just buy her coffee and help somebody else out in the future. Such an awesome thing!

It didn’t even occur to me until the ride home that I should’ve just bought a damn phone charger at one of the gas stations, hijacked an outlet somewhere to charge my phone for a bit, and get my own ride home.

Lesson learned, buy a phone charger instead of trying to call a cab, and the generosity of complete strangers does still exist.

Milky Way and Jupiter over Death Valley

[photos: flickr – Death Valley snow, stars, Ballarat]

[photos: flickr – Death Valley jets]

I bought myself a new wide-angle lens for my birthday this week and it was a new moon phase, so I used it as an excuse to go out to Death Valley again and make photos of stars last weekend. A storm was passing over central California on the way out there, which resulted in blowing snow flurries through Tehachapi and fresh accumulations of powder at higher elevations in the park. Fortunately the clouds had almost all cleared out by the time I got to Stovepipe Wells around midnight.

I set up at Mesquite Dunes for a while but got tired of the cars driving by and headlights getting in my shots. I wound up driving over to the actual stovepipe well site and spent the night there taking photos over the valley. I was sort of surprised at how low on the horizon the Milky Way was, but it started rising and I was able to get good photos of it. The 16-35mm lens does a great job of capturing a huge swath of sky at once.

What I wanted to try was getting photos of geostationary satellites. I had seen a couple of people do it, they should appear as stationary specks against star trails. I aimed south, eyeballed the elevation, and took several photos with around a minute or two of exposure. I’m still not sure if it actually worked, I can’t tell if I actually saw some or if they were just sensor noise, even compared to frames with the lens cap on. I for sure captured some LEO satellites on north-south orbits, they appeared as regular streaks running perpendicular to the star trails. Still need to do some more homework on this.

Snow at Towne Pass

Sunday morning I headed up to Towne Pass. There was still snow everywhere on the vegetation, it looked like giant cotton fields. I wasn’t the only one with this idea, several other people pulled over on the side of the road to play in the snow and take photos. I drove out to Wildrose and the charcoal kilns, there was still snow on the ground up at the higher elevation although it was melting off quick. The road up to Thorndike was gated off after the kilns. The wind would occasionally pick up and kick up snow off the trees which made it felt like it was snowing all over again.

After this I went to find somewhere quiet to go take a nap. I practically pulled an all-nighter Saturday night and wanted some zzz. I wound up going part of the way to Grotto Canyon after figuring out where the road was. In the evening I wanted to out to the Racetrack to take more night photos. The road out there proved to be way rougher than the last time I went, so I turned around after a couple of miles. It was so much bouncing around I lost the whip out of my screwdriver antenna somewhere along the way. The weather wasn’t cooperating either, yesterday the forecast said it would be clear, but clouds had rolled in. I drove back down to Stovepipe Wells for dinner and camped out there again. By 4-5 AM the clouds had cleared off but I was too cold and sleepy to be bothered.

Panamint Valley radar site

Monday I was hoping to go photograph some jets. They weren’t flying over Stovepipe Wells nor Panamint Valley at the time, so I pondered just driving home. I wandered down to Ballarat, I hadn’t really ever explored this side of the valley. I thought it was a complete ghost town but it turns out there’s one dude who lives there, and he has a little general store and lot for RV camping. Otherwise there was still old buildings and vehicle ruins around. From here the Barker Ranch where the Manson family hid out wasn’t too far away, but I didn’t attempt the trip. Along the road was a radar site, presumably to support China Lake activities. On the way out I heard the roar of jets coming from the end of the valley so instead of going home via the south, I headed up to Father Crowley overlook.

F/A-18 Hornet in Rainbow Canyon

There were many people standing along the ridges above Panamint Valley and at the overlook. I hadn’t heard anything flying around, but after 30-40 minutes of waiting one jet flew down through Rainbow Canyon. After another hour or so, two more showed up and were flying overhead. The pace picked up a bit after that, I think I saw about 12 passes going through the canyon or overhead. The ridge next to the rest area had great views of the jets flying through the canyon down below. Several people were there with their long lenses and many others stopped by to watch. It was pretty exciting, the jets were loud, up close and personal. I had a 200mm lens which did a good job after cropping the images.

After 5 PM I packed up and went down to Panamint Springs Resort for dinner, I’d never been there and wanted to give it a try. They make a tasty grilled cheeseburger, even better than what I get over at Stovepipe Wells. It was pretty nice there, a good view with plenty of outdoor seating to watch the landscape. I left and got home around 1 AM, not too bad.

Panamint Springs Resort

Falcon 9 at SLC-4 at night

Actually my first time watching any sort of rocket launch in person. A friend urged me to go watch one of the Space Shuttle launches before the program ended but I never did. (I did get to see the Shuttle Endeavor flyover at Facebook in 2015). I don’t know why I haven’t been down to Vandenberg sooner to watch a SpaceX launch. It’s not terribly far away and it would’ve been easy to take a day off if I needed to. I guess just watching the webcast was easier.

After the maiden flight of the Falcon Heavy I decided to finally go watch one of these things. I had my eye on the SpaceX-Paz launch originally scheduled for the 17th or 18th. I was going to drive down to Lompoc as well as do some exploring of the central California coast. I spent a while studying maps and trip reports from other people to figure out where to go. This page was written by a SpaceX employee was a good resource on making the trip. The night before I was supposed to leave I came down with a terrible cold, didn’t sleep, generally felt like shit and didn’t think I was going to go. Fortunately the launch was delayed until the 21st!

I drove down Tuesday afternoon. It was dark when I got there, but coming in from the north it was super obvious from the bright glow on the horizon where the launch pad was. I was positively giddy when I could see the Falcon 9 sitting on the pad in the distance. I drove out to Surf Beach to get an idea of where things were. The road is gated off beyond the beach, but from there it was a great view of the SLC-4 facility and the Falcon 9 on the pad.

The launch was scheduled for 6:17 AM the next morning so I crashed at a Motel 6 for a few hours. The guide page recommended leaving an hour early to find a good spot, so I left a little before 5 AM. There were already several cars on the shoulder at Ocean Ave & 13th, and that’s where I parked. It was fucking cold, right at 32 F. My puffy ECWCS parka pants came in handy here. I had my HT tuned to 386.3 MHz and was able to pick up the Launch Net just fine. I had decent-ish cell reception so I kept tabs on the Reddit launch page and Twitter.

As it got closer to T-0, it was quickly getting brighter from the sun rising. I had my camera on a tripod, I figured I could get some sort of long exposure shot out of it but didn’t really know what to expect at all. Finally word came over the radio that due to high upper level winds the launch was scrubbed until tomorrow. Bummer.

After the launch was scrubbed I headed back out to Surf Beach. Now there was an Air Force guard parked inside the gate. I took a few more pictures of the rocket, then wandered around on the beach for a while. It turns out when you get up at 5 AM you have a lot of time to do things, I had a whole day to waste. I went back to the motel and got another night, then headed down Highway 1 to Santa Barbra. I’ve never been been there so figured why not. I had fun exploring the coast and took a bunch of pictures. Driving down the entire Highway 1 along the coast is one of my to-do items, so I wound up driving all the way down to Santa Monica and back.

(Why are the freeways in southern California so much nicer and cleaner than ours? White concrete, palm trees, everything just seems clean. The freeways around the Bay Area are busted, patched, dirty, all the signs are dirty, all the medians and intersections are littered with trash and dead homeless people. Also they have many miles of pretty blue ocean, we have the gritty brown Bay that stinks in areas.)

Mission stickers at The Hitching Post

Wednesday night I drove out to Casmalia to have dinner at the Hitching Post, something recommended on the Arring.net page. I had a great top sirloin steak, I can recommend this place. They have one section of dining room that has all sorts of signed aerospace posters and collections of mission patches and stickers hanging up, from all the crews working at the base.

 

Thursday morning I got up a little earlier and left at 4:45 AM. This time there were maybe a dozen cars already pulled off on the side of the road. It was still freezing outside, but still clear. Listening to the radio I could hear things getting closer, then finally the countdown! From where I was I couldn’t see the launch pad for the hill. Then it happened, there was a red glow over the horizon and the Falcon 9 peeked over the top. What I wasn’t expecting was how bright the engine flames were. It just silently and gradually went up in the air, then several seconds later there was a soft rumble that turned into a giant roar! I barely remember the sight of the rocket, but the image of the engine plumes are burned (heh heh) into my memory the most.

Ignition!

I also had no idea what to expect as far as how high it was going to go or if it would curve away. From where I was at, it went straight up really far and I had to tilt my head back to watch it. By now it was bright outside, so my long exposure photo idea didn’t work out. The few photos I did take were greatly overexposed and useless. The 24 mm lens wasn’t anywhere wide enough to capture the entire flight because it went up so high. I didn’t bother swapping out lenses to take anymore photos, I just wanted to watch it with my own eyes and take it in.

Long exposure of launch

The rocket got smaller and smaller, and the engine plume got smaller and smaller as it went further away. I could see the main engine cut off, then see the second state light. It was this tiny puffy spec that eventually vanished. There were 4 things clearly visible after MECO, bright little pinpricks of light clumped close together like morning stars. I assume these were the booster, the two halves of the fairing, and the interstage. They were coming down very slowly, and eventually stopped being visible. The Launch Net over the radio was still reading out how the second stage was going. Soon there was nothing left to see, so I packed up. Amusingly there was a coating of frost on my camera tripod and chair when I picked them up.

 

Titan mural in downown Lompoc

I went back to the motel and napped for a couple of hours. I headed home after that, following highway 1 and 101 back up. One interesting thing about Lompoc is that they have murals all over downtown, and many are space related. I had the idea that I would visit Hearst Castle on the way back, but I was pretty tired from lack of sleep and decided to pass it up.

Takeaways so I don’t forget:

  • 386.3 MHz is useful to listen to. It appears to go online an hour before liftoff.
  • Ocean Ave has the most unobstructed view, cars can really pack in there. The farm roads on the other side are also good spots.
  • Get there at least an hour early for pick of the best spot
  • A wide angle lens isn’t going to cut it here
  • The rocket goes up super high

The next launch is March 28th, hoping to make that one.

The 16th of January I drove out to Death Valley for a few days. Last time I camped up at Wildrose, this time I camped at the NPS campground at Stovepipe Wells. The weather was perfect, a little cool at night, and a little warm during the day. Really not much point to the trip, I just wanted to go somewhere. It was a new moon so it was absolutely dark, save for all the stars.

The first night I spent the night at Mesquite Dunes parking lot letting some long exposure photos go. The next day, Wednesday, I setup my tent at the Stovepipe Wells campground. This was just after MLK Day, I’m told the place had just emptied out. What I wasn’t expecting was the fighter jets flying overhead. Sometimes they would start up right at 7am and sometimes they’d be around in the afternoon. I had problems sleeping, so I’d sleep in what I could, then go lay in the truck bed for a while. I’d go over to the SPW restaurant for their breakfast buffet, which beat cooking for myself.

I had brought along my new Yaesu FT-817, RigBlaster, and Buddipole. I set them up and tried to work HF digital on 20 meters but wasn’t having much luck. I was picking up tons of contacts, but I wasn’t able to get out. I suspect it was the antenna configuration being wrong, and not really knowing how to work all the gear yet. Still more manuals to read before trying again.

The second night I was there, there was a good fire going at the group fire ring at the NPS campground. Some campers and SPW employees were burning some lumber and pallets, so we spent several hours hanging out there shooting the shit while the fire burned.

 

Thursday the 18th I decided to drive over to Furnace Creek and Badwater. I was surprised to see the store, saloon, and cafe at Furnace Creek ranch had been torn down and were being rebuilt. I’m glad I didn’t decide to go hang out over there instead.

After a while I took off down the West Side Road, which was a bit of a mistake. The road was so washboarded, it shook me and the truck to death. Some spots were smooth riding, but the rest was terrible. I drove down to the road leading over to Hanaupah Canyon and turned around. If I had a jeep I probably would’ve went further.

That evening on the way back I stopped at the Mesquite Dunes right at sunset and took some photos of the dunes and colors. I had two bundles of firewood so I lit them up and hung out at my campsite. The jets were even flying at night, looking like they were flying somewhere from the East.

 

Bright and early the next day the fighter jets were flying over at 8 AM. This time they were lower and seemed to be doing some dogfighting. I kind of had enough after three nights so I started driving home. Plus there was a storm rolling in so I didn’t know if that would give me snow to drive through or not. Doing some reading about the jets I found out that not only were they easily seen in the Panamint Valley, but the Father Crowley Overlook/Vista Point was the popular spot to watch them.

From the parking lot there, there’s a dirt road that leads off closer to the canyon. There was a great view overlooking the valley floor and a big canyon right to the side of me. As I was driving in, a jet came shooting out of the canyon right next to the overlook. It was incredibly close and incredibly loud, I felt it rattling the truck. This is clearly the place to be in order to see jets up close and personal. There was also a great view of the Panamint Valley below, so it should afford a good view of jets zooming through there. I waited around for 15-20 minutes in the wind, but nothing else was flying around.

 

I’m sort of glad I left when I did because the next day the Federal government shutdown. It’s unclear what would’ve been affected, I recall last time that some of the roads to like Badwater were closed down.

Next time I go I want to explore the area around Trona. After reading the “Hunt for the Death Valley Germans”, it gave me new ideas of where to explore.

Roadtrips have been kinda light the past few months, I went out to Stovepipe Wells in Death Valley a couple of weeks ago. I should write about it, but I’ve forgotten most of what I did.

This story happened last summer as I was driving across Nevada in the middle of the night. So far I haven’t been sued, summoned, nor contacted about it at all, and haven’t seen it in the news so I guess things turned out okay.

Around 11:30 PM at night I’m cruising over highway 50 listening to music, not suspecting anything. It’s dark and it’s the middle of nowhere, I don’t think I’ve seen another car on the road in hours. This part of Nevada is amazingly sparse and not much is here. I come over a little rise and corner, all of a sudden there’s a guy standing in the other lane of the highway furiously waving a road flare over his head. The thought that this could be some sort of robbery or shenanigans does cross my mind, but I stop anyway because this dude is in the middle of nowhere.

He immediately tells me that he and his girlfriend wrecked their car. They don’t have cell service, she has some sort of medical condition, and they need to call for an ambulance. I don’t have any cell service either, then I hear and see his girlfriend laying on a blanket on the shoulder of the highway. She’s moaning for somebody to get help, and he’s obviously pretty worked up about it. I don’t see the car, it’s presumably rolled off into a ditch, but I can kind of tell where it went off the road. I don’t know how long they’ve been out there, if they told me I forgot. Immediately panic sets in and I don’t know what to do. Hell, I don’t even know where I’m at or what the closest town is. If I have to go for help, do I go back to the small town of Eureka I just passed through however many miles ago, or continue on to the next town?

With no cell service my maps on my phone are useless. I have my old Garmin StreetPilot running with it’s circa-2006 maps and it gives me latitude/longitude that I guess I could give to somebody, but it still doesn’t really me a good idea of where I’m at. Finally I realized I can search for nearby towns and it tells me Eureka is 6.5 miles away, Ely was 70+ miles still to go. I tell him I’ll go back into town and call 911, but he wants to go with me. The girl has limped over to the truck and he wants to put her in the truck bed. So far he’s not acting like he has any injuries, but she’s complaining she’s in pain from head to toe. I tell them okay and he loads her into the bed of the truck, and he gets in and lays down with her. It’s chilly outside so it’s going to be a cold ride. They have one blanket and I get one of my blankets for them, and off we go.

 

This happened well before I took the WFR (wilderness first responder) course, so I have no idea what injuries to be aware of and driving them back probably wasn’t a great idea. Panic was still going strong, I have no idea what I’m doing. At least neither one of them are gushing blood, they’re both conscious, have some bruises and scrapes, and able to walk. Nevertheless they didn’t want to be left in the middle of nowhere while waiting for help, it seemed like a reasonable request.

 

When I first drove through Eureka I actually made a mental note of how nice their fire station is. Despite being a small town (of around 600), it had several bays and looked well kept. On the edge of the town was a highway patrol office. No idea if there’s a hospital. I decide to take them to the fire station because maybe they have people on call inside, and maybe a paramedic. We got there probably 8-10 minutes after I picked them up, and I called 911 on my cell phone from the fire station driveway.

I tell the dispatcher where I’m at and I’ve got a girl who’s been injured and wants an ambulance. They want to know where the wreck was, was it a rollover, and how many people were involved. While I’m on the phone about 5-10 minutes later one of their EMTs arrives in his truck. I think he’s pretty surprised they’re in the bed of my truck and that I brought them here. The girl is agitated about everything, being cold, her legs being cramped up in the truck bed, and doesn’t want the EMT to touch her because she’s in pain. He’s very calm and matter of fact, he tells her something like “I can help you but it’s going to hurt. Just say ‘ow’ when it hurts and don’t try to hit me.”

A few minutes later the sheriff and a couple more EMTs drive up. It turns out their ambulance bay is across the street, so one goes over and brings an ambulance to the driveway. The sheriff asks me to explain what was going on and where the wreck was. The first EMT puts me and the sheriff to work stabilizing the guy’s head while the other two get a gurney out of the ambulance. They get into the back of the truck take over checking over the couple. I hear the guy say he fell asleep while driving and rolled the car. He’s been laying here quietly the whole time and seems to be dozing off after the ambulance arrived. He’d wake up every now and then and ask a question, or people would have to nudge him awake to get an answer.

I hear on the radio they’re trying to decide where to send them. One destination is Elko to the north and the other is Salt Lake City. They’re trying to decide if both will go in the same ambulance, and if they are going to request an airplane or helicopter to fly them out. From what I understood of it they were going to Salt Lake, which is kind of surprising because it’s a couple hundred miles away. They get the girl and guy strapped to backboards and we help load them into the ambulance.

 

It’s now probably 12:30am-1:30am. The sheriff gets all of my contact information and leaves to go check out the wreck. The first EMT that showed up is chatting with me, he tells me it would’ve been better if I had left them at the accident site, but I did a good job and did what I had to do, and thanked me for bringing them in. I mentioned I was going to take a wilderness first aid course soon to hopefully know what to do. He told me he was going to give me some advice for taking the course: there are a lot of things you just have to do that aren’t in the book, like getting patients out of the back of pickups. He asks where I’m headed, I tell him heading out to Great Basin NP. He tells me to be careful as there are lots of critters on the road at night and to please stop if I feel tired.

Back on the road to Ely, I pass by the accident site and see sheriff’s car on the side of the road with all of its lights on. I stop and look, but there’s not much else I can do. Another patrol car arrives around the same time so I leave. Several miles down the highway I meet another patrol car with its lights and siren on, heading at high speed presumably to the wreck site. I was paranoid that I could be contacted about all this in the future if something went wrong, so I pulled over and jotted over a bunch of notes about what happened before I forgot.

So far that’s been the end of it. I haven’t heard anything about it at all, haven’t seen it mentioned in any local news. I’m curious what ever happened to them.

During my WFR class I ask my instructors about it, who are WEMTs. They said due to all the unknowns the best thing would’ve been to leave them where they were, make them warm, give them a full assessment that we were learning about. Ultimately it was up to me because it was my vehicle, I didn’t have to transport them if I didn’t want to. I certainly thought about this whole experience a lot during the WFR course.

I renewed my mail server’s TLS certs the other day and noticed that Alpine had problems verifying my certificate chain. (Yes I still use a text based email).

(Of course after I figured all of this out and changed my Google queries a bit, I found where other people hit the same problem back in 2016 and came to the same conclusion.)

When I ran openssl against my IMAP-SSL server on port 993, the various intermediate certs were okay but I saw this message:

139713136498592:error:14082174:SSL routines:ssl3_check_cert_and_algorithm:dh key too small:s3_clnt.c:3636:

Likewise in the courier-imap log I saw this corresponding entry:

imapd-ssl: couriertls: accept: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

Puzzled in that my other Mail clients like on OS X and my iPhone work just fine, and there isn’t a way to twiddle with forcing SSL/TLS protocols that Alpine uses, I started randomly fiddling knobs.

One thing I learned out of all of this that you can actually use curl to speak to an IMAP server. I tried that to see if it gave me any extra insight and it kind of did:

$ curl "imaps://bwann@example.com"
curl: (35) SSL received a weak ephemeral Diffie-Hellman key in Server Key Exchange handshake message.

This lead me to go fiddle with the dhparams.pem file that Courier IMAP depends on. The monthly cron job that runs /usr/lib/courier-imap/sbin/mkdhparams generates Diffie-Helman parameters using a 768-bit key. This seemed sort of puny, so I edited BITS=768 to say BITS=2048, blew away the old dhparams.pem file and generated a new one with 2048 bits.

Now when I run curl without a password, I get an auth failure instead of an SSL error. Ah hah!

$ curl "imaps://bwann@example.com"
curl: (67) Authentication failed: 78

Now Alpine works just fine.

A/K/A Ubuntu for CentOS kickstart users

I needed to install Ubuntu on a server so I finally got around to figuring out how to do an automated, unattended installation. Specifically, how to install Ubuntu on bare-metal over the network using an existing PXE setup in an all-CentOS environment, as you would in a datacenter. I will attempt to compare CentOS+kickstart installations vs Ubuntu+preseed installations based on my experience. There are some initial similarities in how things are setup, but not too many.

This is not a how-to, but a bridge to get started with Ubuntu. I’ve linked to any scripts I’ve used and where I originally found them.

TL;DR: Overall I’m not impressed with Ubuntu’s way of doing things. While at the end of the day you get an OS on disk, it takes a lot more clever work if you want to add any sort of local customizations particularly around hardware. CentOS/kickstart installations provide much more flexibility to run more things before and during installation, and easier to get in and troubleshoot. Kickstart becomes very handy if you need to setup things (via script, not by hand) beforehand like hardware RAID, flash cards, or conditional partitioning of disks.

PXE / netboot setup

Setting up your PXE environment for Ubuntu is basically identical to handling CentOS kickstarts. You serve the Ubuntu kernel+netboot initrd via HTTP or TFTP, save the same sort of config file to your bootloader of choice (PXELINUX, iPXE, etc), and tweak some different kernel command line options, and that’s it.

Setting up a local release mirror (e.g. Xenial)

(Disclaimer: I’m serving Ubuntu releases and repositories from a CentOS system)

A local release mirror is similar to CentOS is that once you get the files on disk, they’re served up from your HTTP server to clients the same way. Creating an Ubuntu/apt mirror on a CentOS system takes some work because obviously there’s different package managers and layouts involved. You could rsync the entire upstream Ubuntu repo, but this seems to include a lot of non-used stuff. There’s an RPM (in EPEL) and perl script called debmirror that takes care of using dpkg/apt tools and copying the bits you want onto your local repo server.

However there’s a snag with debmirror out of the box that I found. On CentOS, the RPM lays down both the script and a configuration file in /etc/debmirror.conf. It defaults to wanting to download the i386 version of the sid distribution. The comments in the config about command line options overriding the config are complete lies. Arguments are *appended* to what’s in debmirror.conf, so you wind up getting more than what you want.

The problem here (if you’re not aware of that config), is that debmirror will happily mirror your requested release, but then it also tries to download and do GPG checks of the sid release. Because you don’t have sid, it errors on a GPG check failure. Of course the Internet is stupid and says “just disable GPG checks”, instead of actually changing /etc/debmirror.conf.

Setting up local APT packge repositories

This is a local repository where you keep all of your own/third-party DEB packages (e.g. Chef client), seperate from the upstream (e.g. Xenial) release files.

Again, once you have a repository created it’s identical to a local CentOS mirror and served up over HTTP. To generate the package metadata on CentOS, you’d run createrepo. To create Ubuntu package metadata, you’re left to your own devices. There’s a decent shell script[1] out there called reindex_stable.sh which takes care of using dpkg tools to create all of the package indexes and generate SHA checksums. I’ve put my version of this up on Github[2].

Once you have the reindex script working it’s basically the same as running createrepo for CentOS.

  1. http://troubleshootingrange.blogspot.com/2012/09/hosting-simple-apt-repository-on-centos.html
  2. https://github.com/bwann/ubuntu/blob/master/reindex_stable.sh

Installer and configuration

Ok, this is the meat of the installation process. “Preseed” files are Ubuntu’s equivalent to kickstart config files. They answer installer questions as if you were going through them on console. Like kickstart configs, they can be served up over HTTP so the installer can download it.

Preseed files vs kickstart

Technically Ubuntu does let you use kickstart files[1], but it doesn’t support all the directives and options. Under the hood it’s just running shell scripts[2] to transform kickstart directives to preseed directives. Rather than try to shoehorn my existing kickstart configs to meet Ubuntu’s need, I just decided to go native and create my own preseed files.

The internet is littered with copypasta preseed files, just like random modem init string collections from the 90s. I’m not convinced people know exactly what options they’re using and why.  An example of a basic preseed I used up with is up on github[3].

Pros of preseed files:

  • They’re pretty short and succinct, mainly because there’s not a lot of heavy lifting you can do with them.
  • Every possible configuration option available and set during installation will be written to /var/log/installer/cdebconf/questions.dat on an installed system
  • There’s at least an option to change how filesystems are mounted, via UUID, label, or device names. UUIDs suck, use labels.

Cons of preseed files:

  • The closest equivalent to kickstart’s %pre/%pre-install/%post sections are preseed/early_command, partman/early_command, preseed/late_command. You can only have a single command directive in the preseed file, so if you have a bunch of commands to run (or god forbid a script), you have to end each line with a semicolon and backslash. The entire line is consumed as one line, ugh. Or, you’re going to have to use this to download a script holding your bigger scripts.
  • preseed/late_command is the equivalent to “%post --nochroot“. If you want to run commands within the target install, you have to use the in-target wrapper as part of the late_command string.
  1. https://help.ubuntu.com/community/KickstartCompatibility
  2. http://bazaar.launchpad.net/~ubuntu-installer/kickseed/master/files/head:/handlers/
  3. https://github.com/bwann/ubuntu/blob/master/base-preseed16.cfg

SSH to an installing system

One of the big things I miss with Ubuntu’s installer is being able to SSH into a host while it’s being installed. On CentOS, you can pass the inst.sshd option and set a password with sshpw in a kickstart file. This lets you log in while Anaconda is doing its thing to look at logs, troubleshoot your pre/post scripts, attach to the tmux session Anaconda is using, or have your build system interrogate the system to track its process, among other things.

Ubuntu has “network-console” you can enable, but this only starts an interactive installer session when you ssh in. The menu will let you launch a shell and look at logs, but it’s no longer an unattended installation. I get it, I can see the use case, but it’s not what I want. There’s not an easy way to hack this into Ubuntu without some heroic preseed/early_command setup. It’s probably best to re-build your own installer image with OpenSSH, config, and a key baked into it. Otherwise your only option to check on an unattended installation progress is to hop on VGA or serial console.

Bugs!

I hit at least two installer bugs that forced me to change how I did things (as of November 2018):

 

This about wraps up my experience with Ubuntu installs. I’ve installed exactly one system (many times) so I don’t know of any other tricks to throw at it.

It turns out when you run debmirror on CentOS 7 from EPEL, it sneakily (not really) lays down a configuration file in /etc/debmirror.conf. Despite the ‘-d‘ argument, e.g. ‘-d xenial,xenial-security,xenial-updates‘, it happily tries to mirror the sid distribution. I saw other people having this problems and it was maddeningly to finally figure it out at 2 AM. So here’s to all you intrepid Google searchers:

Running debmirror with options something like this:

debmirror -a amd64 --no-source -s main,restricted,universe,multiverse \
  -h archive.ubuntu.com -r /ubuntu --progress -e rsync \
  -d xenial,xenial-security,xenial-updates /mnt/dist/ubuntu/16.04.3/amd64

Does this:

*** Processing arch: amd64
Mirroring to /mnt/dist/ubuntu/16.04.3/amd64 from rsync://anonymous@archive.ubuntu.com/ubuntu/
Arches: amd64,amd64
Dists: sid,xenial,xenial-security,xenial-updates
Sections: main,main/debian-installer,contrib,non-free,main,restricted,universe,multiverse
Pdiff mode: use
Download at most 200 files per rsync call.
Will clean up after mirroring.
Attempting to get lock ...

And it finally dies downloading dists/sid/Release even though we didn’t specify it on the command line.

Ubuntu Release file: using Suite (xenial-updates).
Errors:
 Download of dists/sid/Release failed
Failed to download some Release or Release.gpg files!

The sid coming from /etc/debmirror.conf (the comment at the top about overrides is fucking lies):

# Options specified on the command line override settings in the config
# files.

@dists="sid";
@sections="main,main/debian-installer,contrib,non-free";
@arches="i386";

So change this to whichever dists and architectures you want, or comment these out of the config files.

I haven’t figured out a proper patch/config for this, at this point I’m not sure I really care this much. The internet seems to thing disabling signature checks is a good idea, and they’re WRONG. This is important to ensure integrity of your downloads, to ensure you’re getting the right goods and they’re completely downloaded.

If I stick fiddling with Ubuntu maybe I’ll come out with a better config.

Edit: I learned much, much later that sid is actually the name of the head/upstream/unstable Debian repo, similar to Fedora Rawhide. So the script wasn’t trying to mirror some particular ancient repo, but the latest unstable repo.

Titan Missile Museum

9-megaton warhead on a Titan missile

I forget how I came across it, probably Reddit, but I watched the movie Command and Control. It was a documentary focused on an accident in 1980 where a Titan II missile exploded in the silo outside of Damascus, Arkansas (wiki article). I was surprised I’ve never heard of the incident before because I grew up not far away in Oklahoma. I even asked my parents about it, they said they hadn’t heard of it before either. It’s a pretty good film, it explained a whole string of nuclear weapons safety issues over the years and it’s surprising we haven’t blown ourselves up several times over. I also read the book by Eric Schlosser which the film was based on. Googling around I discovered there’s still a preserved, functional Titan silo complete with an inert missile and warhead in Arizona, so I had to go see it.

I found out there’s a few different tours, the one I took was just a basic one hour tour that went to the control center, another was directed by a former commander, and another was a “top to bottom” tour where you got to go to all levels. I really would like to do the latter, but slots and dates are limited and it didn’t match my schedule. Based on reading the book and watching the film several times I was pretty familiar with the missile and complex. The tour felt rushed, there was so much I wanted to look at and take photos of, but we were hustled through and back out.

The tour starts with a several minute long video that explains the Titan program, the 308th Strategic Missile Wing, and an overview of the launch complex. Next we went outside and into the access portal, a set of stairs and an elevator that went underground. We were told that from the moment the gate on the road opened, crews had three minutes to make it to the portal before alarms went off as a security precaution. First stop underground was another door/gate where personnel would call, read a passcode, then burn the piece of paper with the code in a red can on the wall.

Several floors down put you in front of blast door #6. This puts you into a blast lock, where door #7 on the other side of the room won’t open until door #6 is secure. The blast lock is designed to protect the crew from a surface attack. From here there’s a junction, one way leads to the missile silo, the other leads to the control center, each of which are protected by another blast door.

We went into the control center, which was on level 2. Level 1 above us was a crew quarters, level 3 below us was a mechanical room that contained the air conditioner. All three levels were suspended together from giant springs to protect against blast movement. There was a center console and a row of electronics along one wall, all of which still worked and was powered on.

The docent explained how the crews worked, what their daily routines were, what happened when a drill or launch order happened, and finally simulated what would happen if they launched. A little kid at the center console and the docent turned their launch keys, next thing that happened was a perimeter security alarm bell went off signaling that the silo door had opened, then a fire alarm went off indicating there was (surprise) a fire in the missile silo. After that he said, the missile was away, there’s no way to stop it, nothing else is left to do, and now the entire launch complex was now useless.

We then went over to the missile silo. There were a couple of open doors where we could see the missile from level 2 and look up at the warhead. After this we were shuttled back up to the surface where we were free to roam around topside. The silo door was partially opened with a glass lid, so you could look down into it from a platform. In a small shed were first and second stage engines on display, as well as a fuel truck on the fueling pad.

The gift shop had all sorts of interesting loot for sale, pieces of metal from the missiles, rebar, dosimeters, patches, technical books and diagrams, signs. Also in the lobby was a small exhibit of the Damascus incident, including a big socket like what was dropped down the silo. I want to go back for the top down tour, looking at their schedule they’re booked a few months out, so maybe January or February.

[flickr: trinity site]
[flickr: white sands missile museum park]
[flickr: new mexico museum of space history]
[flickr: titan missile museum]

The very first detonation of a nuclear device happened in New Mexico and was called “Trinity”. Two times a year people can go out onto the White Sands Missile Range to visit the Trinity site. I hit the road and visited the site a couple of weeks ago. It turns out it’s a long drive from the Bay Area to Alamogordo, New Mexico. The first day I put in 12 hours of driving, slept at a rest stop between Phoenix and Tucson, and then about 8 hours the next day.

But first I wanted to make a detour to the Sonoran desert in Arizona. A while back I had found that in 2010 BLM put up these big scary “DANGER – PUBLIC WARNING TRAVEL NOT RECOMMENDED” signs due to smuggling and trafficking on public land along I-8. I wanted to go see one of these things out of curiosity. (Yeah, put up a sign that says don’t go there, I want to go there. Also hooray for Arizona having constitutional open and concealed carry, I felt the freedom wash over me as soon as I hit the border.) Through the Sonoran Desert National Monument along I-8 there actually weren’t that many exits off of the interstate. To get onto BLM land, was one at MM 144 for Vekol Valley Road, MM 140 for Freeman Road, and then a couple around Gila Bend, the next city on the interstate.

2010 BLM travel danger sign

More tame sign on BLM land

I didn’t have a good map of the BLM roads, Google Maps and my Garmin maps were pretty useless for this, and the BLM website is pretty dorky. I nosed around on Freeman Road a bit and only found a tame “Active Federal Law Enforcement Patrol Area” sign. Same at Vekol Valley, except this time I found a paper map I could take. I will say the established gravel roads are pretty good and well marked with numbers. Deciding not to get too far in the middle of nowhere I came back to the highway. Later I found out the big scary warning signs caused such an uproar and embarrassed the government so much they replaced them a few months later with the tamer signs I saw. If anyone knows if there are still big scary signs out there, I’d love to know where!

Trinity site tour

I stayed in Alamogordo and quickly figured out it was a lot farther to the Trinity site than I thought. It turns out WSMR is huge, probably 100 miles wide and even more miles long. The only public entrance was Stallion Gate, at the northern part of WSMR. This was about 120 miles away from Alamogordo. There was an escorted vehicle convoy that left just north in Tularosa which cut directly across the range, cutting the distance down to about 70 miles. However it was slower, so from a time perspective I think it was a wash. The upshot is that you get to see much more of the range as you travel through. Downside is there isn’t much to see except desert scrub and the occasional launch site, and photography or stopping was prohibited.

There were 100-200 vehicles in the convoy and we arrived at the site around 10:45 AM. There was a huge parking lot by the Trinity site and a string of cars coming in from the northern gate. In the parking lot were a couple of vendors selling food and souvenirs, porta-potties, the Jumbo test casing, and a ton of police, military police, and a lot of other Army and Navy folks. A short walk leads to the actual ground zero test site. Within the fenced in area (a couple of acres in size) was a monument to the Trinity test, a tractor-trailer hauling a Fat Man casing, a concrete stub where the test tower was, and string of historical photos hanging on the fence. There was a sign reminding us that there was still some scant radiation here, but 1 hour at the site was less than 1/10th the exposure of a chest x-ray.

There was also a ton of people because everyone got there at the same time. Everyone was jockeying for position to get photos with the monument and milling around the Fat Man trailer. I was amused to overhear this one bit of conversation:

Girl 1: That kid was dabbing while he got his photo taken, that's so disrespectful.
Girl 2: Disrespectful to who?

Yes, disrespectful to who? Nobody is buried here. Anyways the photos along the fence were interesting, it showed how the site looked before testing, the nuclear fireball, and what things looked like after.

Trinitite

If you looked carefully on the ground, you could spot some bits of trinite that hadn’t been buried, but largely it had been picked pretty clean over the years. In theory you’re not supposed to remove it because “theft to the government”, good luck enforcing that. There was a table at the entrance where a docent had a box of trinite you could look at and stick a geiger counter to it.

What was interesting about the site and not really easy to show in a photograph was that the whole test area was in a huge depression. According to the docent, beyond the actual bomb crater and the sand that was turned to trinite, there was about a 4 foot layer of desert sand kicked up and displaced after detonation. It looked like the fenced in area contained the entire depression, it was most visible from the parking lot.

I wanted photographs without people in the background, so I lingered around for a couple of hours until the Alamogordo convoy left. This means I’d have to leave north through the Stallion Gate. After 1 PM things started thinning out and I was mostly able to get the photos I wanted. I missed out on visiting the McDonald House, I didn’t see that there was a separate shuttle that went over to it.

When I left the range and got back on the highway, there was a guy named “Atomic Steve” selling hunks of uranium. There was also a sign directing people to where trinite was for sale. Some may say that’s illegal, but apparently way back when there were considerable mounts to be collected from the land and people are selling it. You’re just not supposed to remove it from the Trinity site.

So pro-tip if you’re wanting to visit only the Trinity site, don’t stay in Alamogordo. Find a place to stay closer to the northwest corner and get there early. From the gate to the site is only about 5 miles. I wanted to check out White Sands National Monument too and wound up discovering the New Mexico Museum of Space History in Alamogordo so that worked out pretty well for me.

White Sands Missile Park

Patriot: If it flies it dies

The same afternoon I left Trinity, I headed south to the White Sands Missile Museum. This was another long haul, about 140 miles from Stallion Gate, through Las Cruces, then over to White Sands. The actual museum closed at 3 PM so I missed it by an hour, but the missile park was still open until sunset. I rolled up to the base gate where police had me park outside the gate and gave me a pass to walk over to the missile park.

On highlight were dozens of missiles and rockets that had been tested at the range. Some were popular, some were unknown to me. There was a Redstone rocket, a Patriot missile system, an Army Pershing missile, various Navy guided missile systems, Nike Ajax and Hercules missiles, and all sorts of other surface-to-air, surface-to-surface, surface-to-ground missiles.

New Mexico Museum of Space History

It turns out there was a lot more early aerospace work done in New Mexico than I realized.  Beyond the Trinity test, Wernher von Braun was stationed in Fort Bliss and they did V2 testing at the White Sands Missile Range. WSMR was where things like the Patriot missile system, Pershing, Redstone rockets were tested. Joe Kittinger did his record setting Project Excelsior skydive from 102,800 feet over New Mexico. Columbia landed here in 1983 as a test for STS-3. Virgin Galactic’s Spaceport America is here. I guess huge swaths of relatively flat, uninhabited, year-round mild weather is pretty useful for this stuff.

Outside the museum building was a collection of missiles and rockets. They had leftovers of a V2 engine from a test, but most impressive was a fully intact F-1 rocket engine which was built for the Apollo program for use in Saturn V rockets. The thrust nozzle material couldn’t have been more thick than a couple of credit cards, and there was an incredible amount of welds that stretched for several feet, every couple of centimeters. I imagine welders at Rocketdyne spent a huge swath of their career just patiently dabbing in welds on a single engine.

Inside the building were mainly exhibits highlighting the aerospace industry in New Mexico as I mentioned earlier. They had several components from the Apollo program on display.

White Sands National Monument

I wish I could’ve spent more time here but I didn’t time it right and I needed to cut my trip short. I made one trip out at 7 PM right after sunset and right before the gates closed. It was pretty tolerable then and the sand made for some neat long exposure photographs. I came back here Saturday after visiting the museum in the dead heat of the afternoon and got cooked. There were loads of people in swimsuits sledding down the sand dunes, treating it like an ocean-less beach. I should’ve waited around a couple of hours for it to cool off before exploring more. I’m told the thing to do is to come out after sunset, setup camp at a campsite, goof around overnight, then leave when the sun comes up. Saving that for next time!

Mileage

10/5
1:56pm  320,502  Fremont, CA  8,431.1 hr
6:18pm  320,782  Tejon
8:35pm  320,909  Fontana, CA
11:05pm 321,092  AZ border
2:09am  321,274  Casa Blanca rest stop
10/6
4:10pm  Deming, NM
10/7
6:35am (M)  321,864  Alamogordo, NM
10:00am (M) 321,940  WSMR staging
10:30am (M) 321,945  Trinity site
10/8
8:23pm  322,573  Tucson, AZ
10/9
9:00pm  Phoenix, AZ
10/10
7:25am  323,243  Tejon rest stop  8,476.7 hr
12:00pm 323,538  Fremont, CA - HOME

« Newer Posts - Older Posts »