Feed on
Posts
Comments

A story about a Cisco 6500 and car jacks

Jun 15th, 2024 by

Many years ago I was out of the country turning up a new site. Along with racks of servers we had a pair of cabinets shipped to us that contained Cisco 6509s, patch panels, with 300+ ports pre-mounted, pre-cabled, and tested from our main US site. This was a weeks long project and we were a couple of days from being completely done and going home, and the site was already taking some customer traffic. The company CEO stops by to check things out, barely looks at one of the 6509s and says “that’s mounted wrong”. We never noticed it, the group that originally racked it never noticed it, but indeed the rear of the 6509’s shelf was one bolt position too low. Not a RU low, just a half inch.

Some people strive for perfection, some people strive for done is good enough. Word came down we had to fix it. This would entail draining all the customer traffic (an 8-12 hour wait itself), unplug all 336 patch cables, fibers, de-rack the 6500, fix the shelf, and reinstall everything. This would have added a day to our trip at the least, provided nothing else went wrong in the process. My manager and I decided to take matters into our own hands.

We went out to our rental cars and got our jacks for changing flat tires. We stuck them under the back of the 6509, gently lifted up the chassis enough to unscrew and fix the supporting shelf (the chassis rack ears were still secure to the rack), and let it back down. While it was serving customer traffic. Problem was solved within an hour, we were very happy with ourselves.

Needless to say our bosses were not happy about our bit of improvisation. But it worked and not a packet was lost that day!

Posted in Uncategorized | No Comments »

FreHD up and running with the TRS-80

May 28th, 2024 by

FreHD menu

I wound up buying a completed FreHD kit for the TRS-80 model 4, along with the self-booting EEPROM. FreHD is a TRS-80 hard drive emulator that plugs into the I/O connector on the bottom of the system. Normally the TRS-80 hard drives required a DOS to be loaded off of a floppy disk first, then the volumes were accessible. My floppy drives are still inoperable, so going this way lets me run programs on it in the meantime.

Installing the FreHD EEPROM

I didn’t realize it at the time but the EEPROM requires some wires to be soldered to it and to some spots on the system board. It works by replacing the casette BASIC ROM (U4) with the new EEPROM which contains the bootloader. I’m ok with soldering a little, but this required a couple of connectors to wee size pads on the circuit board. Fortunately nothing destructive like trace cutting needed to be done, so I was ok with trying it. God bless jeweler’s magnifying headsets, my eyes can’t see this tiny stuff well anymore. Fuck getting old.

I got it all together without any disaster:

Finished install

As I was getting ready to put things together I noticed the I/O edge connector isn’t keyed nor does it have any numbers. Which way does the ribbon cable connect? The manual’s schematics didn’t have any numbering either, but did show all the pins on one side were all connected to ground, and the pins on the other side were connected to other components. Looking at the system board, on the component side, there’s a trace that connects every pin. I finally concluded the “even” pins 2-50 are all the grounds, and “odd” pins 1-49 are signal. So, the leftmost is pin 1 and rightmost is pin 50.

I put the system board back in, buttoned things up and fired it up. After an initial reboot the FreHD LED lit up and the FreHD loader menu popped on the screen. Success! First try!

Learning TRS-80 software

The FreHD SD image comes with several instances of LS-DOS, CP/M, NEWDOS, and various utilities on them. After poking around at different games and programs, I realized the TRS-80 is a much more capable system than I remember. In fact I don’t know much at all about the platform. When I was a kid the extent of my usage was booting TRS-DOS, loading BASIC, going to a long sheet of paper that had a list of BASIC games that were copied from who knows where, LOADing them and running them. I wrote some little BASIC programs, that was about it. We didn’t have hardly any commercial titles, I think just Superscripsit, VisiCalc, and Chicken! (The Superscriptsit did have an audio tape tutorial that taught me about proportional fonts!)

Most importantly I’m happy the thing actually runs software without crashing. Other than the video RAM issue, so far I haven’t seen any corrupted RAM, janky video, janky keys, or lockups.

The games on the system were popular names, if not knockoffs of arcade games. Frogger, Missile Command, Moon Rover, Breakout, etc. They were not some public domain BASIC programs either, they were real titles with real graphics and gameplay. Funnily enough most TRS-80 software seems to have no concept of returning to DOS, you have to hit the o’ orange reset button to get out.

Modem on the TRS-80

I found a couple of comm utilities such as Modem-80 and FastTerm II on the system so of course I had to try hooking up a modem, in this case one of my USR Couriers. First thing I had to do was track down a set of gender changers to go from DB-25 off the TRS-80 to my various DB-9/DB-25 cables to the Courier. The first app I tried was Modem-80. At first I could tell from the LEDs it was communicating with the modem but it wasn’t returning anything. I rigged up a null modem cable to my laptop and verified I could send type stuff on one system and see it on the other. I finally realized the TRS-80 wasn’t raising DTR, so I flipped a DIP switch on the Courier to always assert DTR and I was able to speak to the modem. I don’t know if this was because I wasn’t using a DB-25 to DB-25 cable or not.

I ATDT’d the BBS, lo and behold I got the Wildcat! banner and my login screen!

Dialed up to my BBS with the TRS-80

The ANSI color and graphics understandably didn’t work, but the rest did. 9600 bps seemed to be a bit too fast for it to keep up with as it griped about overruns occasionally. I had problems getting my modems to run at 2400 bps, so I settled at 4800 bps and it was decent. FastTerm II looked interesting too, of note it supported the Ymodem protocol. I found out there are other terminal emulator programs that say they support ANSI something, I need to figure out how to copy them over to the FreHD SD card and see what they do.

Floppy drives still vex me

I thought I might be able to copy Floppy Doctor over to the SD card and use it to work on the floppy drives. Then I read the manual for it and apparently it was released on a self-booting disk that TRS-DOS couldn’t read. Then I just found what looks like maybe an older CMD version, so we’ll see.

With the system booted I tried accessing the TRS-DOS disk I made. The drive would spin and seeks a bit, pause, and then come back with some error like “No disk” or directory not found or something like that. I’m still pouring over the VCF forums for ideas what to try. One thing that looks promising is to write 0x2F to a known track and then use some BASIC to seek the drive to that track and try to read, which should help sort out if the read head even works or if it’s even remotely aligned. I got my oscilloscope the other day, so I’m eager to poke some digital signals!

I do want to get at least one floppy drive working, as I have a few dozen TRS-80 floppies I want to go through.

 

Tags:

Posted in retro | No Comments »

iTerm2 + Qodem + MS Natural Keyboard

May 28th, 2024 by

This is mainly a reminder for myself so I don’t forget what I did. When using Qodem in iTerm2 on my Mac along with a Microsoft Natural Ergonomic 4000 keyboard, the “Alt” key wasn’t working inside Qodem. It was impossible to “Alt-Z” to bring up the help, the dialer screen, or anything. I didn’t have this problem on my Macbook Pro. This is even after the standard swapping of Option and Command modifier keys in System Preferences. After much fiddling and even ssh’ing to/from my laptop to rule out anymore termcap issues like before, I finally figured out in iTerm2, going to Preferences, Profiles, Keys, General and changing “Right Option key” from “Normal” to “Esc+” fixed it. What’s even weirder is that I always use my left-hand modifiers to go into these menus and I have to select “right option key” from the list and not the left? Either way I don’t know what this does or why it matters, I just went with it.

Qodem for Mac (via Homebrew) disables serial port

Also Qodem installed from Homebrew disables the serial port. So if you want to use a real dial-up modem on your Mac, you gotta compile Qodem on your own. Fortunately it’s just a quick and easy ./configure ; make ; make install.  I didn’t see in the git blame for the brew recipe any particular reason for disabling serial support, so maybe I should send a PR for it.

Boooooo serial support disabled by default in Homebrew

ANSI escape bug in Qodem

Bonus: while on the subject of Qodem, I think this one is a straight up bug that I need to submit a patch for (is it even maintained though?). I was testing a new door game GuTTerBoWl and noticed under only Qodem it spewed ANSI escape sequences to the screen. Actual original QModem, SyncTERM, MuffinTerm, and even Minicom didn’t do this, and rendered it correctly. The door author says he didn’t do any changes to the ANSI handling in the door kit (Angel Doorkit v1.0), nor had they seen this sort of output before and I believe them.

GuTTeRBoWl in Qodem

After looking at a capture of the ANSI output I noticed sequences such as ^[1;37m rendered correctly, but all the ones with no number before the semi-colon were not escaped and just sent to the caller raw like ^[;34m. Looking up some info on ANSI escape sequences, after the ^[ you can specify a mode such as change foreground/background text color, intensity, or blinking. If there are multiple modes, you can separate them with semi-colons such as ^[1;37m or ^[1;15m;20. This particular door kit was omitting a number when I presume it meant 0 or to reset modes.

At least according to wiki, a missing number is presumed to be zero (good lord there’s even a whole ECMA spec for this stuff):

All common sequences just use the parameters as a series of semicolon-separated numbers such as 1;2;3. Missing numbers are treated as 0 (1;;3 acts like the middle number is 0, and no parameters at all in ESC[m acts like a 0 reset code). Some sequences (such as CUU) treat 0 as 1 in order to make missing parameters useful.[5]: F.4.2 

 

After figuring this out, it was pretty easy to reproduce this problem in Qodem:

Testing with and without mode numbers

This is what it’s supposed to look like:

GuTTeRBowl properly rendered in SyncTERM

Funny enough I ran a 1.5-something version of this game in 1995, I had forgotten all about it until the author released v4.00 last month.

Posted in retro | No Comments »

The backup tapes live!

May 25th, 2024 by

1995 backups

[photos: flickr – 1995 Tape backup pulls]

Data has been recovered from my 1995 backup tapes! After badly mangling one of my three QIC-80 backup tapes trying to repair it to make it readable, I gave up before I screwed them up even more. Finally a couple of weeks ago I contacted Dmitry Brant, who has experience with tapes, if he still did tape data recovery and he said he does. A few days after sending the tapes to him, he got back to me “with good news and not so good.” One tape had a broken tension band but otherwise good; the second tape had some damage and wrinkles where it jammed in my drive, but was able to recover almost everything; and the third tape that I had wrenched on was too far damaged (the header was too wrinkled) to be salvageable but yet still managed to get a few megabytes off of it.

All in all there was around 250 MB of data recovered. It looks like there was a couple of full-ish backups of my old BBS and Windows 3.11 system, along with several partial or incremental backups throughout 1994 and 1995. I spent several days combing through all the files and it was like a nicely preserved little 1995 time capsule. Among the files were the contents of 27 floppies for the Slackware Linux distribution in 1994, such as AP (apps), N (TCP/IP network), X/XAP/XV (XFree86 implementation of X Windows). I distinctly remember moving these to tape because they took up a lot of my hard drive space at the time. It’s not clear which version of Slackware these came from, but definitely 1994. This was the first Linux distro I used, I seem to recall it was using a pre-1.0 kernel like maybe 0.96 or 1.2.18. Or maybe it was Slackware 1.2.18?

I was most excited about the BBS files after recently re-scratching that itch. It was all there, Wildcat! 4, my ANSI screens, cringy bulletins and welcome messages I wrote when I was 15, a couple hundred files in the files repo, door programs, log files, everything. There were a few dozen off-line QWK message files from a few bulletin boards I frequented, with my cringy messages. Half the stuff I wrote sounds like I was on a sugar rush from Mountain Dew, which I probably was. My graphics collection, which consisted of a few dozen .PCX and .GIF files. Unfortunately I didn’t seem to have a full working backup of Windows, I would have liked to see what I had there.

The BBS was named after the very small town in Oklahoma that I lived in, with much irony. It didn’t really have much of a theme or niche, other than trying to provide some sales and technical information for my gig building/selling/repairing computers. Despite being in the middle of nowhere I did have a dozen or so regular callers, mainly other BBS friends from around the country.

Door menu

I don’t know what I was thinking with most of the door programs I was running, nobody wanted that stuff. Star Trek: TNG was all the rage then, on top of trivia doors I had a few oddball Enterprise and TNG graphics.

Upload scanner written in MS-DOS batch

I was really into writing MS-DOS batch files because I was too poor to buy anyone’s utilities. One thing I had written was a batch file that figured out the file extension of uploads, .ZIP, .ARJ, .EXE, .LHZ, .Z, and then called the right archive tool to unpack and inspect it. (Still looks legit today). Another was this complicated batch file upload and virus scanner which shot raw ANSI to the caller as it progressed. I seem to recall it worked but it was brittle to work on. Normal people just bought somebody else’s program for $25 or whatever to do this for them.

Mirror of Software Creations, Apogee, and Id shareware

LORD upgrade bulletin

My graphics collection wasn’t very large, and was mostly in the .PCX graphic format. I had just gotten a 1024 x 768 SVGA monitor and a Logitech handheld black and white scanner.

The very first .GIF file I ever downloaded!

.GIF files were also new and novel to me, 800 x 600 pixels! 256 color! Photorealistic! They really made that Trident video card pop with what it could do. It’s funny to me because around that time .GIF files were relatively large, probably took thousands of dollars worth of scanner and PC to create them, probably came from CompuServe or a large membership BBS that cost $25/month and $2/hour to access, on top of the phone bill. (And they were all probably scans of copyrighted stuff which got a lot of boards in trouble). Now we regularly encourage taking and throwing away 12 megapixel HDR movies.

A single 256-color 640 x 480 image may have ran around 256 kilobytes in size. With a 14.4k modem, it downloaded about 1,440 characters per second, that image would have taken 177 seconds, or a hair under 3 minutes to download!

320 x 200 GIF

800 x 600 x 256 color GIF

Rusty n Edie’s BBS ads featuring their cat

640 x 480 GIF

Also right around this exact same time (April 1995) the Oklahoma City bombing happened. I had a couple of .GIFs of the FBI sketches going around looking for suspects that I had downloaded a couple days after it happened.

Oklahoma City bombing suspects sketch

There were a few apps still intact, like NCSA Mosaic and Netscape 1.0! Mosaic was the first browser I used, both of these were pre-HTTP/1.1 and pre-Javascript.

NCSA Mosaic 2.0 alpha 7

Netscape v1.0

 

Overall I’m very excited about the haul. I had low expectations with the recovery effort, but thrilled data was salvaged from 29 years ago. I’ll probably go through and upload a lot of the original BBS file areas to my new BBS and otherwise make them accessible, as there’s quite a few vintage DOS/Windows 3.11 apps in there.

Posted in retro, Uncategorized | No Comments »

I moved my desk!

May 24th, 2024 by

TL;DR so awesome, so much room for activities I should have done this years ago

This is such an exciting game-changing, lifestyle change! I moved my desk!

Ever since I moved in years ago into my small apartment my desk has always been in my bedroom, where in my place in Redmond it had been in a 2nd bedroom-turned-office. I think at the time I said I didn’t want the clutter of my desk in the living room, and I primarily used a laptop for everything, so to bedroom exile it went. Despite having my larger monitor(s) and an actual chair, I used my desktop setup less and less, only actually using it for doing photo imports. Even during COVID and the resulting work from home I continued to work from my couch and coffee table on laptops. I did not want to wake up, roll over to my computer, and stay in the same room all day long!

The problem was I wound up cluttering up a shelf and the corner of my living room anyways and all the computers made my bedroom hot in the summer, even with a window open. To the point I would turn off systems in ikeacluster and some nights sleep on the couch, where my A/C was located. It also turns out the cats’ litter boxes create a ton of clay dust so my electronics got absolutely filthy over time and required regular dust blasting and vacuuming.

I knew if I was going to continue to do any WFH I need to move my desk where it was usable and comfortable again. I procrastinated for months, dreading re-wiring everything, taking down my Internet connection, and having something inevitably failing during the process. Last week I finally sucked it up, took extra backups, moved the shelf into the bedroom and moved the desk and all the computers into the livingroom.

Before hooking everything up, I opened up the cases, stood in the doorway and gave them all a good air blasting. It kicked up so much dust I got three separate “smoke detected” warnings from my smoke detector! All of the hard drives survived, but the PSU in one of my Linux machines did smell like smoke and went to heaven when I flipped it on, as about I expected. Pro-tip: order spare power supplies for your weird size PC cases.

After getting everything back together, what a difference it made! I took out an Ikea chair I didn’t use, but it all freed up so much floor space in both the bedroom and livingroom! Somehow the new configuration just worked a lot better than I expected. I had room for an even bigger shelf in the bedroom which was satisfied by a trip to IKEA. The bedroom was immediately WAY cooler and dead silent now. The living room now gets noticeably warmer if I don’t have the balcony door open, but at least I have more options to deal with the heat and hopefully this means I don’t have to run my space heater so much in the winter. Now I’m getting around to adding some RGB rizz and vibes to the setup, because why not.

Posted in Uncategorized | No Comments »

Reviving the USRSTATS BBS door

May 14th, 2024 by

TL;DR I hex edit a binary to make it work with my “newer” Courier modems

TL;DR 2: Less than 12 hours later posting I find an updated version of USRSTAT2.EXE from 1997 that fixed my bug and makes my hacking irrelevant

Way back on the US Robotics BBS (USR BBS) they had a door program that would display information about your modem connection from their BBS’s perspective. I always kind of assumed this was some door that talked to their Total Control modem rack over the network and gleamed connection statistics via SNMP or something. I used to run Total Controls later on at my ISP and they had a vast amount of information you could collect over the network and actually had a page written in PHP that would display connection speed to my callers.

Fast forward to the present with my new BBS and it got me to thinking about the old door on USR BBS. I really care about modem connections now, trying to squeeze as much performance out of VoIP to demonstrate it can be done. I knew call diagnostics stats were available on my US Robotics Courier modems via AT I6, I4, I11, just how do you display that to a caller? Browsing various BBS door archives then I found a few programs that said they’d display stats but I assumed they only worked with things like Supra/Diamond modems or were written for specific BBS software. It also seemed kind of crazy to have a door program send +++ commands to take over the modem while a caller as on. So I didn’t put much thought into it, written off as a forgotten wish.

Calling around to some new BBSs the other night I called up Another Millennium (949-59-31337, cute) and saw they had a USRstats page that looked like what I remembered from USR BBS! I looked around at what they were doing it with, it looks like they are running a version of USRSTATS for Maximus from 1995, which is its own compiled add-on.

USRSTATS – MEX on Another Millennium BBS

MODST

This got me back to looking at the doors that were out there like Modem Stats (MODST120.ZIP) and USRStats Generic (STGEN107.ZIP). Modem Stats was the first I got working. Despite being developed/tested apparently in a Remote Access + Diamond Supra modem environment, it worked just fine with Wildcat! and my USR Courier modems. Hooray!! It was a bit basic and just displayed raw ATI6 output to the caller, which I understand now was so that it provided generic output that would work with any modem. But it worked, finally!

Modem Stats (MODSTATS)

STGEN

Mentioned in the Modem Stats documentation was how it was inspired by STGEN, written by Joe Frankiewicz. So I went looking at STGEN and what it did. This looked more promising but when I got it working it would display part of a screenful of ANSI-formatted data and then abruptly end:

Broken USRSTAT2.EXE

Spending more time reading docs and playing with STGEN (and the modified STGEN-MC.EXE) I realized that STGEN handles comms with the BBS software, caller, and modem, but it saves raw AT-command output from the modem and feeds this to USRSTAT2.EXE, which actually parses the output and makes the pretty ANSI screens to display to the caller. If one were so inspired they could write their own USRSTAT2 replacement and generate whatever screens they wanted. The QuickBASIC source code to STGEN and STGEN-MC are included so you can even modify those till your heart’s content.

(STGEN108 contains both the original v1.07 source code written by Joseph C. Frankiewicz, and a binary+source of a revision called STGEN-MC written by Michael Conley 12/14/95. I’m running the STGEN-MC version but may refer to it as STGEN)

But why was USRSTAT2.EXE only displaying part of a screen? The fact that it cut off after displaying “Preemphasis” made me think it had a problem parsing the modem output, and indeed when I ran USRSTAT2 by hand while connected it threw a message saying “USRSTATS trapped error.”

Now I’m stuck, I don’t know what it’s dying on, there’s no source code for USRSTAT2 included, and a bunch of Google and BBS archive searches don’t turn up anything. It seems I have the latest version of USRSTAT2 that exists. I went back and looked at the USRSTATS MEX version for Maximus and wondered if that could be compiled as a standalone binary I could use with Wildcat. I noticed that the MEX version had a bunch of .LOC files that were output captures for various USR Courier models that it had been tested against. I wonder what would happen if I fed one of those to my USRSTAT2.EXE?

SUCCESS!

USRSTAT2 happily generated a full connection report from the files in the MEX version, including the frequency response table:

Correct USRSTAT2 output

Correct USRSTAT2 output

Now the question was why? I combed over the example VEVR1195.LOC file compared to the output STGEN-MC was grabbing from my modem. Command output was ordered differently, so using a text editor I moved blocks of output around, that didn’t help. Because USRSTAT2 was dying at “Preemphasis” I started looking at that line in the modem output. Ah hah! At least one line of modem output from my 1997-era Courier was slightly different than the 1995 Courier when STGEN/USRSTAT2 were written:

1995 Courier ATI11 output:
...
Preemphasis (-dB ) 8/8
...

1997 Courier ATI11 output:
...
Preemphasis Index 0/0
...

I manually changed that line in my modem output and boom, USRSTAT2 produced a full report! Now the question moved to “how do I fix this output on the fly?” STGEN-MC does a SHELL "USRSTAT2.EXE" directly, so there’s no way to modify the temp file before it’s fed to the report generator. I pondered re-compiling STGEN-MC to fix up the modem output on the fly, or having ChatGPT whip me up a C or Pascal shim to replace USRSTAT2.EXE, fix up the modem output, and call the real USRSTAT2.EXE.

After sleeping on it I wondered if I could get away with just hex editing USRSTAT2.EXE and fudge the string it’s looking for? So that’s what I did. Using WinHex, I found the instances that contained ‘Preemphasis (dB)’ and replaced them with ‘Preemphasis Index’, making sure to make the new string fit in the same spot.

 

USRSTAT2.EXE – Before

Fixing two strings

USRSTAT2.EXE – After hax

And it worked!!! It read in the output from my newer Courier modem and didn’t crash. I now have a functional STGEN-MC and USRSTAT2 door that produces pretty modem reports for callers. I know nothing about patching Windows binaries, so I don’t know how to distribute what’s about a 10 byte change.

I did notice at the very tail end of STGEN.DOC “Source code for the USRSTAT2.EXE module is NOT being released at this time, as development of that module WILL be continuing.” I have no idea if a new version has been released since then, if there is I haven’t found it.

 

As an aside, I didn’t know who Joseph Frankiewicz was until now. I was Googling his name to find out more about USRSTAT2 and found his name in an old German US Robotics FAQ usrfaq.txt file, where he talks about the USR BBS and USRSTAT and identifies as working at US Robotics. I found out through forum posts it turns out he was either the sysop of the USR BBS and/or an engineer with a ton of modem knowledge that interacted a lot with sysops. It would seem he wrote the original USR BBS door or at least the first PCBoard version if it, found as ST234B.ZIP “USR STATS V.234 BETA 5/22/94”. This zip file includes more documentation about his original USRSTAT.EXE program. I have no idea if he’s still developing software or still around.

Update 5/14/2024 7:00 PM

Less than 12 hours after I patched my USRSTAT2.EXE and typed up this post, I found USRST419.ZIP on sak.sk through random googling. This included USRSTAT2.EXE version 4.19 dated 2/28/97 which the change notes says it has fixes for “Total Control x2 modems, Courier x2 modems, newer Sportster modems, and fixed colorization of the Preemphasis fields.” This new version works right away with the STGEN-MC door and my Courier modems, making all my clever hacking completely obsolete. My ego is crushed a bit but I’m glad I found a newer version.

Taking a peek at the v4.19 binary it looks like it now can string compare three variants of the field I was having problems with, “Preemphasis (-dB)” “Preemphasis     (-dB)” and “Preemphasis Index”. I’m sure it has more string comparisons to handle other newer modems but I didn’t check them that closely.

USRSTAT2 v4.19 2/28/97

Google is very frustrating to search for anything named ‘USR’ because it desperately wants to add inflection even with quotes, and a few decades of indexing unix things with “/usr” sure isn’t helping.

Tags:

Posted in retro | No Comments »

Adventures in Let’s Encrypt

May 13th, 2024 by

(Rant warning) TL;DR I gripe at how complicated it gets and offer no solutions. I really do like what Let’s Encrypt offers. Just getting there figuring out what options work and don’t work is work. I don’t know how the muggles manage it.

TL;DR 2: HTTP-01 was out because of internal sites. DNS-01 was the only option, but I don’t use 3rd party DNS with APIs to handle automated challenge updates. Wound up installing a standalone ACME-DNS server for challenge responses.

I finally got annoyed enough at my TLS certificates that I started seriously trying to use Let’s Encrypt and ACME. I only have a couple of normal public-facing websites running on port 443 on the Internet, but internally I have a small army of Ubiquiti EdgeRouters, switches, wireless bridges, UniFi wireless controllers, Raspberry Pis, and other software with web servers on ports other than 80/443 that all need certificates. For years I’ve ran my own private CA to issue certificates, but it’s the same problem as commercial certificates to issue them and load them on all of my devices. Some browsers like Chrome now bitch at self-signed certificates in some cases too, so that’s not really a fix either.

One could say “but ha ha only suckers use web interfaces on routers”, which is true, but on occasion I do use them I’m reminded of the stupid problem of a long expired certificate and have to jump through the browser warning hoops every time that yes I’m yolo’ing to this allegedly sketchy device. A lot of the Ubiquiti stuff is only manageable over WebUI. This goes double if I’m on a new device that doesn’t have my private CA root certificates installed, or Android which makes it really difficult to install a private root CA. Triple combo pain if you’ve accidentally configured HTTP Strict Transport Security with a long lifetime to cover sub-domains and you try to reach something internally with a hostname with that domain with a bad certificate and the browser is like fuck you I won’t let you visit this site at all! This is where Chome’s ‘thisisunsafe’ override really comes in handy! I just want stuff to work and go about my day man, and not leak passwords.

No to HTTP-01

I can’t just throw certbot or acme.sh everywhere and call the problem solved. First of all not all of my devices are exposed to the Internet to accept HTTP-01 challenges from random sources, not to mention the non port-443 services. (God bless them for having a mix of IPv4 and IPv6 probe sources to handle IPv6-only endpoints which helps.) For the whole existence of Let’s Encrypt every year I thought about this problem I would look up how to do run my own ACME server with my private CA, groan at the apparent effort learning the whole ACME protocol and leave it.

DNS-01 with caveats

That leaves me with DNS-01 challenges. The problem here is that I don’t use a cloudy/third-party DNS provider that has an API where I can automatically update TXT records for automatic certificate renewals. I run straight up BIND and further my authoritative servers each use independent replicated master files with no slaving. This means any kind of dynamic updates to my DNS would have to go to each DNS server. I do have a few dynamically-updated A/AAAA records in sub-zones and for years I’ve just been running nsupdate twice, one for each authoritative server, and this has worked fine. ACME clients I’ve seen don’t support nsupdate to multiple servers, so this would be a hack to carry around.

I’m not casually replacing BIND nor throwing it all on the cloud. This recently lead to me to thinking ok fine maybe it’s not so bad doing a master/slave of my dynamic zones, that way an ACME client would only have to update one. This then lead me what to do about zone keys distribution. I’d have to copy the same master TSIG key around to all of my devices, or create a TSIG key per domain, sub-zone, or per device/A/AAAA record, which gets tedious and unpalatable.

acme-dns

I retreated and thought surely others have hit this problem too. This lead me to the acme-dns server project. It’s a little standalone DNS server that does nothing but serve up TXT records and has a simple REST API. I set it up on my internal IPv6 network so all of my internal devices can reach the API, and expose the DNS server on port 53 to the Internet so DNS-01 challenges are queryable. In my master zone files I delegate a sub-domain via NS record to the acme-dns IP address, and then create CNAMEs that point at that sub-domain so all challenges go to the acme-dns server. This is where I praise Let’s Encrypt for having IPv6 probes, they can reach my acme-dns server without having to burn a public IPv4 address just for it.

It took me a while to figure out how to actually use the thing. Certbot requires yet another thing to be installed, an acme-dns hook program. Have I mentioned how complicated this whole ecosystem is? acme.sh already includes a hook. Rant 1: good lord that thing is one massive unit of a Bash script. Rant 2: I really do not like it when installation instructions are “here just curl | sh”. It doesn’t just download a single file, it downloads several directories of files and shoves stuff into your crontab. Who knows what else it did. Must find DEB/RPM packages of that sucker.

Ok so now for each and every hostname+FQDN you want a certificate you want, you have to hit the /register endpoint of the acme-dns server first with a curl POST request. This generates a “username”, password, and a string for a sub-domain. This only exists in the acme-dns server database. If for example I had gw1.example.com, I would now add a record in my zone file that says “_acme-challenge.gw1.example.com.  IN CNAME asdf-asdf-asdf-asdf-asdf-asdf.acme-dns.example.com.” *.acme-dns.example.com is already delegated via NS record to the acme-dns instance. This is tedious and annoying to do for a bunch of hostnames but it’s for the greater good and only has to be done once fortunately. By default acme-dns uses SQLite (or Postgres), so either way back that sucker up or you’ll have to re-generate every single one of your domain usernames when something dies.

Then for each and every hostname, take the username/password/subdomain, feed them into environment variables and then run acme.sh to issue the certificates. Witness the gigantic scripts in action!  Stuff going to the CA, TXT records being fed to acme-DNS, stuff going to the DNS server, stuff coming back from the CA, more stuff going back and forth!  If you’re lucky you get a few certificate and key files left. If you’re unlucky, good luck troubleshooting which step in this whole process broke down.

Finally, certificates!

Now you have certificates, what to do with them! This is another whole bear of a problem to tackle because there’s an infinite amount of web servers and directories to insert certificates into. Again there’s a whole ecosystem of Certbot/acme.sh deployment hooks that try to handle your webserver. Also remember by default this is all happening within your home directory, so keys have to be copied to secure system directories owned by root too. This is where I’m at now. I have some devices like Ubiquiti EdgeSwitches that can’t run an ACME client directly, so I have to rig up things to scp over the certificates.

I hope all of this just magically works and auto-renews in 90 days, what a pain to set up!

Posted in Uncategorized | No Comments »

TRS-80 video fixed!

May 13th, 2024 by

Good news! A new video RAM chip fixed the lingering video artifacts I had. I looked up the IC model that was in it “HM6116LP-2” and found some on eBay. I had to wait a couple of weeks for them to arrive from China, but popped the first one in and it worked like a charm. Now I have four extra SRAM, what to do.

After scant Reddit and YouTube review browsing I ordered up a Rigol DHO814 oscilloscope to take on floppy drive repair. It’s a bit of a letdown that it seems like it’s just a glorified Android tablet with inputs, but maybe they all are these days.

Original chip with problems in center

Tags:

Posted in retro | No Comments »

TRS-80 update – floppy drives

May 7th, 2024 by

I finally got around to working on the TRS-80 model 4 again, specifically troubleshooting the floppy drives. Last time I tried to boot it, the drive light would come on and it didn’t sound like it was doing anything. I suspected the belt on the bottom had deteriorated/and or lost tension and wasn’t spinning the disc.

I pulled the drives out out and the spindle drive belts were intact, had tension, and moved freely. The stepper motors that moved the head were another story. Following this YouTube video on restoring TRS-80 floppy drives, first thing I did was swab down the rails and apply some lube. On drive 0, the stepper motor was completely seized up and wouldn’t move. At first I wondered if this was because there was no power and there was some sort of clutch holding it in place. Some YT videos suggested that the stepper motor and head assembly should move freely with very little effort. This was the case with drive 1, I could easily slide the head back and forth with my fingertip (video), especially after lubricating the slide rails.

For the record I identified I have a model 4 gate array system, 26-1069A. The two 5.25″ floppy drives are from Texas Peripherals, model 10-5355-001. The drive PCBs are marked “Tandy Corp” 1983, 10 5053-012. I actually have an original Tandy 5.25″ head cleaning disk somewhere, but I couldn’t find it when it came time to work on the drives. These disks are amazingly spendy now on eBay!

On drive 0 I worked the stepper motor coupling clamp back with my fingers and forth for a few minutes and eventually it worked loose, the head was able to travel the full range of motion without any effort. On the Vintage Computer Federation forums there were several threads about TRS-80 stepper motors, in particular I followed this one which had some cassette BASIC code that would manually spin the floppy and move the head carriage back and forth between track 0 and 39. Drive 0 seemed to be functional, it would spin the drive motor and would seek the drive the full distance over and over again (albeit a bit noisy). Drive 1 would spin the drive motor, but the stepper motor angrily chattered and jerked in place (video) like it wanted to move but was stuck. A common problem with these drives seems to be the coupling clamp would come loose and slip when the stepper motor moved, best I can see there’s no slip and the screw is snug.

I have a bunch of our old TRS-80 disks, I found a couple I think had TRS-DOS 1.3 on them and tried to boot. No go. Maybe my 40 year old floppy disks were shot? Following a couple (#1, #2) of Adrian’s Digital Basement videos, I wound up getting a 5.25″ 360k PC floppy drive off of eBay to hook up to the 486 to make some new TRS-80 boot disks. For good measure since I’m going through all this effort I bought some new 5.25″ DS/DD floppy disks from floppydisks.com. I downloaded Dave Dunfield’s ImageDisk and made a few different TRS-80 boot disks with TRSDOS 6.21, 1.3, and Floppy Doctor. Unfortunately after a week of waiting for stuff to show up none of them booted in my TRS-80 either.

So now I’m back at figuring out what to do next. It seems likely that wrestling with the stepper motor on drive 0 I could have knocked the drive out of alignment. There are several pages and forum posts that describe how to check and re-align them but I’m short an oscilloscope at the moment. I saw at least one person just advocating yolo’ing it by turning the alignment screw until it boots. Because I kinda want to learn how to use a scope, that might be my next sub-project.

FreHD

I’d like to keep both floppy drives in the system to keep it original, so I’m not keen on replacing one with a disk emulator. I did decide to opt for ordering a FreHD hard drive emulator because ultimately I’d like to be able to use this thing and not be at the whim of 40 year old floppy disks and disk drives. I learned there’s an EEPROM that can be installed in a model 4 along with some bodge wires so it’ll boot directly from the FreHD without need of loading a DOS from a floppy first. This might save me if I can’t get the floppy drives working.

Computerfacts!

By sheer dumb luck of googling for part numbers of IC chips I ran across the Sam’s Computerfacts for the TRS-80 Model 4 Gate Array on archive.org. I’ve never seen a Computerfacts book before, but it seems to be the Chilton/Haynes of old computer repair. Most of the manuals and schematics I’ve been looking at were for the original Model 4, so they didn’t quite match up. This is exciting because this manual matches my motherboard and TPI floppy drives exactly! I wish I had found this a long time ago, it provides a ton of troubleshooting workflows, including aligning the floppy drives and checking track 0 seeking.

Video artifacts

A secondary problem on the TRS-80 is video artifacts. I have several rows of ( ( ( ( ( ( ( ( ( that stay on the screen. It doesn’t appear to affect operation, altho it does jumble text when it scrolls up into the affected rows. Seems like it could be some bad RAM or video RAM. Some enterprising individuals at least in model III have gone as far as taking the hex values of the garbage characters, subtracting from what it should be and identifying exactly which RAM chips need replacing. In my model 4 there’s a bank of 8 chips of RAM, and only 1 chip of video RAM. The Computerfacts lists video RAM as a commonly replaced part in troubleshooting video issues, so I ordered up some 4016 chips on eBay from China. Hopefully they’re not fake and fix my issue, else it could be the regular RAM. I don’t think it’s the character generation ROM, which is a good thing because I don’t know how I’d get a replacement.

Tags:

Posted in retro | No Comments »

389,000 miles on the truck – still going

Jan 13th, 2024 by

Heater line repair

The pandemic, work from home, and layoffs have put a damper on my driving. I would easily put 15-20k miles a year on the truck, in Feburary 2022 I hit 380,000 miles, and I just rolled 389,000 miles, so 9k in 2 years. I figured I’d pass 400,000 by now. I also now have my dad’s Ram truck too which gets some of the miles.

Right before I left for Christmas I had a leaking heater hose connector, where the hose meets the heater core in the firewall. I thought ah hah, I’ll just slap another clamp on there. And it worked, I drove it for a while, got the engine hot, leak stopped, great. I’m not sure how long it had been leaking, occasionally I’d smell coolant, but it wasn’t until a cool day that I just happened to see some steam coming out when I was getting something out of the passenger side.

Two days later after running some errands, the day before I was supposed to fly out, it started leaking again and by the time I pulled into my driveway the hose completely broke off and dumped all the coolant in a giant cloud. What I hadn’t realized until I watched YouTube later was that there was a plastic quick disconnect fitting between the hose and the heater core, that’s what snapped off. Had I known that thing was there I would’ve just taken it directly to the mechanic the first time.

I was really worried that I might have finally done the truck in, this may have caused it to overheat and warp something. Either way it was gonna have to set a while till I got back. Last week I finally got it towed to a mechanic and fortunately it was a simple fix of replacing the hoses and fittings, no other damage was done. It’s been running fine since. I also had them replace the leaking valve cover gaskets, so between the two of them hopefully this fixes my coolant and oil consumption.

To date, here’s all the things I’ve replaced or fixed on the truck in it’s 20 year lifespan. I feel like I go through a lot of batteries. The transmission is the major thing I’ve had to fix, first a rebuild way back in 2010-something and then finally had to replace it entirely. AAA is great for towing, would recommend.

The clear coat on the roof has started to go, I’m debating if I want to spend the money to get it re-sprayed. I’d like to keep this truck as a backup as I’ve learned having two vehicles is pretty handy and want to trade the Ram in on something that’s 4wd.

– At least 4 sets of plugs and wires
– heater hoses, disconnect fittings
– valve cover gaskets
– 2 fuel pumps
– 2 water pumps
– 2 sets of O2 sensors
– At least 4 sets of tires
– 5 batteries
– 2 alternators
– 3 sets of brake pads
– 1 remanufactured transmission
– 1 transmission rebuild
– 1 differential rebuild
– 1 a/c compressor and system
– 1 starter
– 1 throttle body assembly
– 1 evap vent solenoid
– 1 set front wheel bearings
– 2 sets shocks
– 1 oil pressure sensor/sender
– 1 rear main seal
– 2 windshields
– 2 catalytic converters
– 1 upper control arm
– 1 blower motor resistor
– 2 serp belts

Tags:

Posted in Uncategorized | No Comments »

Older Posts »