Posts Tagged Cloud computing
This post is a quick overview of my GitHub repo (pdp-11-in-gcp) and how it works to create a fully functional UNIX system from 1976 (UNIX V6) “in the cloud”. It has everything you need to run your own piece of UNIX history.
For the most part, this is an automation of the instructions from http://gunkies.org/wiki/Installing_Unix_v6_(PDP-11)_on_SIMH
This assumes that you have a functioning GCP account with billing enabled, and have at least skimmed earlier posts in this series.
This repo includes several scripts and configuration files:
* launch-pdp11.sh – The master script creates a place to run the SIMH emulator, and builds the emulator. Part of this process is loading another script on to the GCP instance.
* update-os-build-simh.sh – This script is copied on to the GCP Ubuntu instance and gets the SIMH PDP-11 emulator running in the instance. When this script completes, you have a running Ubuntu system with a PDP-11 emulator ready to install v6 UNIX.
The end of the launch-pdp11.sh script provides instructions on how to install V6 UNIX into the emulator. This requires manually running three commands while logged into the GCP Ubuntu instance. Due to limitations of EXPECT, there are a few places where you will need to manually halt the emulator (^E).
* simh-master/BIN/pdp11 tboot.ini – This starts the emulator and does a “tape boot” from an emulated tape image and copies the minimal root filesystem on to the emulated RK disk (which is a file on the Ubuntu host).
* simh-master/BIN/pdp11 buildunix.ini – This script uses extreme expect hackery to do LOTS of customization of the kernel to support an RK disk
* simh-master/BIN/pdp11 normalboot.ini – boots the fully functional PDP-11 with all software. Use this for all subsequent boots of the UNIX guest
One of the most fun parts of this project was dealing with SIMH’s internal EXPECT function. In the “olden days” you had to change the kernel source code to configure tables for each device driver that you wanted included in a new kernel. I’ll show some of that in the next post.
Let’s get started recovering the site. See Part 1 for the background. Note that I actually did this recovery in February 2015, and some software may have changed since then.
1. Dump the DB of the infected site in the test SQL dump format. This creates a human readable (and editable) file on my laptop.
There are all kinds of tutorials out there on dumping a SQL DB using phpMyAdmin. They are all better than I could write. This one, for example.
2. Examine and edit the DB dump file to remove any obvious damage. Is it worthwhile to continue?
For this I used Emacs. Yes, Emacs. You can use any test editor that you understand well, that has a “repeat this edit” or a general “search and replace” function. It must handle long lines, as each DB record is on a single loooong line. It helps if the editor can deal with escape characters. To make a long story short, the damage was almost immediately obvious. I was able to find the suspect lines and ^K (kill) them pretty quickly. For large values of “quickly”. There were about 1500 damaged or bogs records. Using search/replace and making a “fine pattern and kill line” worked wonders.
OK, after about 45 minutes of editing, I’ve got a clean database. All the records that I see are (probably) valid WordPress code/values or (probably) valid user records, or image pointers. It’s worthwhile to continue.
However, there’s still some cleanup, and this is a raw mySQL dump. I can’t import this into WordPress.com, yet. For that I need a WXR format dump, and this WordPress version was so old, that WXR isn’t even supported. I need a modern WordPress install somewhere that will accept the old MySQL dump and then allow a WXR export.
3. Install stand-alone WordPress somewhere (but how, and where?)
I’m going to use this new environment to examine the site in a sandboxed environment and get a chance at some forensics and to more completely assess the damage. This will also be the bridge between the raw mySQL dump and the WXR file that I import into WordPress.com later.
I expected that installing a new host and WordPress to take the most time of the entire process. In the olden days I would start with a physical host, do a full Linux install, add mySQL, Apache, etc and eventually WordPress. I don’t want to take this much time.
What’s the fastest, easiest way to get a full-blown WordPress setup? Turns out, the cloud is a pretty good place to start.
[Sorry for the sporadic posting. I’ve had more travel in the past 7 weeks than the last 2 years. I should be back to a more regular schedule soon.]
The “cloud” is still a new and curious beast for a lot of us, especially people who grew up in a more traditional hosting model. We have several generations of IT workers who have learned everything about hosting on our own hardware and networks. The flexibility of the cloud is a game-changer, and I’m continually learning new places where “conventional wisdom” will lead you down a difficult path.
Netflix has been kind enough to post their five key lessons from their cloud experiences on their tech blog. While these lessons may look simple and perhaps obvious in retrospect, there are two that really hit home with me:
1. Prepare to unlearn everything you know about running applications in your own datacenters.
3. The best way to avoid failure is to fail constantly.
First, an entire generation (or maybe two or three) of system and network administrators learned all of what we know about scale and reliability by running our own applications on our own servers in our own datacenters using our own networks. There are thousands of person-centuries of of experience that have created best (or at least “good”) practice on how to be successful in this model, but this has done very little to prepare us to be successful using cloud resources. In fact, it might even be working against us.
We’ve all got a lot to un-learn.
Second, in the olden days, uptime was king, and a high time between reboots (or crashes) was considered a mark of a capable system administrator. Failure was to be avoided at all costs, and testing failover (or disaster recovery) was done infrequently, if at all, due to the high impact and high costs. We did all get used to a more frequent reboot cycle, if only to be able to install all the needed security patches, but that was just a small change in focus, not a complete sea change.
In computing clouds, it is a given and an expectation that instances will fail at random, and the solution is to have an agile application, not to focus on high availability or increasing hardware reliability. Just as there is continuous development, testing and deployment, there needs to be continuous failover testing. Netflix created a tool (Chaos Money) specifically to force random failures in their production systems! That’s right, they are constantly creating failures, just to continuously test their failover methods, in the live production system.
That’s a) really hardcore, b) really scary and c) really cool.
That’s one way to put your reputation on the line. And it points out just how you need to do some very non-intuitive things, and unlearn decades of good practice to be successful in the cloud.