On 2015-03-16 20:43, Clem Cole wrote:
On Mon, Mar 16, 2015 at 3:13 PM, Johnny Billquist <bqt at softjar.se
<mailto:bqt at softjar.se>> wrote:
And one of the tricks were to reduce disk seek times during a
backup. And that is done by actually not backing up files in the
logical order of the blocks in the file, but in the order they
appear on disk, which is why I said the format is slightly tricky.
That was a common trade off - the reason being that backup was done way
more often then restore. But the problem was that the a lot of the
those formats had a big issues when you had a tape error (UNIX tp and
the PDP-10 dumper formats were notorious for this problem).
Tape errors gives you problems no matter which way you cut it. The BRU format allows you to deal with the rest of the file/tape properly anyway.
But as you say a threaded format like that, you don't need a PhD to
figure it out. You just need to be careful and recognize there is a
bunch of housekeeping needed when you pull things back in. Today its a
little easier, but on an 11/40 class (shared I/D space) that would be
even more difficult, since you don't have a lot of room in active memory
to keep those tables.
Nitpick: The 11/40 "class" machines did not have split I/D space.
Split I/D space was the 11/45,11/50,11/55 and 11/70, before we move into "modern" PDP-11s...
The 11/40 is really a primitive model, where most things you even could have was optional. The basic CPU do not have an MMU, no EIS, no stack limit, not to mention that the 11/40 never even had the capability of an FPP. It had the FIS instead...
(And no I/D space, nor supervisor mode, nor 22-bit addressing.)
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
On Mon, Mar 16, 2015 at 3:13 PM, Johnny Billquist <bqt at softjar.se> wrote:
And one of the tricks were to reduce disk seek times during a backup. And that is done by actually not backing up files in the logical order of the blocks in the file, but in the order they appear on disk, which is why I said the format is slightly tricky.
That was a common trade off - the reason being that backup was done way more often then restore. But the problem was that the a lot of the those formats had a big issues when you had a tape error (UNIX tp and the PDP-10 dumper formats were notorious for this problem).
But as you say a threaded format like that, you don't need a PhD to figure it out. You just need to be careful and recognize there is a bunch of housekeeping needed when you pull things back in. Today its a little easier, but on an 11/40 class (shared I/D space) that would be even more difficult, since you don't have a lot of room in active memory to keep those tables.
Clem
On 2015-03-16 19:52, Clem Cole wrote:
On Mon, Mar 16, 2015 at 2:28 PM, Johnny Billquist <bqt at softjar.se
<mailto:bqt at softjar.se>> wrote:
I obviously don't know what Tom might have done. It's definitely
possible he did. But depending on when this was, it might have been
another format called DSC (Disk Save and Compress), which is what
RSX used up until V3.something. Which is, I think, almost mid-80s.
Johnny,
As I said, neither do it and I don't have the code on-line to look
anymore, but given the dates you mention, DSC sounds more reasonable.
His work would have been 82/83 ish IIRC. I seem to remember that the
tapes from Mitel came from an RSX system that one of our
officemates/fellow grad students brought with him, telling us the tapes
were an RSX backup format. When we mounted them and started to poke
around, the tools we had would not work without modification. Since I
was getting ready to bug out/graduate, I remember Tom dealt with it, I
just don't remember how he did it.
Yeah. And I don't know at all how the DSC format looked like, but I believe it was simpler than BRU. One of the reasons for DEC to create BRU was to improve speed of backups. And one of the tricks were to reduce disk seek times during a backup. And that is done by actually not backing up files in the logical order of the blocks in the file, but in the order they appear on disk, which is why I said the format is slightly tricky.
This is a good example of knowing enough to be dangerous. I know a
little about RSX, but never dived in deep enough to be sure of any of
some of those details, so if you suggest that it was likely DSC, I
definitely bow to your knowledge.
:-)
That said, a quick look at:
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and, assuming no tape errors .... I suspect that it should not be that
hard to take any of the UNIX ansi tape readers that are available and
hack this format in them. Interesting to see the RAD50 stuff in there.
That takes me back to painful times.
Yeah, I have already studied that document at a great length.
The format is not hard. But the fact that you need to analyze some meta-information in order to understand where the disk blocks in the backup saveset actually fits into the restored file makes it a bit more interesting.
It also looks like the tape directory is in the front few blocks of the
tape, which was pretty typical of a number of tape formats, particularly
ones for backup. [This makes writing the tapes easier (faster) but it
also make recovery harder if you lose blocks in the directory].
Yes. Like said. It's not rocket science, but it is a bit more complicated than you might first realize.
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
On Mon, Mar 16, 2015 at 2:28 PM, Johnny Billquist <bqt at softjar.se> wrote:
I obviously don't know what Tom might have done. It's definitely possible he did. But depending on when this was, it might have been another format called DSC (Disk Save and Compress), which is what RSX used up until V3.something. Which is, I think, almost mid-80s.
Johnny,
As I said, neither do it and I don't have the code on-line to look anymore, but given the dates you mention, DSC sounds more reasonable. His work would have been 82/83 ish IIRC. I seem to remember that the tapes from Mitel came from an RSX system that one of our officemates/fellow grad students brought with him, telling us the tapes were an RSX backup format. When we mounted them and started to poke around, the tools we had would not work without modification. Since I was getting ready to bug out/graduate, I remember Tom dealt with it, I just don't remember how he did it.
This is a good example of knowing enough to be dangerous. I know a little about RSX, but never dived in deep enough to be sure of any of some of those details, so if you suggest that it was likely DSC, I definitely bow to your knowledge.
That said, a quick look at: http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and, assuming no tape errors .... I suspect that it should not be that hard to take any of the UNIX ansi tape readers that are available and hack this format in them. Interesting to see the RAD50 stuff in there. That takes me back to painful times.
It also looks like the tape directory is in the front few blocks of the tape, which was pretty typical of a number of tape formats, particularly ones for backup. [This makes writing the tapes easier (faster) but it also make recovery harder if you lose blocks in the directory].
Clem
On 2015-03-16 18:32, Clement T. Cole wrote:
Yes. It should be by easy to do, although I thought Tom put the BRU support code in there because I remember one the folks we worked (Mital in Canada IIRC) used RSX somewhere and he had the deal with it when then sent us tapes. But may be it was not BRU. I don't remember as that worked started as I was leaving UCB and Tom had become the new mr. 9-track and Thus I did not write that code (I did write the original RT11 support with which he started).
I obviously don't know what Tom might have done. It's definitely possible he did. But depending on when this was, it might have been another format called DSC (Disk Save and Compress), which is what RSX used up until V3.something. Which is, I think, almost mid-80s.
BRU has been documented enough that it's not hard to write a program to extract the files from it, but it's is not totally trivial, as the files are somewhat scrambled in the saveset. I wrote a program to analyze and manipulate BRU tapes under Unix, to be able to repair tapes that had become corrupted (tape blocks scrambled).
When Tom took over he wrote a whole new program because the CAD group had had to start to deal with VMS so much and my original ANSI reader for RT11 tapes was pretty lame. When he was done, I'm pretty sure he could handle VMS save sets in his version because that what the CAD group @ DEC would use. I remember that HP used some funky format from the HP3000 which I ended up decoding using dd and some shell scripts and when never used again as we got them to switch to tar. IBM was always a different format (usually in EBCDIC). I remember once getting an ANSI labeled tape from them but in EBCDIC and broke all our tools. AT&T could also be funny. Different groups there used different tools even within UNIX so I got pretty good handling strange formats since they often sent tapes in binary and had endianness mixed in to add to the confusion. At the time, I became the go to person in Cory Hall (and often Evans folks would come to see me too) because I had spent time in a mainfra
m
e shop long before UCB and learned a lot of tricks and had a tool kit for same.
VMS backup savesets are totally incompatible with BRU savesets, though (sad). But I know I've seen Unix tools to read VMS tapes and savesets.
But both VMS and RSX use normal ANSI format tapes, so the first level of just getting to the savesets as such, is easy on Unix. Plenty of tools already exist to deal with ANSI labelled tapes.
Johnny
Somewhere I have a script called mtaapita - mag tapes are a pain in the ass - which I wrote to help pull the first few records apart and try to figure out how the tape was formatted. As I said, I used to have a pretty good collection of mag tape tools but they are not online anymore ( undoubtedly on a 9-track Unix dump tape in my basement).
Sent from my iPad
On Mar 16, 2015, at 12:25 PM, Johnny Billquist <bqt at softjar.se> wrote:
On 2015-03-16 14:08, Clem Cole wrote:
The format is defined in:
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and yes UNIX has tools that grok it.
Worst case it any ANSI tape reader will get the raw data off as files
with strange names and you may need a shell script to put some of the
files back together after decoding the directory.
My old housemate (and SPICE2/3 author) Tom Quarles wrote an really good
ANSI tape reader for Unix years ago. It did VMS format by default but
seem to remember he added RT11 and RSX support to it also. Since we had
to write tapes to export to those systems also.
Sadly, I used to have a directory of "magtape utilities" but I no longer
have it online. I'll try to poke around to see if I can find it next
weekend if no one else shouts out sooner.
BRU tapes are proper ANSI format tapes, so as far as that goes, the whole saveset as such can certainly be accessed with any tool that deals with ANSI tapes.
However, BRU savesets is another level, below this. To extract files from those savesets will require other software. I don't think any exists for Unix, but you can certainly write it.
Johnny
Clem
On Mon, Mar 16, 2015 at 1:05 AM, Cory Smelosky <b4 at gewt.net
<mailto:b4 at gewt.net>> wrote:
All,
Is it possible to extract files from a BRU RSX tape on UNIX, or is
the best path BRUREAD to VMS, and then zip/tar?
--
Cory Smelosky
http://gewt.net Personal stuff
http://gimme-sympathy.org Projects
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
below
On Mon, Mar 16, 2015 at 1:47 PM, <Paul_Koning at dell.com> wrote:
I remember the ANSI format produced by OS/360. It faintly resembled real ANSI labels, but the text was encoded in 8 bit ASCII , an IBM invention that no one else ever used before or since. I think they took regular 7 bit ASCII and moved the top bit up one so, for example, A was encoded as 0x81 rather than 0x41 as in real ASCII. After that discovery, I switched back to IBM standard labels (IBM proprietary, in EBCDIC), or more usually, unlabeled since IBM didn t do useful label processing anyway.
I think I remember getting a few of these from the IBM Cambridge folks.
On Mar 16, 2015, at 1:32 PM, Clement T. Cole <clemc at ccc.com> wrote:
... IBM was always a different format (usually in EBCDIC). I remember once getting an ANSI labeled tape from them but in EBCDIC and broke all our tools.
I remember the ANSI format produced by OS/360. It faintly resembled real ANSI labels, but the text was encoded in 8 bit ASCII , an IBM invention that no one else ever used before or since. I think they took regular 7 bit ASCII and moved the top bit up one so, for example, A was encoded as 0x81 rather than 0x41 as in real ASCII. After that discovery, I switched back to IBM standard labels (IBM proprietary, in EBCDIC), or more usually, unlabeled since IBM didn t do useful label processing anyway.
paul
Yes. It should be by easy to do, although I thought Tom put the BRU support code in there because I remember one the folks we worked (Mital in Canada IIRC) used RSX somewhere and he had the deal with it when then sent us tapes. But may be it was not BRU. I don't remember as that worked started as I was leaving UCB and Tom had become the new mr. 9-track and Thus I did not write that code (I did write the original RT11 support with which he started).
When Tom took over he wrote a whole new program because the CAD group had had to start to deal with VMS so much and my original ANSI reader for RT11 tapes was pretty lame. When he was done, I'm pretty sure he could handle VMS save sets in his version because that what the CAD group @ DEC would use. I remember that HP used some funky format from the HP3000 which I ended up decoding using dd and some shell scripts and when never used again as we got them to switch to tar. IBM was always a different format (usually in EBCDIC). I remember once getting an ANSI labeled tape from them but in EBCDIC and broke all our tools. AT&T could also be funny. Different groups there used different tools even within UNIX so I got pretty good handling strange formats since they often sent tapes in binary and had endianness mixed in to add to the confusion. At the time, I became the go to person in Cory Hall (and often Evans folks would come to see me too) because I had spent time in a mainframe shop long before UCB and learned a lot of tricks and had a tool kit for same.
Somewhere I have a script called mtaapita - mag tapes are a pain in the ass - which I wrote to help pull the first few records apart and try to figure out how the tape was formatted. As I said, I used to have a pretty good collection of mag tape tools but they are not online anymore ( undoubtedly on a 9-track Unix dump tape in my basement).
Sent from my iPad
On Mar 16, 2015, at 12:25 PM, Johnny Billquist <bqt at softjar.se> wrote:
On 2015-03-16 14:08, Clem Cole wrote:
The format is defined in:
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and yes UNIX has tools that grok it.
Worst case it any ANSI tape reader will get the raw data off as files
with strange names and you may need a shell script to put some of the
files back together after decoding the directory.
My old housemate (and SPICE2/3 author) Tom Quarles wrote an really good
ANSI tape reader for Unix years ago. It did VMS format by default but
seem to remember he added RT11 and RSX support to it also. Since we had
to write tapes to export to those systems also.
Sadly, I used to have a directory of "magtape utilities" but I no longer
have it online. I'll try to poke around to see if I can find it next
weekend if no one else shouts out sooner.
BRU tapes are proper ANSI format tapes, so as far as that goes, the whole saveset as such can certainly be accessed with any tool that deals with ANSI tapes.
However, BRU savesets is another level, below this. To extract files from those savesets will require other software. I don't think any exists for Unix, but you can certainly write it.
Johnny
Clem
On Mon, Mar 16, 2015 at 1:05 AM, Cory Smelosky <b4 at gewt.net
<mailto:b4 at gewt.net>> wrote:
All,
Is it possible to extract files from a BRU RSX tape on UNIX, or is
the best path BRUREAD to VMS, and then zip/tar?
--
Cory Smelosky
http://gewt.net Personal stuff
http://gimme-sympathy.org Projects
On 2015-03-16 14:08, Clem Cole wrote:
The format is defined in:
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and yes UNIX has tools that grok it.
Worst case it any ANSI tape reader will get the raw data off as files
with strange names and you may need a shell script to put some of the
files back together after decoding the directory.
My old housemate (and SPICE2/3 author) Tom Quarles wrote an really good
ANSI tape reader for Unix years ago. It did VMS format by default but
seem to remember he added RT11 and RSX support to it also. Since we had
to write tapes to export to those systems also.
Sadly, I used to have a directory of "magtape utilities" but I no longer
have it online. I'll try to poke around to see if I can find it next
weekend if no one else shouts out sooner.
BRU tapes are proper ANSI format tapes, so as far as that goes, the whole saveset as such can certainly be accessed with any tool that deals with ANSI tapes.
However, BRU savesets is another level, below this. To extract files from those savesets will require other software. I don't think any exists for Unix, but you can certainly write it.
Johnny
Clem
On Mon, Mar 16, 2015 at 1:05 AM, Cory Smelosky <b4 at gewt.net
<mailto:b4 at gewt.net>> wrote:
All,
Is it possible to extract files from a BRU RSX tape on UNIX, or is
the best path BRUREAD to VMS, and then zip/tar?
--
Cory Smelosky
http://gewt.net Personal stuff
http://gimme-sympathy.org Projects
The format is defined in: http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/rsx/dec…
and yes UNIX has tools that grok it.
Worst case it any ANSI tape reader will get the raw data off as files with strange names and you may need a shell script to put some of the files back together after decoding the directory.
My old housemate (and SPICE2/3 author) Tom Quarles wrote an really good ANSI tape reader for Unix years ago. It did VMS format by default but seem to remember he added RT11 and RSX support to it also. Since we had to write tapes to export to those systems also.
Sadly, I used to have a directory of "magtape utilities" but I no longer have it online. I'll try to poke around to see if I can find it next weekend if no one else shouts out sooner.
Clem
On Mon, Mar 16, 2015 at 1:05 AM, Cory Smelosky <b4 at gewt.net> wrote:
All,
Is it possible to extract files from a BRU RSX tape on UNIX, or is the best path BRUREAD to VMS, and then zip/tar?
--
Cory Smelosky
http://gewt.net Personal stuff
http://gimme-sympathy.org Projects
