On 2015-01-08 17:57, Mark Pizzolato - Info Comm wrote:
On Thursday, January 08, 2015 at 8:38 AM, Johnny Billquist wrote:
On 2015-01-08 17:02, Paul_Koning at Dell.com wrote:
On Jan 7, 2015, at 10:43 PM, Johnny Billquist <bqt at softjar.se> wrote:
So the question is - what do VMS do when talking to another VMS
system, and how does VMS decide when to use the extra abilities?
If there isn t a spec, the alternative would be to capture an FTP session
with a tool like Wireshark, and reverse engineer things. That tends to be a
pain but it can be done if all else fails.
Yeah. At this point I suspect that would be the easiest. Sending a really small
file over should tell me enough, I think.
Anyone who have both a VMS client and server, and some machine
inbetween that can run tcpdump or similar, and who could get me a dump of
a simple file?
Please keep in mind that any passwords you enter in the session will also be
in such a log...
I did the Wireshark trace and doing a GET there is an initial exchange of STRU O VMS and if the comes back with a "200 OK" the client knows that the server is VMS and probably viceversa. Then the data delivered in the RETR (separate TCP session as normal) is prefixed with a binary blob of data. The format of that blob is important. I recalled that MadGoat FTP also implemented this functionality (and interoperated with it - Matt Madison (of MadGoat) ended up working for the MultiNet folks for a while). In any case, over the years MGFTP became completely open source and is still maintained by Hunter Goatley (the Goat part of MadGoat). The source is available, but digging through the bliss code seemed a little harder than asking Hunter Goatley directly which I've done. I'll let you know what he says.
Aha! That is really good information. Now I just wonder if it always tries this trick, or only if the other side identifies itself as a VMS system. Could you do the same test against a Unix system and see if it tries the STRU command there too?
The binary blob is what I need to figure out next...
Meanwhile, the HGFTP stuff is available at: http://vms.process.com/ftp/vms-freeware/fileserv/hgftp.zip
Thanks. Might be worth looking into.
As of now, Text transfers to and from RSX works fine, and I've extended the directory output a little. I'm wondering if I should bother trying to fool things like web browsers next, as they also tries to be clever and tries to understand a Unix-style directory output.
Johnny
On Thursday, January 08, 2015 at 8:38 AM, Johnny Billquist wrote:
On 2015-01-08 17:02, Paul_Koning at Dell.com wrote:
On Jan 7, 2015, at 10:43 PM, Johnny Billquist <bqt at softjar.se> wrote:
...Note where I said "page structure" above. :-) I've read RFC 959
(as well as the various additions) backwards and forwards way too many
times by now.
The page structure would be a way of doing this. It will definitely not work
with any other client out there. But on the other hand, I'm mostly interested
in the efficient transport of files between two RSX systems for this part
anyway, so it will be towards my own client only.
But I doubt VMS does it this way, and I was curious about if I could
possibly do something that would be compatible with VMS, since enough
commonality in file storage exists between RSX and VMS for this to actually
be useful.
So the question is - what do VMS do when talking to another VMS
system, and how does VMS decide when to use the extra abilities?
If there isn t a spec, the alternative would be to capture an FTP session
with a tool like Wireshark, and reverse engineer things. That tends to be a
pain but it can be done if all else fails.
Yeah. At this point I suspect that would be the easiest. Sending a really small
file over should tell me enough, I think.
Anyone who have both a VMS client and server, and some machine
inbetween that can run tcpdump or similar, and who could get me a dump of
a simple file?
Please keep in mind that any passwords you enter in the session will also be
in such a log...
I did the Wireshark trace and doing a GET there is an initial exchange of STRU O VMS and if the comes back with a "200 OK" the client knows that the server is VMS and probably viceversa. Then the data delivered in the RETR (separate TCP session as normal) is prefixed with a binary blob of data. The format of that blob is important. I recalled that MadGoat FTP also implemented this functionality (and interoperated with it - Matt Madison (of MadGoat) ended up working for the MultiNet folks for a while). In any case, over the years MGFTP became completely open source and is still maintained by Hunter Goatley (the Goat part of MadGoat). The source is available, but digging through the bliss code seemed a little harder than asking Hunter Goatley directly which I've done. I'll let you know what he says.
Meanwhile, the HGFTP stuff is available at: http://vms.process.com/ftp/vms-freeware/fileserv/hgftp.zip
In the days before we all had internet access, I had uucp transfers between VMS machines (via DECUS UUCP) which also preserved file attributes. I don't recall how I determined if the peer system was a VMS one, but I do recall that I also prefixed the file data with the attributes. The attributes I sent were a FDL (File Definition Language) text blob which I was then able to use directly with some FDL library API to create the desired file and I then did block I/O to populate the contents.
- Mark
On 2015-01-08 17:02, Paul_Koning at Dell.com wrote:
On Jan 7, 2015, at 10:43 PM, Johnny Billquist <bqt at softjar.se> wrote:
...Note where I said "page structure" above. :-)
I've read RFC 959 (as well as the various additions) backwards and forwards way too many times by now.
The page structure would be a way of doing this. It will definitely not work with any other client out there. But on the other hand, I'm mostly interested in the efficient transport of files between two RSX systems for this part anyway, so it will be towards my own client only.
But I doubt VMS does it this way, and I was curious about if I could possibly do something that would be compatible with VMS, since enough commonality in file storage exists between RSX and VMS for this to actually be useful.
So the question is - what do VMS do when talking to another VMS system, and how does VMS decide when to use the extra abilities?
If there isn t a spec, the alternative would be to capture an FTP session with a tool like Wireshark, and reverse engineer things. That tends to be a pain but it can be done if all else fails.
Yeah. At this point I suspect that would be the easiest. Sending a really small file over should tell me enough, I think.
Anyone who have both a VMS client and server, and some machine inbetween that can run tcpdump or similar, and who could get me a dump of a simple file?
Please keep in mind that any passwords you enter in the session will also be in such a log...
Johnny
On Jan 7, 2015, at 10:43 PM, Johnny Billquist <bqt at softjar.se> wrote:
...Note where I said "page structure" above. :-)
I've read RFC 959 (as well as the various additions) backwards and forwards way too many times by now.
The page structure would be a way of doing this. It will definitely not work with any other client out there. But on the other hand, I'm mostly interested in the efficient transport of files between two RSX systems for this part anyway, so it will be towards my own client only.
But I doubt VMS does it this way, and I was curious about if I could possibly do something that would be compatible with VMS, since enough commonality in file storage exists between RSX and VMS for this to actually be useful.
So the question is - what do VMS do when talking to another VMS system, and how does VMS decide when to use the extra abilities?
If there isn t a spec, the alternative would be to capture an FTP session with a tool like Wireshark, and reverse engineer things. That tends to be a pain but it can be done if all else fails.
paul
On Jan 8, 2015, at 6:10 AM, Sampsa Laine <sampsa at mac.com> wrote:
On 8 Jan 2015, at 13:07, Johnny Billquist <bqt at softjar.se> wrote:
...
Which text format, EBCDIC or ASCII? :P
The joke was more aimed at you stating that text files are the one universal format, which of course they are, until you meet IBM.
Or CDC which has 4 or more of its own, all different from either EBCDIC or ASCII.
paul
On 8 Jan 2015, at 13:59, Johnny Billquist <bqt at softjar.se> wrote:
Now, wait a minute.
So you have problems with GUI ftp clients and servers. So you go for kermit for file transfers. Which means going command line.
If you go command line, what's wrong with command line ftp, which will work without the silly issues of GUI tools?
You can create scripts for ftp as well...
And ftp deals just as well with the different types of files as kermit do.
Johnny
Well, basically, I don't really use FTP.
I mostly use SFTP when updating my websites, hosting setup etc, and for that I use a graphical client since the hosting is Unixy so everything works nicely - I just happened to try connect to a VMS box with Transmit.app and it was seriously borked, so I thought I'd mention it.
When I was mostly playing with Alphas and IA64 boxes, I'd use a set of scripts around command-line SFTP because it wasn't as broken as Transmit :)
Now I'm usually logged into the following:
- GORVAX over Telnet (VAX)
- HILANT over Telnet (VAX)
- CHIMPY over SSH (AXP)
Anyway, I don't run SSH on my VAXes (and something in somewhere* router is messing with the FTP, even in passive mode) so I started playing around with Kermit scripts and eventually found them so convenient that I now mostly use Kermit over SSH instead of the SFTP scripts.
Summa summarum: I use SFTP in normal situations for file transfer where available, most of the time. If it's not, I use Kermit over Telnet. And I run Kermit over SSH as well since it does no harm AND gives me the ability to pull or push files in-band should I feel the need to :)
Sampsa
* My current connection in Lebanon is NAT'd from the ISP, god knows what they might be doing to the FTP connection
On 2015-01-08 11:34, Sampsa Laine wrote:
On 8 Jan 2015, at 12:17, John H. Reinhardt <johnhreinhardt at yahoo.com> wrote:
On 1/7/2015 9:23 PM, Sampsa Laine wrote:
On 8 Jan 2015, at 04:03, Johnny Billquist <bqt at softjar.se> wrote:
Aha. Interesting output. That was sortof what I had a vague memory remembering it looking like. Now I need to decide if I want to do it similar.
Do people think file protection (for example) is useful to see? What about owner? I certainly have the information available, but I have not displayed it so far.
I would try to make it look as close to the *nix implementations, that way automated / GUI front-ends are more likely to work nicely with the server.
For example, I have a very nice GUI file transfer program called Transmit.app on OS X, but it REALLY gets confused when talking to VMS..
sampsa
I had the same problem with my Mac FTP utilities. I tried Panic's Transmit, Yummy FTP and a couple others. Yummy *almost* works except for file versions. But I've been using Filezilla <https://filezilla-project.org/> now for a few months and it seems to work pretty well. My biggest problem with it has been training it to recognize file types for automatic transfers (ASCII vs binary/image). It has Mac, Windows and Linux versions. I have V3.8.1 The latest is 3.10 but it won't work on my OS X 10.6.8 (Snow Leopard) system.
John H. Reinhardt
I pretty much gave up on this and now just use Kermit to move files back and forth when needed, I have a set of scripts that automatically log me in to all my systems with Kermit invoking the telnet or ssh connection - this way I can transfer a file at any given moment.
Now, wait a minute.
So you have problems with GUI ftp clients and servers. So you go for kermit for file transfers. Which means going command line.
If you go command line, what's wrong with command line ftp, which will work without the silly issues of GUI tools?
You can create scripts for ftp as well...
And ftp deals just as well with the different types of files as kermit do.
Johnny
On 2015-01-08 11:07, Sampsa Laine wrote:
Guys,
Just for fun I thought I'd like to attach a virtual RX02 drive to GORVAX (it's running SIMH).
Any of you guys know of the correct dd parameters to use to create a blank disk image?
If you want to be really sure, look up the documentation, but unless I remember wrong, it's 256K, so you'd do:
dd if=/dev/zero of=floppy bs=1024 count=256
as one example.
Johnny
On 2015-01-08 10:22, Hans Vlems wrote:
Johnny,
Just finished reading the ftp discussion and cannot figure whether you' ve found a VMS ftp server...
I can offer two, one on axp (real iron) the other is a simple vax. VMS 8.3 and 6.1, both run DEC's tcpip/ ucx product.
I haven't had a chance to test any myself, but the output provided in the thread have given me at least something to ponder.
It could be interesting to hit some in the future when/if I want to test for compatibility issues. But I can get back if that becomes relevant.
Johnny
Hans
Verzonden vanaf mijn BlackBerry 10-smartphone.
Origineel bericht
Van: Johnny Billquist
Verzonden: donderdag 8 januari 2015 08:40
Aan: hecnet at Update.UU.SE
Beantwoorden: hecnet at Update.UU.SE
Onderwerp: Re: [HECnet] VMS FTP server?
On 2015-01-08 04:47, Sampsa Laine wrote:
On 8 Jan 2015, at 05:43, Johnny Billquist <bqt at softjar.se> wrote:
A different question is how to deal with binary file transfers to (for example) Unix systems. I have some different options there as well, but I haven't figured one out yet that does the right thing under all circumstances.
How about writing an archiver that produces a simple binary file as output but retains all the RSX specific stuff within the archive, sort of like ZIP's -V option on VMS and then using something like Kermit over say Telnet to transfer those files around?
That is already sortof existing. You can use LBR to stuff any files into
a library, and then I've written something similar to uuencode/uudecode
that can process any file, creating a text file, which is the only
universal file format. :-)
I'm sure there must be a Kermit port for RSX-11 out there somewhere..
Of course. But Kermit isn't exactly efficient. Not to mention that you
cannot, on an RSX system, do kermit over telnet...
Johnny
On 2015-01-08 08:44, Sampsa Laine wrote:
On 8 Jan 2015, at 09:39, Johnny Billquist <bqt at softjar.se
<mailto:bqt at softjar.se>> wrote:
I'm sure there must be a Kermit port for RSX-11 out there somewhere..
Of course. But Kermit isn't exactly efficient. Not to mention that you
cannot, on an RSX system, do kermit over telnet...
Johnny
My idea was to extend the RSX-11 kermit port to include access over
telnet rather than using FTP.
Urgh.... No, I don't think so. Kermit-11 is rather messy to start with, and there isn't much free memory. I've mucked in Kermit-11 in the past. It's a good enough tool if you have a serial line, but it's not what I'd use if I have some other alternative.
Just throwing this out there since FTP currently seems quite Unix-oriented..
It's not. However, I want to try and be a bit more general than just passing bits between two RSX systems. Which means I should consider those other types of systems as well, when it comes to non-text files.
As for graphic UI interfaces that talk ftp, and can't deal with non-Unix systems - I think I'm going to ignore them. It's as broken as sometimes using ftp in a web browser is (files from windows machines anyone?)
Johnny
