26 Nov
2022
26 Nov
'22
4:56 a.m.
On Nov 25, 2022, at 10:18 PM, Johnny Billquist
<bqt(a)softjar.se> wrote:
I would just assume that SINR% returns when a record have been read. Size be damned.
Again, it's not a stream of bytes. It is records. An analogy with tape is really not
that far fetched.
If your read is larger than the record, you'll just get the whole record. If the read
is shorter, then some data will be lost.
Another analogy would be UDP, but with guaranteed delivery, order and no duplication.
Johnny
DECnet-Utrix did allow for stream of bytes but it was for all I/O on the connection and
you couldn’t run any of the standard DECnet protocols using this mechanism (I don’t think
anyone ever used this):
For normal use with standard DECnet protocols you would use SOCK_SEQPACKET to maintain
message boundaries. For this you would use SOCK_STREAM.
On Write:
Take whatever data you have and send it as a single record (both start of message and end
of message bits set)
Depending on flow control and resource usage you may get a different sequence of messages
each time the
application is run.
On Read:
Message boundary information is discarded and the inbound queue in the socket is just a
sequence of bytes.
When a short read is issued on a SEQPACKET socket any extra data is discarded so that
each read always starts
at the beginning of a record. When a short read is issued on a STREAM socket it leaves
the remaining bytes in the
socket ready for the next read.
The original idea was that maybe we could run some of the standard TCP-based protocols
(e.g. telnet) over a DECnet
infrastructure allowing us exercise the kernel level code before the DECnet protocols were
available. As it turned out we didn’t need the early testing and it was less than a page
of code.
John.
On 2022-11-26 03:34, Thomas DeBellis wrote:
I'm just talking about what a system call
/might/ do and at what level. I could talk about what Unix (C, C++, Java) does, but most
recently I've programmed Tops-20. Aside from SOUTR% setting PSH, the /real/ question
that I'm trying to get to the bottom of is what breaks the SINR% out early.
SINR% will return either a full buffer or a partly full buffer if an end-of-record is
detected. For a magnetic tape, the hardware provides this. For TCP, the PSH flag defines
the record boundary; I know this from having looked at the monitor code, Wireshark'ed,
Etc.).
Clearly, the SINR% can also return early on DECnet and DAP depends on this. But the
question remains: what is performing the equivalent DECnet function of the PSH?
This was the bug I caused. It stems from an optimization that I tried to transfer from
Kermit-20. As background, in June 2019, I fixed a bug for Tops-20 NFT/FAL breaking on
very long directory listings. I mean /*huge*/ listings. Part of the problem was that
DAP was using memory very inefficiently (4K buffers, always) and could run out.
Like some other bug I've bumped into, the problem was multi-faceted, with DAP
tickling a problem in GLXMEM, which I had to go fix before finally converging on the DAP
issue. Briefly, not all DAP messages are going to use the the maximum amount of memory
and the fix was to trim unused memory and run the garbage collector more often. For
example:
[Sending Configuration message (Flg=0 Cnt=17. Rem=0.)]
[Allocated Receive buffer (Words=1033. Len=10000.)]
[Trimmed 1019. words from receive buffer]
[Received Configuration message segment (Flg=2 Cnt=18.)]
So that's fine and it works, Etc., Etc. I had wondering about the computational
efficiency of trimming and hitting the garbage collector so much. What I do in Kermit is
determine exactly how much data is in the monitor's buffer and read that. It works
because of the way the protocol works. Anyway, I figured if I always knew what to
allocate, I'd never need to do any segment trimming with the consequence garbage
collection.
Since DAP isn't lock-step to the extent that Kermit is, you can get into a race
condition where you check the available data in the monitor, but by the time you have
allocated the perfectly sized segment, more data will have come in which blows the SINR%
up with an IOX10 error (Record is longer than user requested).
So, oops. My bad. (which is what I had suspected)
The 'fix' was to just chuck the code that I had put in and live with the garbage
collection overhead until I figure something more clever out. Or just please leave it
alone.
But I got to wondering how a DECnet SINR% knows to wake up _early_. Something in the
protocol has to tell it, right?
------------------------------------------------------------------------
On 11/25/22 8:53 PM, Johnny Billquist wrote:
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt(a)softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se Well, if you are talking about what the system
calls might do in TOPS-20, then I can't make any comments.
I would guess/assume that the %SOUTR would flush out any buffered data then.
But down at the DECnet level, it's packets. DECnet don't have a concept of a
stream of bytes.
And yeah, looks correct. A file that is protected from accessing would give you some sort
of error. Someone could check the MAC and MIC code, if they are really curious, to verify
the error.
Johnny
On 2022-11-26 02:02, Thomas DeBellis wrote:
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se Yes, agreed.
However, I'm not sure if you and I are talking about the same level in the network
stack. I had meant to only talk about the network stack in an implied and tangential
manner.
What I was getting is that SOUTR% is used in TCP to push accumulated data out. There is
no idea of a record, it's just doing a push. From the code that I am seeing in FAL
and NFT, it appears that SOUTR% is being used in the same way. I should have clarified
that both SOUT% and SOUTR% view the data as a stream of bytes (of varying widths, of
course). There is only a record on those media that support it, magnetic tape as far as I
remember.
I /think/ I may have fixed my most recent bug. I had put a data transfer optimization in
from some code that I wrote in Kermit-20 for the NRT and PTY driver that I had picked up
from my FTP server. It tickles a completely understandable run condition which can not
occur in Kermit but can in DAP because Kermit is acknowledging individual packets at the
application level while DAP does not.
Well, that's what I think so far while I'm testing. One hopeful result is that
an anonymous directory listing against MIM:: works now. Well, works /mostly/. I'm
getting the error below:
NNQUERY.C;208 Remote status - File protection violation (MAC:4
MIC:125 STV:0)
.
.
.
NNQUERY.C;199 Remote status - File protection violation (MAC:4
MIC:125 STV:0)
NNQUERY.MAP;285;P775656 17 34816(8) 18-Oct-2022 09:59:35
I think this is correct, because when I logged into MIM:: and checked further, I got:
$ *dsp DU1:[DECNET]NNQUERY.C;208*
?DSP-F-OPNINP, Error opening DU1:[DECNET]NNQUERY.C;208 as input
-RMS-E-ER$PRV, Privilege violation
I don't remember protected files like that in DU1:[DECNET], but it's been awhile
since I've tested against multiple systems (last May, in fact)
On 11/25/22 6:47 PM, Johnny Billquist wrote:
> There is, as far as I know, no equivalent to push, since in DECnet, connections are
never handled as a stream of bytes, but as packets, and delivery is always immediate, and
the sending side is directly informed about the receiver actually receiving the packet.
It's essentially much more synchronous than TCP/IP. DECnet don't even necessarily
buffer on the sending side. At least with RSX, transmits goes directly from the task out
on the wire. And the send is only flagged complete when the receiving side have received
it. But on receiving side it might be in an internal buffer and not actually delivered to
the recipient task.
>
> But I'm sure Paul remember more details.
>
> However, one thing you do have in DECnet, which TCP/IP don't really do well, is
that you can send out of bounds, urgent data in DECnet.
>
> Johnny
>
> On 2022-11-26 00:05, Thomas DeBellis wrote:
>> In TCP, something called the 'push flag' can be set to cause accumulated
data in the monitor (which may be one or more packets) to be sent over the network. It is
often abbreviated as "PSH".
>>
>> The Tops-20 FTP client and server both set PSH on the last packet of data from a
file to send it on its way. This is done with a SOUTR% JSYS, which stands for *S*tring
*OUT* *R*ecord. This is an overloading of the hardware record concept which is more
commonly associated with nine-track magnetic tape.
>>
>> PSH can also be detected on input by doing a SINR%, which stops the network read
early, instead of waiting for a full buffer of data (which might, in fact, never come).
>>
>> I believe DECnet transport implements the same semantics, but I would like to
double check that. Where would I go in which part of what specification? Or does anybody
know?
>>
>>
>>
>> _______________________________________________
>> HECnet mailing list -- hecnet(a)lists.dfupdate.se
>> To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se
>
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se