I'm just talking about what a system call might do and at what level.  I could talk about what Unix (C, C++, Java) does, but most recently I've programmed Tops-20.  Aside from SOUTR% setting PSH, the real question that I'm trying to get to the bottom of is what breaks the SINR% out early.

SINR% will return either a full buffer or a partly full buffer if an end-of-record is detected.  For a magnetic tape, the hardware provides this.  For TCP, the PSH flag defines the record boundary; I know this from having looked at the monitor code, Wireshark'ed, Etc.).

Clearly, the SINR% can also return early on DECnet and DAP depends on this.  But the question remains: what is performing the equivalent DECnet function of the PSH?

This was the bug I caused.  It stems from an optimization that I tried to transfer from Kermit-20.  As background, in June 2019, I fixed a bug for Tops-20 NFT/FAL breaking on very long directory listings.  I mean huge listings.   Part of the problem was that DAP was using memory very inefficiently (4K buffers, always) and could run out.

Like some other bug I've bumped into, the problem was multi-faceted, with DAP tickling a problem in GLXMEM, which I had to go fix before finally converging on the DAP issue.  Briefly, not all DAP messages are going to use the the maximum amount of memory and the fix was to trim unused memory and run the garbage collector more often.  For example:

[Sending Configuration message (Flg=0 Cnt=17. Rem=0.)]
[Allocated Receive buffer (Words=1033. Len=10000.)]
[Trimmed 1019. words from receive buffer]
[Received Configuration message segment (Flg=2 Cnt=18.)]

So that's fine and it works, Etc., Etc.  I had wondering about the computational efficiency of trimming and hitting the garbage collector so much.  What I do in Kermit is determine exactly how much data is in the monitor's buffer and read that.  It works because of the way the protocol works.  Anyway, I figured if I always knew what to allocate, I'd never need to do any segment trimming with the consequence garbage collection.

Since DAP isn't lock-step to the extent that Kermit is, you can get into a race condition where you check the available data in the monitor, but by the time you have allocated the perfectly sized segment, more data will have come in which blows the SINR% up with an IOX10 error (Record is longer than user requested).

So, oops.  My bad.   (which is what I had suspected)

The 'fix' was to just chuck the code that I had put in and live with the garbage collection overhead until I figure something more clever out.  Or just please leave it alone.

But I got to wondering how a DECnet SINR% knows to wake up early.  Something in the protocol has to tell it, right?

Well, if you are talking about what the system calls might do in TOPS-20, then I can't make any comments.

I would guess/assume that the %SOUTR would flush out any buffered data then.

But down at the DECnet level, it's packets. DECnet don't have a concept of a stream of bytes.

And yeah, looks correct. A file that is protected from accessing would give you some sort of error. Someone could check the MAC and MIC code, if they are really curious, to verify the error.

  Johnny

On 2022-11-26 02:02, Thomas DeBellis wrote:

Yes, agreed.

However, I'm not sure if you and I are talking about the same level in the network stack.  I had meant to only talk about the network stack in an implied and tangential manner.

What I was getting is that SOUTR% is used in TCP to push accumulated data out.  There is no idea of a record, it's just doing a push.  From the code that I am seeing in FAL and NFT, it appears that SOUTR% is being used in the same way.  I should have clarified that both SOUT% and SOUTR% view the data as a stream of bytes (of varying widths, of course).  There is only a record on those media that support it, magnetic tape as far as I remember.

I /think/ I may have fixed my most recent bug.  I had put a data transfer optimization in from some code that I wrote in Kermit-20 for the NRT and PTY driver that I had picked up from my FTP server.  It tickles a completely understandable run condition which can not occur in Kermit but can in DAP because Kermit is acknowledging individual packets at the application level while DAP does not.

Well, that's what I think so far while I'm testing.  One hopeful result is that an anonymous directory listing against MIM:: works now.  Well, works /mostly/. I'm getting the error below:

    NNQUERY.C;208   Remote status - File protection violation (MAC:4
    MIC:125 STV:0)

    .
    .
    .

    NNQUERY.C;199   Remote status - File protection violation (MAC:4
    MIC:125 STV:0)
    NNQUERY.MAP;285;P775656         17 34816(8) 18-Oct-2022 09:59:35

I think this is correct, because when I logged into MIM:: and checked further, I got:

    $ *dsp DU1:[DECNET]NNQUERY.C;208*
    ?DSP-F-OPNINP, Error opening DU1:[DECNET]NNQUERY.C;208 as input
    -RMS-E-ER$PRV, Privilege violation

I don't remember protected files like that in DU1:[DECNET], but it's been awhile since I've tested against multiple systems (last May, in fact)


On 11/25/22 6:47 PM, Johnny Billquist wrote:

There is, as far as I know, no equivalent to push, since in DECnet, connections are never handled as a stream of bytes, but as packets, and delivery is always immediate, and the sending side is directly informed about the receiver actually receiving the packet. It's essentially much more synchronous than TCP/IP. DECnet don't even necessarily buffer on the sending side. At least with RSX, transmits goes directly from the task out on the wire. And the send is only flagged complete when the receiving side have received it. But on receiving side it might be in an internal buffer and not actually delivered to the recipient task.

But I'm sure Paul remember more details.

However, one thing you do have in DECnet, which TCP/IP don't really do well, is that you can send out of bounds, urgent data in DECnet.

  Johnny

On 2022-11-26 00:05, Thomas DeBellis wrote:

In TCP, something called the 'push flag' can be set to cause accumulated data in the monitor (which may be one or more packets) to be sent over the network.  It is often abbreviated as "PSH".

The Tops-20 FTP client and server both set PSH on the last packet of data from a file to send it on its way. This is done with a SOUTR% JSYS, which stands for *S*tring *OUT* *R*ecord. This is an overloading of the hardware record concept which is more commonly associated with nine-track magnetic tape.

PSH can also be detected on input by doing a SINR%, which stops the network read early, instead of waiting for a full buffer of data (which might, in fact, never come).

I believe DECnet transport implements the same semantics, but I would like to double check that.  Where would I go in which part of what specification?  Or does anybody know?



_______________________________________________
HECnet mailing list -- hecnet@lists.dfupdate.se
To unsubscribe send an email to hecnet-leave@lists.dfupdate.se

_______________________________________________
HECnet mailing list -- hecnet@lists.dfupdate.se
To unsubscribe send an email to hecnet-leave@lists.dfupdate.se