[HECnet] Re: DAP Configuration BUFSIZ operator

------------------------------------------------------------------------ On 4/25/23 9:59 AM, Thomas DeBellis wrote: Thanks!!  The RSX NFT /D switch reminds me that the list of features that I want to lift from RSX NFT and put into Tops-20 NFT just keeps getting longer... Right now, I'm in the middle of changing the buffering mechanism for Tops-20 DAP to use correctly allocated sizes from a post-build static buffer header instead of a maximally allocated and trimmed dynamic buffer and then compacted post-build. The latter current method only allows me to buffer up 170 messages, most of which are not very large at all.  I would assume that I should be seeing far more than that given that I have 448 pages of buffer memory to play with.  Admittedly, some of that is being taken up by linked list overhead, but it still seems like a flagrant waste of memory. Of course, using that much main storage either way back in the day would almost certainly have gotten me 'a good talking to' (I.E., yelled at) for being a memory hog and paging the system into oblivion.  One assumes that there would be a configurable parameter to use instead of what I'm doing (grabbing as much as I can get my hands on in the section zero address space). > ------------------------------------------------------------------------ > On 4/24/23 9:51 PM, Johnny Billquist wrote: > > A quick check on some different systems that are online on HECnet at > the moment didn't reveal any with a bufsiz of 0 or greater than 4096. > > I checked TOPS-20, Ultrix (a few different versions), VMS (including > V9.2), RSTS/E and RSX. > > With RSX, it's fairly easy to check: > > .nft venti2::/id > NFT -- Version V9.0 > Local NFARs V8   DAP V7.1 Buffer size=  2064. OS=RSX-11M+ FS=FCS-11 > DC=Yes > Remote FAL  V2   DAP V5.6 Buffer size=  4096. OS=TOPS-20 FS=TOPS-20 > DC=Yes > > >   Johnny >> ------------------------------------------------------------------------ >> On 2023-04-25 00:13, Thomas DeBellis wrote: >> >> After I got done 'enough' with the auto-mounting code for FAL >> (meaning, I didn't make the modifications to the access control job >> because I bumped into .MSIMF), I turned to fixing a bug that I >> bumped into a few years ago. While Tops-20 does not support >> wildcarding devices, it does support wildcarding all structures that >> the system currently has online via the DSK*: moniker. >> >> So, let's say you're a 60+ who is having those occasional lapses of >> memory and you /really/ can't remember a single thing about >> something you think you might have somewhere. This is, of course, >> completely forgetting that you wanted to find something in the first >> place... you can ask the EXEC to do something like, DIRECT >> DSK*:<*>*SOMETHING*.*.0.  You can also hand that to FIND to have it >> look in all those files. >> >> Some experimentation with NFT shows that Tops-20 FAL supports >> structure wildcarding.  BUT!!  If you're being nosy or arejust >> plain hacking and do DIRECT DSK*:<*>*.*.*, it crashes with a GLXLIB >> out of memory error (ASE, Out of Address Space)... >> >> Researching this shows that Tops-20 DAPLIB tries to buffer messages >> before dumping them over the wire and that the decision it makes to >> dump is file count based.  That appeared to not work properly >> because of two issues.  First, each file was actually generating >> four separate messages: a Name, an Attributes, a Date/Time >> Attributes and a File Protection Attributes.  Second, the buffer >> builder was always allocating a buffer for the maximum size message, >> 4090 characters (1032 thirty-six bit words). >> >> That's pretty excessive, considering the typical sizes. Some >> instrumentation shows the below: >> >>     Message     Allocated     Actual %Excess >>     Name           4,090       18         99.56% >>     Attributes     4,090       17         99.58% >>     Date/Time      4,090       40         99.02% >>     Protection     4,090        7         99.83% >> >> I changed the buffering algorithm to dump based on available >> memory.  The starting available memory is 448 pages, so I take ¾ of >> that to devote to DAP buffers and ¼ (122 pages) to do the dumping. >> That fixed the problem.  Then I changed the memory allocation to >> trim unused words from the end of messages. That's not working as >> well as I expected, probably because I need compact (or 'burp') the >> memory.  Another alternative would be to build all messages in a >> single area and then only allow a chunk in the linked list for what >> I need.  No burping. >> >> A concern is what happens with the Configuration BUFSIZ operator and >> what to do if it is 0 (unlimited).  The DAP specification specifies >> that an operand of zero means that if one of the two systems has >> an  unlimited buffer size, it sends a 0 and the two systems >> will use  the  nonzero buffer size as the maximum.  If both systems >> send 0, thereis no limit on the length  of messages which may be sent. >> >> My question is whether zero is absurd.  Does any implementationsend >> that? >> >> It probably doesn't matter that much for sending, but I'm also >> thinking that I might hypothetically have a similar problem on >> receive.  Obviously I don't because the current Tops-20 DAP won't >> negotiate a size larger than 4096 bytes.  Is anything running larger >> than that? Would it be silly to?