4 Oct
2024
4 Oct
'24
12:05 a.m.
I thought I would update everyone with where things stand with the
Tops-20 finger server.
Johnny and I agreed that, by default, the response to a DECnet finger
query is a sequences of records, each no longer than 132 bytes (more
typically 90), terminated by a line-feed. If the remote finger client
or operating system can handle larger buffers, then it can connect with
optional data=STREAM. Tops-20 will then dump everything in one giant
record. Only the Tops-20 finger client can handle this, although I
suspect maybe a VAX client might also be able to do it.
I had been making steady progress until I started stress testing it,
meaning having a session active on MIM::, TOMMYT::, and VENTI2::
(local), each having a finger command in ready to go and then pushing
carriage return on all three as simultaneously as possible.
This started generating errors on MIM:: that the "object wasn't
available". What was going on was that the structure of the Tops-20
finger server really wasn't architected to have real time response for
that much curiosity. It only opens a single SRV:117 (finger) object at a
time, waits for a connection, reads the data, hands it off to an idle
finger client fork via a pipe, and then gets a new SRV:117 object.
In other words, it isn't until the finger client is started with the
connection redirected to DECnet that the server is ready to accept
another connection. That has what I would consider to be noticeable
latency, particularly on failure. An error doesn't really matter for
SMTP as it is a background task and just tries again, later. An
interactive finger on the other hand, has a user sitting there waiting
for a response, so it seemed to me that this wasn't really going to cut it.
I took the model of the Tops-20 FAL server, which has a single control
fork, looking for illness in sub-forks, all of which open their own FAL
server object. The new finger controller now starts separate FNGSRV
sub-server forks, creating a FINGER sub-fork for each, gets all the
communications lined up, and starts all the FNGSRV sub-forks to listen
for connections.
This has the advantage of not clobbering the system on FNGSRV startup
because resources are gotten or creating sequentially, so there isn't
much for a FNGSRV sub-fork to do except wait for a connection and manage
its own single FINGER sub-fork.
Startup resource allocation looks like this:
[Fork FNGSRV opening PIP:1;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:1.1 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:2;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:2.2 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:3;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:3.3 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
The resulting JFN's being:
JFN File Mode Bytes(Size)
2 FNGSRV.EXE.330 Read, Execute
3 FINGER.EXE.116 Read, Execute
15 PIP:1;RECORD-LENGTH:200 Append 0.(8)
16 PIP:1.1 Read 0.(8)
17 SRV:117 Read, Append 0.(8)
20 PIP:2;RECORD-LENGTH:200 Append 0.(8)
21 PIP:2.2 Read 0.(8)
22 SRV:117 Read, Append 0.(8)
23 PIP:3;RECORD-LENGTH:200 Append 0.(8)
24 PIP:3.3 Read 0.(8)
25 SRV:117 Read, Append 0.(8)
The resulting fork structure is:
=> FNGSRV (2): HALT at STARTS+13, 0.02719
Fork 12: HALT at 0, 0.00018
Fork 13: HALT at 0, 0.00016
Fork 10: HALT at 0, 0.00012
Fork 11: HALT at 0, 0.00012
Fork 6: HALT at 0, 0.00008
Fork 7: HALT at 0, 0.00007
So fork 2 is the finger server controller, forks 12, 10, and 6 are
finger server sub-forks, and forks 13, 11, and 7 are the respective
Tops-20 finger clients. Times are in tens of microseconds, the maximum
resolution that Tops-20 supports. What can be seen is that fork
creation is happening in sub-millisecond time. This was not the case in
the 1980's with KL10's (I /think/), and modifications were necessary to
Tops-20 and the EXEC to capture the increased resolution.
I guess I'll have another version ready in about two weeks.
4 Oct
4 Oct
1:38 a.m.
Sounds like a good improvement.
However, I saw something about mail here, that you should be aware of...
My MAIL11 implementation is doing the queuing of mails, because that is
normal with SMTP, and I just carried over the same behavior to all parts
of the system.
However, the "standard" MAIL11 application for RSX as well as RSTS/E ,
along with the DECUS MAIL for RSX (and I suspect also VMS) do not behave
that way. If the connecting fails, the sending of mail immediately fail,
and it is not queued for retrying. So it would appears that the mail for
TOPS-20 you look at is also potentially flawed, which could cause issues.
Of course, in that case, you'd normally have a human sitting in front of
the terminal, who would then try again...
Johnny
On 2024-10-04 00:05, Thomas DeBellis wrote:
I thought I would update everyone with where things
stand with the
Tops-20 finger server.
Johnny and I agreed that, by default, the response to a DECnet finger
query is a sequences of records, each no longer than 132 bytes (more
typically 90), terminated by a line-feed. If the remote finger client
or operating system can handle larger buffers, then it can connect with
optional data=STREAM. Tops-20 will then dump everything in one giant
record. Only the Tops-20 finger client can handle this, although I
suspect maybe a VAX client might also be able to do it.
I had been making steady progress until I started stress testing it,
meaning having a session active on MIM::, TOMMYT::, and VENTI2::
(local), each having a finger command in ready to go and then pushing
carriage return on all three as simultaneously as possible.
This started generating errors on MIM:: that the "object wasn't
available". What was going on was that the structure of the Tops-20
finger server really wasn't architected to have real time response for
that much curiosity. It only opens a single SRV:117 (finger) object at a
time, waits for a connection, reads the data, hands it off to an idle
finger client fork via a pipe, and then gets a new SRV:117 object.
In other words, it isn't until the finger client is started with the
connection redirected to DECnet that the server is ready to accept
another connection. That has what I would consider to be noticeable
latency, particularly on failure. An error doesn't really matter for
SMTP as it is a background task and just tries again, later. An
interactive finger on the other hand, has a user sitting there waiting
for a response, so it seemed to me that this wasn't really going to cutit.
I took the model of the Tops-20 FAL server, which has a single control
fork, looking for illness in sub-forks, all of which open their own FAL
server object. The new finger controller now starts separate FNGSRV
sub-server forks, creating a FINGER sub-fork for each, gets all the
communications lined up, and starts all the FNGSRV sub-forks to listen
for connections.
This has the advantage of not clobbering the system on FNGSRV startup
because resources are gotten or creating sequentially, so there isn't
much for a FNGSRV sub-fork to do except wait for a connection and manage
its own single FINGER sub-fork.
Startup resource allocation looks like this:
[Fork FNGSRV opening PIP:1;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:1.1 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:2;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:2.2 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:3;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:3.3 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
The resulting JFN's being:
JFN File Mode Bytes(Size)
2 FNGSRV.EXE.330 Read, Execute
3 FINGER.EXE.116 Read, Execute
15 PIP:1;RECORD-LENGTH:200 Append 0.(8)
16 PIP:1.1 Read 0.(8)
17 SRV:117 Read, Append 0.(8)
20 PIP:2;RECORD-LENGTH:200 Append 0.(8)
21 PIP:2.2 Read 0.(8)
22 SRV:117 Read, Append 0.(8)
23 PIP:3;RECORD-LENGTH:200 Append 0.(8)
24 PIP:3.3 Read 0.(8)
25 SRV:117 Read, Append 0.(8)
The resulting fork structure is:
=> FNGSRV (2): HALT at STARTS+13, 0.02719
Fork 12: HALT at 0, 0.00018
Fork 13: HALT at 0, 0.00016
Fork 10: HALT at 0, 0.00012
Fork 11: HALT at 0, 0.00012
Fork 6: HALT at 0, 0.00008
Fork 7: HALT at 0, 0.00007
So fork 2 is the finger server controller, forks 12, 10, and 6 are
finger server sub-forks, and forks 13, 11, and 7 are the respective
Tops-20 finger clients. Times are in tens of microseconds, the maximum
resolution that Tops-20 supports. What can be seen is that fork
creation is happening in sub-millisecond time. This was not the case in
the 1980's with KL10's (I /think/), and modifications were necessary to
Tops-20 and the EXEC to capture the increased resolution.
I guess I'll have another version ready in about two weeks.
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt(a)softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
2:29 a.m.
And just if anyone wonders - yes, that also means you cannot send mail
to users on systems that are not currently online.
A thing that I always found a bit annoying/irritating.
Another reason why I wrote the mail handling for BQTCP to also do the
writing and sending as two separate steps, with a queuing in between.
Here is how it looks from VMS:
MAIL> sen
To: josse::bqt
%MAIL-E-LOGLINK, error creating network link to node JOSSE
-SYSTEM-F-UNREACHABLE, remote node is not currently reachable
MAIL> sen
To: anke::bqt
%MAIL-E-LOGLINK, error creating network link to node ANKE
-SYSTEM-F-NOSUCHOBJ, network object is unknown at remote node
MAIL>
It instantly tries to connect, and if that don't work, it immediately fails.
Johnny
On 2024-10-04 01:38, Johnny Billquist wrote:
Sounds like a good improvement.
However, I saw something about mail here, that you should be aware of...
My MAIL11 implementation is doing the queuing of mails, because that is
normal with SMTP, and I just carried over the same behavior to all parts
of the system.
However, the "standard" MAIL11 application for RSX as well as RSTS/E ,
along with the DECUS MAIL for RSX (and I suspect also VMS) do not behave
that way. If the connecting fails, the sending of mail immediately fail,
and it is not queued for retrying. So it would appears that the mail for
TOPS-20 you look at is also potentially flawed, which could cause issues.
Of course, in that case, you'd normally have a human sitting in front of
the terminal, who would then try again...
Johnny
On 2024-10-04 00:05, Thomas DeBellis wrote:
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt(a)softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
I thought I would update everyone with where
things stand with the
Tops-20 finger server.
Johnny and I agreed that, by default, the response to a DECnet finger
query is a sequences of records, each no longer than 132 bytes (more
typically 90), terminated by a line-feed. If the remote finger
client or operating system can handle larger buffers, then it can
connect with optional data=STREAM. Tops-20 will then dump everything
in one giant record. Only the Tops-20 finger client can handle this,
although I suspect maybe a VAX client might also be able to do it.
I had been making steady progress until I started stress testing it,
meaning having a session active on MIM::, TOMMYT::, and VENTI2::
(local), each having a finger command in ready to go and then pushing
carriage return on all three as simultaneously as possible.
This started generating errors on MIM:: that the "object wasn't
available". What was going on was that the structure of the Tops-20
finger server really wasn't architected to have real time response for
that much curiosity. It only opens a single SRV:117 (finger) object at
a time, waits for a connection, reads the data, hands it off to an
idle finger client fork via a pipe, and then gets a new SRV:117 object.
In other words, it isn't until the finger client is started with the
connection redirected to DECnet that the server is ready to accept
another connection. That has what I would consider to be noticeable
latency, particularly on failure. An error doesn't really matter
for SMTP as it is a background task and just tries again, later. An
interactive finger on the other hand, has a user sitting there waiting
for a response, so it seemed to me that this wasn't really going to
cutit.
I took the model of the Tops-20 FAL server, which has a single control
fork, looking for illness in sub-forks, all of which open their own
FAL server object. The new finger controller now starts separate
FNGSRV sub-server forks, creating a FINGER sub-fork for each, gets all
the communications lined up, and starts all the FNGSRV sub-forks to
listen for connections.
This has the advantage of not clobbering the system on FNGSRV startup
because resources are gotten or creating sequentially, so there isn't
much for a FNGSRV sub-fork to do except wait for a connection and
manage its own single FINGER sub-fork.
Startup resource allocation looks like this:
[Fork FNGSRV opening PIP:1;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:1.1 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:2;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:2.2 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:3;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:3.3 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
The resulting JFN's being:
JFN File Mode Bytes(Size)
2 FNGSRV.EXE.330 Read, Execute
3 FINGER.EXE.116 Read, Execute
15 PIP:1;RECORD-LENGTH:200 Append 0.(8)
16 PIP:1.1 Read 0.(8)
17 SRV:117 Read, Append 0.(8)
20 PIP:2;RECORD-LENGTH:200 Append 0.(8)
21 PIP:2.2 Read 0.(8)
22 SRV:117 Read, Append 0.(8)
23 PIP:3;RECORD-LENGTH:200 Append 0.(8)
24 PIP:3.3 Read 0.(8)
25 SRV:117 Read, Append 0.(8)
The resulting fork structure is:
=> FNGSRV (2): HALT at STARTS+13, 0.02719
Fork 12:HALT at 0, 0.00018
Fork 13: HALT at 0, 0.00016
Fork 10:HALT at 0, 0.00012
Fork 11: HALT at 0, 0.00012
Fork 6: HALT at 0, 0.00008
Fork 7: HALT at 0, 0.00007
So fork 2 is the finger server controller, forks 12, 10, and 6 are
finger server sub-forks, and forks 13, 11, and 7 are the respective
Tops-20 finger clients. Times are in tens of microseconds, the
maximum resolution that Tops-20 supports. What can be seen is that
fork creation is happening in sub-millisecond time. This was notthe
case in the 1980's with KL10's (I /think/), and modifications were
necessary to Tops-20 and the EXEC to capture the increased resolution.
I guess I'll have another version ready in about two weeks.
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se
5 Oct
5 Oct
3:54 a.m.
I'll get to mail again after I finish up with the Tops-20 finger
server. That being said, there is a MAIL11 implementation for Tops-20
and some of the comments are 'interesting'. I'll leave it at that, but
I will see what it queues were and why.
Meanwhile, I started putting some timers in to handle hangs and so forth.
If I decide to punt a connection because the finger client hangs, what
would be an appropriate NSP disconnect code? Maybe 40? (Link aborted
due to data loss)
------------------------------------------------------------------------
On 10/3/24 7:38 PM, Johnny Billquist wrote:
Sounds like a good improvement.
However, I saw something about mail here, that you should be aware of...
My MAIL11 implementation is doing the queuing of mails, because that
is normal with SMTP, and I just carried over the same behavior to all
parts of the system.
However, the "standard" MAIL11 application for RSX as well as RSTS/E ,
along with the DECUS MAIL for RSX (and I suspect also VMS) do not
behave that way. If the connecting fails, the sending of mail
immediately fail, and it is not queued for retrying. So it would
appears that the mail for TOPS-20 you look at is also potentially
flawed, which could cause issues.
Of course, in that case, you'd normally have a human sitting in front
of the terminal, who would then try again...
Johnny
On 2024-10-04 00:05, Thomas DeBellis wrote:
I thought I would update everyone with where
things stand with the
Tops-20 finger server.
Johnny and I agreed that, by default, the response to a DECnet finger
query is a sequences of records, each no longer than 132 bytes (more
typically 90), terminated by a line-feed. If the remote finger
client or operating system can handle larger buffers, then it can
connect with optional data=STREAM. Tops-20 will then dump everything
in one giant record. Only the Tops-20 finger client can handle this,
although I suspect maybe a VAX client might also be able to do it.
I had been making steady progress until I started stress testing it,
meaning having a session active on MIM::, TOMMYT::, and VENTI2::
(local), each having a finger command in ready to go and then pushing
carriage return on all three as simultaneously as possible.
This started generating errors on MIM:: that the "object wasn't
available". What was going on was that the structure of the Tops-20
finger server really wasn't architected to have real time response
for that much curiosity. It only opens a single SRV:117 (finger)
object at a time, waits for a connection, reads the data, hands it
off to an idle finger client fork via a pipe, and then gets a new
SRV:117 object.
In other words, it isn't until the finger client is started with the
connection redirected to DECnet that the server is ready to accept
another connection. That has what I would consider to be noticeable
latency, particularly on failure. An error doesn't really matter
for SMTP as it is a background task and just tries again, later. An
interactive finger on the other hand, has a user sitting there
waiting for a response, so it seemed to me that this wasn't really
going to cutit.
I took the model of the Tops-20 FAL server, which has a single
control fork, looking for illness in sub-forks, all of which open
their own FAL server object. The new finger controller now starts
separate FNGSRV sub-server forks, creating a FINGER sub-fork for
each, gets all the communications lined up, and starts all the FNGSRV
sub-forks to listen for connections.
This has the advantage of not clobbering the system on FNGSRV startup
because resources are gotten or creating sequentially, so there isn't
much for a FNGSRV sub-fork to do except wait for a connection and
manage its own single FINGER sub-fork.
Startup resource allocation looks like this:
[Fork FNGSRV opening PIP:1;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:1.1 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:2;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:2.2 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
[Fork FNGSRV opening PIP:3;RECORD-LENGTH:200 for writing]
[Fork FNGSRV opening PIP:3.3 for reading]
[Fork FNGSRV opening SRV:117 for reading, writing]
The resulting JFN's being:
JFN File Mode Bytes(Size)
2 FNGSRV.EXE.330 Read, Execute
3 FINGER.EXE.116 Read, Execute
15 PIP:1;RECORD-LENGTH:200 Append 0.(8)
16 PIP:1.1 Read 0.(8)
17 SRV:117 Read, Append 0.(8)
20 PIP:2;RECORD-LENGTH:200 Append 0.(8)
21 PIP:2.2 Read 0.(8)
22 SRV:117 Read, Append 0.(8)
23 PIP:3;RECORD-LENGTH:200 Append 0.(8)
24 PIP:3.3 Read 0.(8)
25 SRV:117 Read, Append 0.(8)
The resulting fork structure is:
=> FNGSRV (2): HALT at STARTS+13, 0.02719
Fork 12: HALT at 0, 0.00018
Fork 13: HALT at 0, 0.00016
Fork 10: HALT at 0, 0.00012
Fork 11: HALT at 0, 0.00012
Fork 6: HALT at 0, 0.00008
Fork 7: HALT at 0, 0.00007
So fork 2 is the finger server controller, forks 12, 10, and 6 are
finger server sub-forks, and forks 13, 11, and 7 are the respective
Tops-20 finger clients. Times are in tens of microseconds, the
maximum resolution that Tops-20 supports. What can be seen is that
fork creation is happening in sub-millisecond time. This was not the
case in the 1980's with KL10's (I /think/), and modifications were
necessary to Tops-20 and the EXEC to capture the increased resolution.
I guess I'll have another version ready in about two weeks.
_______________________________________________
HECnet mailing list -- hecnet(a)lists.dfupdate.se
To unsubscribe send an email to hecnet-leave(a)lists.dfupdate.se
77
days inactive
79
days old
3 comments
2 participants
participants (2)
-
Johnny Billquist -
Thomas DeBellis