Time for a new release announcement of TCP/IP for RSX-11M-PLUS.
This is version 2.16 of BQTCP/IP.
Highlights:
. Fixed an issue in TCP, where if the system is severely loaded, a TCP
socket could get stuck.
. Fixed DNS resolver issues that could create problems at boot time, if
DHCP was taking time to get a configuration.
. Fixed various issues in MAIL/MAILD.
. Improved logging in many daemons.
Detailed information on things that have been done since the last release:
UDP:
. Improved UDDRV IO timeout handling.
TCP:
. Bugfix. If we run out of IPPOOL when doing a TCP transmit, that socket
can get stuck. We need to retry allocation later.
DNS:
. Change resolver to just handle if no IP is known to have DNS resolving
fail, and each time requested, try and figure it out.
. If resolver starts up and we do not have an IP addres, make sure
resolver exits again.
BQTLIB:
. Updated BQTLIB documentation, PDP-11 C AIO functions.
. Added asynchronous I/O handling to PDP-11 C in bqtlib.
. Reworked DECnet access in PDP-11 C significantly.
. Bugfix in C DECnet library. Connection block setup used the wrong
format codings, leading to connection failures to some operating
systems. This affected MAILD.
MAIL/MAILD:
. Fixed mail reader handling when at end or beginning of mails.
. Bugfixes in DECnet mail handling.
. Rewritten DECnet handling in MAILD to use new API from BQTLIB.
. Improved MAILD error handling for many situations.
. Added session id in locally received mail logs.
. If MAILD sends mails using SMTP, and the remote side close the
connection during a session, we should reopen the connection for the
next mail to be delivered.
. Improved MAILD logging. In some situations, the wrong network error
code was logged.
LPT:
. Added the ability to select what is the default form for LPT despooler
via the queue form parameter.
MLTNET:
. Changed MLTNET to do immediate ACKs, to improve performance if other
end is using nagle.
FTPD:
. Added task name to FTPD console log entries.
. Improved FTP/FTPD file handling to ensure created files always have at
least RD protection for user and system.
HTTPD:
. Trim out binary data from HTTPD error log.
TELNETD:
. Added TELNET$LOG logical for if telnetd should be logging.
. Improved TNC2 task to also understand result from RT:
. Improved TNC2 task to also understand result from HT:
IFCONFIG:
. Added allocation fail statistics to IPPOOL.
IPGEN:
. Bugfix in IPGEN. C header files should be installed to PDP11C$INCLUDE:
if it exists.
NTP:
. Bugfix in NTP building. It was referring to $TMADJ, which might be a
non-existing symbol.
Some additional notes:
As usual, I would recommend people to update as soon as possible.
The changes are somewhat critical, but will also lead to a much better
experience.
The patches to the TT: driver cannot be applied automatically, but
requires users to apply the patches themselves, and then run SYSGEN to
generate a new system.
Once added, the TNC2 task can be run at login, and will define logical
names for the user telling where he is connected from, if using telnet
or LAT.
TNC2 can get information about remote connections over DECnet as well,
but this requires updates to DECnet. Such patches are not available
separately at this time, but are included in the RSX image provided from
Johnny Billquist.
The TT: driver patches also allows the updated MCR to give more
information with the DEV command (SHOW TERMINAL in DCL).
The patched TT: driver also makes is possible to get telnetd fully
vectorized, as this version provides two more addresses that are
required by telnetd to access information in the kernel.
The other patches to RSX can be applied automatically by IPGEN, either
if used interactively when answering YES to the question about applying
RSX patches, or by running IPGEN explicitly to do the patches, with the
command:
@IPGEN PATCH
Specific information about the patches:
LAT: Fixes a memory leak, and adds the ability to read where a terminal
connection comes from when using LAT, using SF.GMC.
RMSDAP: Fixes a bug in getting the file protection, so the XAB gets
filled in correctly for remote files.
RMSDSP: Fixes that some numbers were displayed in signed octal, which
should have been displayed in decimal or unsigned octal, depending on
number.
DCL: Added terminal attributes for COLOR.
MCR: Too many fixes to be listed here...
INS: Fixes that users cannot circumvent protection on common regions.
HEL: Fix that users can login with session ID, or with directory, in
addition to name and UIC.
ACNT: Add no password change attribute to accounts.
PSW: Add no password change handling.
SYL (SYSLOG): Add terminal idle tracking on accounts without idle logout.
ECL: If the receiving machine is very slow, and the sending machine is
very fast, and the receiver announce several large buffers available,
ECL cannot keep up, and drops packets. This is a problem with the DECnet
flow control, as it is used in RSX. The simple solution is to allow more
outstanding buffers when receiving. A more complex solution would be to
change how RSX DECnet do flow control, but that would require rewriting
a fair chunk of the ECL module.
NMVACP: Fix handling of "show known nodes" command, which could skip
some nodes.
NVP: Add ability to use session ID or directory name for user identity
in DECnet nodename specifications.
EPM: Fix handling of ethernet multicast.
NTDEMO: Fix that hosts without names should display DECnet address.
NCP: Parse of additional information types in NICE messages.
As usual, the distribution is available from:
ftp://mim.stupi.net/bqtcp.dskftp://mim.stupi.net/bqtcp.tap
!!! BQTCP is also available through RPM !!!
(As an additional note, if there are any problems communicating with Mim
using port 21, the ftp service is also available at port 10021)
The documentation is also available through ftp on Mim, or also at
http://mim.stupi.net/tcpipdoc
I hope people find this update useful.
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt(a)softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
I'm new to hecnet...
Old timer here. I'm the guy who rewrote decwar to run again on tops-10
See my github.
Anyway. I alrdy have a rsts10.1 system up on hecnet. trying to get
vax8650 up running vms 4.7.
I do want to get my tops20 panda working but having alot of probelms
with klh10.
Please give me some pointers on getting panda online please
thanxs
more to come
drforbin
Gentlepeople,
I figured it was about time to move PyDECnet past the "beta 1.1" stage. I sorted through the various bugfixes I had not yet commmitted and did so, the built kits called "release candidate 1". The code is currently running on node PYTHON, and kits can be found on the downloads page at http://akdesign.dyndns.org:8080/resources/public/index.html
I'd like to get the "real" release out soon, so it would be great if some of you could take this version for a test drive. I'd also like to hear if there were any bugfixes I was supposed to do that I missed.
By the way, this version requires Python 3.7 or later.
Last but not least, I moved the source control to GitHub: https://github.com/pkoning2/pydecnet . The Subversion repository is now obsolete; I will take it off line soon. The full history should be in the Git version (it was converted from the Subversion repository).
paul
I was just reading about an alternate Python implementation called PyPy, which uses a "just in time" compiler. With that it's supposed to be potentially a lot faster than regular Python, for code that does a fair amount of computation.
While PyDECnet does plenty of I/O, things like packet parsing are fairly time consuming. I've optimized it a bit but the structure I adopted only allows for so much.
I tried PyPy to see if it supports PyDECnet. The answer is yes, after I removed some code from two files that wasn't even used in the first place...
Just to see if it makes a big difference, I did a large loop test: "ncp loop nod 0 count 50000 len 400". On my Mac (ARM) laptop, that takes 6.3 seconds with Python 3.13, and 1.2 seconds with PyPy.
More testing is needed but this is interesting.
paul
Hi,
I'm including this in HECnet because it might be of general interest.
I have been creating the odd object here and there on a test router, using the mirror.py application class as a starting example, for listening to and talking to VMS processes that have opened up a task connection to them.
I can do something like:
$ open/read/write fio node::"=TIMESTAMP"
$ write fio "UTC"
$ read fio ts
$ close fio
$ show sym fio
The object waits for a message - the timezone required - and then returns with a message containing the VMS time string in the form dd-mmm-yyyy:hh:mm:ss.cc
It all work very well.
The dispatch method in the pydecnet application class code sends an accept when it gets a connect and then sends a message containing the timestamp when it receives a data message containing the timezone required.
A simpler version of this, in which the timestamp in UTC is sent immediately without waiting for a timezone string, would not work. I was trying to execute a conn.send_data(...) immediately after a conn.accept() and getting a 'WrongState' exception there and then. What am I missing?
Regards
Keith
I believe I did send an earlier version of this to the list yesterday,
as I even saw Thomas comment on it, but somehow I can't find it in the
mailing list archive at lists.dfupdate.se now, so in order to follow up
on those questions, I'm sorry to bother you with what is likely a partial
repeat post.
My Maneframe-6 data centre is back in operation; thanks for allthe support
andfeedback, includingthe serendipitous information on DECnet over CI
which I don't use (I'dlike to have an emulated CI-20 interface as well,
even in the form of an Am2900-based emulator running the CI20 microcode as
provided with TOPS-20.
My apologies everyone for having youspendtime on what seems tohave been
a totally self-inflicted problem on my part. My earlier "moving around"
involved trying to find the appropriate commands to bring down the bridge
to its default state in order to perform a clean restart. It turns out that
I failed to bring down some tap interfaces before deleting the bridge,
which led to
a defective bridge configuration being left running. Then I
tried restarting pydecnet and that's when the problems showed up. Thanks to
Thomas, Johnny and David for putting me on the right track.
Several of you asked for how I configured the KLH10 networkinterfaces, and
I remarked on that while hinting at my incomplete testing procedure. I
should have taken down the bridge completely, and restarted the KLH10
instances, which would have resolved theproblem, before asking the list:
>The KLH10 instances obtain their tap interfaces as part of their
>startup configuration, so I have to restart them whenever I have reset
>the bridge which I haven't done in a few days now. This is a sample net
>device onfiguration line from one host:
Asto that question however, the network configuration looksasfollows:
devdef ni0 564 ni20 dedic=true ifc=tap2 ifmeth=tap+bridge
ipaddr=192.168.9.72 dpdelay=3
then I havea local file in SYSTEM: with the node number used by
7-CONFIG.CMD
which I believe makes
KLH10 set the correctethernet addresson itsinerface.
$TYPE (FILE) NODE.IND.1
RARITY 1.802
This is because I want to maintain only a single 7-CONFIG.CMD to be copied
to all hosts, and it therefore contains the line
node @SYSTEM:NODE.IND
I findthe @indirect_file feature of COMND very convenient for this purpose
as SETSPD has no explicit command for reading another file.
This is another local file of potential interest (such as "IPNI#0"in case I
havemultiple networkinterfaces, which I don't yet have):
$TY INTERNET.ADDRESS.2
IPNI#0,192 168 9 72,PACKET-SIZE:1500,DEFAULT,PREFERRED
I have yet to check a few things before can say for sure that all issues
are
resolved, butthankyou for yourquestions and feedback, it was very helpful.
I have encountered a few different problems during this troubleshooting
process,
such as being unable to connect to my kaskal server even internally on my
home network, while simultaneously being able to reach it from the outside,
but that seems also to
be solved right now. and just in case any of you
would
like to be able to log in on my virtual machines, you are welcome to ask
via
private mail,stating your preferred username,max 39 characters (no "."
periods
or ^V:s please, CHECKD may choke on ^V:s in directory names,but I think DEC
removed the ability to prefix arbitrary characters with ^V in directory
names
after one of our users created a subdirectory containing a colon ":" and
our
staff reported the CHECKD issue which may in reality have been with the
colon
itself rather than the ^V prefix). Besides being reachable via HECnet you
can
run "ssh rarity(a)kaskal.pdcs.org" to be connected to a non-logged-in EXEC.
Visitors are welcome to drop by just to SEND HLLO!
Why would you want a subdirectory name with a colon in it,you may ask.
Well,
as the canonical place for storing a bunch of EMACS ".:EJ" files of course.
TECO must be the only programming language where modem line noise serves as
perfectly valid source code.
I have a PDP-10
installation of CMU's ISPS in my planned work pipeline in
case anyone is interested. Lots of fascinating stuff found when we cleaned
out our magtape collection two years ago.
/AndersAndersson
My zpologies everyone for having youspendtime on what seems tohave been
a totally self-inflicted problemonmypart. My earlier "moving around"
invlvedtrying to findheapprpriatecommands to bring downthe bridge toits
defaultstatein orderto performacleanrestart. Itturnsoutthat I failed to
bringdown some tapinterfacesbefore deleing thebridge,which led to a
defective bridge configuration being left running. Then I tried restarting
pydecnet and that's when theproblems showed up. Thanks to Thomas, Johnny
and David for putting me on the right track.
Several of you askedfor how I configuredthe KLH10 networkinterfaces, and
I remarked on that while hinting on my icompletetesting procedure.Ishould
have taken down the bridgecompletely, andrestarted the KLH10 instances,
which sould have resolved theproblem before asking the list:
>The KLH10 instances obtain their tap interfaces as part of their
>startup configuration, so I have to restart them whenever I have reset
>the bridge which I haven't done
in a few days now. This is a sample net
>device onfiguration line from one host:
Asto that question however, the network configuration looksasfollows:
devdef ni0 564 ni20 dedic=true ifc=tap2 ifmeth=tap+bridge
ipaddr=192.168.9.72 dpdelay=3
then I havea local file in SYSTEM with the node number usedby 7-CONFIG.CMD
which I believe makes KLH10 setthe correctethernet addresson itsinerface.
$TYPE (FILE) NODE.IND.1
RARITY 1.802
I haveyet to chck a fewthings before can say forsurethatthe issueis
resolved,
butthankyou fior yourquestions and feedback, itwasvery helpful.
/AndersAndersson
Thank you foryourfeedback Yes,the KLH10istancesall run on thsame Linux Mint
host that runs the apydecanet router calledKSKAL,and I haveno additioal
hardware on that internal segment which lookslike this now:
oot@kaskal:/home/andersa# brctl show
bridge name bridge id STP enabled interfaces
bridge0 8000.26d039e4c0f4 no tap1
tap2
tap3
If that doesn't indicate that thethreetap interfaceshavbeen addedto
the bridge,I don't know what does. Are there any other diagnostics you
want me to check? I dont know whether the following add anything relevant:
root@kaskal:/home/andersa# ifconfig bridge0
bridge0: flags=4163 mtu 1500
inet 192.168.9.78 netmask 255.255.255.0 broadcast 192.168.9.255
inet6 fe80::4cf6:1fff:fe01:f257 prefixlen 64 scopeid 0x20
ether 26:d0:39:e4:c0:f4 txqueuelen 1000 (Ethernet)
RX packets 557072 bytes 143985475 (143.9 MB)
RX errors 0 dropped 450854 overruns 0 frame 0
TX packets 106193 bytes 4586565 (4.5 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions
0
root@kaskal:/home/andersa# ifconfig tap1
tap1: flags=4163 mtu 1500
inet6 fe80::6032:eff:fedb:7b94 prefixlen 64 scopeid 0x20
ether 62:32:0e:db:7b:94 txqueuelen 1000 (Ethernet)
RX packets 176227 bytes 46414104 (46.4 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 318825 bytes 90356527 (90.3 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root@kaskal:/home/andersa# ip tuntap show
tap0: tap persist
tap1: tap
tap2: tap
tap3: tap
The threeTOPS-20 hosts arealso awareof and can communicate with each
other. When I run tcpdump -i bridge0 I do see DECnet, DEC LAT and TCP/IP
ARP traffic to or from the three TOPS-20 hosts pass by,according to the
ethertype code as well as decoding done by tcpdump.
The KLH10 instances obtain their tap interfaces as part of their
startup configuration, so I have to restart them whenever I have reset
the bridge which I haven't done in a few days now. This is a sample net
device onfiguration line from one host:
devdef ni0 564 ni20
dedic=true ifc=tap2 ifmeth=tap+bridge
ipaddr=192.168.9.72
dpdelay=3
The number of the interface specified with ifc= doesn't seem to matter the
machine
still gets assigned the next unassiged tap interface in order.And it did
work previously.
Eventually I hope to put all the bridge and pydecnet configuration into a
hecnet.service file andstart it from systemd.
Now I plan to restartpydecnet while logging bridg0 traffic using tcpdmpto
see
what comes up.
/AndersAndersson
YES! The Maneframe-Six data centre is back in operation! thanks for all
your support! And we giotome extrainfo on DECnet via CI which wasn't
plannd.thanks to serendipity!.
/Anders Andersson
pydecnet Q.Q.648Uusedto workfor me in the past,and I HAD three instances of
KLH10 running TOPS-20, talking to each and other HECnet nodes with no
visible problem.
I have sincemovedafewthings around and I'mno longerabletoget the old
configuratinstowork.
"Moved around" essentially means updating theufw firewall settings and
adjusted afew system settings such asthe SSH service parameers. When I
moved thestart script forpydecnetI hadto createasymbolic link to keep
thedecnet/applications/mirror.py script accessible (I uncommentedit in
thehopeof gaining some extradiagnostisthat way. But no, my routerKASKAL
is nolonger visible to my KLH10 hosts.
This is the kaskal.conf file(wheremost lines are ommwns):
# Configuration file
circuit tap-0 Ethernet tap:tap0 --console Plugh --random-address
#circuit tap-1 Ethernet pcap:tap0 --console Plugh --random-address
#circuit eth-1 Ethernet pcap:en1
#circuit bridged-1 Ethernet udp:7101:127.0.0.1:7001 --random
#circuit mul-0 Multinet localhost:7000
--latency 47
#circuit mul-1 Multinet localhost:700:connect
#circuit dmc-0 Multinet --mode=connect
--remote-address=hecnet-1-1023.stupi.net --remote-port=7700 --cost=3
circuit dmc-0 Multinet --mode=connect
--remote-address=hecnet-1-1023.stupi.net --remote-port=7704 --cost=3
#circuit dmc-0 DDCMP tcp:12345:localhost:32154 --cost 3
#circuit dmc-1 DDCMP udp:12345:localhost:32154 --cost 3
#circuit dmc-2 DDCMP serial:/dev/tty.usbserial-FTVSKM26:19200 --t3 120
--qmax 2
#circuit gre-0 GRE localhost
circuit dmc-1 Multinet --mode=listen -4 --local-port=7700 --cost=2
routing 1.808 --type l2router
node 1.808 KASKAL
node @flogsta.dat
node @nodenames.dat
system --ident "Flogsta HECnet router"
# This replaces the default built-in mirror object which is
# implemented as a Python module within PyDECnet by a functionally
# equivalent one that runs as a subprocess.
object --number 25 --name MIRROR --file decnet/applications/mirror.py
### end of kaskal.conf ###
Aftersetting up the virtual network
segment usng theseincantations
brctl addbr bridge0
ifconfig bridge0 192.168.9.78 up
ip tuntap add dev tap0 mode tap
ifconfig tap0 up
brctl addif bridge0 tap0
andstarting the KLH10 instances (if they weren't already running)
the following line is run under sudo:
pydecnet -d --uid decnet --gid 137 -e DEBUG -L logs/decnet.log kaskal.conf
http.
conf
This sems to be a relevant portion of thelogfile:
2022-08-25 14:25:13.770: MainThread: command line: /usr/local/bin/pydecnet
-d -
-uid decnet --gid 137 -L logs/decnet.log kaskal.conf http.conf
2022-08-25 14:25:15.446: KASKAL: Logging configuration reloaded
2022-08-25 14:25:15.446: KASKAL: Now running as daemon
2022-08-25 14:25:15.446: KASKAL: Running in as uid 129, gid 137
2022-08-25 14:25:15.447: KASKAL: Error starting datalink TAP-0
Traceback (most recent call last):
File
"/usr/local/lib/python3.8/dist-packages/decnet-1.0.596-py3.8.egg/decnet/d
atalink.py", line 73, in start
c.open ()
File
"/usr/local/lib/python3.8/dist-packages/decnet-1.0.596-py3.8.egg/decnet/e
thernet.py", line 220, in open
ioctl (fd, TUNSETIFF, ifr)
PermissionError: [Errno 1] Operation not permitted
2022-08-25 15:11:26.565: MainThread: Starting DECnet/Python DECnet/Python
V1.0-5
96
2022-08-25 15:11:26.565: MainThread: command line: /usr/local/bin/pydecnet
-d -
-uid decnet --gid 137 -L logs/decnet.log kaskal.conf http.conf
2022-08-25 15:11:28.185: KASKAL: Logging configuration reloaded
2022-08-25 15:11:28.185: KASKAL: Now running as daemon
2022-08-25 15:11:28.185: KASKAL: Running in as uid 129, gid 137
Myguess is that ioctl UNSETIFF on theethernetinterfacerquirsroottosucceed,
but
effctive UID is nowprobably 129 (my DECnet pseudou-user. Anychanceof
setting
the interface parameters beforetransfering control to UID 129)? I
haven'tlooked
hrough thecodeto findout.
Removing the --uid decnet option however lets pydecnet run as root,and now
the
error diagnstics re gonefromthelog file,but still I
can'treach KSKAL or any
other HECnetnode frommy local machines:
OPR>ENTER (command subset) NCP
NCP>LOOP NODE KASKAL::
NCP>
20:05:32 NCP
Request # 92 Accepted
20:05:32 NCP
Request # 92; Loop Node Failed, Mirror connect failed
Remote Node = 1.808 (KASKAL)
Unlooped count = 25701
NCP>EX
$I DEC
Local DECNET node: RARITY. Nodes reachable: 3.
Accessible DECNET nodes are: FLUSHY RARITY RBDASH
$
$INFORMATION (ABOUT) DECNET MIM::
%Node MIM is unreachable
$
So, now Ihavetwo problems. My pydecnet runs as root, which shouldn't be too
muchof an issue untilIstart experimentingwithcustom DECnetserver scripts on
the outer and more importantly,my configuraton is still dysfunctional,as I
have no connectivity to or via the router.
i intendthe HTTP server to beavailableat http://kaskal.pdcs.org:8036/
though
I'mnot sure my setupisstable enough for it to work allthetime,Ihavea ZTE
broadband routerreporting its IP address to dyndns.One of the KLH10 hosts
should bereachable via "ssh
rarity(a)kaskal.pdcs.org" (without apassord) to a
non-logged-in EXEC where atleast theINFORMATION (ABOUT) DECNET command is
availabe, in case anyonewants to see itscurrent status.
I can aaccess it locally however,and NSP Characteristics lists maybe a
thousand
nodes in a table.
So,theremaining issuewill likely require me to dig a little deeper in the
pydecnet debugging options, any particular feature I could investigate?
Currently, most recent portion of the log file:
2024-05-15 20:13:34.779: KASKAL: Starting node KASKAL
2024-05-15 20:13:34.779: KASKAL: Starting timer subsystem
2024-05-15 20:13:34.779: KASKAL: Starting datalink layer
2024-05-15 20:13:34.779: KASKAL: Ethernet TAP-0 hardware address is
86-fa-6f-77-4b-56
2024-05-15 20:13:34.780: KASKAL: Started datalink TAP-0
2024-05-15 20:13:34.780: KASKAL: Started datalink DMC-0
2024-05-15 20:13:34.780: KASKAL: Started datalink DMC-1
2024-05-15 20:13:34.780: KASKAL: Starting MOP layer
2024-05-15 20:13:34.780: KASKAL: Starting mop for
_TapEth TAP-0
2024-05-15 20:13:34.781: KASKAL: Initialized loop handler for TAP-0
2024-05-15 20:13:34.781: KASKAL: Initialized sysid handler for TAP-0
2024-05-15 20:13:34.781: KASKAL: Started MOP circuit TAP-0
2024-05-15 20:13:34.781: KASKAL: Starting Routing layer
2024-05-15 20:13:34.781: KASKAL: Started Routing circuit TAP-0
2024-05-15 20:13:34.781: KASKAL: Started Routing circuit DMC-0
2024-05-15 20:13:34.782: KASKAL: Started Routing circuit DMC-1
2024-05-15 20:13:34.785: KASKAL: Starting NSP
2024-05-15 20:13:34.785: KASKAL: Starting Session Control
2024-05-15 20:13:34.786: https: Starting https server on port 8443
2024-05-15 20:13:34.787: KASKAL: Starting http server on port 8000
2024-05-15 20:13:35.245: KASKAL: L2 attached state changed to True
2024-05-15 20:13:44.820: KASKAL: Designated router on TAP-0 will be self, 5
second delay
2024-05-15 20:13:49.833: KASKAL: Designated router on TAP-0 is self
Yetwhen I try to talk to KASKAL viathat very virtualetenet it hasTAP-0
on:
NCP>
10:01:59 NCP
Request # 93 Accepted
10:01:59 NCP
Request # 93; Loop Node Failed, Mirror connect failed
Remote Node = 1.808 (KASKAL)
Unlooped count = 25701
NCP>
Should I use some other server than mirror.py?
