I have learned the hard way to never trust anything I write until I've run tests on it...  The worst part is that I keep having to relearn it.  The hard way...

These days, I try to write the tests with the code, pounding on the edge cases, first.

Part of the reason that I'm coding diagnostic packet decoding (for configuration negotiation, for example) is to eventually put together some 'fuzz' tests.  Assuming I break something, it should enable me to more quickly understand why something blew up.  That's the hope, anyway.  If nothing else, I've gained a better understanding of DAP.
On 5/25/23 12:39 PM, Rob Jarratt wrote:


-----Original Message-----
From: Paul Koning <paulkoning@comcast.net>
Sent: 24 May 2023 19:36
To: The Hobbyist DECnet mailing list <hecnet@lists.dfupdate.se>
Subject: [HECnet] Re: DECnet Regression Tests?




On May 24, 2023, at 2:15 PM, Paul Koning <paulkoning@comcast.net>

wrote:




On May 19, 2023, at 8:23 PM, Johnny Billquist <bqt@softjar.se> wrote:

There are some test scripts and tools in the RSX DECnet sources, but

they

are really RSX specific, and for testing RSX against different types of

systems.

I've never used them, and I don't know what would be required for them

to even be usable.

Apart from those, I don't think I've seen anything.
But there is something for RSX, which might be answering parts of your

question, but it's nothing general.

PyDECnet comes with a large body of unit tests.

And that suggests an interesting notion.  Many of those tests are of the

form:

1. Construct a test packet
2. Submit that test packet to the system being tested (PyDECnet) 3. If a
response is expected, check that it arrives and contains what we expect 4.
Check the resulting sytem state

Those could, in many cases, be adapted to do "black box" testing of other
DECnet implementations.  The key difference is that as written the tests
typically poke at a single protocol layer, and check state by looking into

the

internals of the layer.  A black box test would have to wrap the test

packets in

the necessary lower layer envelopes to get them to the destination, and

the

state checks would be limited to what is visible via the management

interface

(counters and state) or other application APIs.

Those would not be terribly difficult things to do.  For example, PyDECnet
already contains lower layer envelope descriptions, so to take, say, an

NSP

test packet and wrap it in routing and datalink layer envelopes is not

hard to

do, nor is unwrapping the reply.

One part of my PyDECnet tests is random packet tests, where I construct
random crud to send to a layer to verify that it doesn't die.  Obviously
random packets don't do anything useful, and typically would just get
discarded, but it's well known that such abuse tests often break code that
isn't sufficiently paranoid.  It would be entertaining to subject other

DECnets

to such abuse.

I usually write a lot of automated tests, but for my own user mode router I
didn't do that, which makes me feel very bad. It would be interesting to
subject my router to a fuzz test to see how it copes
.

	paul


_______________________________________________
HECnet mailing list -- hecnet@lists.dfupdate.se To unsubscribe send an

email

to hecnet-leave@lists.dfupdate.se

_______________________________________________
HECnet mailing list -- hecnet@lists.dfupdate.se
To unsubscribe send an email to hecnet-leave@lists.dfupdate.se