Paul Koning wrote:
... Well, level 1 routers gives you pretty much the same isolation as
areas.
Unless you consider the relative size of the level 1 routing messages
a
problematic burden compared to level 2 routing messages. However, I'm
not even sure you can make the system not transmit level 1 routing
messages on all interfaces, meaning they will go out anyway, even if
you
only have another area at the other end of the line.
I think if you have a point to point out of area link you won't see L1
routing messages, but you probably will see them on an Ethernet.
Hmm. I have not tried, so I wouldn't be able to tell for sure. You might be right.
The routing message size isn't an issue in Phase IV because the data can
be sent in pieces. That was another important change from Phase III,
which always sends the route data for all nodes (up to 255 of them)
whether there is a change or not. 255 nodes just barely fits into a
standard DDCMP packet, but 1023 nodes would not. (Nor in a standard
Ethernet frame, for that matter.) And it's wasteful to send data that
hasn't changed. So Phase IV has a way to send routing data for selected
nodes.
Looking at traffic, I'd say that systems seems to be sending out routing packet updates although nothing is changing.
Johnny
Hi, Paul. I see that I'll have to be way more careful with how I choose my words around you... :-)
Paul Koning wrote:
...
The 2.0.0 (phase IV) routing spec talks about the "on Ethernet"
cache
instead, and describes it in a way that makes it help only for
directly
attached nodes. The "previous hop" flavor was a generalization in,
I
think, 2.1.0 (Phase IV+).
Hmm, yuck. Now I'll have to try to remember some details, as well as
extrapolating some stuff.
No, as far as I can remember, the end-nodes will only remember (or
know)
of one level 1 router. It will pick the one with the highest priority
in
case there are more than one on the local segment. That will become
the
designated router, and will be used for all traffic I believe.
The way this works is that there are two multicast addresses used on
Ethernet segments. (Well, three later on, a separate one for all L2
routers.) One is "all routers", one is "all endnodes". Routers listen
to the first, endnodes to the second. Hellos are sent to "all routers"
so ONLY routers hear hellos. Both endnodes and routers send hellos
(different types). So routers build a list of all the nodes on each
interface. The routers on an Ethernet pick one to be the designated
router, and only that router sends a hello to "all endnodes". That's
how endnodes know the DR.
Not really.
end-nodes send endnode-hello messages to ab:00:00:03:00:00
l1 routers send router-hello messages to ab:00:00:03:00:00
l2 routers send router-hello messages to ab:00:00:03:00:00, ab:00:00:04:00:00 and 09:00:2b:02:00:00 (exactly which of these last two addresses are sent to seems to differ from node to node, and I haven't figured out exactly how yet).
(looking at live traffic right now)
I can give you a longer snapshot of live traffic if you want to see more details...
So, obviously, routers and endnodes both send, and listen to the same messages. If we ignore the weird other addresses of L2 routers, they in fact all send messages on the same channel.
So, they all also pick up all the same information.
But, in addition to the hello messages, which all pick up and process, and which are rather short messages sent fairly often, routers also send level 1 and level 2 routing information messages. These are all just thrown away by end-nodes. And level 1 routers more or less just throw away level 2 routing messages as well.
End-nodes will, however, just snoop hello-messages, and know about
other
nodes that are on the same ethernet segment (I'm not sure if this
generalizes to other mediums than ethernet, I think the manuals only
talked about ethernet). So, you can actually have an ethernet segment
with only end-nodes, and communication will actually work anyway. Even
though if you look at adjacent nodes with NCP, you'll not see any.
That's the ethernet cache unless I read things wrong here, btw.
No, endnodes don't do any snooping (nor do routers). DECnet
intentionally was designed to rely on advertisements (hello messages)
and those were done in a way that limits overhead. So end nodes hear
only hellos from a single system (the designated router).
Sorry. Very sloppy wording on my part.
By "snooping" just meant that they receive the hello multicast messages, and figure it out from there.
In essence, I meant snooping, since the messages aren't sent explicitly to them, but they do not go into promiscous mode and look at all traffic that pass over the ethernet.
But no, end-nodes will actually hear hello messages from everybody on the same ethernet segment. From that, they remember all the nodes that they can hear in their cache, and also remembers which is the routing node with highest priority (priority of the router is carried in the hello message).
If an end node is asked to talk to some address, it consults the cache.
If it's in the cache, the data there is used. If not, the message goes
to the designated router if one is known. Finally, if there is no DR
(single endnode-only LAN) then it sends directly to the destination
using the calculated MAC address.
Yes, it starts by consulting the cache, and if the destination is in there, it is used.
If not, it is sent to the designated router.
I think that if no designated router exist, and it's not in the cache, any traffic will just result in an error return right away.
Incoming traffic fills in the previous-hop cache, so once you get past
the first message, end node traffic goes via the optimal path.
No. The hello messages fills the cache. That's how it can work even if there is no router.
What I find "horrible" about this is that end-nodes will (atleast in
VMS) will actually happily even talk with machines in another area, if
they happen to be sitting on the same ethernet segment (this has
bitten
me with my bridge and HECnet).
Yes, that was considered a feature. It's a side effect of the way the
previous hop cache works, and when we realized this we decided it was a
good thing so that's how it ended up.
Well... I don't know if I'd call it good. But that's probably because it spoiled a feature I had wanted to do in my bridge... :-)
Anyway, the comment is that the path might not be optimal. End nodes
pick the level 1 router with the highest priority. If you have two
level
1 routers, the one you didn't communicate with might have been more
optimal, but you'll never know. And the same is true for area routers.
For out of area traffic you're definitely right, the first leg is to the
best L2 router for the destination area, which may not be the best way
to get to the final destination. (In a sensibly layed-out hierarchical
network, the difference should not be large.) But end nodes do
communicate just as efficiently as routers, thanks to the previous hop
cache. And in fact, for the multi area Ethernet case, they do better
than routers because routers will not talk directly to an out of area
node, even if it's on the same Ethernet.
Hmm... I don't think so. The first leg to another area from an end-node will be to the designated level 1 router. From there it will travel to the designated level 2 router, only then will it go by the most efficient path to the right area (which might not in fact be the best for the actual end destination), and only once inside that area will it go by the most efficient path to the destination.
It all breaks down if you have several L1 or L2 routes on an area. (And by break down I mean it can go suboptimal ways, it will still definitely work just fine.)
Johnny
Node 52.1 will be named SUN
Sent from my iPhone
On Jun 30, 2010, at 11:31, Johnny Billquist <bqt at softjar.se> wrote:
Noted.
Johnny
Brian Hechinger wrote:
I would like to claim area 52.
-brian
...
Well, level 1 routers gives you pretty much the same isolation as
areas.
Unless you consider the relative size of the level 1 routing messages
a
problematic burden compared to level 2 routing messages. However, I'm
not even sure you can make the system not transmit level 1 routing
messages on all interfaces, meaning they will go out anyway, even if
you
only have another area at the other end of the line.
I think if you have a point to point out of area link you won't see L1
routing messages, but you probably will see them on an Ethernet.
The routing message size isn't an issue in Phase IV because the data can
be sent in pieces. That was another important change from Phase III,
which always sends the route data for all nodes (up to 255 of them)
whether there is a change or not. 255 nodes just barely fits into a
standard DDCMP packet, but 1023 nodes would not. (Nor in a standard
Ethernet frame, for that matter.) And it's wasteful to send data that
hasn't changed. So Phase IV has a way to send routing data for selected
nodes.
paul
...
The 2.0.0 (phase IV) routing spec talks about the "on Ethernet"
cache
instead, and describes it in a way that makes it help only for
directly
attached nodes. The "previous hop" flavor was a generalization in,
I
think, 2.1.0 (Phase IV+).
Hmm, yuck. Now I'll have to try to remember some details, as well as
extrapolating some stuff.
No, as far as I can remember, the end-nodes will only remember (or
know)
of one level 1 router. It will pick the one with the highest priority
in
case there are more than one on the local segment. That will become
the
designated router, and will be used for all traffic I believe.
The way this works is that there are two multicast addresses used on
Ethernet segments. (Well, three later on, a separate one for all L2
routers.) One is "all routers", one is "all endnodes". Routers listen
to the first, endnodes to the second. Hellos are sent to "all routers"
so ONLY routers hear hellos. Both endnodes and routers send hellos
(different types). So routers build a list of all the nodes on each
interface. The routers on an Ethernet pick one to be the designated
router, and only that router sends a hello to "all endnodes". That's
how endnodes know the DR.
End-nodes will, however, just snoop hello-messages, and know about
other
nodes that are on the same ethernet segment (I'm not sure if this
generalizes to other mediums than ethernet, I think the manuals only
talked about ethernet). So, you can actually have an ethernet segment
with only end-nodes, and communication will actually work anyway. Even
though if you look at adjacent nodes with NCP, you'll not see any.
That's the ethernet cache unless I read things wrong here, btw.
No, endnodes don't do any snooping (nor do routers). DECnet
intentionally was designed to rely on advertisements (hello messages)
and those were done in a way that limits overhead. So end nodes hear
only hellos from a single system (the designated router).
If an end node is asked to talk to some address, it consults the cache.
If it's in the cache, the data there is used. If not, the message goes
to the designated router if one is known. Finally, if there is no DR
(single endnode-only LAN) then it sends directly to the destination
using the calculated MAC address.
Incoming traffic fills in the previous-hop cache, so once you get past
the first message, end node traffic goes via the optimal path.
What I find "horrible" about this is that end-nodes will (atleast in
VMS) will actually happily even talk with machines in another area, if
they happen to be sitting on the same ethernet segment (this has
bitten
me with my bridge and HECnet).
Yes, that was considered a feature. It's a side effect of the way the
previous hop cache works, and when we realized this we decided it was a
good thing so that's how it ended up.
Anyway, the comment is that the path might not be optimal. End nodes
pick the level 1 router with the highest priority. If you have two
level
1 routers, the one you didn't communicate with might have been more
optimal, but you'll never know. And the same is true for area routers.
For out of area traffic you're definitely right, the first leg is to the
best L2 router for the destination area, which may not be the best way
to get to the final destination. (In a sensibly layed-out hierarchical
network, the difference should not be large.) But end nodes do
communicate just as efficiently as routers, thanks to the previous hop
cache. And in fact, for the multi area Ethernet case, they do better
than routers because routers will not talk directly to an out of area
node, even if it's on the same Ethernet.
paul
Brian Hechinger wrote:
On Mon, Jun 28, 2010 at 12:10:31PM -0700, Johnny Billquist wrote:
Should I just use one area? What's the best way to set that up?
Areas in DECnet are not really meant to be associated with physical location, but rather with organizational. So keep it within one area.
Ok, sounds good to me.
For physically different locations, you might want to use level 1 routers, though. But as the internet nowadays is so fast, using just a bridge, and pretend that it's all one ethernet segment also works just fine.
My upstream blows (as well as latency in general) so I'd like to avoid
putting any more traffic across it than is absolutely nessesary.
In that case you should definitely not use my bridge...
I suppose it's not a huge deal at this point since the SIMH instance at
the colo will be the only thing running most likely for a little while.
That being said, how should I set it up? L1? L2? Does anyone want to
use me as a hub?
L1? L2? As in layers in the network stack?
Also, is there a list of what areas are already in use so I can pick an
unused one?
Already answered by others, I noticed... :-)
Johnny
Paul Koning wrote:
Johnny, the original DECnet manuals showed pictures of area routers
that
were interconnected by WAN links. Each site had its own area number.
In
fact
the name "area routing" implies clearly that the concept was meant to
set up
DECnet networks that were geographically separated.
I don't remember those pictures, but this is an issue that generated a
lot of debate and a lot of annoyance in the DECnet architecture group.
The pictures, atleast in the RSX manuals, are very simplistic. Like five nodes, maybe four areas, and spread between the west and east coast of the US.
...
One reason for separating systems in different areas was that
rebooting
the
pc's would generate so many DECnet state up and down messages that
PDP-
11's
and the older VAX systems choked in their console output.
That's a good one. (Actually, the way to deal with that is to turn the
logging to console off for that event...) Yes, that's a valid example.
The purpose of area routing is to avoid having things get too large.
There were various opinions on how big an area (or Phase III network)
could reasonably be. Originally the limit was assumed to be about 32,
without any basis that I know of. That too produced a pile of debate,
and later on it became obvious you could go much higher. But clearly
you couldn't run a flat Phase III style network with 10,000 nodes.
Well, phase III networks could never be more than 255 nodes. :-)
For that reason area routing was introduced, so each instance of the
routing algorithm (in-area and level 2) would deal with a reasonably
small topology. Again debates broke out whether a full area (1000
nodes) was acceptable. It turned out yes, but certainly keeping it
smaller, say to 100-200, would be a good thing if possible.
When phase IV came, and the node address was extended to 16 bits, it's obvious (I think) that you didn't want a flat 16-bit address space.
So then it becomes a question of how to split that address up. By that time, I think they had come to the conclusion that a single network could definitely be more than 255 nodes. 1023 seemed like a good size, leaving 63 areas. (Neither node 0, nor area 0 is permitted.)
Geography isn't a consideration unless the links are slow enough to make
route change propagation an issue. Corporate organization (different
departments) CERTAINLY isnt' a consideration, though some people tried
to argue that it was valid to do so.
Well, level 1 routers gives you pretty much the same isolation as areas. Unless you consider the relative size of the level 1 routing messages a problematic burden compared to level 2 routing messages. However, I'm not even sure you can make the system not transmit level 1 routing messages on all interfaces, meaning they will go out anyway, even if you only have another area at the other end of the line.
Johnny
Paul Koning wrote:
...
Level 1 routers knows where all machines in the same area are located,
and knows the most efficient way to any node within the same area.
However, it only keeps track of the closest area router, and knows
nothing about any other area.
Area routers work just like level 1 routers, but they also have an
area
routing table, so they know the most efficient path to an area router
for any area.
So, an end node only knows what machines are on the same segment, and
which is the closest level 1 router. Level 1 routers knows where all
machines are in the same area, and knows where the closest area router
is.
Area routers knows where every machine is on the local area, and also
where all area routers are.
Right... except for end nodes. Those know what *routers* exist on the
same segment. End nodes don't know (directly) about other end nodes.
To make that case efficient, end nodes have a "previous hop cache".
Whenever an end node hears from another node, it remembers the previous
hop (the last router on the path) and sends future traffic for that
destination to that router. This way it can pick any router for the
initial packets but will quickly move to the optimal one. It also
tracks directly connected endnodes in that cache, because a router that
forwards a packet onto the same segment as it arrived will leave the
"intra-Ethernet" bit in the header set, which tells the endnode that
destination is directly reachable.
The 2.0.0 (phase IV) routing spec talks about the "on Ethernet" cache
instead, and describes it in a way that makes it help only for directly
attached nodes. The "previous hop" flavor was a generalization in, I
think, 2.1.0 (Phase IV+).
Hmm, yuck. Now I'll have to try to remember some details, as well as extrapolating some stuff.
No, as far as I can remember, the end-nodes will only remember (or know) of one level 1 router. It will pick the one with the highest priority in case there are more than one on the local segment. That will become the designated router, and will be used for all traffic I believe.
End-nodes will, however, just snoop hello-messages, and know about other nodes that are on the same ethernet segment (I'm not sure if this generalizes to other mediums than ethernet, I think the manuals only talked about ethernet). So, you can actually have an ethernet segment with only end-nodes, and communication will actually work anyway. Even though if you look at adjacent nodes with NCP, you'll not see any. That's the ethernet cache unless I read things wrong here, btw.
What I find "horrible" about this is that end-nodes will (atleast in VMS) will actually happily even talk with machines in another area, if they happen to be sitting on the same ethernet segment (this has bitten me with my bridge and HECnet).
Now, this means that a machine that sits in one area will not
neccesarily take the shortest path to a machine in another area. In
fact, it will not neccesarily take the shortest path even to another
machine in the same area. But that's another story. :-)
??? It certainly will, where "shortest" is defined as "least path
*cost*", not necessarily fewest hops.
Sorry. Yes, you are right. I should have been more careful with my words. It was the path cost I was thinking of. I should have said cheapest, not shortest.
Anyway, the comment is that the path might not be optimal. End nodes pick the level 1 router with the highest priority. If you have two level 1 routers, the one you didn't communicate with might have been more optimal, but you'll never know. And the same is true for area routers.
Johnny
...
Level 1 routers knows where all machines in the same area are located,
and knows the most efficient way to any node within the same area.
However, it only keeps track of the closest area router, and knows
nothing about any other area.
Area routers work just like level 1 routers, but they also have an
area
routing table, so they know the most efficient path to an area router
for any area.
So, an end node only knows what machines are on the same segment, and
which is the closest level 1 router. Level 1 routers knows where all
machines are in the same area, and knows where the closest area router
is.
Area routers knows where every machine is on the local area, and also
where all area routers are.
Right... except for end nodes. Those know what *routers* exist on the
same segment. End nodes don't know (directly) about other end nodes.
To make that case efficient, end nodes have a "previous hop cache".
Whenever an end node hears from another node, it remembers the previous
hop (the last router on the path) and sends future traffic for that
destination to that router. This way it can pick any router for the
initial packets but will quickly move to the optimal one. It also
tracks directly connected endnodes in that cache, because a router that
forwards a packet onto the same segment as it arrived will leave the
"intra-Ethernet" bit in the header set, which tells the endnode that
destination is directly reachable.
The 2.0.0 (phase IV) routing spec talks about the "on Ethernet" cache
instead, and describes it in a way that makes it help only for directly
attached nodes. The "previous hop" flavor was a generalization in, I
think, 2.1.0 (Phase IV+).
Now, this means that a machine that sits in one area will not
neccesarily take the shortest path to a machine in another area. In
fact, it will not neccesarily take the shortest path even to another
machine in the same area. But that's another story. :-)
??? It certainly will, where "shortest" is defined as "least path
*cost*", not necessarily fewest hops.
paul
