Paul Koning wrote:
"Jason" == Jason Stevens <neozeed at gmail.com> writes:
Jason> Ah I had thought fragmentation was a 'feature' of TCP not
Jason> UDP.. well then that would take care of it then.
There are two things that serve similar purposes and have confusingly
similar names.
"Segmentation" is what TCP does to break user data into convenient
size chunks. That chunk size is MSS (Max Segment Size), it's
negotiated at TCP connection setup.
"Fragmentation" is what IPv4 does to break datagrams into chunks that
fit on the wire (that size is MTU, Max Transmission Unit).
Also note that MTU can differ from hop to hop over the internet. So a packet that goes over the first hop as one piece might need to be fragmented to get over the next hop.
Segmentation is more efficient than fragmentation, so with TCP the
normal approach is to pick MSS such that the resulting packets end up
no larger than MTU. In the case of UDP, the packet size is set by the
application (because each application send results in exactly one UDP
packet), so fragmentation will occur if the application sends stuff
bigger than MTU - header size.
No. Segmentation isn't more efficient, as such. However, it behaves much better in the face of lost packets.
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Paul Koning wrote:
"Jason" == Jason Stevens <neozeed at gmail.com> writes:
>> As for fragmentation... Now I assume you are talking about the
>> encapsulation of ethernet packets in UDP packets. Those will be a
>> bit larger still, and will almost certainly be fragmented when
>> sent over the internet, yes. I don't see a problem with that. Do
>> you?
>> Jason> Well if you were trying to send the whole 1500 bytes of data +
Jason> headers in the UDP packet won't it cut stuff off? No, not unless it's either IPv6, or you set the "don't fragment" flag
in the IP header. IPv4 will fragment oversized packets no matter
what's inside, and indeed this is the only way for random size
UDPgrams to get where they are going. It should work fine. Note that
fragmentation is often not all that efficient. Compared with the
performance of old 10Mb/s DEC Ethernet gear, that's unlikely to be an
issue.
Actually, since there isn't a good way of finding out how large packets you can send, fragmentation is almost neccesary. It can occur anywhere along the way from source to destination, not only at the start or end.
And fragmentation is actually the most efficient way of getting data across. The problem is if one fragment is lost, which will cause retransmission of a lot of fragments. That's the reason TCP tries to avoid it.
But with large IP packets, you get a lower protocol overhead compared to actual data transferred.
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Jason Stevens wrote:
On Thu, Feb 26, 2009 at 8:19 AM, Johnny Billquist <bqt at softjar.se> wrote:
Jason Stevens wrote:
Right now testing, although I'm making a 'stub' version to run under
windows so I can setup a bridge on a windows hosting box I have...
Hmm. I suspect that might turn out to be very hard, if not impossible...
One of the serious problems is that you need to be able to send out packets
with faked source MAC addresses. And all tools I've ever worked with in
Windows which need to use some other MAC address have been tricky to say the
least, not to mention that you need to reconfigure, and for some, also
reboot the machine to use another MAC address. Transparent ethernet access
from Windows seems to be difficult.
There is libpcap for windows, you can do it... although I was going to
remove the physical access, and just have it forward the packets... A
bridge in the sky as it were.
Well, normally libpcap is used to recieve packets, so it don't neccesarily help with sending them.
Second, sending packets is one thing, sending packets with a "fake" source MAC address is yet another thing (as a short note, the DEUNA and DELUA ethernet controllers for the Unibus can never do this. They set the source MAC address from the controller, no matter what you place in the packet you want to send).
But please report if you have success with this.
I noticed from the source there is no way to handle packets of 1500
bytes, as it would invariably require fragmentation... is
LAT/MOP/DecNET all smaller then 1500 bytes?
I'm not sure how you mean. The code bridge ethernet. Ethernet packets are
(normally) a maximum of 1518 bytes. 1500 bytes of data, 6 bytes source and
destination MAC, two bytes with protocol number, and four bytes of CRC.
Is there somewhere where you have identified that the bridge program don't
handle 1500 bytes of data?
I'm not really interested in jumbo ethernet frames. No DEC equipment
relevant here supports it anyway, so there is no need.
As for fragmentation... Now I assume you are talking about the encapsulation
of ethernet packets in UDP packets. Those will be a bit larger still, and
will almost certainly be fragmented when sent over the internet, yes. I
don't see a problem with that. Do you?
Well if you were trying to send the whole 1500 bytes of data + headers
in the UDP packet won't it cut stuff off? I know TCP will do the auto
fragment & reassemble, so you won't notice it... I haven't stressed
the thing out yet, just some simple ICMP stuff at the moment.. I
should add, I've added code to send thru ARP/RARP & IP since I don't
have any idea on configureing networking on VMS..
No. Two different things.
TCP intentionally sends short enough packets that they don't get fragmented. TCP then keeps track of everything to actually recreate a stream of bytes at the other end. IP will fragment packets if needed, no matter if you use UDP, TCP or any other protocol that is transported on top of IP.
That is, unless the datagram have the DF flag set (don't fragment), or the IP implementation haven't implemented fragmentation. They should, but there are always the odd implementation that might not do it.
Anyways I was thinking of adding compression support (zlib style)... I
should test it some more as I figure if the packet compresses somewhat
it'll be under the 1500 snaplen for ethernet....
Feel free. I'm not sure there is a point of doing so, but anyway.
If you want to avoid fragmentation of packets, you need to find out what the
minimum MTU are along the whole path between two nodes. And your packet then
needs to fit both the data to carry as well as the UDP and IP header within
that packet size. The minimum size that is guaranteed by IP to not be
fragmented is 576 bytes. For UDP that means a size of the data part of just
548 bytes.
And that means you need to do fragmentation yourself, since you'll never be
able to squeeze 1500 byte packets into 548 bytes with any kind of certainty.
And if you start doing the fragmentation yourself, you will in fact add to
the overhead, since then you'll create a separate packet for each fragment,
all with their IP *and* UDP headers. And you'll need to do fragment
reassembly, and what will you do if some fragment is missing? Do
retransmissions? Now we're getting into a very complex protocol with
acknowledgements and whatever. And don't forget the cpu overhead of doing
the compression/decompression of each packet. And if you start to have
fragments, acknowledgements and so on, you'll have to add information to
each packet to handle all this information, which means extra overhead there
as well. In the end, you should not be surprised if your packets end up
being larger than the current ones.
Yeah that's what I was trying to avoid.... I mean it's not that hard
to sequence things, and have it note fragment numbers, and of course
being aware of the sequence and reassembly.... But then at that point,
it'd be easier to just use TCP.. As for the compression I was thinking
more so of loading a HECnet enabled version of SIMH on an iPhone... I
know not the most 'usefull' but I think it'd be cool to have a pocket
VAX on the network...
But compression of the network traffic is still not a gain, even on an iPhone (I think).
There are several reasons I prefer using UDP over TCP for HECnet. Not the least that it's okay to actually drop packets, or get them out of order. Using UDP also adds minimal overhead. And it's naturally packet oriented. If I were to use TCP, I'd have to reassemble the stream into packets. That means I also need to transfer packet size before each packet, and a lot of "boring" extra work.
And in the "normal" case, each packet is still just generating a single,
unfragmented IP packet anyway.
Like I said. Feel free to play with it, and see what you can do. But
personally I don't think it's a good idea. You add complexity, processing
overhead, protocol overhead, and possibly a whole new level of communication
overhead, to probably no gain at any corner, but at a cost of having a
protocol that makes it harder to troubleshoot. :-)
Ok enough rambling for now.
I hope my ramblings don't scare you off, or turns you off from
experimenting. Having fun and experimenting is after all what HECnet is all
about.
Thanks for the input, yeah I need to play more with it, I do like the
realative simplicity of it... If anything now that I think I got SIMH
tied into it ok, I wanted to add Qemu support, and whatever emulators
I can also find to build up some emulated network.. And again tying
back to a stubbed windows server that I have on the internet. In the
age of multi Ghz cpu's I don't think libz would be that much
overhead..
Lot larger memory footprint, more cpu usage, maybe larger packets. :-)
I still hate watching resources wasted, even if todays machines are fast. :-)
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
"Jason" == Jason Stevens <neozeed at gmail.com> writes:
Jason> Ah I had thought fragmentation was a 'feature' of TCP not
Jason> UDP.. well then that would take care of it then.
There are two things that serve similar purposes and have confusingly
similar names.
"Segmentation" is what TCP does to break user data into convenient
size chunks. That chunk size is MSS (Max Segment Size), it's
negotiated at TCP connection setup.
"Fragmentation" is what IPv4 does to break datagrams into chunks that
fit on the wire (that size is MTU, Max Transmission Unit).
Segmentation is more efficient than fragmentation, so with TCP the
normal approach is to pick MSS such that the resulting packets end up
no larger than MTU. In the case of UDP, the packet size is set by the
application (because each application send results in exactly one UDP
packet), so fragmentation will occur if the application sends stuff
bigger than MTU - header size.
paul
On Thu, Feb 26, 2009 at 1:53 PM, Paul Koning <Paul_Koning at dell.com> wrote:
"Jason" == Jason Stevens <neozeed at gmail.com> writes:
>> As for fragmentation... Now I assume you are talking about the
>> encapsulation of ethernet packets in UDP packets. Those will be a
>> bit larger still, and will almost certainly be fragmented when
>> sent over the internet, yes. I don't see a problem with that. Do
>> you?
>>
Jason> Well if you were trying to send the whole 1500 bytes of data +
Jason> headers in the UDP packet won't it cut stuff off?
No, not unless it's either IPv6, or you set the "don't fragment" flag
in the IP header. IPv4 will fragment oversized packets no matter
what's inside, and indeed this is the only way for random size
UDPgrams to get where they are going. It should work fine. Note that
fragmentation is often not all that efficient. Compared with the
performance of old 10Mb/s DEC Ethernet gear, that's unlikely to be an
issue.
paul
Ah I had thought fragmentation was a 'feature' of TCP not UDP.. well
then that would take care of it then.
"Jason" == Jason Stevens <neozeed at gmail.com> writes:
As for fragmentation... Now I assume you are talking about the
encapsulation of ethernet packets in UDP packets. Those will be a
bit larger still, and will almost certainly be fragmented when
sent over the internet, yes. I don't see a problem with that. Do
you?
Jason> Well if you were trying to send the whole 1500 bytes of data +
Jason> headers in the UDP packet won't it cut stuff off?
No, not unless it's either IPv6, or you set the "don't fragment" flag
in the IP header. IPv4 will fragment oversized packets no matter
what's inside, and indeed this is the only way for random size
UDPgrams to get where they are going. It should work fine. Note that
fragmentation is often not all that efficient. Compared with the
performance of old 10Mb/s DEC Ethernet gear, that's unlikely to be an
issue.
paul
On Thu, Feb 26, 2009 at 8:19 AM, Johnny Billquist <bqt at softjar.se> wrote:
Jason Stevens wrote:
Right now testing, although I'm making a 'stub' version to run under
windows so I can setup a bridge on a windows hosting box I have...
Hmm. I suspect that might turn out to be very hard, if not impossible...
One of the serious problems is that you need to be able to send out packets
with faked source MAC addresses. And all tools I've ever worked with in
Windows which need to use some other MAC address have been tricky to say the
least, not to mention that you need to reconfigure, and for some, also
reboot the machine to use another MAC address. Transparent ethernet access
from Windows seems to be difficult.
There is libpcap for windows, you can do it... although I was going to
remove the physical access, and just have it forward the packets... A
bridge in the sky as it were.
I noticed from the source there is no way to handle packets of 1500
bytes, as it would invariably require fragmentation... is
LAT/MOP/DecNET all smaller then 1500 bytes?
I'm not sure how you mean. The code bridge ethernet. Ethernet packets are
(normally) a maximum of 1518 bytes. 1500 bytes of data, 6 bytes source and
destination MAC, two bytes with protocol number, and four bytes of CRC.
Is there somewhere where you have identified that the bridge program don't
handle 1500 bytes of data?
I'm not really interested in jumbo ethernet frames. No DEC equipment
relevant here supports it anyway, so there is no need.
As for fragmentation... Now I assume you are talking about the encapsulation
of ethernet packets in UDP packets. Those will be a bit larger still, and
will almost certainly be fragmented when sent over the internet, yes. I
don't see a problem with that. Do you?
Well if you were trying to send the whole 1500 bytes of data + headers
in the UDP packet won't it cut stuff off? I know TCP will do the auto
fragment & reassemble, so you won't notice it... I haven't stressed
the thing out yet, just some simple ICMP stuff at the moment.. I
should add, I've added code to send thru ARP/RARP & IP since I don't
have any idea on configureing networking on VMS..
I've only used decnet in the 'wild' once, but we moved from the VAX to
PC's... but then our VAX was weird as it ran Novell Netware..... I
wish I had saved the tapes, as netware for the vax has to be super
ultra rare...
Yeah. That would have been fun to have kept around.
I recall it being like the netware for OS/2 it's a shame they priced
themselves out of the market...
Anyways I was thinking of adding compression support (zlib style)... I
should test it some more as I figure if the packet compresses somewhat
it'll be under the 1500 snaplen for ethernet....
Feel free. I'm not sure there is a point of doing so, but anyway.
If you want to avoid fragmentation of packets, you need to find out what the
minimum MTU are along the whole path between two nodes. And your packet then
needs to fit both the data to carry as well as the UDP and IP header within
that packet size. The minimum size that is guaranteed by IP to not be
fragmented is 576 bytes. For UDP that means a size of the data part of just
548 bytes.
And that means you need to do fragmentation yourself, since you'll never be
able to squeeze 1500 byte packets into 548 bytes with any kind of certainty.
And if you start doing the fragmentation yourself, you will in fact add to
the overhead, since then you'll create a separate packet for each fragment,
all with their IP *and* UDP headers. And you'll need to do fragment
reassembly, and what will you do if some fragment is missing? Do
retransmissions? Now we're getting into a very complex protocol with
acknowledgements and whatever. And don't forget the cpu overhead of doing
the compression/decompression of each packet. And if you start to have
fragments, acknowledgements and so on, you'll have to add information to
each packet to handle all this information, which means extra overhead there
as well. In the end, you should not be surprised if your packets end up
being larger than the current ones.
Yeah that's what I was trying to avoid.... I mean it's not that hard
to sequence things, and have it note fragment numbers, and of course
being aware of the sequence and reassembly.... But then at that point,
it'd be easier to just use TCP.. As for the compression I was thinking
more so of loading a HECnet enabled version of SIMH on an iPhone... I
know not the most 'usefull' but I think it'd be cool to have a pocket
VAX on the network...
And in the "normal" case, each packet is still just generating a single,
unfragmented IP packet anyway.
Like I said. Feel free to play with it, and see what you can do. But
personally I don't think it's a good idea. You add complexity, processing
overhead, protocol overhead, and possibly a whole new level of communication
overhead, to probably no gain at any corner, but at a cost of having a
protocol that makes it harder to troubleshoot. :-)
Ok enough rambling for now.
I hope my ramblings don't scare you off, or turns you off from
experimenting. Having fun and experimenting is after all what HECnet is all
about.
Thanks for the input, yeah I need to play more with it, I do like the
realative simplicity of it... If anything now that I think I got SIMH
tied into it ok, I wanted to add Qemu support, and whatever emulators
I can also find to build up some emulated network.. And again tying
back to a stubbed windows server that I have on the internet. In the
age of multi Ghz cpu's I don't think libz would be that much
overhead..
if anything HECnet strikes me as a neat way to plug in a bunch of
various emulators & stuff together...
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Jason Stevens wrote:
Right now testing, although I'm making a 'stub' version to run under
windows so I can setup a bridge on a windows hosting box I have...
Hmm. I suspect that might turn out to be very hard, if not impossible...
One of the serious problems is that you need to be able to send out packets with faked source MAC addresses. And all tools I've ever worked with in Windows which need to use some other MAC address have been tricky to say the least, not to mention that you need to reconfigure, and for some, also reboot the machine to use another MAC address. Transparent ethernet access from Windows seems to be difficult.
I noticed from the source there is no way to handle packets of 1500
bytes, as it would invariably require fragmentation... is
LAT/MOP/DecNET all smaller then 1500 bytes?
I'm not sure how you mean. The code bridge ethernet. Ethernet packets are (normally) a maximum of 1518 bytes. 1500 bytes of data, 6 bytes source and destination MAC, two bytes with protocol number, and four bytes of CRC.
Is there somewhere where you have identified that the bridge program don't handle 1500 bytes of data?
I'm not really interested in jumbo ethernet frames. No DEC equipment relevant here supports it anyway, so there is no need.
As for fragmentation... Now I assume you are talking about the encapsulation of ethernet packets in UDP packets. Those will be a bit larger still, and will almost certainly be fragmented when sent over the internet, yes. I don't see a problem with that. Do you?
I've only used decnet in the 'wild' once, but we moved from the VAX to
PC's... but then our VAX was weird as it ran Novell Netware..... I
wish I had saved the tapes, as netware for the vax has to be super
ultra rare...
Yeah. That would have been fun to have kept around.
Anyways I was thinking of adding compression support (zlib style)... I
should test it some more as I figure if the packet compresses somewhat
it'll be under the 1500 snaplen for ethernet....
Feel free. I'm not sure there is a point of doing so, but anyway.
If you want to avoid fragmentation of packets, you need to find out what the minimum MTU are along the whole path between two nodes. And your packet then needs to fit both the data to carry as well as the UDP and IP header within that packet size. The minimum size that is guaranteed by IP to not be fragmented is 576 bytes. For UDP that means a size of the data part of just 548 bytes.
And that means you need to do fragmentation yourself, since you'll never be able to squeeze 1500 byte packets into 548 bytes with any kind of certainty. And if you start doing the fragmentation yourself, you will in fact add to the overhead, since then you'll create a separate packet for each fragment, all with their IP *and* UDP headers. And you'll need to do fragment reassembly, and what will you do if some fragment is missing? Do retransmissions? Now we're getting into a very complex protocol with acknowledgements and whatever. And don't forget the cpu overhead of doing the compression/decompression of each packet. And if you start to have fragments, acknowledgements and so on, you'll have to add information to each packet to handle all this information, which means extra overhead there as well. In the end, you should not be surprised if your packets end up being larger than the current ones.
And in the "normal" case, each packet is still just generating a single, unfragmented IP packet anyway.
Like I said. Feel free to play with it, and see what you can do. But personally I don't think it's a good idea. You add complexity, processing overhead, protocol overhead, and possibly a whole new level of communication overhead, to probably no gain at any corner, but at a cost of having a protocol that makes it harder to troubleshoot. :-)
Ok enough rambling for now.
I hope my ramblings don't scare you off, or turns you off from experimenting. Having fun and experimenting is after all what HECnet is all about.
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Right now testing, although I'm making a 'stub' version to run under
windows so I can setup a bridge on a windows hosting box I have...
I noticed from the source there is no way to handle packets of 1500
bytes, as it would invariably require fragmentation... is
LAT/MOP/DecNET all smaller then 1500 bytes?
I've only used decnet in the 'wild' once, but we moved from the VAX to
PC's... but then our VAX was weird as it ran Novell Netware..... I
wish I had saved the tapes, as netware for the vax has to be super
ultra rare...
Anyways I was thinking of adding compression support (zlib style)... I
should test it some more as I figure if the packet compresses somewhat
it'll be under the 1500 snaplen for ethernet....
Ok enough rambling for now.
Marc Chametzky wrote:
I don't have a current use for HECnet other than the novelty of being connected. I just got my SIMH VAX back up and running, now on a dedicated virtual host running under ESXi. I need to tweak my IP address and send that to Bob to update my MultiNet circuit.
I think for most people it's more about fun than actual real use...
I'd love to get it to the point that I've got a guest account for people to play what games I've got installed (including a collection of Infocom games I put on there).
Speaking of infocom, do people know about ZEMU?
On MIM::
.zem /li
-- System games --
Game Release Serial Inform Z-Machine
ADVENT 5 961209 6.05 5
AMFV 77 850814 4
BALLERINA 1 991128 6.21 8
BALLYHOO 97 851218 3
BEYONDZOR 57 871221 5
BOMBER 3 971123 6.13 5
BORDERZON 9 871008 5
BUREAUCRA 116 870602 4
CUTTHROAT 23 840809 3
DEADLINE 27 831005 3
DOGSLIFE 1 981015 6.15 5
DREAMHOLD 5 041231 6.21 8
ENCHANTER 29 860820 3
HEROINE 3 001211 6.21 8
HITCHHIKE 31 871119 5
HOLLYWOOD 37 861215 3
INFIDEL 22 830916 3
LEATHER 59 860730 3
LURKING 203 870506 3
MOONMIST 9 861022 3
NEVER 10 000928 6.21 5
NORDANDBE 19 870722 4
PLANETFAL 37 851003 3
PLUNDERER 26 870730 3
SAMEGAME 1 980731 6.15 5
SEASTALKE 16 850603 3
SHERLOCK 21 871214 5
SORCERER 15 851108 3
SPACEZ 1 980710 6.15 5
SPELLBREA 87 860904 3
STARCROSS 17 821021 3
STATIONFA 107 870430 3
SUSPECT 14 841005 3
SUSPENDED 8 840521 3
TRINITY 12 860926 4
TROLL 3 980518 6.15 5
WISHBRING 69 850920 3
WITNESS 22 840924 3
ZDUNGEON 13 040826 6.14 5
ZOKOBAN 1 990810 6.21 5
ZORK1 88 840726 3
ZORK2 48 840904 3
ZORK3 17 840727 3
ZTUU 16 970828 6.13 5
.
If someone have any problems, or find any bugs, just send them to me. :-)
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
