On Sun, May 19, 2013 at 06:34:14PM +0200, Johnny Billquist wrote:
On 2013-05-19 18:14, Brian Hechinger wrote:
On Sun, May 19, 2013 at 05:02:18PM +0100, Robert Jarratt wrote:
Well, if the separator is | then it isn't a CSV file :-)
While I do understand the origins of the term CSV, the C really doesn
stand for character. Tabs are used quite commonly as well as commas. Any
character is valid, so long as it's accepted by all consumers of said
file.
The C standing for comma is antiquated and outdated and should be
changed. In my not so humble opinion, anyway. :)
He says, to a crowd of people running antiquated and outdated software
and hardware... :-D
Maybe I worded that wrong. An antiquated and outdated term that was
incorrect shortly after it was created. There's a big difference there. :)
Anyway, while I think I agree that technically, C stands for comma, I
also think | is a better separator in this case.
Feel free to call it anything. The acronym for the format is less
important. We could call it BSV then.
I vote HSV. HECnet Separated Values. That way we can use whatever the
HEC we want. :)
I honestly don't know what I think on that subject. Or, actually, I do.
I think it would be even better to just have key-value pairs. Why
overcomplicate things?
The problem with key-value pairs is they work great for single sets of
data but quickly turn into either INI files or something that quickly
aproaches being similar to JSON. CSV (sorry, HSV) is nothing more than a
simple way to store multiple sets of key-value pairs.
Honestly, this is all a moot argument considering the fact that your
data format needs to match whomever you want to be able to read it. If
you don't care that I can read it, well then it doesn't really matter
does it? :)
Any unknown key is just ignored. We could expand with new keys without
having the code synced. And then the values can hold any character
except newline.
This is pretty much what header indexed CSV gives us.
Actually I am thinking that I wouldn't mind writing the program myself, but
don't wait for me, I have a host of other things I want to do as well.
I'm always open to writing anything. :)
Knock one out for RSX? :-)
Gladly! Once I have more time (give it a month or so) I'd be more than
happy to take a stab at writing stuff for RSX/RSTS/etc.
I'll try and get back with some more thoughts about file formats and
whatnot in a while. My pizza just arrived...
Mmmm, pizza. I just had home made Feijoada. :)
-brian
On 19 May 2013, at 19:17, Dave McGuire <mcguire at neurotica.com> wrote:
I like it.
-Dave
--
Dave McGuire, AK4HZ
New Kensington, PA
On May 19, 2013, at 11:03 AM, Brian Hechinger <wonko at 4amlunch.net> wrote:
CSV as a data format is actually quite powerful, flexible and simple.
In order for it to work, however, we need to agree on some ground rules.
The ground rules are simply this:
1) The first line must *always* be the header. This is not optional.
This has already sort of been defined, i.e.:
ADDR |NAME |OWNER |EMAIL |HARDWARE |OS |LOCATION |NOTES
2) The field separator must be chosen.
I think we've settled on "|", no?
3) The names of the columns must be decided on.
See 1
4) The column names should be case insensative.
No issue with that.
5) The columns need to be defined required or optional.
Do we do this in the header or in the documentation?
6) This all needs to be documented somewhere.
I'm happy to write a doc describing the format we eventually settle on.
I like it.
-Dave
--
Dave McGuire, AK4HZ
New Kensington, PA
On May 19, 2013, at 11:03 AM, Brian Hechinger <wonko at 4amlunch.net> wrote:
CSV as a data format is actually quite powerful, flexible and simple.
In order for it to work, however, we need to agree on some ground rules.
The ground rules are simply this:
1) The first line must *always* be the header. This is not optional.
2) The field separator must be chosen.
3) The names of the columns must be decided on.
4) The column names should be case insensative.
5) The columns need to be defined required or optional.
6) This all needs to be documented somewhere.
The reason for the header being required is you can do column name to
column position mapping. This makes the CSV column layout not fixed. The
columns can be in whichever order each person finds appropriate.
#2 is pretty much already chosen as | seems to be the most common one
(one of my favorites as it rarely appears in text).
There are some already commonly used column names at this point, so we
should make a list of what those are and which we are going to use.
I think the rest are self explanatory. :)
Let's discuss!!
-brian
On 2013-05-19, at 9:34 AM, Johnny Billquist <bqt at softjar.se> wrote:
I honestly don't know what I think on that subject. Or, actually, I do. I think it would be even better to just have key-value pairs. Why overcomplicate things?
Any unknown key is just ignored. We could expand with new keys without having the code synced. And then the values can hold any character except newline.
You mean just like a Windows .ini file? (ducks)
j/k
Ian
-----Original Message-----
From: owner-hecnet at Update.UU.SE [mailto:owner-
hecnet at Update.UU.SE] On Behalf Of Johnny Billquist
Sent: 19 May 2013 17:34
To: hecnet at Update.UU.SE
Cc: Brian Hechinger
Subject: Re: [HECnet] CSV Proposal
On 2013-05-19 18:14, Brian Hechinger wrote:
On Sun, May 19, 2013 at 05:02:18PM +0100, Robert Jarratt wrote:
Well, if the separator is | then it isn't a CSV file :-)
While I do understand the origins of the term CSV, the C really doesn
stand for character. Tabs are used quite commonly as well as commas.
Any character is valid, so long as it's accepted by all consumers of
said file.
The C standing for comma is antiquated and outdated and should be
changed. In my not so humble opinion, anyway. :)
He says, to a crowd of people running antiquated and outdated software and
hardware... :-D
Anyway, while I think I agree that technically, C stands for comma, I also
think
| is a better separator in this case.
Feel free to call it anything. The acronym for the format is less
important. We
could call it BSV then.
I didn't want to start a big debate about this, it was just a little joke. I
would be more than happy to use the pipe symbol as the separator.
I have no issue with this except that allowing the columns to appear
in any order, while nice and flexible, makes it harder to write the
software. It does not seem worth the effort to have that flexibility
given the low enthusiasm for writing this software, so a
simplification would be to fix the column order.
It really isn't that hard to write. I've implemented such a thing in
many languages (including very stupid ones) and it is *always* worth the
effort.
It wouldn't be a big effort for me on the platforms I write for today. It
would be harder for me on platforms I programmed a long time ago (or even
never in the case of RSX) and never having programmed for Datatrieve, but
that is precisely why I would like to do it, although I am not at all sure
about the RSX hurdle.
Is there a language on VAX that would make it easier to port it to RSX? Or
can Datatrieve be accessed over DECnet so the scraper code runs on VAX and
populates Datatrieve on RSX?
I honestly don't know what I think on that subject. Or, actually, I do.
I think it would be even better to just have key-value pairs. Why
overcomplicate things?
Any unknown key is just ignored. We could expand with new keys without
having the code synced. And then the values can hold any character except
newline.
Actually I am thinking that I wouldn't mind writing the program
myself, but don't wait for me, I have a host of other things I want to
do as
well.
I'm always open to writing anything. :)
Knock one out for RSX? :-)
I'll try and get back with some more thoughts about file formats and
whatnot
in a while. My pizza just arrived...
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
On 2013-05-19 18:14, Brian Hechinger wrote:
On Sun, May 19, 2013 at 05:02:18PM +0100, Robert Jarratt wrote:
Well, if the separator is | then it isn't a CSV file :-)
While I do understand the origins of the term CSV, the C really doesn
stand for character. Tabs are used quite commonly as well as commas. Any
character is valid, so long as it's accepted by all consumers of said
file.
The C standing for comma is antiquated and outdated and should be
changed. In my not so humble opinion, anyway. :)
He says, to a crowd of people running antiquated and outdated software and hardware... :-D
Anyway, while I think I agree that technically, C stands for comma, I also think | is a better separator in this case.
Feel free to call it anything. The acronym for the format is less important. We could call it BSV then.
I have no issue with this except that allowing the columns to appear in any
order, while nice and flexible, makes it harder to write the software. It
does not seem worth the effort to have that flexibility given the low
enthusiasm for writing this software, so a simplification would be to fix
the column order.
It really isn't that hard to write. I've implemented such a thing in many
languages (including very stupid ones) and it is *always* worth the effort.
I honestly don't know what I think on that subject. Or, actually, I do. I think it would be even better to just have key-value pairs. Why overcomplicate things?
Any unknown key is just ignored. We could expand with new keys without having the code synced. And then the values can hold any character except newline.
Actually I am thinking that I wouldn't mind writing the program myself, but
don't wait for me, I have a host of other things I want to do as well.
I'm always open to writing anything. :)
Knock one out for RSX? :-)
I'll try and get back with some more thoughts about file formats and whatnot in a while. My pizza just arrived...
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt at softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
On Sun, May 19, 2013 at 05:02:18PM +0100, Robert Jarratt wrote:
Well, if the separator is | then it isn't a CSV file :-)
While I do understand the origins of the term CSV, the C really doesn
stand for character. Tabs are used quite commonly as well as commas. Any
character is valid, so long as it's accepted by all consumers of said
file.
The C standing for comma is antiquated and outdated and should be
changed. In my not so humble opinion, anyway. :)
I have no issue with this except that allowing the columns to appear in any
order, while nice and flexible, makes it harder to write the software. It
does not seem worth the effort to have that flexibility given the low
enthusiasm for writing this software, so a simplification would be to fix
the column order.
It really isn't that hard to write. I've implemented such a thing in many
languages (including very stupid ones) and it is *always* worth the effort.
Actually I am thinking that I wouldn't mind writing the program myself, but
don't wait for me, I have a host of other things I want to do as well.
I'm always open to writing anything. :)
-brian
-----Original Message-----
From: owner-hecnet at Update.UU.SE [mailto:owner-
hecnet at Update.UU.SE] On Behalf Of Brian Hechinger
Sent: 19 May 2013 16:03
To: HECnet Mailing List
Subject: [HECnet] CSV Proposal
CSV as a data format is actually quite powerful, flexible and simple.
In order for it to work, however, we need to agree on some ground rules.
The ground rules are simply this:
1) The first line must *always* be the header. This is not optional.
2) The field separator must be chosen.
3) The names of the columns must be decided on.
4) The column names should be case insensative.
5) The columns need to be defined required or optional.
6) This all needs to be documented somewhere.
The reason for the header being required is you can do column name to
column position mapping. This makes the CSV column layout not fixed. The
columns can be in whichever order each person finds appropriate.
#2 is pretty much already chosen as | seems to be the most common one
(one of my favorites as it rarely appears in text).
There are some already commonly used column names at this point, so we
should make a list of what those are and which we are going to use.
I think the rest are self explanatory. :)
Let's discuss!!
-brian
Well, if the separator is | then it isn't a CSV file :-)
I have no issue with this except that allowing the columns to appear in any
order, while nice and flexible, makes it harder to write the software. It
does not seem worth the effort to have that flexibility given the low
enthusiasm for writing this software, so a simplification would be to fix
the column order.
Actually I am thinking that I wouldn't mind writing the program myself, but
don't wait for me, I have a host of other things I want to do as well.
Regards
Rob
CSV as a data format is actually quite powerful, flexible and simple.
In order for it to work, however, we need to agree on some ground rules.
The ground rules are simply this:
1) The first line must *always* be the header. This is not optional.
2) The field separator must be chosen.
3) The names of the columns must be decided on.
4) The column names should be case insensative.
5) The columns need to be defined required or optional.
6) This all needs to be documented somewhere.
The reason for the header being required is you can do column name to
column position mapping. This makes the CSV column layout not fixed. The
columns can be in whichever order each person finds appropriate.
#2 is pretty much already chosen as | seems to be the most common one
(one of my favorites as it rarely appears in text).
There are some already commonly used column names at this point, so we
should make a list of what those are and which we are going to use.
I think the rest are self explanatory. :)
Let's discuss!!
-brian
On Sun, May 19, 2013 at 02:31:25PM +0200, Johnny Billquist wrote:
In a way it might be important to remember that my nodename database
extensions right now is just a fun project on my behalf without much
defined needs. Just like most things people do on HECnet it is totally
voluntary, and can be considered a pet project without any real demand.
You just described everything we're doing here. :)
I have not really seen any need for this stuff, so it's just for my
enjoyment. I personally found the INFO.TXT files unsatisfying, so I'm
doing something else. If it is meaningful or useful is not my primary
concern. It satisfies me. If it later turns out to be useful, that is
really cool. The same thing can be said of HECnet in general.
I repeat my earlier statement here. :)
All that said, yes, I could definitely consider a more strict format of
how something like location is defined. However, you will never see me
use XML if I can ever avoid it. And it is a really bad fit with older
computer systems, as it quickly becomes so big.
The same can be said of JSON. If we really want to get fine grained
information in fields, I would do that natively in Datatrieve. Why on
earth would I use a silly serializable external representation when I
already have the data in a database?
I am also against XML and JSON, but for mainly the same reason. CSV can
and will do everything we need here so why overcomplicate it? We aren't
questioning your need for the database or why we want to save things
externally. This is more a means to an end. How do we (meaning all of us
who's names aren't Johnny Billquist) update said database. Giving us
database access is one way, but I'm sure not everyone wants to bother
with that. The INFO.TXT file is a great way for those people to get data
into the database. The more information is in the database, the more
useful the database will end up being.
(And while JSON might be easier for a human to read, it's about as
unfriendly a XML when it comes to actually manipulating by hand. You
need tools, and those tools are also just out of the question on
something like a PDP-11. And I will not write my own.)
JSON is easier to read when you use it for such things as config files.
When you start using it to store rows of data and can very quickly
become less readable, at least when compared to CSV.
Right. And my main concern is actually that people do not have enough
interest to make this work. So it is either having a project by a single
individual, or else have something that never takes off.
There are at a minimum two people who are interested to make this work.
I'm one and you are the other. We've proven over the years that we can
be a force to be reckoned with if we so choose. :)
I think also it's not about it "taking off". It's something we put out
there that people use or don't use. We can't (and don't want to) force
anyone into using these tools. We just want them to be available if they
do.
But that is just what I think.
You think too much. :)
So at this point I do not believe in a distributed effort of some sort.
I also do not believe in doing things in something ugly like XML or
JSON. I have a database. I can change the schema of that, if needed.
That is the easy part. The hard part is having data *in* the database
that is up to date, and in a conformant shape. Neither of those problems
are addressed by having a schema as such, nor by using some other
representation.
I'll repeat here what I said before. The proposal is to simply have a
manner of getting data into the database. A properly defined schema for
some file that can be parsed is certainly a good idea in my opinion.
I will start a new thread for the CSV proposal as this one os getting a
tad off track. :)
-brian