Discussion List Archives

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ddlm-group] Semantics of whitespace-delimited values

Let me take up one of Simon's comments:

"...we could suggest that CIF applications started to turn to the dictionary rather than syntax to determine the exact nature of a data item".

What is not perhaps appreciated is that the application programmer accessing the CIF file searching for a numeric value for a particular dataname has already consulted the dictionary when writing the program (with the minor exception of e.g. pretty-printers as noted before).  Consulting the very same dictionary at runtime is pointless as these meanings are never supposed to change. So I would suggest to Simon that there is no problem nudging *application* programmers to accept the dictionary definitions as they already have done so in order to write correct calculations, the only problem will be nudging CIF APIs to do their best to return a number if asked (and your survey would suggest that the majority already do).  My position is very strongly pro application programmer - if they are asking my API for a number, I am not going to second-guess them unless they have asked me to by providing a dictionary as well.

I'd be happy to see period and question mark added to the EBNF as primitive productions, this is a simple change.

On 9 July 2015 at 07:46, SIMON WESTRIP <simonwestrip@btinternet.com> wrote:
Dear all

I extended the mini survey of current applications a little and looked  closer at some of the less-liberal parsers:

one of the applications I've looked at did not complain when I included some non-ASCII text in the CIF, while another complained about a data value constructed as '''z''' (valid CIF1), and one displayed rather quirky behaviour with regard to semicolon-delimited strings, rejecting the contained 'value' if it had a leading newline but not if it had a leading space -
all of these particular applications complained about delimited numbers to the extent that the application stopped processing.

Based on this (albeit limited) survey of some current well- known CIF applications, regarding the introduction of CIF2 it would definitely be prudent to indicate that 'yes indeed' the interpretation of CIF1.1 wihitespace-delimited values retains significance in CIF2. However, if possible I think it would be in the interests of flexibility and unambiguity if somehow we could suggest that CIF applications started to turn to the dictionary rather than syntax to determine the exact nature of a data item (afterall, as I see it, that's one very strong  motivation for developing CIF2 in the first place - and is the preferred approach in CIF1 too).  Thankfully (from my point of view) this isn't even an issue for the majority of applications I have looked at (they simply grab the data however they've found it and make use of it if they can, or they carefully validate the data against the dictionary). So what is challenging me is how we achieve this - i.e. nudging some applications to be a little more flexible (which in my experience is what many  'users' would most appreciate)., while at the same time maintaining the convention that numbers especially are still presented in an undelimited (uncluttered) fashion. I've no convincing answer to this yet.

Regarding the ? and . 'null' values I hesitate to suggest that we could take these out of the issue altogether by making them CIF key tokens - I hesitate because I suspect that some applications simply ignore their significance anyway and incorrect usage rarely presents a real problem (and also I haven't yet attempted to see if its actually possible to define them in this way in any case:-) - so its probably unnecessary and may even seem like a new complication to applications that were not particularly aware of, or bothered by, the significance of these tokens in the first place.



From: "Bollinger, John C" <John.Bollinger@STJUDE.ORG>
To: Group finalising DDLm and associated dictionaries <ddlm-group@iucr.org>; SIMON WESTRIP <simonwestrip@btinternet.com>
Sent: Wednesday, 8 July 2015, 16:21
Subject: RE: [ddlm-group] Semantics of whitespace-delimited values

Thanks Simon, James, and John.  I am uncertain how many distinct parsers are represented by the reports so far, but it seems there must be at least five.
I think we agree that parsers and applications should be permitted, if not required, to distinguish between the values . and '.', and between the values ? and '?'.  We also seem to agree that it is not useful to insist that parsers or applications refuse to interpret quoted values as numbers, although some CIF 1.1 parsers in fact do so at their own discretion, and some warn instead of refusing (with even that relying on taking the position that numbers are not supposed to be quoted).
Not being enamored of special cases, and not wanting CIF 2.0 to rule out CIF interpretation practice that is accepted and common in CIF 1.1 applications, I find myself favoring CIF 2.0 taking the position that in general, it is permitted but not required to interpret any string value differently when it is presented in whitespace-delimited form than when it is presented in any of the other forms.  The conventions for the special values . and ? could then be taken to apply on a domain-wide basis, whereas the convention for the form of numbers could be taken to apply on a more selective basis (per-DDL, per-dictionary, or even per-definition).  An implication of this position, however, is that whether or not a value is presented whitespace-delimited becomes a property of that value that a fully general CIF 2.0 parser must make available to its clients.  Moreover, for better or for worse, future dictionaries could establish additional items or data types whose values are required to be presented unquoted.
We could perhaps characterize that more specifically, maybe by saying that the exact form of values presented in any of the quoted forms is significant, or something along those lines, whereas values presented in whitespace-delimited form may afford equivalent alternative expressions.  That doesn’t exactly fit . and ?, but perhaps some similar statement could do so better.
John C. Bollinger, Ph.D.
Computing and X-Ray Scientist
Department of Structural Biology
St. Jude Children's Research Hospital

T +61 (02) 9717 9907
F +61 (02) 9717 3145
M +61 (04) 0249 4148
ddlm-group mailing list

Reply to: [list | sender only]
International Union of Crystallography

Scientific Union Member of the International Council for Science (admitted 1947). Member of CODATA, the ICSU Committee on Data. Member of ICSTI, the International Council for Scientific and Technical Information. Partner with UNESCO, the United Nations Educational, Scientific and Cultural Organization in the International Year of Crystallography 2014.

ICSU Scientific Freedom Policy

The IUCr observes the basic policy of non-discrimination and affirms the right and freedom of scientists to associate in international scientific activity without regard to such factors as ethnic origin, religion, citizenship, language, political stance, gender, sex or age, in accordance with the Statutes of the International Council for Science.