Discussion List Archives

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Treatment of Greek characters in CIF2

I strongly recommend that all non-CIF2 data be transformed into CIF2 format. We have an opportunity to help with CIF1 -> CIF2 transition via CIF1 -> CIF2-JSON -> CIF2 even. The METADATA should reflect that this was done. Let's be progressive, not regressive. If need be, software can revert that to CIF1.

Is Brian McMahon listening? If so, Brian, can you chime in here?

Bob


On Thu, Apr 20, 2017 at 8:42 AM, Bollinger, John C <John.Bollinger@stjude.org> wrote:


On Thursday, April 20, 2017 12:38 AM, James Hester wrote:
> According to section 2.2.7.4.13 - 17 of International Tables Vol G, by default Greek and some other non-ASCII characters can be represented in text datavalues using a backslash notation <backslash><ascii character>, e.g. \a is alpha.   Different markup conventions are possible on a per-dictionary or per-definition basis. In CIF2, these characters can be represented natively, but legacy CIF applications presented with a datavalue containing non-ASCII values may not be prepared to typeset or present them appropriately.  On the other hand, it would seem inefficient to define separate Unicode-aware datanames for every text value simply to avoid legacy problems.
>
> I view this as a secondary issue in that very few if any machine-interpretable datavalues rely on the backslash convention (perhaps a few enumerated values for X-ray tube radiation type?).
The problem arises only with CIF2 (obviously). As a starting suggestion, what do you think of the following as a notional additional paragraph for the above-mentioned section 2.2.7.4?
> (2.2.7.4.18) Whenever an application is required to convert a datavalue from a CIF2 datafile containing code points outside the ASCII range to a datavalue containing only ASCII codepoints, the appropriate markup as per paragraphs 2.2.7.4.13-16 should be substituted, provided that the relevant definition meets the requirements of paragraph 2.2.7.4.13. If no markup is defined for the Unicode code point, no CIF1 equivalent value exists and application behaviour is undefined.
>
> I toyed with the idea of allowing '\Uxxxxxx' for arbitrary Unicode code points, but (i) this would clash with '\U' for capital upsilon and (ii) is not expected by legacy applications and so would therefore require that they be updated, in which case adapting them to just ingest Unicode would be more straightforward.
>
>
> Thoughts?

I think that to whatever extent and in whatever context the proposed specifications are natural, they do not require any special more force or support behind them.  On the other hand, where they are _not_ natural, they _should not have_ any force or support behind them.  Consider in particular that ITVG 2.2.7.4.13 conditions use of the markup codes on permission from the relevant dictionary, therefore any rule that directs conversions to those coded forms without regard to items' definitions is inappropriate.

I'm inclined to say that the best available approach is for software to use (some encoding of) Unicode as the internal representation of CIF data, and to convert that as needed to appropriate external forms, where the relative propriety of different external forms is context-dependent.  CIF 1.1 with traditional CIF markup codes (+/- some convention for general Unicode characters) is a widely-recognized and used external form, so it may often be the best choice, but that's no justification for asserting that it should be the only choice, especially given its well-known limitations.

I acknowledge that it poses a clear risk of data corruption and/or misinterpretation if different external forms cannot easily be distinguished from each other, but I don't think that's a problem that we can reasonably hope to solve by fiat.


John


________________________________

Email Disclaimer: www.stjude.org/emaildisclaimer
Consultation Disclaimer: www.stjude.org/consultationdisclaimer
_______________________________________________
cif-developers mailing list
cif-developers@iucr.org
http://mailman.iucr.org/cgi-bin/mailman/listinfo/cif-developers



--
Robert M. Hanson
Larson-Anderson Professor of Chemistry
St. Olaf College
Northfield, MN
http://www.stolaf.edu/people/hansonr


If nature does not answer first what we want,
it is better to take what answer we get.

-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900

_______________________________________________
cif-developers mailing list
cif-developers@iucr.org
http://mailman.iucr.org/cgi-bin/mailman/listinfo/cif-developers

Reply to: [list | sender only]
International Union of Crystallography

Scientific Union Member of the International Science Council (admitted 1947). Member of CODATA, the ISC Committee on Data. Partner with UNESCO, the United Nations Educational, Scientific and Cultural Organization in the International Year of Crystallography 2014.

International Science Council Scientific Freedom Policy

The IUCr observes the basic policy of non-discrimination and affirms the right and freedom of scientists to associate in international scientific activity without regard to such factors as ethnic origin, religion, citizenship, language, political stance, gender, sex or age, in accordance with the Statutes of the International Council for Science.