[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Reply to: [list | sender only]
Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... .. .
- To: Group finalising DDLm and associated dictionaries <ddlm-group@iucr.org>
- Subject: Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... .. .
- From: "Bollinger, John C" <John.Bollinger@STJUDE.ORG>
- Date: Wed, 23 Jun 2010 17:11:22 -0500
- Accept-Language: en-US
- acceptlanguage: en-US
- In-Reply-To: <alpine.BSF.2.00.1006231550410.30894@epsilon.pair.com>
- References: <AANLkTilyJE2mCxprlBYaSkysu1OBjY7otWrXDWm3oOT9@mail.gmail.com><AANLkTilolZk4SzLF8mzqOz4EagFJcEHDKOAblGMnoqpW@mail.gmail.com><alpine.BSF.2.00.1006212120510.91069@epsilon.pair.com><AANLkTiklvzlKquqlRQIrpPGZjJfuRzLqiv2E6Stcq6wd@mail.gmail.com><alpine.BSF.2.00.1006212241210.4105@epsilon.pair.com><AANLkTilACXxnPRtJXEjGD39eleDl9dxlAcwar8j9MBPr@mail.gmail.com><alpine.BSF.2.00.1006220753471.87930@epsilon.pair.com><8F77913624F7524AACD2A92EAF3BFA54166122951E@SJMEMXMBS11.stjude.sjcrh.local><AANLkTikih0j6-vyLDPMOqcTkoiK545yE28y4fU9JTUa2@mail.gmail.com><20100623103310.GD15883@emerald.iucr.org><8F77913624F7524AACD2A92EAF3BFA541661229521@SJMEMXMBS11.stjude.sjcrh.local><alpine.BSF.2.00.1006231033360.56372@epsilon.pair.com><8F77913624F7524AACD2A92EAF3BFA541661229523@SJMEMXMBS11.stjude.sjcrh.local><alpine.BSF.2.00.1006231406010.30894@epsilon.pair.com><8F77913624F7524AACD2A92EAF3BFA541661229526@SJMEMXMBS11.stjude.sjcrh.local><alpine.BSF.2.00.1006231550410.30894@epsilon.pair.com>
On Wednesday, June 23, 2010 3:00 PM, Herbert J. Bernstein wrote: > John is mistaken about the value of the transmission check for most >code pages. Most of the Cyrillic code pages have quite distinctive >printable charcaters for these characters values, and none of them would >return the transmission check as the intended characters. At worst you >would get :: in place of the :<tc>: (e.g. with KOI0 or KOI7) Try it with >a few encodings and you will see it works pretty well. I chose the >accented o's to get in a region of the code tables that are well >populated, so for most encodings you would really see the problem, >but even the case of :: is pretty clear. I think you misunderstand my point, but perhaps I'm just not getting it. Here's what I think I understand about the transmission check proposal: the idea is to 1) include a fixed, well-known code at a recognizable position in the magic number. 2) The code will be constructed using a sufficient number of chosen non-ASCII characters such that 3) decoding the byte stream according to the wrong encoding will result in a transmission check string that differs from the expected, well-known one. Have I got it? Supposing that I understand the proposal, I quite agree that if a CIF encoded in UTF-8 and bearing the proposed transmission check signature were misinterpreted as being encoded in pretty much any other encoding, then the TC would reveal the mistake. And that would be valuable. The proposed scheme and specific signature would also work for a CIF encoded in ISO-8859-1, ISO-8859-15, and perhaps a couple others of the ISO-8859 family, except that it would not distinguish these one from another. As I understand it, however, that the same procedure could not be used for CIFs using most other encodings. At best, different TC codes would be required for most encodings. For example, KOI8-R does not have a way to express *any* of the characters having Unicode code points U+00F2 - U+00F6, so CIF text encoded in KOI8-R simply could not include the TC signature at all. There are no codes for the requisite characters. That does not stop detection of UTF-8 misinterpreted as KOI8-R; rather, it makes it impossible to even attempt the reverse. I am unaware of a single non-ASCII character that is representable in substantially all encodings we might reasonably expect to be used for CIF text. A different TC signature could be chosen to use for KOI8-R, ISO-8859-various, Shift-JIS, etc. Each would provide a decent check that a CIF encoded in the corresponding form was not misinterpreted as being in some other form, provided that the intended encoding were also known (for looking up the appropriate TC code). That's doable, but it's suddenly a lot more complicated, requiring a lot of bookkeeping. As I said, I do not oppose the transmission check idea. If UTF-8 were indeed designated the canonical encoding for CIF among many alternatives, then I would expect TC to be applicable to a large number of the important cases. I doubt there is a simple alternative that is significantly more general, but that's what I would prefer if such could be found. Regards, John -- John C. Bollinger, Ph.D. Department of Structural Biology St. Jude Children's Research Hospital Email Disclaimer: www.stjude.org/emaildisclaimer _______________________________________________ ddlm-group mailing list ddlm-group@iucr.org http://scripts.iucr.org/mailman/listinfo/ddlm-group
Reply to: [list | sender only]
- Follow-Ups:
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... .. . (Herbert J. Bernstein)
- References:
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (James Hester)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (James Hester)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (Herbert J. Bernstein)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (James Hester)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (Herbert J. Bernstein)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (James Hester)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. . (Herbert J. Bernstein)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. . (Bollinger, John C)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. . (James Hester)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. . (Brian McMahon)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. . (Bollinger, John C)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. . (Herbert J. Bernstein)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .. . (Bollinger, John C)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .. . (Herbert J. Bernstein)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... . (Bollinger, John C)
- Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .. ... (Herbert J. Bernstein)
- Prev by Date: Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .. .
- Next by Date: Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... .. .
- Prev by thread: Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .. ...
- Next by thread: Re: [ddlm-group] options/text vs binary/end-of-line. .. .. .. .... .. .
- Index(es):