Re: [ddlm-group] Finalizing DDLm

Inasmuch as we appear to be discussing, rather than voting, please allow me to clarify my position:

I am _not_ concerned about whether a DDLm-conformant dictionary does or does not have rules to say that a particular category is or is not allowed to be presented "unrolled".  I am concerned with how to handle two important cases:

 1.  Existing legacy data files that have "rolled" or "unrolled" loops that do not conform to the new dictionary rules; and

 2.  Applications that are confronted with a data file, portions of which are not in dictionaries to which that application has access.

If an application has a dictionary handy and that dictionary says something relevant about the rolledness or unrolledness of a loop, then I am reluctantly willing to accept the DDLm specification requiring the issuance of a warning or an error message.  Some application writers may decide not to do that, but that is a different discussion.

What I am concerned about is the very practical issues above -- of doing something useful with user data that either does not conform to this stricture as presented in a dictionary or on which the dictionaries available to the application are silent.  I am proposing that, rather than requiring an application to throw up its hands and die, we try to maximize the useful work to be accomplished and try to do something sensible with the data, i.e. roll that which is unrolled or unroll that which is rolled, if it allows the work of the application to get done.

I have yet to hear of a reason not to adopt that approach for the cases listed above.  Once we have those two cases settled, I would be happy to discuss the subtleties of whether the List attribute itself should be modified or not, but first, please, let us deal with this practical issue.

Regards,
   Herbert

P.S. to Nick:  It is the current DDLm specification that would require every application writer to read the dictionary in order to process a CIF, else we would have no way to tell whether rolled or unrolled presentation was in conformance with the dictionary.  The list attribute is in the dictionary, not the data file.  The discussion we are having is orthogonal to the question of whether the DDLm specification requires the reading of the dictionary.

P.S. to James:  I have read Nick's argument and on the DDLm specification
issue and stick to voting for 2.  If we change the specification then
strict adherence will no longer require List categories to be presented
as looped data, and no more or less dictionary reading will be required
than is required by the current specification, but users will be
annoyed by one less warning/error message they are not likely to understand or be able to do anything about.  However, no matter how
that vote comes out, we really do need to deal with the practical issue
above -- there are an awful lot of PDB mmCIF data files.

=====================================================
 Herbert J. Bernstein, Professor of Computer Science
  Dowling College, Kramer Science Center, KSC 121
       Idle Hour Blvd, Oakdale, NY, 11769

                +1-631-244-3035
                yaya@dowling.edu
=====================================================

On Wed, 14 Apr 2010, Nick Spadaccini wrote:

This doesn?t actually make things clearer or easier James. I will repeat it again.

Strict adherence to the formal specification of DDLm REQUIRES List categories to be presented as looped data, even
if there is only one row.

If the IUCr wishes to universally adopt the case where instances of List categories that contain only one row may
be presented as a Set category then it can do so as an accepted extension. This of course requires every
application writer to necessarily read the dictionary to establish if the data is really a Set category or
possibly a List category. Once the IUCr adopts this extension to DDLm within its implementation, I would assume
every application writer would be required to adhere to it.

On 14/04/10 3:29 PM, "James Hester" <jamesrhester@gmail.com> wrote:

     Dear all,

     Both John and Herb have come out in favour of allowing one-row loops to be unrolled.  Nick and I are
     both sceptical about the value of this idea.  We have a few options:

     1.  Disallow loop unrolling altogether (as in DDL1).
     2.  Allow loop unrolling for all DDLm dictionaries
     3.  Add a category-scope DDLm attribute stating that one-row loops in this category and child
     categories may be unrolled.  If it appears in the 'Head' category of the dictionary, it would mean
     that all categories in the dictionary could be unrolled.

     We have not discussed option 3: it basically means deferring the decision on loop unrolling to the
     dictionary writers.  It also means that programmers of generic CIF software will need to be prepared
     for either behaviour, so in that sense it is slightly more burdensome than option 2.

     Unless the silent majority would like to contribute further thoughts on this matter, I suggest that we
     vote and move on.  I discern that the voting so far would be:

     Option 1: James, Nick
     Option 2: John, Herb
     Option 3: ?

     (Some comments on John's post are inserted below).

     James.
     On Thu, Apr 1, 2010 at 10:41 AM, John Westbrook <jwest@rcsb.rutgers.edu> wrote:
           Hi all,

           Coming in late on this in support of Herb's position.

           I  have never understood the necessity of marking a category as
           a 'list' type in the dictionary in the early CIF DDL,
           and in DDLm I find this even more confusing.   Given
           that DDLm supports a category key which provides a
           well defined basis for each category, this alone
           would seem to provide the appropriate expression of
           cardinality.


     Absolutely agree, my objection is not to the loss of some packet ordering information, this is
     explicitly excluded from the infoset produced by the parser in any case.


           The choice of exporting a category with a single row as
           a collection of keyword-value pairs or  using a table
           format via a loop_  seems like a presentation style
           matter rather than dictionary issue.


     It is more than a presentation issue, as you have lost the information that those key-value pairs
     belong together, and so you need to refer to your dictionary to reconstitute them as a group.  And if
     you allow the possibility of unrolling single-row loops for all categories, then significant extra
     work is done to check, and if necessary, transform the internal representation back to a canonical
     looped form.  This reconstruction of the canonical form is highly desirable in a DDLm context, where
     we often wish to apply dREL operations to all packets in a loop.
      

           As Herb has observed, the vast majority of DDL2 files opt
           for key-value output for any category with a single row.
           I do not see what additional semantics are conveyed by
           regulating the manner data presentation in these cases.


     See above - some semantic information is lost.
      

           John

         

_______________________________________________
ddlm-group mailing list
ddlm-group@iucr.org
http://scripts.iucr.org/mailman/listinfo/ddlm-group