[Fwd: Re: Question about DDLm]

Hi Nick, David and Syd:

(our discussion may be suitable for cif-developers?)

Before addressing David and Nick's comments, let me make a few assertions:

1. The DDL files contain text descriptions, which guide the programmer
in constructing programs which understand the machine-readable subset
of DDL attributes.  Once the machine-relevant human-readable content
has been programmed, no further programming should be necessary.

2. I distinguish two classes of dictionary use: validation and
generation.  Validation can be done once when a DDL domain dictionary
is published to check consistency with the DDL specifications, and has
historically been optional.  Generative capabilities in DDL1 and DDL2
are restricted to production of default values but are obviously far
richer in DDLm.

As far as enCIFer is concerned, there is no way that even the most
dictionary-conversant program could have survived the change to the
DDL.  The latest DDL1 dictionary added the enumerated value 'su' to
_type_conditions together with a human-readable description that was
machine-significant - that is, without human reprogramming there was
no way that a DDL1-conversant program would understand what 'su' meant
as a _type_condition in a definition block.  While a program that
always loaded the DDL dictionary would see that 'su' was one of the
accepted values for _type_condition, at the time that program was
written, only 'esd' would permit bracketed values to appear after a
data value.

Now to David's thoughts regarding the correct way to process CIF files
using dictionaries.

When reading in CIF files, the DDL *domain* dictionary is obviously
necessary for both validation and generation.  As for reading in the
DDL dictionary, the DDL1 dictionary generates only
'_enumeration_default' values for '_list', '_list_level',
'_list_mandatory','_type_conditions'  and '_type_construct' of which
only '_list' could sensibly change, and even then it would have
wide-ranging implications and so is unlikely.  In short, there is no
reason to read DDL1 dictionaries for generative purposes.  The DDL2
dictionary doesn't specify enumeration defaults for dictionary
attributes, so has no updateable generative sections and therefore
doesn't have to be read in as part of constructing the domain
dictionary.

So we come to DDLm.  The draft DDLm dictionary contains a number of
defaults for domain dictionary attributes, some of which could be
changed more or less easily - I'm thinking _import.if_miss, for
example.  The only method which is defined is how to construct the
_import_list.id attribute from individual components (this is actually
a human-readable description masquerading as a method, for reasons I
can go into if anyone is interested and will take up with Nick and Syd
shortly).  In any case, the DDLm dictionary is indeed necessary to
fully construct the DDL domain dictionary, both to understand
importation and to correctly assign default attribute values.  For
example, if the need arose to add a human-readable 'comment' field
into _import_list.id, the DDLm dictionary file could be changed and a
smart program would still be able to unpack the fields it needed
without producing any error.  Again, as soon as machine-significant
entries are added (as opposed to human-significant only) the program
would need to be human-edited.

Getting back to the issue at hand, the problem that I think David has
correctly identified is that there is no data item for the CIF data
block to specify to which domain dictionary(s) it conforms.

Now for the steps:

(David) 1. Program loads the latest version of DDLm.

No.  It should load the version to which the domain dictionary
conforms, so it needs to know what the domain dictionary is.

(David) 2. Program loads an audit CIF dictionary, followed by the CIF
itself.  At this stage the CIF is scanned just for the audit items
which include information on the dictionaries used to prepare the CIF.

Implicit in this is that the audit CIF dictionaries/data items are
specified externally to the data block.  The logical place is in the
DDL specification, in order to keep all human-readable
machine-significant information in one place.  This breaks the nice
symmetry of the DDL dictionary being the dictionary for the DDL domain
dictionary only.  I suspect that, in the past, programmers have just
looked at the domain dictionary (which is where these data items were
located) and hard-coded the dataname.

(David) 3. Program constructs a top level (calling) CIF that imports
the necessary dictionaries to create the virtual dictionary.

I would rephrase this as saying that the full DDL domain dictionary is
constructed (no need to reference the CIF data file any further) by
reading the DDL domain dictionary, finding the conformant DDL
dictionary, loading it and applying any generative methods to itself,
then applying any generative methods to the DDL domain dictionary
(i.e. importation).

(David) 4. Program reads the CIF again using the virtual CIF
dictionary, extracting the required crystallographic information.

No nead to read again, simply to apply the necessary methods (but
re-reading is one way to do it).

COMMENTS BY DAVID BROWN

If I understand your suggestion, my scheme would be replaced by the following