Re: [ddlm-group] Multi block principles

On Wed, 17 Nov 2021 at 03:41, Bollinger, John C <John.Bollinger@stjude.org> wrote:

I'm not sure I follow. If a powder diffraction experiment splits the structures of, say, 3 component phases over 3 blocks + 1 block for the invariable information, all conforming to 'Base', isn't that information identical to presenting it all in a single block (no longer conforming to 'Base') with appropriate key data names added?

Perhaps, then, it is me who does not follow.  Let me try again.

I do not think it is the intention to allow CIF data to be split _arbitrarily_ among data blocks, but if so, then I do object to the “arbitrarily” part.  Up to now, different CIF data items have been associated with each other at a basic level by appearing in the same data block.  I understand that some communities may have layered additional conventions on top of that, but no such additions have been baked into CIF or any of our DDLs.  I want to maintain the data block as a coherent unit of data, such that splitting this …

data_combined

_set_category_1.x  5.00

_set_category_2.a  'example'

... into this …

data_1

_set_category_1.x  5.00

data_2

_set_category_2.a  'example'

... loses information (that the two items are associated with each other) except in cases that are explicitly provided for by dictionaries and conveyed by data explicitly presented among the relevant collection of data blocks.

The message that one can split data across different blocks should not be de-contextualized or oversold.

What is a zero-column key?  Is that like an implicit key with no actual values stored?

I’m not sure it’s a concept that anyone else uses, but it’s a simple generalization of standard ideas:

A key for a relation consists of some subset of attributes of that relation (columns).  No two distinct rows of the relation can match in every key attribute, so in the absence of other constraints, as many rows can be present as there are distinct combinations of values drawn from the key attributes’ domains.

Now suppose we want a relation that is restricted to a single row.  One way to do that would be to give the relation an attribute whose domain contains only one value, and to designate that as the only key attribute. But in most cases that’s artificial and untidy.

There is a cleaner and simpler alternative: designate a key consisting of _zero_ attributes.  There is only one distinct combination of zero values: the empty set / tuple / dictionary. Therefore, a zero-attribute (zero-column) key affords only one row.  If we contemplate adding key attributes to a category, then adding them to an existing zero-attribute key is both logically and structurally simpler than converting a category that does not have a key at all into one that does.

For DDLm dictionaries, that concept could be applied to give category keys to Set categories without defining any new attributes in those categories.  Where Set categories need to be changed, possibly dynamically, into Loop categories, that’s made simpler if the fundamental difference is quantitative (the number of key attributes) rather than qualitative.

So how about reversing it, and the child categories instead identify their parent categories using a new DDLm attribute?

Of course, we already have exactly this for Loop – Loop relationships, and we will need to use it for Loop – Set_turned_into_Loop relationships, at least conceptually.  It takes the form of the _name.linked_item_id of an item with _type.purpose Link.  I think it’s sensible to handle Loop – Set relationships analogously.  It’s too late to design a single mechanism that could handle both, but that would have been ideal.

This would still require extension dictionaries to add information to core categories from time to time. One example might be an imaginary twinning dictionary that introduces 'twin_id' in category 'twin'. Until this dictionary, the 'refln' category implicitly assumed a single value of this identifier, so the dictionary would redefine 'refln' to also depend on 'twin', as would 'diffrn_refln' and some others.  The test comes when e.g. the modulated structure dictionary does not know about the existence of the 'twin' dictionary and redefines 'refln' its own way.

This missing information is, however, not a problem as it simply retains the meaning of 'single individual twin' for a modulated structure. If someone wants to describe a twinned modulated structure, then the modulated structures dictionary categories can be updated accordingly, and as long as the 'Base' schema is retained legacy software will be OK.  We still retain the option of explicitly defining the parent/child data names for complex situations.

I acknowledge that we can expect that extensions will still sometimes need to define modifications to core categories.  I am ok with that in principle, and although it will require some care in practice, I think it is workable.

On reflection, the original extension dictionary mechanism (adding explicit key data names to child categories) was really just creating these dependencies in child categories, but at the cost of proliferation of extension dictionaries (e.g. a modulated-structure-twin dictionary, a modulated-structures-laue-twin dictionary etc.). It seems much neater to simply gradually expand the lists of "parent" categories in child categories within the single dictionary as the need arises. If we are agreeable with this approach I'll draft a definition for a new DDLm attribute that we can discuss.

As may be evident from my previous comments, I’m not sure I recognize a distinction between an original extension mechanism and a new one.  At minimum, I guess I have not been viewing the multi-block proposal as a dictionary extension mechanism, though upon reflection, I see how it has a form of that rolled in.  If you are satisfied that we have enough common ground to consider specifics of a DDLm attribute then I would be happy to have that conversation.

Best regards,

John