Re: DDLm, dREL, images and NeXus

Dear Colleagues,

   This is a distillation of an email conversation James Hester and I

have been having since the Osaka meeting.  We both feel that it

would be helpful if others were to join in and express their views.

We have been discussing the interaction among DDLm, dREL, images and

NeXus.  We agree on most points, and disagree on a few, and hope,

by opening up the discussion, to arrive at a consensus.

   What is driving this discussion is a need to understand how best to

manage image data in the context of both imgCIF and NeXus, and to do

so in a way that is consistent with the recent adoption of DDLm as

the target framework for new work on CIF dictionaries.

   It must be clearly understood that it is highly unlikely that a

single standard will ever be adopted for crystallographic diffraction

images, much less for the broader context of pixel-based data in

structural biology.  The best we can hope for right now is to have

some number of clearly defined image data frameworks, and agreed

algorithms for conversion among them.  There are many frameworks

to consider, but two that are very close to achieving the goal of becoming

inter-operable in the immediate future are imgCIF and NeXus.  What is

missing is a formal language within which to specify how to move between

them.

   We could, of course, just come up with a verbal description of how

to move between imgCIF and NeXus and a couple of example conversion

programs written ad hoc in whatever language might come to mind.  However,

the effort being expended on dREL, the supporting language for DDLm,

suggests the possibility of building on dREL as a base to do this job

by extending dREL to have the capability of working with NeXus (dREL

is already capable of dealing with CIF).  James has made the counter

proposal of leaving dREL as just a CIF-specific language and keeping

the CIF-NeXus conversion algorithm specification as a matter for a

different language and/or API.  James has also suggested the further

step of stripping out the built-in functions from dREL and dealing with

just a very stable dREL language specification in one instance and a

perhaps evolving API (list of builtin functions available in dREL) on the

other:

"My comment at this stage would be that defining a coupling mechanism

between CIF and a given language is not a large task, due to the

simplicity of the CIF syntax, whereas adding lots of stuff to dREL

would be a serious task and has some important downsides (loss of

simplicity being an important one). Apropros the

simplicity of the coupling mechanism, I (predictably) quite like my

Python model of a CIF file as a hash table of CIF data block objects

indexed by datablock name, and the datablock objects are themselves

hash tables of strings/lists of strings indexed by dataname.  This

model would appear to translate pretty easily into most other

languages.  What then remains is some syntactic sugar (the use of

square brackets to do key-based lookup is nice in dREL) which can be

replaced in another language by a few standard methods."

There was a lot more to the discussion, but let us try to settle a

direction:

Should we be trying to extend dREL to support more than just CIF,

specifically NeXus, making something we might call dREL++, or should the 

language for this broader task be something distinct from dREL with a 

distinct name.  In practice, in either case, I suspect all of this will be

built on a python base, or something similar, as James suggests, but

names do matter,

Comments please.

Regards,

   Herbert

=====================================================

  Herbert J. Bernstein, Professor of Computer Science

    Dowling College, Kramer Science Center, KSC 121

         Idle Hour Blvd, Oakdale, NY, 11769

                  +1-631-244-3035

                  yaya@dowling.edu

=====================================================

_______________________________________________

comcifs mailing list

comcifs@iucr.org

http://scripts.iucr.org/mailman/listinfo/comcifs