[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Fwd: Re: DDLm, dREL, images and NeXus
I was under the impression the dREL use case was primarily for
calculation and validation methodologies embedded within a
relational database. It was a "fit for purpose" language that
didn't include any external IO routines, although they were added to
the dREL shell interpreter for convenience.
Consequently conversion of CIF content to NeXus/HDF5 seems way out of
language scope to me. I am sure if the designers intended
a general purpose programming language to be used, then they
would have embedded python in its totality and been done with it.
Personally I see dREL as a promissing technology in the way that
fusion reactors have been promissing for the past 50 years, and
probably the next 50 too. Consequently I would be hesitant
to base a technology around waiting for dREL to come of age.
What would be more useful (from a search and archival perspective)
would be something like a standardised mapping from CIF names to NeXus
NXDetector.NXdata names and vice-versa - for data values that don't map
onto already defined NeXus elements.
It would be useful also to map those into the sample_parameter,
dataset_parameter and datafile_parameters of the STFC's
ICAT metadata catalogue. And even the metadata attributes of the TARDIS
spec and the RDF of eCrystals. If that mapping was in CIF or some other
format, then everyone could write their own conversion tools in their
language of choice (and then use NXTranslate? to convert to HDF5).
Just for the record, there a couple of non-authoritative, kludged
pseudo-schema's here:
http://cima.chem.usyd.edu.au:8095/schemaFusion/html/index.html
but they are only intelligible if you view them with firefox.
Actually, probably it would be nice if all the various diffractometer
company frame header parameters could be mapped to a standard as well.
Herbert, I believe you have already written an imageCIF to NeXus (XML?)
conversion tool and I would be interested to know which CIF items were
mapped into which NeXus fields and whether the NeXus files hold
multiple images or links to external filesets and what do you do with
image parameters that don't fit exist within the imageCIF vocabulary.
I understand that HDF5 has one of the most efficient structured file access
protocols around, so presumably you would embed an image within the NeXus
file?
Or are there granularity issues? Should a complete imageCIF file be
embedded as a single NX_CHAR/NX_BINARY record?
As datasets are becoming larger over time, both in the number and size of
images,the simple act of looking at files within directories can be
troublesome, especially over remote networks.
Pascal Calvat, the author of the JUX SRB browser, was suggesting to archive
complete datasets in SRB repositories as a single very large file. I think
he was actually refusing to handle more than 500 files in a folder/directory
with JUX. But apparently SRB and iRODS have backends that can let you
peer into such stored HDF5 files in any case, so I am not sure if his
arguments still hold.
Doug
On Thu, 11 Dec 2008, Herbert J. Bernstein wrote:
> Dear Colleagues,
> There are many frameworks
> to consider, but two that are very close to achieving the goal of becoming
> inter-operable in the immediate future are imgCIF and NeXus. What is
> missing is a formal language within which to specify how to move between
> them.
>
> We could, of course, just come up with a verbal description of how
> to move between imgCIF and NeXus and a couple of example conversion
> programs written ad hoc in whatever language might come to mind. However,
> the effort being expended on dREL, the supporting language for DDLm,
> suggests the possibility of building on dREL as a base to do this job
> by extending dREL to have the capability of working with NeXus (dREL
> is already capable of dealing with CIF).
> James has made the counter
> proposal of leaving dREL as just a CIF-specific language and keeping
> the CIF-NeXus conversion algorithm specification as a matter for a
> different language and/or API. James has also suggested the further
> step of stripping out the built-in functions from dREL and dealing with
> just a very stable dREL language specification in one instance and a
> perhaps evolving API (list of builtin functions available in dREL) on the
> other:
> "My comment at this stage would be that defining a coupling mechanism
> between CIF and a given language is not a large task, due to the
> simplicity of the CIF syntax, whereas adding lots of stuff to dREL
> would be a serious task and has some important downsides (loss of
> simplicity being an important one). Apropros the
> simplicity of the coupling mechanism, I (predictably) quite like my
> Python model of a CIF file as a hash table of CIF data block objects
> indexed by datablock name, and the datablock objects are themselves
> hash tables of strings/lists of strings indexed by dataname. This
> model would appear to translate pretty easily into most other
> languages. What then remains is some syntactic sugar (the use of
> square brackets to do key-based lookup is nice in dREL) which can be
> replaced in another language by a few standard methods."
>
> There was a lot more to the discussion, but let us try to settle a
> direction:
>
> Should we be trying to extend dREL to support more than just CIF,
> specifically NeXus, making something we might call dREL++, or should the
> language for this broader task be something distinct from dREL with a
> distinct name. In practice, in either case, I suspect all of this will be
> built on a python base, or something similar, as James suggests, but
> names do matter,
>
> Comments please.
>
> Regards,
> Herbert
>
> =====================================================
> Herbert J. Bernstein, Professor of Computer Science
> Dowling College, Kramer Science Center, KSC 121
> Idle Hour Blvd, Oakdale, NY, 11769
>
> +1-631-244-3035
>
[email protected]
> =====================================================
> _______________________________________________
> comcifs mailing list
>
[email protected]
>
http://scripts.iucr.org/mailman/listinfo/comcifs
-------------------------------------------------------
Reply to: [list |
sender only]
