Re: [ddlm-group] Relationship asmong CIF2, STAR,CIF1 and Python. .

---

From: Herbert J. Bernstein <yaya@bernstein-plus-sons.com>
To: Group finalising DDLm and associated dictionaries <ddlm-group@iucr.org>
Sent: Sunday, 16 January, 2011 12:26:28
Subject: Re: [ddlm-group] Relationship asmong CIF2, STAR, CIF1 and Python. .

Dear Simon,

  If I read your message correctly, you seem to be saying that the design objectives of CIF2 are:

  1.  enchance CIF as a data source
  2.  accomodate DDLm
  3.  do 1 and 2 in a way that does not complicate CIF as a data source

  I don't understand what you mean by "enhance CIF as a data source". This is the first time I recall hearing that objective.  Please clarify.

In terms of present practice, I had thought that the major current uses of CIF to be:

  1.  As a language for publication of papers in IUCr journals;
  2.  As a language for submission of data to CCDC;
  3.  As a language for submission of data to the PDB;
  4.  As a data harvest language for CCP4; and
  5.  As a language for formatting image data from Dectris detectors,
with some emerging use for data management of synchrotron data
  ( Please fill in uses I have missed ...)

which would seem to me to represent a large current investment in data management software that depends critically on stability and reliability of CIF representation of both numeric data and text.  As I understand it, DDLm and dREL can contribute to these current uses by enhancing the reliabiity of validation of data in these contexts, especially in uses 1, 2 and 3, above.  To me, that would  seem to change the objectives to:

  1.  Accomodate DDLm and dREL; and
  2.  Do this in a way that keeps required changes to existing
archives to a minimum; and
  3.  Do this in a way that allows as much existing CIF software
as possible to continue to operate reliably; and
  4.  To the extent that changes will be needed in archives and
software, provide a clearly understood mechanism for making
those changes, with as much support software as possible; and
  5.  Subordinate to the above, add new features to CIF that may
encourage broader use and more software support

Please tell me what I have misunderstood in this.

Regards,
  Herbert

=====================================================
Herbert J. Bernstein, Professor of Computer Science
  Dowling College, Kramer Science Center, KSC 121
        Idle Hour Blvd, Oakdale, NY, 11769

                +1-631-244-3035
                yaya@dowling.edu
=====================================================

On Sun, 16 Jan 2011, SIMON WESTRIP wrote:

> I believe that the CIF2 syntax changes enhance CIF as data source,
> even with the restriction to the contents of  ' and " delimited values,
> which
> I suspect will be the main source of incompatability between archived CIFs
> and CIF2. What we havent acheived is 100% compatability of CIF1 with CIF2,
> but having agreed that CIF2 is distinct from CIF1 and minimized the
> incompatability,
> I do not see the changes as disruptive or retrograde.
>
> DDLm methods will not be applicable to a significant percentage of a CIF
> data source
> (those 'free text' fields used to report experimental details etc.). I
> believe other approaches
> will be needed to enhance their content (referencing other data items from
> an item etc).
> Unicode support in CIF2 is beneficial in this area, and the new alternative
> delimiters may
> prove convenient. The list and table structures may also be exploited.
>
> So from my point of view, though I understood the aim of CIF2 was to
> accommodate DDLm,
> it turns out that it does offer a little bit more. I would still like the
> problem of including all delimiters
> in a value to be solved, and line-folding to be included. On my 'wish list'
> are value concatenation
> and a value-referencing mechanism for use with CIF as a data source.
>
> I know this doesnt really answer your question. What we are *not* trying to
> do is complicate
>  CIF as a data source?
>
> Cheers
>
> Simon
>
> ____________________________________________________________________________
> From: Herbert J. Bernstein <yaya@bernstein-plus-sons.com>
> To: Group finalising DDLm and associated dictionaries <ddlm-group@iucr.org>
> Sent: Sunday, 16 January, 2011 2:05:59
> Subject: Re: [ddlm-group] Relationship asmong CIF2, STAR, CIF1 and Python. .
>
> Dear Simon,
>
>   I have reviewed the record on this, and I have trouble finding
> a strong basis for most of the decisions made other than arguments
> from the authority of established past practice in STAR and dREL.
> Now that those arguments have turned out not to have been what they
> seemed, I have trouble accepting the resulting conclusions without
> new, clearly stated arguments based in the functionality we are
> trying to achieve, but it is now not even clear what functionality we
> are
> trying to achieve.
>
>   Perhaps you can help me -- what are we trying to do in defining
> CIF2?
>
>   Regards,
>     Herbert
>
>
> At 11:55 PM +0000 1/15/11, SIMON WESTRIP wrote:
> >That's a question I asked myself when I first joined this discussion group
> :-)
> >
> >To be honest Herbert, I do not feal qualified to contest anything to do
> with
> >dREL, nor DDLm, given that I havent worked on them and respecting the
> >years of effort that went into development thus far. The same is
> >true with respect to some of
> >the changes to CIF syntax that invalidate CIF1: I have accepted that
> >they are necessary
> >to facilitate implementation of dREL methods.
> >
> >Cheers
> >
> >Simon
> >
> >
> >
> >From: Herbert J. Bernstein <yaya@bernstein-plus-sons.com>
> >To: Group finalising DDLm and associated dictionaries <ddlm-group@iucr.org>
> >Sent: Saturday, 15 January, 2011 23:14:36
> >Subject: Re: [ddlm-group] Relationship asmong CIF2, STAR, CIF1 and Python.
> .
> >
> >Why not?  It is almost that right now.
> >
> >
> >At 11:08 PM +0000 1/15/11, SIMON WESTRIP wrote:
> >>True - but I can't see dREL becoming pyREL at this stage?
> >>
> >>
> >>
> >>From: Herbert J. Bernstein
> >><<mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.com>
> >>To: Group finalising DDLm and associated dictionaries
> >><<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org>
> >>Sent: Saturday, 15 January, 2011 22:57:17
> >>Subject: Re: [ddlm-group] Relationship asmong CIF2, STAR, CIF1 and Python.
> .
> >>
> >>Dear Simon,
> >>
> >>  But dREL already shares much of Python syntax and data structures,
> >>but, being significantly mutated, lacks the software support and
> >>documentation that Python has.  Anyone who has to work with the
> >>methods in a DDLm dictionary would be much better off if we
> >>simply made Python work with DDLm.  We would gain large libraries
> >>of pre-written utilities, tools to test code fragments interactively,
> >>and a lot more time to do science or whatever we are actually
> >>funded to do.
> >>
> >>  Regards,
> >>    Herbert
> >>
> >>
> >>
> >>At 10:35 PM +0000 1/15/11, SIMON WESTRIP wrote:
> >>>As far as I can see, parsing DDLm into an object stucture is fairly
> >>>uncomplicated;
> >>>the hurdle is parsing the dREL script as a method of the object.
> >>>Unless working with python, I'm not sure that adopting python syntax
> >>>for DDLm/CIF
> >>>is of any great benefit; likewise for dREL.
> >>>
> >>>That said, I have yet to actually do anything with DDLm, let alone
> >>>dREL, so I may be
> >>>well off the mark. But even if this is the case, I suspect there
> >>>will be non-python programmers out
> >>>there that have cause to work with CIF and similarly will see no
> >>>obvious benefit in
> >>>CIF sharing python syntax (especially if it only adopts it for one
> >>>set of delimiters at the
> >>>data-source level).
> >>>
> >>>Cheers
> >>>
> >>>Simon
> >>>
> >>>
> >>>
> >>>From: Herbert J. Bernstein
> >>><<mailto:<mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.c
> om><mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.com>
> >>>To: Group finalising DDLm and associated dictionaries
> >>><<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-gr
> oup@iucr.org>ddlm-group@iucr.org>
> >>>Sent: Saturday, 15 January, 2011 21:16:59
> >>>Subject: Re: [ddlm-group] Relationship asmong CIF2, STAR, CIF1 and
> Python. .
> >>>
> >>>At 12:43 PM +0000 1/15/11, Brian McMahon wrote:
> >>>>It might be worth remarking (again) that dREL is being developed as a
> >>>>canonical methods description language, and not necessarily the runtime
> >  >>>methods evaluator of choice for future applications. It may be that in
> >>>>practice future methods are initially developed and most frequently
> >>>>executed directly in Python or some other language. As I see it, the
> >>>>goal of CIF and DDL evolution is not to exclude such a possibility.
> >>>
> >>>If we are trying to be Python friendly and much of dREL is derived
> >>>from a Jython implementation, I don't understand why we are not
> >>>conforming dREL, DDLm and CIF2 to Python conventions as closely as
> >>>possible.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>At 12:43 PM +0000 1/15/11, Brian McMahon wrote:
> >>>>On Thu, Jan 13, 2011 at 05:35:21PM -0600, Bollinger, John C wrote:
> >>>>>
> >>>>>  (snip)
> >>>>>
> >>>>>  CIF2 <=> CIF1:
> >>>>>  To the greatest extent feasible, well-formed CIF1 documents should be
> >>>>>  well-formed CIF2 documents (modulo a CIF version identification
> >>>>>  signature) having the same meaning.
> >>>>
> >>>>Agreed.
> >>>>
> >>>>>  CIF2 <=> STAR:
> >>>>>  Inasmuch as CIF1 is derived from STAR, I think it appropriate for
> CIF2
> >>>>>  to look first to STAR, including its post-CIF1 development, for new
> >>>>>  features it may need.  Even if CIF2 is not 100% compatible with STAR,
> it
> >>>>>  is worthwhile to avoid diverging without compelling reason.
> >>>>
> >>>>Agreed
> >>>>
> >>>>>  CIF2 <=> Python:
> >>>>>  I see no particular reason for any formal relationship here beyond
> >>>>>  Python's role as the indirect inspiration for CIF2's new
> >>>>>  triple-quote syntax.  I am wary of the idea of tying CIF tightly to
> >>>>>  a particular language.  CIF2 documents are not and never will be
> >>>>>  Python programs.  I could imagine embedding Python in CIF or vise
> >>  >>>  versa, but I have seen no evidence to suggest that greater
> similarity
> >>>>>  between the two languages' syntax and semantics would benefit efforts
> >>>>>  such as those.
> >>>>
> >>>>Agreed. As I mention elsewhere, there is a greater influence on the
> >>>>prototype dREL (arising from the initial Jython implementation), and
> >>>>the list and table data types doubtless arise from that also.
> >>>>
> >>>>It might be worth remarking (again) that dREL is being developed as a
> >>>>canonical methods description language, and not necessarily the runtime
> >>>>methods evaluator of choice for future applications. It may be that in
> >>>  >practice future methods are initially developed and most frequently
> >>>>executed directly in Python or some other language. As I see it, the
> >>>>goal of CIF and DDL evolution is not to exclude such a possibility.
> >>>>
> >>>>Regards
> >>>>Brian
> >>>>_______________________________________________
> >>>>ddlm-group mailing list
> >>>><mailto:<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto
> :ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:<mailto:ddlm-group@iucr.or
> g>ddlm-group@iucr.org><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
> >>>><<<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.i
> ucr.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listin
> fo/ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group><<http://
> scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.org/mailma
> n/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/ddlm-group>
> http://scripts.iucr.org/mailman/listinfo/ddlm-group
> >>>
> >>>
> >>>--
> >>>=====================================================
> >>>  Herbert J. Bernstein, Professor of Computer Science
> >>>    Dowling College, Kramer Science Center, KSC 121
> >>>        Idle Hour Blvd, Oakdale, NY, 11769
> >>>
> >>>                  +1-631-244-3035
> >>>
> >>><mailto:<mailto:<mailto:yaya@dowling.edu>yaya@dowling.edu><mailto:yaya@d
> owling.edu>yaya@dowling.edu><mailto:<mailto:yaya@dowling.edu>yaya@dowling.e
> du><mailto:yaya@dowling.edu>yaya@dowling.edu
> >>>=====================================================
> >>>_______________________________________________
> >>>ddlm-group mailing list
> >>><mailto:<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:
> ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:<mailto:ddlm-group@iucr.org
> >ddlm-group@iucr.org><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
> >>><<<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iu
> cr.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinf
> o/ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group><<http://s
> cripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.org/mailman
> /listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/ddlm-group>h
> ttp://scripts.iucr.org/mailman/listinfo/ddlm-group
> >  >>
> >>>
> >>>_______________________________________________
> >>>ddlm-group mailing list
> >>><mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-gro
> up@iucr.org>ddlm-group@iucr.org
> >>><<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iuc
> r.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo
> /ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group
> >>
> >>
> >>--
> >>=====================================================
> >>  Herbert J. Bernstein, Professor of Computer Science
> >>    Dowling College, Kramer Science Center, KSC 121
> >>        Idle Hour Blvd, Oakdale, NY, 11769
> >>
> >>                  +1-631-244-3035
> >>
> >><mailto:<mailto:yaya@dowling.edu>yaya@dowling.edu><mailto:yaya@dowling.ed
> u>yaya@dowling.edu
> >>=====================================================
> >>_______________________________________________
> >>ddlm-group mailing list
> >><mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-grou
> p@iucr.org>ddlm-group@iucr.org
> >><<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr
> .org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/
> ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group
> >>
> >>
> >>_______________________________________________
> >>ddlm-group mailing list
> >><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
> >><http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.
> org/mailman/listinfo/ddlm-group
> >
> >
> >--
> >=====================================================
> >  Herbert J. Bernstein, Professor of Computer Science
> >    Dowling College, Kramer Science Center, KSC 121
> >        Idle Hour Blvd, Oakdale, NY, 11769
> >
> >                  +1-631-244-3035
> >                  <mailto:yaya@dowling.edu>yaya@dowling.edu
> >=====================================================
> >_______________________________________________
> >ddlm-group mailing list
> ><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
> ><http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.o
> rg/mailman/listinfo/ddlm-group
> >
> >
> >_______________________________________________
> >ddlm-group mailing list
> >ddlm-group@iucr.org
> >http://scripts.iucr.org/mailman/listinfo/ddlm-group
>
>
> --
> =====================================================
>   Herbert J. Bernstein, Professor of Computer Science
>     Dowling College, Kramer Science Center, KSC 121
>         Idle Hour Blvd, Oakdale, NY, 11769
>
>                   +1-631-244-3035
>                   yaya@dowling.edu
> =====================================================
> _______________________________________________
> ddlm-group mailing list
> ddlm-group@iucr.org
> http://scripts.iucr.org/mailman/listinfo/ddlm-group
>
>