Herbert,
I had assumed that the DDLm would be backwardly compatible, but not
forwardly compatible. The _alias mechanism allows a DDLm program to
read CIF1 and CIF2 files and store the information under DDLm names.
There may be some problems with datanames that include reserved
characters, but the lack of a magic number at the head of the file will
warn the program of this possibility. Once the file is read, all the
features of DDLm are available. The program can then output a CIF3 (or
CIFm) file without further problem.
Whatyou want is to have the DDLm-based program write out a CIF1 or
CIF2, according to where it found the input or according to some other
instruction. This could be done again by consulting the _aliases, but
of course there may not be datanames for all the items it calculates.
By outputing the information under the appropriate _alias dataname, it
should be possible to output a CIF1 or CIF2. The categories should
take care of themeselves. The only place there would be difficulty is
in the loops, but the DDLm rules aer stronger than the CIF1 and CIF2
rules, so the loops should not violate the less stringent earler
standard. In any case I am not sure how your import..as would work in
this mode (I would have thought an export..as would be more
appropriate). The problem is that in the relevant items there are
always two, and sometimes as many as four, _aliases which means that
dREL will become extremely cumbersome and would have to know which
_import to apply.
But this is not really what we are talking about the moment. If we
want compatiblity at the syntax level we should be worrying about the
use of reserved characters in the older datanames (which we have
already discussed), not how we should modify dREL. It would be better
to use _aliases even if we have to provide information about their
original categories. This is by far the cleanest place - when we are
reading or writing a CIF - not when we are in the middle of applying a
method.
David
Herbert J. Bernstein wrote:
Dear David,
The alias mechanism in the DDLm document of 2008 does
not seem to do the job of getting around the restrictions
that were imposed for CIF2 on the CIF files themselves,
invalidating basically any tag that does not conform
to the necessities of identifer syntax in a dREL method.
What I am proposing as a way for a dREL method itself
to cope with an arbitrary CIF1 tag, allowing true
CIF1 documents handled. This is important to, for
eaxmple, ensure that when a DDLm method is populating
a category in a CIF1 document, that it populates the
actual original CIF1 category and not the category
with the CIF2 alias name. This will allow the power
of DDLm and dREL to be applied to documents that still
need to be processed by existing software that is
not yet aware of new CIF2 names for various categories
and tags, and therefore should not automatically migrate
to the alias name. The current form of alias seems
to be purely a one-way path.
Is there a later extension of alias after the 2008
document?
Regards,
Herbert
=====================================================
Herbert J. Bernstein, Professor of Computer Science
Dowling College, Kramer Science Center, KSC 121
Idle Hour Blvd, Oakdale, NY, 11769
+1-631-244-3035
yaya@dowling.edu
=====================================================
On Mon, 17 Jan 2011, David Brown wrote:
This is a pssing comment on the first
paragraph of Herbert's email to set
the record straight.
David Brown
Herbert J. Bernstein wrote:
What I propose is that we add one new statement to dREL,
similar to the python import .. as ..., which allows a module in
python that actually exists under one name to be used in a chunk
of pythin code as if it had a different name. One possible
syntax could be to simply extent the python import statment
allowing elides in the identifier name, such as
import cell\[\] as CELL
or some other quoting mechanism to allow CIF1 names that are not
valid internally in dREL to be handled by dREL algorithms.
We discussed this many moons ago. For the most part Herbert's
suggestion is
already accommodated by the _alias feature of DDLm. This allows any
CIF1 or
CIF2 dataname to be recognized as equivalent to a DDLm dataname at the
timw
of reading. Thus a CIF1 or CIF2 item will be treated exactly the same
way
in the program as if it were in a file written using a DDLm
dictionary.
There may be one or two places where a DDLm driven program might have
difficulty, but they will not be common. None come immediately to
mind.
The main problem will be trying to read the many archived CIFs that are
not
well formed, but there should be no need for a major reformatting of
the
existing archive.
## End of my comments
This does not mean that a DDLm dictionary would somehow have
to contain the identifiers that have been banned under CIF2. The
dictionary could work entirely under CIF2 rules. It just means
that the data being manipulated under control of the dictionary
could include unmodified CIF1 data files and that the algorithms
in dREL could be applied to the entire range of CIF1 tags.
There is no import statement currently in dREL, so there is
not conflict with the existing spec. Other than providing a
symbol table translation, there should be no impact of this
change on the structure and syntax of dREL.
While this is an othogonal feature, I do intend to also
propose that DDLm dictionaries we used to process CIF1 files
that include CIF1-style string, e.g. the famous 'O'', so that
the promise made for so many years on the IUCr web site of being
able to process archived CIF1 data files with DDLm can be kept.
I think CIF1 (both DDL1 and DDL2 CIF) coupled to DDLm could be
a very powerful and useful combination that could be introduced
with minimal disruption to existing software. I understand that
others may prefer a more drastic change. I hope they will see
the value in allowing this more cautious and conservative
approach to go forward under the CIF banner, even if they
themselves intend to pursue their own, more drastic set of
changes, also under the CIF banner.
If this is agreeable, I will undertake to have a demonstration
version of DDLm/dREL fully coupled to CIF1 using the import ...
as ... statement to show and discuss with those who are
interested in Madrid.
Regards,
Herbert
=====================================================
Herbert J. Bernstein, Professor of Computer Science
Dowling College, Kramer Science Center, KSC 121
Idle Hour Blvd, Oakdale, NY, 11769
+1-631-244-3035
yaya@dowling.edu
=====================================================
On Sun, 16 Jan 2011, Herbert J. Bernstein wrote:
Dear Simon,
If I read your message correctly, you seem to be
saying that the design objectives of CIF2 are:
1. enchance CIF as a data source
2. accomodate DDLm
3. do 1 and 2 in a way that does not complicate
CIF as a data source
I don't understand what you mean by "enhance CIF as
a data source". This is the first time I recall
hearing that objective. Please clarify.
In terms of present practice, I had thought that the
major current uses of CIF to be:
1. As a language for publication of papers in IUCr
journals;
2. As a language for submission of data to CCDC;
3. As a language for submission of data to the
PDB;
4. As a data harvest language for CCP4; and
5. As a language for formatting image data from
Dectris detectors,
with some emerging use for data management of
synchrotron data
( Please fill in uses I have missed ...)
which would seem to me to represent a large current
investment in data management software that depends
critically on stability and reliability of CIF
representation of both numeric data and text. As I
understand it, DDLm and dREL can contribute to these
current uses by enhancing the reliabiity of
validation of data in these contexts, especially in
uses 1, 2 and 3, above. To me, that would seem to
change the objectives to:
1. Accomodate DDLm and dREL; and
2. Do this in a way that keeps required changes to
existing
archives to a minimum; and
3. Do this in a way that allows as much existing
CIF software
as possible to continue to operate reliably; and
4. To the extent that changes will be needed in
archives and
software, provide a clearly understood mechanism for
making
those changes, with as much support software as
possible; and
5. Subordinate to the above, add new features to
CIF that may
encourage broader use and more software support
Please tell me what I have misunderstood in this.
Regards,
Herbert
=====================================================
Herbert J. Bernstein, Professor of Computer Science
Dowling College, Kramer Science Center, KSC 121
Idle Hour Blvd, Oakdale, NY, 11769
+1-631-244-3035
yaya@dowling.edu
=====================================================
On Sun, 16 Jan 2011, SIMON WESTRIP wrote:
I believe that the CIF2 syntax changes
enhance CIF as data source,
even with the restriction to the
contents of ' and " delimited values,
which
I suspect will be the main source of
incompatability between archived CIFs
and CIF2. What we havent acheived is
100% compatability of CIF1 with CIF2,
but having agreed that CIF2 is distinct
from CIF1 and minimized the
incompatability,
I do not see the changes as disruptive
or retrograde.
DDLm methods will not be applicable to a
significant percentage of a CIF
data source
(those 'free text' fields used to report
experimental details etc.). I
believe other approaches
will be needed to enhance their content
(referencing other data items from
an item etc).
Unicode support in CIF2 is beneficial in
this area, and the new alternative
delimiters may
prove convenient. The list and table
structures may also be exploited.
So from my point of view, though I
understood the aim of CIF2 was to
accommodate DDLm,
it turns out that it does offer a little
bit more. I would still like the
problem of including all delimiters
in a value to be solved, and
line-folding to be included. On my 'wish
list'
are value concatenation
and a value-referencing mechanism for
use with CIF as a data source.
I know this doesnt really answer your
question. What we are *not* trying to
do is complicate
CIF as a data source?
Cheers
Simon
___________________________________________________________________________
_
From: Herbert J. Bernstein
<yaya@bernstein-plus-sons.com>
To: Group finalising DDLm and associated
dictionaries <ddlm-group@iucr.org>
Sent: Sunday, 16 January, 2011 2:05:59
Subject: Re: [ddlm-group] Relationship
asmong CIF2, STAR, CIF1 and Python. .
Dear Simon,
I have reviewed the record on this,
and I have trouble finding
a strong basis for most of the decisions
made other than arguments
from the authority of established past
practice in STAR and dREL.
Now that those arguments have turned out
not to have been what they
seemed, I have trouble accepting the
resulting conclusions without
new, clearly stated arguments based in
the functionality we are
trying to achieve, but it is now not
even clear what functionality we
are
trying to achieve.
Perhaps you can help me -- what are we
trying to do in defining
CIF2?
Regards,
Herbert
At 11:55 PM +0000 1/15/11, SIMON WESTRIP
wrote:
>That's a question I asked myself when I
first joined this discussion group
:-)
>
>To be honest Herbert, I do not feal
qualified to contest anything to do
with
>dREL, nor DDLm, given that I havent
worked on them and respecting the
>years of effort that went into
development thus far. The same is
>true with respect to some of
>the changes to CIF syntax that
invalidate CIF1: I have accepted that
>they are necessary
>to facilitate implementation of dREL
methods.
>
>Cheers
>
>Simon
>
>
>
>From: Herbert J. Bernstein
<yaya@bernstein-plus-sons.com>
>To: Group finalising DDLm and
associated dictionaries
<ddlm-group@iucr.org>
>Sent: Saturday, 15 January, 2011
23:14:36
>Subject: Re: [ddlm-group] Relationship
asmong CIF2, STAR, CIF1 and Python.
.
>
>Why not? It is almost that right now.
>
>
>At 11:08 PM +0000 1/15/11, SIMON
WESTRIP wrote:
>>True - but I can't see dREL becoming
pyREL at this stage?
>>
>>
>>
>>From: Herbert J. Bernstein
>><<mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.com>
>>To: Group finalising DDLm and
associated dictionaries
>><<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org>
>>Sent: Saturday, 15 January, 2011
22:57:17
>>Subject: Re: [ddlm-group] Relationship
asmong CIF2, STAR, CIF1 and Python.
.
>>
>>Dear Simon,
>>
>> But dREL already shares much of
Python syntax and data structures,
>>but, being significantly mutated,
lacks the software support and
>>documentation that Python has. Anyone
who has to work with the
>>methods in a DDLm dictionary would be
much better off if we
>>simply made Python work with DDLm. We
would gain large libraries
>>of pre-written utilities, tools to
test code fragments interactively,
>>and a lot more time to do science or
whatever we are actually
>>funded to do.
>>
>> Regards,
>> Herbert
>>
>>
>>
>>At 10:35 PM +0000 1/15/11, SIMON
WESTRIP wrote:
>>>As far as I can see, parsing DDLm
into an object stucture is fairly
>>>uncomplicated;
>>>the hurdle is parsing the dREL script
as a method of the object.
>>>Unless working with python, I'm not
sure that adopting python syntax
>>>for DDLm/CIF
>>>is of any great benefit; likewise for
dREL.
>>>
>>>That said, I have yet to actually do
anything with DDLm, let alone
>>>dREL, so I may be
>>>well off the mark. But even if this
is the case, I suspect there
>>>will be non-python programmers out
>>>there that have cause to work with
CIF and similarly will see no
>>>obvious benefit in
>>>CIF sharing python syntax (especially
if it only adopts it for one
>>>set of delimiters at the
>>>data-source level).
>>>
>>>Cheers
>>>
>>>Simon
>>>
>>>
>>>
>>>From: Herbert J. Bernstein
>>><<mailto:<mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.c
om><mailto:yaya@bernstein-plus-sons.com>yaya@bernstein-plus-sons.com>
>>>To: Group finalising DDLm and
associated dictionaries
>>><<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-gr
oup@iucr.org>ddlm-group@iucr.org>
>>>Sent: Saturday, 15 January, 2011
21:16:59
>>>Subject: Re: [ddlm-group]
Relationship asmong CIF2, STAR, CIF1 and
Python. .
>>>
>>>At 12:43 PM +0000 1/15/11, Brian
McMahon wrote:
>>>>It might be worth remarking (again)
that dREL is being developed as a
>>>>canonical methods description
language, and not necessarily the
runtime
> >>>methods evaluator of choice for
future applications. It may be that in
>>>>practice future methods are
initially developed and most frequently
>>>>executed directly in Python or some
other language. As I see it, the
>>>>goal of CIF and DDL evolution is not
to exclude such a possibility.
>>>
>>>If we are trying to be Python
friendly and much of dREL is derived
>>>from a Jython implementation, I don't
understand why we are not
>>>conforming dREL, DDLm and CIF2 to
Python conventions as closely as
>>>possible.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>At 12:43 PM +0000 1/15/11, Brian
McMahon wrote:
>>>>On Thu, Jan 13, 2011 at 05:35:21PM
-0600, Bollinger, John C wrote:
>>>>>
>>>>> (snip)
>>>>>
>>>>> CIF2 <=> CIF1:
>>>>> To the greatest extent
feasible,
well-formed CIF1 documents should be
>>>>> well-formed CIF2 documents
(modulo a CIF version identification
>>>>> signature) having the same
meaning.
>>>>
>>>>Agreed.
>>>>
>>>>> CIF2 <=> STAR:
>>>>> Inasmuch as CIF1 is derived
from
STAR, I think it appropriate for
CIF2
>>>>> to look first to STAR,
including
its post-CIF1 development, for new
>>>>> features it may need. Even if
CIF2 is not 100% compatible with STAR,
it
>>>>> is worthwhile to avoid
diverging
without compelling reason.
>>>>
>>>>Agreed
>>>>
>>>>> CIF2 <=> Python:
>>>>> I see no particular reason for
any formal relationship here beyond
>>>>> Python's role as the indirect
inspiration for CIF2's new
>>>>> triple-quote syntax. I am wary
of the idea of tying CIF tightly to
>>>>> a particular language. CIF2
documents are not and never will be
>>>>> Python programs. I could
imagine
embedding Python in CIF or vise
>> >>> versa, but I have seen no
evidence to suggest that greater
similarity
>>>>> between the two languages'
syntax
and semantics would benefit efforts
>>>>> such as those.
>>>>
>>>>Agreed. As I mention elsewhere,
there is a greater influence on the
>>>>prototype dREL (arising from the
initial Jython implementation), and
>>>>the list and table data types
doubtless arise from that also.
>>>>
>>>>It might be worth remarking (again)
that dREL is being developed as a
>>>>canonical methods description
language, and not necessarily the
runtime
>>>>methods evaluator of choice for
future applications. It may be that in
>>> >practice future methods are
initially developed and most frequently
>>>>executed directly in Python or some
other language. As I see it, the
>>>>goal of CIF and DDL evolution is not
to exclude such a possibility.
>>>>
>>>>Regards
>>>>Brian
>>>>_______________________________________________
>>>>ddlm-group mailing list
>>>><mailto:<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto
:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:<mailto:ddlm-group@iucr.or
g>ddlm-group@iucr.org><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
>>>><<<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.i
ucr.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listin
fo/ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group><<http://
scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.org/mailma
n/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/ddlm-group>
http://scripts.iucr.org/mailman/listinfo/ddlm-group
>>>
>>>
>>>--
>>>=====================================================
>>> Herbert J. Bernstein, Professor of
Computer Science
>>> Dowling College, Kramer Science
Center, KSC 121
>>> Idle Hour Blvd, Oakdale, NY,
11769
>>>
>>> +1-631-244-3035
>>>
>>><mailto:<mailto:<mailto:yaya@dowling.edu>yaya@dowling.edu><mailto:yaya@d
owling.edu>yaya@dowling.edu><mailto:<mailto:yaya@dowling.edu>yaya@dowling.e
du><mailto:yaya@dowling.edu>yaya@dowling.edu
>>>=====================================================
>>>_______________________________________________
>>>ddlm-group mailing list
>>><mailto:<mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:
ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:<mailto:ddlm-group@iucr.org
>ddlm-group@iucr.org><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
>>><<<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iu
cr.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinf
o/ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group><<http://s
cripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.org/mailman
/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/ddlm-group>h
ttp://scripts.iucr.org/mailman/listinfo/ddlm-group
> >>
>>>
>>>_______________________________________________
>>>ddlm-group mailing list
>>><mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-gro
up@iucr.org>ddlm-group@iucr.org
>>><<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iuc
r.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo
/ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group
>>
>>
>>--
>>=====================================================
>> Herbert J. Bernstein, Professor of
Computer Science
>> Dowling College, Kramer Science
Center, KSC 121
>> Idle Hour Blvd, Oakdale, NY,
11769
>>
>> +1-631-244-3035
>>
>><mailto:<mailto:yaya@dowling.edu>yaya@dowling.edu><mailto:yaya@dowling.ed
u>yaya@dowling.edu
>>=====================================================
>>_______________________________________________
>>ddlm-group mailing list
>><mailto:<mailto:ddlm-group@iucr.org>ddlm-group@iucr.org><mailto:ddlm-grou
p@iucr.org>ddlm-group@iucr.org
>><<http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr
.org/mailman/listinfo/ddlm-group><http://scripts.iucr.org/mailman/listinfo/
ddlm-group>http://scripts.iucr.org/mailman/listinfo/ddlm-group
>>
>>
>>_______________________________________________
>>ddlm-group mailing list
>><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
>><http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.
org/mailman/listinfo/ddlm-group
>
>
>--
>=====================================================
> Herbert J. Bernstein, Professor of
Computer Science
> Dowling College, Kramer Science
Center, KSC 121
> Idle Hour Blvd, Oakdale, NY,
11769
>
> +1-631-244-3035
>
<mailto:yaya@dowling.edu>yaya@dowling.edu
>=====================================================
>_______________________________________________
>ddlm-group mailing list
><mailto:ddlm-group@iucr.org>ddlm-group@iucr.org
><http://scripts.iucr.org/mailman/listinfo/ddlm-group>http://scripts.iucr.o
rg/mailman/listinfo/ddlm-group
>
>
>_______________________________________________
>ddlm-group mailing list
>ddlm-group@iucr.org
>http://scripts.iucr.org/mailman/listinfo/ddlm-group
--
=====================================================
Herbert J. Bernstein, Professor of
Computer Science
Dowling College, Kramer Science
Center, KSC 121
Idle Hour Blvd, Oakdale, NY,
11769
+1-631-244-3035
yaya@dowling.edu
=====================================================
_______________________________________________
ddlm-group mailing list
ddlm-group@iucr.org
http://scripts.iucr.org/mailman/listinfo/ddlm-group
____________________________________________________________________
_______________________________________________
ddlm-group mailing list
ddlm-group@iucr.org
http://scripts.iucr.org/mailman/listinfo/ddlm-group
_______________________________________________
ddlm-group mailing list
ddlm-group@iucr.org
http://scripts.iucr.org/mailman/listinfo/ddlm-group
|
