Discussion List Archives

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Imgcif-l] CBF - Adding binary proprietary format header dump

Dear Justin,

   Brian McMahon of the IUCr maintains a nice system of "prefixes"
for such cases (e.g. the pdbx prefix you see in imgCIF entries for
the PDB specific tags).  I suggest that we ask Brian to register
RAYONIX as a prefix, rather than our handling it as a category.
This would then allow you both to create categories specific to
your needs and to add tags within existing categories as needed.

   If  categories and tags with prefixes become of general interest,
then the same categories and tags without the prefixes can be proposed
as part of the community dictionaries with aliases to the categories
and tags in your private dictionary.

   You are welcome to create files that use your private dictionary
combined with the imgCIF dictionary, but as a software engineer, you
should understand that other people are not likely to be able to
read this information without significant support.

   An alternative, if you are trying to produce a CBF that will be
properly handled by the packages in use in the community, including
CBFlib, is to work through the changes you propose as part of the
community dictionary at the same time you are working with the
prefix version to get your code fully debugged.  For example, right
now the currently proposed draft dictionary has

_array_data.header_contents

Definition:

         This item is a text field for use in minimal CBF files to carry
                essential header information to be kept with image data
                in _array_data.data when the tags that normally carry the
                structured metadata for the image have not been populated.

                Normally this data item should not appear when the full set
                of tags have been populated and _diffrn_data_frame.details
                appears.

Type: text

Mandatory item: no

Category: array_data


_array_data.header_convention

Definition:

         This item is an identifier for the convention followed in
                constructing the contents of _array_data.header_contents

                The permitted values are of an image creator identifier
                followed by an underscore and a version string.  To avoid
                confusion about conventions, all creator identifiers
                should be registered with the IUCr and the conventions
                for all identifiers and versions should be posted on
                the MEDSBIO.org web site.


Type: code

Mandatory item: no

Category: array_data

It sounds like your needs could be met by adding

_array_data.header_binary_contents

Definition:

   This item is a binary field for use in a minimal CBF files to carry
                essential header information to be kept with image data
                in _array_data.data when the tags that normally carry the
                structured metadata for the image have not been populated.

                Normally this data item should not appear when the full set
                of tags have been populated and _diffrn_data_frame.details
                appears.

                This is an alternative and/or supplement to
                _array_data.header_contents

Type: binary

Mandatory item: no

Category: array_data

and changing the definition of

_array_data.header_convention

Definition:

         This item is an identifier for the convention followed in
                constructing the contents of _array_data.header_contents
                and/or _array_data.header_binary_contents

                The permitted values are of an image creator identifier
                followed by an underscore and a version string.  To avoid
                confusion about conventions, all creator identifiers
                should be registered with the IUCr and the conventions
                for all identifiers and versions should be posted on
                the MEDSBIO.org web site.


Type: code

Mandatory item: no

Category: array_data

In this case, it would seem that RAYONIX would be the appropriate creator
identifier.  You would not need another binary ID since this would be
coupled to the binary ID of the row in ARRAY_DATA with the image itself.

I suspect if you follow this approach you will have the fewest problems
to resolve with existing processing programs.

Regards,
   Herbert



At 10:07 AM -0500 9/26/07, Justin Anderson wrote:
>Thanks to everyone for the input.
>
>We have decided to register the "rayonix" category to hold our binary
>header.  There is information that we would like to save in our binary
>header which does not translate to ASCII but which would be useful for
>diagnostic purposes.  The marccd program suite does include a
>"dump_header" program which writes out most of the binary header data in
>text format but the results are not user friendly nor are they
>comprehensive.  The intent of writing our binary header to the CBF file
>is not to get around writing data to the CIF header but, rather, to
>include additional information which could be useful later.  The users
>will not need to read the binary header.
>
>I am assuming we can use any subcategories we wish with the "rayonix"
>category.  We were thinking the header would reference the image array
>by giving it the same .array_id.  Since we are only writing one binary
>item then we assume the default .binary_id of "1" will suffice.  We will
>use CBFLib to write the binary header data so that it will have a MIME
>as well.
>
>Regards,
>
>
>Justin Anderson                    Rayonix, LLC (Formerly Mar USA)
>Software Engineer                  justin@rayonix.com
>1880 Oak Ave. Ste. 120             Evanston, IL, USA 60201
>877.627.9729                       847.869.1548
>
>
>On 09/25/2007 08:18 PM, Herbert J. Bernstein wrote:
>>  One important point -- while, strictly speaking, there should be no harm
>>  in adding additional binary sections anywhere in a CBF, I believe people
>>  have gotten used to finding the image as the first (and in all current
>>  cases as the only) binary section.  So, it might be a good idea to put
>>  such supplemental binary header binary sections after the image, so that
>>  packages that are looking for the image don't get confused.
>>
>>  Personally I would agree with Harry that an ascii translation of binary
>>  header would be easier to work with in checking the format.  That could
>>  appear before the image with the risk of confusion.
>>
>>  Regards,
>>    Herbert
>>  =====================================================
>>   Herbert J. Bernstein, Professor of Computer Science
>>     Dowling College, Kramer Science Center, KSC 121
>>          Idle Hour Blvd, Oakdale, NY, 11769
>>
>>                   +1-631-244-3035
>>                   yaya@dowling.edu
>>  =====================================================
>>
>>  On Mon, 24 Sep 2007, harry powell wrote:
>>
>>>  Hi
>>>
>>>  I don't understand why you'd want to add the binary header, to be
>>>  honest. I can see why you might want to have the contents of the
>>>  header as an ASCII comment section - but I think this should just be
>>>  a check while developing the file format.
>>>
>>>  Could you elaborate why you'd want it there?
>>>
>>>>  Hello everyone,
>>>>
>>>>  We here would, in fact, like to add our binary header to the CBF file.
>>>>  Should that just go into an additional binary section, with MIME
>>>>  header
>>>>  and all of that?  If so, is there any recommendation as to how we can
>>>>  refer to it in the CIF headers so that I can distinguish between the
>>>>  binary image data and our binary header data?  My inclination is to
>>>>  add
>>>>  the ".header_convention" column to my "_array_data" loop and fill
>>>>  it in
>>>>  with "RAY_1.0" or something like that for the our binary header and
>>>>  with
>>>>    an empty value "" for the image data.
>>>>
>>>>  e.g. :
>>>>
>>>>  loop_
>>>>  _array_data.array_id
>>>>  _array_data.binary_id
>>>>  _array_data.header_convention
>>>>  _array_data.data
>>>>    ARRAY1 1 RAY_1.0
>>>>  ;
>>>>  --CIF-BINARY-FORMAT-SECTION--
>  >>> <<MIME header for Rayonix binary header dump>>
>>>>  <<Rayonix binary header dump>>
>>>>  --CIF-BINARY-FORMAT-SECTION----
>>>>  ;
>>>>    ARRAY2 2
>>>>  ;
>>>>  --CIF-BINARY-FORMAT-SECTION--
>>>>  <<MIME header for image binary data>>
>>>>  <<Image binary data>>
>>>>  --CIF-BINARY-FORMAT-SECTION----
>>>>  ;
>>>>
>>>>
>>>>
>>>>
>>>>  Regards,
>>>>
>>>>  --
>>>>  Justin Anderson                    Rayonix, LLC (Formerly Mar USA)
>>>>  Software Engineer                  justin@rayonix.com
>>>>  1880 Oak Ave. Ste. 120             Evanston, IL, USA 60201
>>>>  877.627.9729                       847.869.1548
>>>>  _______________________________________________
>>>>  imgcif-l mailing list
>>>>  imgcif-l@iucr.org
>>>>  http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>>  Harry
>>>  --
>>>  Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre,
>>>  Hills Road, Cambridge, CB2 2QH
>>>
>>>
>>>
>>>
>>>  _______________________________________________
>>>  imgcif-l mailing list
>>>  imgcif-l@iucr.org
>>>  http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>>
>>  _______________________________________________
>>  imgcif-l mailing list
>>  imgcif-l@iucr.org
>>  http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>
>>
>_______________________________________________
>imgcif-l mailing list
>imgcif-l@iucr.org
>http://scripts.iucr.org/mailman/listinfo/imgcif-l


-- 
=====================================================
  Herbert J. Bernstein, Professor of Computer Science
    Dowling College, Kramer Science Center, KSC 121
         Idle Hour Blvd, Oakdale, NY, 11769

                  +1-631-244-3035
                  yaya@dowling.edu
=====================================================

Reply to: [list | sender only]
International Union of Crystallography

Scientific Union Member of the International Science Council (admitted 1947). Member of CODATA, the ISC Committee on Data. Partner with UNESCO, the United Nations Educational, Scientific and Cultural Organization in the International Year of Crystallography 2014.

International Science Council Scientific Freedom Policy

The IUCr observes the basic policy of non-discrimination and affirms the right and freedom of scientists to associate in international scientific activity without regard to such factors as ethnic origin, religion, citizenship, language, political stance, gender, sex or age, in accordance with the Statutes of the International Council for Science.

Useful links