Re: [Imgcif-l] CBF Format

["Herbert J. Bernstein" <yaya@bernstein-plus-sons.com>]

Dear Harry,

   Your point on more or less is well taken, and is why I think it
would be better to use a transliteration of the binary header into
an equivalent ascii miniheader for debugging purposes, but if you
are comfortable with hexdump or od in a pipe to more, there is
no harm in putting a verbatim copy of the binary header into the
CBF for debugging.  The extra copy of the information is for the
developer more than for the end user.

   Regards,
     Herbert


At 2:01 PM +0100 9/13/07, Harry Powell wrote:
>Hi Herb
>
>You've missed the point completely of what I wrote. _Of_course_ you 
>could put the binary header in, but, to quote my last message -
>
>>>so you can't just put the header information as it appears in the
>>>native format and expect people to be able to read it by more-or-lessing
>>>the file.
>
>Perhaps I should have said "more" or "less" -ing, to make it plainer 
>that I meant using the programs of those names to output the headers 
>to the screen. As Charlie Brown used to say (and probably still 
>does) - "good grief!"
>
>>Dear Harry,
>>
>>  Actually, if you want the CCD header stictly
>>as binary, rather than putting in an ASCII
>>translation, you could do that as well, by
>>putting it in as CBF binary section, just
>>as we will be doing with thumbnails, but
>>for debugging, a straight ASCII translation
>>of the header before the reorganization
>>into the imgCIF categories can be very useful.
>>
>>  Regards,
>>    Herbert
>>
>>=====================================================
>>Herbert J. Bernstein, Professor of Computer Science
>>   Dowling College, Kramer Science Center, KSC 121
>>        Idle Hour Blvd, Oakdale, NY, 11769
>>
>>                 +1-631-244-3035
>>                 yaya@dowling.edu
>>=====================================================
>>
>>On Thu, 13 Sep 2007, Harry Powell wrote:
>>
>>>Hi Herb
>>>
>>>not strictly true. The MarCCD headers are binary, Mar IP headers are
>>>ASCII, so you can't just put the header information as it appears in the
>>>native format and expect people to be able to read it by more-or-lessing
>>>the file.
>>>
>>>Further to what I wrote yesterday, my experience of native MarCCD headers
>>>is that they contain sufficient information to process the image (and
>>>indeed full datasets) without reference to external files/lab books etc.
>>>So if this information were to be written to a MarCCD CBF, that would be a
>>>very good start.
>>>
>>>>  Actually you can have your miniCBF header and full
>>>>header, too, even for MAR.  If you look at the
>>>>file converted_flat_orig.cbf in the CBFlib data
>>>>files kit, you will see;
>>>>
>>>>_diffrn_frame_data.id FRAME1
>>>>_diffrn_frame_data.detector_element_id ELEMENT1
>>>>_diffrn_frame_data.array_id ARRAY1
>>>>_diffrn_frame_data.binary_id 1
>>>>_diffrn_frame_data.details
>>>>;
>>>>DETECTOR = MAR 345;
>>>>PIXEL SIZE = 0.15 0.15;
>>>>WAVELENGTH = 1.080000;
>>>>DISTANCE = 240;
>>>>OSCILLATION AXIS = PHI;
>>>>PHI = 0;
>>>>OSCILLATION RANGE = 1;
>>>>PROGRAM = mar345 Version 0.8.6;
>>>>DATE = Thu Dec  4 10:23:48 1997;
>>>>SCANNER = 26;
>>>>FORMAT = 2300  PCK 5290000;
>>>>HIGH = 155;
>>>>PIXEL = LENGTH 150  HEIGHT 150;
>>>>OFFSET = ROFF 5.000  TOFF 0.000;
>>>>MULTIPLIER = 1.000;
>>>>GAIN = 1.000;
>>>>RESOLUTION = 1.761;
>>>>PHI.. = START 0.000  END 1.000  OSC 1;
>>>>OMEGA = START 0.000  END 0.000  OSC 0;
>>>>CHI = 0.000;
>>>>TWOTHETA = 0.000;
>>>>CENTER = X 1150.000  Y 1150.000;
>>>>MODE = TIME;
>>>>EXPOSURE TIME = 20.00;
>>>>COUNTS = START 19.35 END 19.29  NMEAS 9;
>>>>COUNTS.. = MIN 19.27  MAX 19.38;
>>>>COUNTS.... = AVE 19.30  SIG 0.03;
>>>>INTENSITY = MIN 1  MAX 249051  AVE 207.8  SIG 951.09;
>>>>HISTOGRAM = START 0  END 2023  MAX 28307;
>>>>GENERATOR = ROTATINGANODE  kV 10.0  mA 20.0;
>>>>MONOCHROMATOR = GRAPHITE  POLAR 0.000;
>>>>COLLIMATOR = WIDTH 0.30  HEIGHT 0.30;
>>>>;
>>>>
>>>>I added this to extra copy of header information
>>>>to the output of convert_image when I was working
>>>>with Chris on the ADSC header.  I proved so
>>>>useful there, that I put it in for MAR as
>>>>well and would suggest it for all detectors.
>>>>
>>>>=====================================================
>>>>Herbert J. Bernstein, Professor of Computer Science
>>>>   Dowling College, Kramer Science Center, KSC 121
>>>>        Idle Hour Blvd, Oakdale, NY, 11769
>>>>
>>>>                 +1-631-244-3035
>>>>                 yaya@dowling.edu
>>>>=====================================================
>>>>
>>>>On Wed, 12 Sep 2007, Harry Powell wrote:
>>>>
>>>>>Hi
>>>>>
>>>>>I'd like to concur with Nick. As far as Mosflm is concerned, the miniCBF
>>>>>is not a CBF. In my opinion, it should not be called "anything" CBF.
>>>>>Developers of some other integration programs have been rather less polite
>>>>>(privately) in their descriptions of the miniCBF.
>>>>>
>>>>>My understanding is that the miniCBF was developed for the Pilatus
>>>>>detector for processing with XDS - which does not use image header
>>>>>information except for the image size and compression algorithm (and it
>>>>>gets these from the MIME part).
>>>>>
>>>>>If you include the CIF fields that are present in images from the ADSCs at
>>>>>Diamond, you will not be going far wrong - see an example header at
>>>>>
>>>>>  	http://www.mrc-lmb.cam.ac.uk/harry/herb.header
>>>>>
>>>>>(this is a file I prepared for Herb a little while ago, and not linked to,
>>>>>so you need to use this URL). Note that it also includes the normal ADSC
>>>>>header information as a properly-formed CIF comment (i.e. free text
>>>>>delimited by ";" on lines on their own), but this is not parsed by Mosflm.
>>>>>MarCCD headers, being binary encoded, would probably not be usefully
>>>>>reproduced in the CBF header!
>>>>>
>>>>>The beamline and data acquisition staff at Diamond expect all their PX
>>>>>area detectors to write CBFs in the same form with the same header
>>>>>information as proper CIF items; the image compression algorithm is up to
>>>>>the detector manufacturer, as long as it is one of those supported by
>>>>>CBFlib.
>>>>>
>>>>>>From a personal point of view, I think it would be really nice 
>>>>>>if all CBFs
>>>>>(at least in the same field) conformed to the same format!
>>>>>
>>>>>>Thanks for working on CBF support for the Mar CCD.  The image processing
>>>>>>program LABELIT expects full CBF, not miniCBF.  The MIME area is not
>>>>>>parsed.  You are correct in identifying the "non-optional" data items.
>>>>>>This is as far as my expertise goes...it would be quite helpful if some
>>>>>>Mar datasets written in CBF format were to be made available.  The most
>>>>>>useful test cases would be those where the direct beam is offset from
>>>>>>the detector center, but not at a 45- or 90-degree position angle.  Full
>>>>>>datasets would be most useful.
>>>>>>
>>>>>>Nick Sauter
>>>>>>
>>>>>>Justin Anderson wrote:
>>>>>>
>>>>>>>Hello everyone,
>>>>>>>
>>>>>>>My name is Justin Anderson with Rayonix (formerly MarUSA).  I have been
>>>>>>>given the task of adding CBF format support to the marccd program.  I
>>>>>>>was hoping you could help me with a few questions that have arisen.  I
>>>>>>>truly appreciate it.
>>>>>>>
>>>>>>>Now for the questions...
>>>>>>>
>>>>>>>1)  Are miniCBF files meant to be read by imaging programs such as
>>>>>>>marccd or are they expected to be converted to full CBF format first?
>>>>>>>
>>>>>>>2)  What is the minimum set of fields that is necessary for an official
>>>>>>>miniCBF and CBF file?  In the "Simple Example Header" in section 2 of
>>>>>>>the Draft Format
>>>>>>>(http://www.iucr.org/iucr-top/cif/cbf/cbf_definition.html) it does
>>>>>>>mention at one point that "the above" data items are optional.  Does
>>>>>>>that imply that the items below are required?
>>>>>>>
>>>>>>>3)  Some items can exist in both the header area and the MIME area.  One
>>>>>>>such example is the image size (e.g. fastest and second-fastest number
>>>>>>>of pixels).  When reading a CBF file, should I expect to get that
>>>>>>>information from the MIME, the CBF header area or would it be 
>>>>>>>found in both?
>>>>>>>
>>>>>>>I appreciate any help you can give me.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>Justin Anderson                    Rayonix, LLC (Formerly Mar USA)
>>>>>>>Software Engineer                  justin@rayonix.com
>>>>>>>1880 Oak Ave. Ste. 120             Evanston, IL, USA 60201
>>>>>>>877.627.9729                       847.869.1548
>>>>>>>
>>>>>>>_______________________________________________
>>>>>>>imgcif-l mailing list
>>>>>>>imgcif-l@iucr.org
>>>>>>>http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>>>>>>
>>>>>>
>>>>>>--
>>>>>>Nicholas K. Sauter, Ph.D.
>>>>>>Lawrence Berkeley National Laboratory
>>>>>>1 Cyclotron Road
>>>>>>Berkeley, CA 94720
>>>>>>
>>>>>>nksauter@lbl.gov
>>>>>>Voice: 510-486-5713
>>>>>>Fax: 510-486-5909
>>>>>>_______________________________________________
>>>>>>imgcif-l mailing list
>>>>>>imgcif-l@iucr.org
>>>>>>http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>>>>>
>>>>>
>>>>>Harry
>>>>>--
>>>>>Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills
>>>>>Road, Cambridge, CB2 2QH
>>>>>
>>>>>_______________________________________________
>>>>>imgcif-l mailing list
>>>>>imgcif-l@iucr.org
>>>>>http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>>>>
>>>>
>>>
>>>Harry
>>>--
>>>Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills
>>>Road, Cambridge, CB2 2QH
>>>
>>
>
>Harry
>--
>Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills
>Road, Cambridge, CB2 2QH


-- 
=====================================================
  Herbert J. Bernstein, Professor of Computer Science
    Dowling College, Kramer Science Center, KSC 121
         Idle Hour Blvd, Oakdale, NY, 11769

                  +1-631-244-3035
                  yaya@dowling.edu
=====================================================
_______________________________________________
imgcif-l mailing list
imgcif-l@iucr.org
http://scripts.iucr.org/mailman/listinfo/imgcif-l