[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Reply to: [list | sender only]
Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- To: The Crystallographic Binary File and its imgCIF application to image data <imgcif-l@iucr.org>
- Subject: Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- From: Jon Wright <wright@esrf.fr>
- Date: Tue, 22 Jun 2010 16:41:07 +0200
- In-Reply-To: <alpine.BSF.2.00.1006220803370.87930@epsilon.pair.com>
- References: <4C20A497.3080200@esrf.fr><alpine.BSF.2.00.1006220803370.87930@epsilon.pair.com>
Thanks Herbert - perhaps the problem is that I am using 0.7.9, and also see the error on cif2cbf. Will try upgrading. Best Jon Herbert J. Bernstein wrote: > Puzzling. This image doe copy fine with cif2cbf. I'll work on it > after the eye doctor. -- Herbert > > ===================================================== > Herbert J. Bernstein, Professor of Computer Science > Dowling College, Kramer Science Center, KSC 121 > Idle Hour Blvd, Oakdale, NY, 11769 > > +1-631-244-3035 > yaya@dowling.edu > ===================================================== > > On Tue, 22 Jun 2010, Jon Wright wrote: > >> >> Hi Herbert, Graeme, >> >> I had a quick look with the pycbf bindings and ran into a problem with the >> example image (http://www.ccp4.ac.uk/xia/image.cbf). I get a slew of warnings >> and finally an error coming from CBFlib: >> >> coral24:~/cbflib_jun2010/CBFlib_0.7.9/pycbf % python tst.py image.cbf 2>&1 | >> more >> CBFlib: warning input line 26 (1) -- invalid character >> CBFlib: warning input line 26 (2) -- invalid character >> CBFlib: warning input line 26 (3) -- invalid character >> .. >> CBFlib: warning input line 26 (1978) -- invalid character >> CBFlib: warning input line 26 (1979) -- invalid character >> CBFlib: error input line 26 (1) -- value without tag >> >> >> The code in python is essentially: >> >> import pycbf >> o = pycbf.cbf_handle_struct() >> o.read_file("image.cbf", pycbf.MSG_DIGEST) >> >> In c this translates into something like the following, and gives the same >> output: >> >> #include "cbf_simple.h" >> #include <stdlib.h> >> #include <stdio.h> >> int main (int argc, char *argv []) >> { >> FILE *in; >> cbf_handle cbf; >> if (!(in = fopen (argv[1], "rb"))) { >> fprintf (stderr,"Couldn't open the input cbf file %s\n", argv[1]); >> exit (1); >> } >> cbf_failnez (cbf_make_handle (&cbf)); >> cbf_failnez (cbf_read_widefile (cbf, in, MSG_DIGEST)); >> } >> >> >> A workaround is to copy the binary section into a valid file and then read >> that file, but I don't think it is what Graeme is after. Code to do that is >> attached in case it is useful in the meantime. >> >> Sorry I can't be more useful, >> >> Jon >> >> >> >> >> >> >> -------- Original Message -------- >> Subject: RE: [Imgcif-l] Reading CBF files from Python >> Date: Mon, 21 Jun 2010 14:20:38 +0100 >> From: <Graeme.Winter@Diamond.ac.uk> >> To: <wright@esrf.fr> >> CC: <Graeme.Winter@Diamond.ac.uk> >> References: >> <4854F2500EA8C4478A508D2D92973E5206D6B5F9@EXCHANGE25.fed.cclrc.ac.uk> >> <4C1F2B5C.30400@esrf.fr><4854F2500EA8C4478A508D2D92973E5206D6B5FA@EXCHANGE25.fed.cclrc.ac.uk> >> <4C1F3784.6060107@esrf.fr> >> >> Hi Jon, >> >> Thanks - I think that the problem is more than this image does not >> behave like a real cbf, though having some similarities. However I tried >> building pycbf as you suggest and the build failed - I think the >> configuration is not right in the way the cctbx folks use cbflib. >> >> I have attached the image (picture would be a better word) - when I try >> to load it in to cctbx using the MiniCBFImage loader it fails indicating >> I think that it is poorly structured. >> >> I am sure that there is code in cbflib which will do exactly what I >> need, the only problem is gaining access to it! >> >> Best wishes, >> >> Graeme >> >> >> >> -----Original Message----- >> From: imgcif-l-bounces@iucr.org [mailto:imgcif-l-bounces@iucr.org] On >> Behalf Of Jon Wright >> Sent: 21 June 2010 10:57 >> To: The Crystallographic Binary File and its imgCIF application to image >> data >> Subject: Re: [Imgcif-l] Reading CBF files from Python >> >> Hi Graeme, >> >> To use the binding the wrapper needs to be compiled, which should just >> mean typing "python setup.py build" in the pycbf subfolder. Then you >> just need to drop pycbf.py and _pycbf.so somewhere on your python path >> (import sys; print sys.path). The binding runs entirely via calls to >> CBFlib, so if there is something useful in there, it should be possible >> to include it. Herbert can probably help more for knowing where to look. >> >> If you can send an example image off-list I'll try to take a look >> >> Best >> >> Jon >> >> Graeme.Winter@Diamond.ac.uk wrote: >>> Hi Jon, >>> >>> Thanks - I see that this is bundled with cctbx and Labelit, which was >>> awesome. However it seems that the modules from pycbf aren't available >>> as standard :o( - I am slightly hamstrung by the fact that I use the cctbx >>> python, so having access to other python extensions presents a problem. >>> However I *really need* cctbx :o) - xia2 won't work without it now. >>> >>> My rate limiting step right now is the reading and writing of the byte >>> offset compression. I am going to guess that if I messed with the python >>> paths and such I should be able to find your implementation. Do >>> you have a method in there which will allow me to just pack and unpack >>> the byte_offset compressed regions? >>> >>> This is the data in question: >>> >>> ###CBF: Version July 2008 generated by XDS >>> >>> data_FRAME.cbf >>> >>> _array_data.header_convention "XDS special" >>> _array_data.header_contents >>> ; >>> ; >>> >>> _array_data.data >>> ; >>> --CIF-BINARY-FORMAT-SECTION-- >>> Content-Type: application/octet-stream; >>> conversions="x-CBF_BYTE_OFFSET" >>> Content-Transfer-Encoding: BINARY >>> X-Binary-Size: 9450352 >>> X-Binary-ID: 1 >>> X-Binary-Element-Type: "signed 32-bit integer" >>> X-Binary-Element-Byte-Order: LITTLE_ENDIAN >>> X-Binary-Number-of-Elements: 9437184 >>> X-Binary-Size-Fastest-Dimension: 3072 >>> X-Binary-Size-Second-Dimension: 3072 >>> (massive binary blob) >>> >>> The bit which is causing me problems is that the cbflib adaptor raises >>> an exception as most of the "proper" cbf stuff (i.e. the CIF) is not >>> present. Erk. >>> >>> Any suggestions will be pounced apon! >>> >>> Many thanks, >>> >>> Graeme >>> >>> -----Original Message----- >>> From: imgcif-l-bounces@iucr.org [mailto:imgcif-l-bounces@iucr.org] On >>> Behalf Of Jon Wright >>> Sent: 21 June 2010 10:06 >>> To: The Crystallographic Binary File and its imgCIF application to image >>> data >>> Subject: Re: [Imgcif-l] Reading CBF files from Python >>> >>> Hi Graeme, >>> >>> There is a swig based python binding included with CBFlib in the pycbf >>> subfolder. It should work for reading files (see pycbf_test1.py). >>> Further work was needed for writing. Let me know if you run into problems. >>> Improvements are most welcome. >>> >>> Best >>> >>> Jon >>> >>> Graeme.Winter@Diamond.ac.uk wrote: >>>> Hi Folks, >>>> Does anyone know if CBF images can be read from Python? I coded up >>>> something in Python to do this but it's a spot slow to say the least. >>>> My interest is partly in reading the cbf "image" files calculated by XDS >>>> to explain what it has been up to. I could code something up in 'c' or a >>>> bridge to CBFlib but that would be hardish to maintain, so a >>>> solution using existing Python libraries would suit me better. >>>> I looked quickly at PyCIFrw but this appears to be mostly about the CIF >>>> side of things rather than the MIME encoded chunks. Please correct >>>> me if I am wrong. I also tried iotbx.detectors.cbf.CBFImage but hit an >>>> exception which probably relates to this being more of a picture than >>>> a diffraction image. >>>> Thanks, >>>> Graeme >>>> Dr. Graeme Winter >>>> Software and MX Support Scientist >>>> Diamond Light Source >>>> +44 1235 778091 (work) >>>> +44 7786 662784 (work mobile) >>>> >>>> >>>> >>> _______________________________________________ >>> imgcif-l mailing list >>> imgcif-l@iucr.org >>> http://scripts.iucr.org/mailman/listinfo/imgcif-l >>> >> _______________________________________________ >> imgcif-l mailing list >> imgcif-l@iucr.org >> http://scripts.iucr.org/mailman/listinfo/imgcif-l >> >> >> >> -- >> >> This e-mail and any attachments may contain confidential, copyright and or >> privileged material, and are for the use of the intended addressee only. If >> you are not the intended addressee or an authorised recipient of the >> addressee please notify us of receipt by returning the e-mail and do not use, >> copy, retain, distribute or disclose the information in or attached to the >> e-mail. >> >> Any opinions expressed within this e-mail are those of the individual and not >> necessarily of Diamond Light Source Ltd. >> >> Diamond Light Source Ltd. cannot guarantee that this e-mail or any >> attachments are free from viruses and we cannot accept liability for any >> damage which you may sustain as a result of software viruses which may be >> transmitted in or with the message. >> >> Diamond Light Source Limited (company no. 4375679). Registered in England and >> Wales with its registered office at Diamond House, Harwell Science and >> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom >> >> >> >> >> >> >> >> >> >> >> >> --------------090102040500010601060009 >> Content-Type: text/plain; >> name="read_graeme.py" >> Content-Transfer-Encoding: 7bit >> Content-Disposition: inline; >> filename="read_graeme.py" >> >> >> import pycbf, sys, tempfile, time, numpy as np >> >> def fix_im( barecbf, template, tempname ): >> temp = open(tempname, "wb") >> tag = "--CIF-BINARY-FORMAT-SECTION--" >> good = open(template,"rb").read() >> s = good.find(tag) + len(tag) >> temp.write( good[:s] ) >> eg = s + good[s:].find("--CIF") >> bin = open(barecbf, "rb").read() >> s = bin.find(tag)+len(tag) >> e = s + bin[s:].find("--CIF") >> temp.write( bin[s:e] ) >> temp.write( good[ eg: ] ) >> temp.flush() >> temp.close() >> >> >> >> >> def read_binary(name ): >> object = pycbf.cbf_handle_struct() >> object.read_file(name,pycbf.MSG_DIGEST) >> object.rewind_datablock() >> object.select_datablock(0) >> object.rewind_category() >> categories = object.count_categories() >> for i in range(categories): >> object.select_category(i) >> category_name = object.category_name() >> rows=object.count_rows() >> cols = object.count_columns() >> loop=1 >> object.rewind_column() >> while loop is not 0: >> column_name = object.column_name() >> try: >> object.next_column() >> except: >> break >> for j in range(rows): >> object.select_row(j) >> object.rewind_column() >> for k in range(cols): >> name=object.column_name() >> object.select_column(k) >> typeofvalue=object.get_typeofvalue() >> if typeofvalue.find("bnry") > -1: >> s=object.get_integerarray_as_string() >> ar = np.fromstring(s,np.int32) >> else: >> value=object.get_value() >> # print "Val:",value,i >> del(object) >> return ar >> >> start = time.time() >> fix_im(sys.argv[1], sys.argv[2], sys.argv[3] ) >> print "fixing",time.time()-start >> start = time.time() >> image = read_binary(sys.argv[3]) >> print "reading",time.time()-start >> print image.max(), image.min(), image.mean() >> >> --------------090102040500010601060009 >> Content-Type: text/plain; charset="us-ascii" >> MIME-Version: 1.0 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: inline >> >> _______________________________________________ >> imgcif-l mailing list >> imgcif-l@iucr.org >> http://scripts.iucr.org/mailman/listinfo/imgcif-l >> >> --------------090102040500010601060009-- >> >> > _______________________________________________ > imgcif-l mailing list > imgcif-l@iucr.org > http://scripts.iucr.org/mailman/listinfo/imgcif-l _______________________________________________ imgcif-l mailing list imgcif-l@iucr.org http://scripts.iucr.org/mailman/listinfo/imgcif-l
Reply to: [list | sender only]
- Follow-Ups:
- References:
- [Imgcif-l] [Fwd: RE: Reading CBF files from Python] (Jon Wright)
- Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python] (Herbert J. Bernstein)
- Prev by Date: Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- Next by Date: Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- Prev by thread: Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- Next by thread: Re: [Imgcif-l] [Fwd: RE: Reading CBF files from Python]
- Index(es):