Discussion List Archives

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CIF-JSON new draft

>
> Since we are targeting this as a low overhead representation, which I associate with performance considerations, I am prepared to entertain arguments about performance impact.  I am not, however, prepared to accept unsupported assertions about performance.

Fair enough, here is a microbenchmark for you. First using Python2.7:

$ python -m timeit -s 'import json,sys' 'f=open("numbers.json"); numbers = json.load(f)["numbers"]'
100 loops, best of 3: 10.2 msec per loop
$ python -m timeit -s 'import json,sys' 'f=open("strings.json"); numbers = [float(x) for x in json.load(f)["strings"]]'
10 loops, best of 3: 58.1 msec per loop

In this case using strings is 5x slower.

Now with Python 3.5:

$ python3 -m timeit -s 'import json,sys' 'f=open("numbers.json"); numbers = json.load(f)["numbers"]'
100 loops, best of 3: 12.7 msec per loop
$ python3 -m timeit -s 'import json,sys' 'f=open("strings.json"); numbers = [float(x) for x in json.load(f)["strings"]]'
10 loops, best of 3: 27.4 msec per loop

The difference is smaller, but still >2x.

The input files were prepared using this script:

import json
import random
numbers = [round(random.uniform(-30, 30), 3) for _ in range(100000)]
with open('numbers.json', 'w') as f:
    json.dump({'numbers':numbers}, f)
with open('strings.json', 'w') as f:
    json.dump({'strings':[str(x) for x in numbers]}, f)


$ du -h strings.json numbers.json
984K    strings.json
788K    numbers.json


Marcin
_______________________________________________
cif-developers mailing list
cif-developers@iucr.org
http://mailman.iucr.org/cgi-bin/mailman/listinfo/cif-developers

Reply to: [list | sender only]
International Union of Crystallography

Scientific Union Member of the International Science Council (admitted 1947). Member of CODATA, the ISC Committee on Data. Partner with UNESCO, the United Nations Educational, Scientific and Cultural Organization in the International Year of Crystallography 2014.

International Science Council Scientific Freedom Policy

The IUCr observes the basic policy of non-discrimination and affirms the right and freedom of scientists to associate in international scientific activity without regard to such factors as ethnic origin, religion, citizenship, language, political stance, gender, sex or age, in accordance with the Statutes of the International Council for Science.

Useful links