Discussion List Archives

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: update of mirrors

  • To: Multiple recipients of list <epc-l@iucr.org>
  • Subject: Re: update of mirrors
  • From: Brian McMahon <bm@iucr.org>
  • Date: Fri, 4 Sep 1998 09:26:28 +0100 (BST)
Dear Yves
 
> Just by curiosity I have looked this morning to all mirrors to have an
> idea of the time needed to update the different mirrors.

Here is an update on the position:

Tests have demonstrated that we can mirror using wget but with anonymous ftp
protocol at least an order of magnitude faster, and I shall recommend that
all the mirrors switch to that approach.

But first I need to clean up my WWW directory tree for two reasons:
   (1) to delete or move to another place all the junk and ancillary files
       that have crept into various directories;
   (2) to resolve soft links that point to other files on my local
       filesystem.
(1) is tiresome, but not altogether essential, since it just means the junk
is copied to the mirrors and clutters up their filesystem, but is never seen
by users. (2) is more difficult. They didn't cause a problem until now,
because wget/http followed the links on my system, and downloaded the files
as hard files to the mirrors. wget/ftp will preserve the symbolic links,
rendering in most cases useless locations on the mirrors.

I propose to resolve (2) by changing most of the symlinks (which point to
structure-factor files) into URLs to our ftp site. But this means building a
new ftp hierarchy, and superimposing this on the existing archive file
structure. Since we are also rearranging the archive structure, the details
are quite complex.

It's a project I have high on my list of priorities (the new archive
structure is itself important for the efficient management of our
publications archive into the future), but it needs a suitable long, quiet
period for implementation. I can't yet give a firm schedule for it.

As soon as it's complete and working, I shall vigorously encourage all the
mirrors to adopt the new mirroring scheme and update their sites frequently
(since the overhead will be so much less, a daily update will usually be
feasible).

The Israeli site is not only out of date, it's also incomplete. The South
African site is hampered by very slow communications and complications with
working through a proxy. There is a Japan site, known to the local Japanese
community, which does update regularly but is unwilling to "go public" until
more efficient mirroring is implemented. The other sites are working at
least tolerably well.

I look forward to the day - I hope not far off - when all the sites can be
relied upon to be quite up to date.

Regards
Brian

Reply to: [list | sender only]
International Union of Crystallography

Scientific Union Member of the International Science Council (admitted 1947). Member of CODATA, the ISC Committee on Data. Partner with UNESCO, the United Nations Educational, Scientific and Cultural Organization in the International Year of Crystallography 2014.

International Science Council Scientific Freedom Policy

The IUCr observes the basic policy of non-discrimination and affirms the right and freedom of scientists to associate in international scientific activity without regard to such factors as ethnic origin, religion, citizenship, language, political stance, gender, sex or age, in accordance with the Statutes of the International Council for Science.