is cache-only-prefixes an nnpfs limitation?

Harald Barth haba at pdc.kth.se
Mon Apr 10 08:38:35 CEST 2006


> The HPC folks are almost ready to kill for this (and cacheless writes).
> Right now they'll have to resort to linking against arlad to do it.

Oh, yes we will ;-) The reason is that our users very often read small
files (the "application") often and big files (the "data") once. Then
they write big files (the "result") once. This behaviour is very
differet from for example a class of students who read and write small
files (the "program") often and read big files (the "compiler") often
and almost never write big files. Some time in the future I hope we
can hint the cache what to do. If the HPC folks would write stuff in C
I could tell them to open(2) with O_DIRECT and then implement some
other cache strategy for that, but most of these applictions are "behind"
FORTRAN I/O.

> BTW, what's the current status on ways for the user to communicate such
> hints to the fs on various OS:es?  I just love Windows in this respect,
> they have tons of flags. They even have FILE_OPEN_FOR_FREE_SPACE_QUERY.  I
> wonder if it's ever used.

In libc we could use open(2) with O_DIRECT, I just do not know it it
would help my FORTRAN HPC folks. The C and C++ HPC folks could probably
be teached to use O_DIRECT but I suspect that some other means (like
an environment variable) might be necessary.

Hm. I don't know what happens if I use O_DIRECT in Arla today if it is
ignored or passed down on the cache file open.

Harald.


More information about the Arla-drinkers mailing list