Cache Manager

[Artur Grabowski]

From: wwshen at eecs.umich.edu
Subject: Re: Cache Manager
Date: Fri, 20 Mar 1998 22:53:04 -0500 (EST)

> 
> > No, many programs will get confused if files are changed while they
> > are reading (or executing them) so the local programs will not see the
> > modifications before the file is closed.
> 
>    So you mean that AFS is "last close wins". The first client opens
> a file and the 2nd client opens it later and closes it before the 1st
> closes. Then after the 1st client closes the final version of the file
> is decided by the 1st client. 

That's probably correct.

> So why do we need the callback? I am still not clear about this. 

The callback is needed so you can know when your cached file is updated.
The point is that you are absolutely sure when your cache-file is up-to-date.
The files are cached locally even when they are not used and they stay in the
cache until they get garbage-collected (because of full cache) or until you
get a callback. The normal use for a filesystem is mostly reads. In that case
it's good to know that the file you are about to read is the same that on the
other computers in your cell. The problem with mutiple writers to the same
file is minimal. And it will be solved by locking.

>    I am trying to reading your source code. As far as I know about AFS,
> the file open system call, issued by the application program, is
> intercepted by a small "hook" installed in the workstation's kernel.
> The user's program is suspended and the intercepted request is divereted
> to the Cache Manager, implemented as a user-level process on the
> workstation. The cache manager then forwards the file request to a file
> server, receives the file, stores it on the local disk, .....

That's not completly correct. The systems we use have an interface called VFS.
VFS is an abstraction layer above the filesystems and all syscalls on files
go through it.

The procedure is something like this: (simplified)

 - The application program does a syscall, (on a file from arla)
 - The syscall is interpreted by the kernel,
 - The kernel find out on which filesystem the file is, (our filesystem is xfs)
 - The kernel calls a VFS-operation om that filesystem, (xfs)
    (in most cases xfs already has the file in the local cache, but
     let's say it doesn't)
 - xfs sends a message to arlad through a device (/dev/xfs0),
 - arlad interprets the message,
    (in most cases the information needed is already in arlas cache)
 - arlad fetches the relevant information from the AFS-server.
 - arlad writes down the information on a file in a local filesystem.
 - arlad tells the xfs (throguh the device) about the location of the
   local cache file.
 - xfs calls the operation requested from the syscall on the local
   cache file and returns the result to the syscall code.
 - the syscall answers to the application.

This was heavily simplified because there are caches on almost all levels
of this procedure. And the VFS interface doesn't implement specific syscalls,
most syscalls are done by serveral simple VFS operations.

(It's not as hard as it sounds. You just have to separate all the different
layers of abstraction, when implementing this.)

> So could
> you tell me which files and functions in arald implement the beginning
> function of the cache manager? I think your detailed explanation will
> help me to understand how you do it?

It's not so easy to show on specific functions. Most of the functionality
is separated by different layers of abstraction.

A fast explaination about the different parts of Arla:

rx - the network protocol.
rxkad - authentication and encryption for rx.
rxdef - the generated rpc-protocol used to talk to the AFS servers.

xfs - The kernel module that implements the VFS, a device-driver and a 
      special syscall.
xfs/<system>/xfs_messages.c - The part of xfs that is responsible for
      processing messages to/from arlad.
xfs/<system>/{xfs_vfsops.c,xfs_vnodeops.c} - The VFS interface.
xfs/<system>/xfs_dev.c - the device used to talk to arlad.
xfs/<system>/xfs_syscalls.c - the syscall 

arlad/messages.c - the part of arla that is resposible for processing messages
      to/from xfs.
arlad/fcache.c - manager of local cache files.
arlad/inter.c - a generic interface for the cache manager
arlad/kernel.c - the interface used to talk to the device in xfs.

The other files are mostly utilities or the part of arlad that talks with
the AFS-server. And I don't think I'm the right person to explain that.

//art