VOStore interface

[Reagan Moore]

The differentiation between the VOStore and VOSpace interfaces is 
becoming unclear.  The latest draft implies that properties that were 
originally associated with VOSpace would now be supported by VOStore.

One way to better understand how VOSpace and VOStore should interact 
is to define the state information that will be managed by each 
system, and the set of operations that will change that state 
information.  An example of such a characterization is the Storage 
Resource Manager (developed at LBNL) which is being used in the EGEE 
(Enabling Grids for E-Science) and OSG (Open Science Grid) projects. 
The SRM is also being proposed as a standard data access interface 
within the Global Grid Forum.  The SRM interface is based on support 
for file access.

A second Global Grid Forum effort, OGSA-DAI, is designing a data 
access interface based on support for database queries.  This 
interface is being extended to support access to files.  Both 
interfaces are implementing WSDL services for supporting data access.

Let's look at the current VOStore and VOSpace proposal:

VOStore                                     VOSpace
Storage of objects                          management of virtual file system
data stored under unspecified ID?
no user home directory                      User home directory
directory hierarchy                         Directory hierarchy
Unique file name within storage             User-defined file names
                                             Mapping VOSpace name to 
VOStore name
                                             List files for user
Restrict access by user identity?
Identify files with URIs
Access controls on local file name          Access controls on VOSPace name

This characterization mixes name space, mixes access controls, does 
not provide consistent identity, does not allow consistent 
management.  For instance, if a URI is being provided for file 
identity within the VOStore interface, then there is no need for 
user-specified names within VOSTore.  A second issue is the 
assumption that file access can be restricted by user identity.  This 
means that the VOStore must manage the owner for each file, access 
controls for each file.  File systems usually do this by creating 
accounts for each user name and applying Unix permissions.  Is this 
capability to be provided now by both VOSpace and VOStore?  We need a 
cleaner separation of capabilities.

Let's look at the Storage Resource Broker data grid separation of 
local storage management from the virtual file system management:

Local storage system                        SRB name space
Storage of objects                          management of virtual file system
data stored under SRB ID
no user home directory                      User home directory
directory indirection structure             Directory hierarchy
Unique file name within storage             User-defined file names
                                             Mapping SRB name to local file name
                                             List files for user
Access through SRB ID, controlled by SRB
                                             Identify files by URIs
                                             Access controls on SRB name

Note the local storage system directory indirection structure is 
needed to allow users to store 5 million images in a single virtual 
file system directory.  Also, storage of data under an ID 
corresponding to the virtual file system is needed to assure 
consistency between the state information managed by the virtual name 
space and the files in the local storage system.

The SRB system allows a user to create files within their own file 
system under their account ID, then turn on permission for the SRB 
account to read their files.  They can then register their local 
files into the SRB name space, preserving both the directory 
structure and local file name within the virtual file system name 
space.  This inverts the problem of managing choice of names for 
local versus virtual files.  The user is able to impose their choice 
of names at either level, but operations specifying local file names 
are done independently of the VOStore interface.

The set of operations that are performed upon the local storage 
system can be restricted to Unix file operations, or augmented to 
include SQL commands for databases.  The VOStore interface would then 
need the ability to map operations from the virtual name space to 
both types of storage systems.

Reagan Moore