Logical storage units in VOSpace 1.1

[Paul Harrison]

Hi,

I think that data replication is an important functionality of  
VOSpace, but I think that introducing the concept of logical storage  
units in this fashion into the "public" api might not be very easy to  
use in practice without knowledge of the underlying storage system,  
and additionally is contrary to one of the aims of the VOSpace design  
of trying to hide as much as possible about the internals from the  
outside world. The use case that you describe could also be handled  
in a more easy-to-reason-about way by having "move to fast storage"  
and "move to slow storage" functions in the api, or having similar  
hints in the various get api calls .Perhaps a compromise using a  
similar api to the one you suggest, is that the "hardware units" are  
generic classes of unit rather than each vospace defining its own set  
of proprietary hardware units. VOSpaces that want to, simply map the  
generic classes onto specific internal hardware units transparently.   
The VOSpace then hides all the details of exactly where items are  
stored.

Paul Harrison

On 13.08.2007, at 19:32, Matthew Graham wrote:

> Hi,
>
> A request has from our friends at SDSC to include references to the  
> actual storage units that data is being deposited on. The use case  
> is data replication so, for example, I want to move/copy a data  
> object from a slow tape archive to an ultrafast disk but both  
> hardware units are within the same VOSpace or I want to retrieve a  
> data object from the ultrafast disk copy and not the slow tape one.
>
> I think that we can incorporate this easily into our existing data  
> model. We will refer to hardware units as logical storage units  
> with the implication that they are identified via a logical  
> identifier (URI) that is set by the particular VOSpace  
> implementation. To get the list of available storage units from a  
> VOSpace, we will need a method: getLogicalStorageUnits() which will  
> return a list of URIs. These URIs may be resolvable to a  
> description of the storage unit.
>
> The logical storage unit identifier will be an optional argument in  
> the <transfer> entity so that as part of the data transfer  
> negotiation, the user can specify a list of storage units that they  
> want the data transferred to/from. The identifier will also be an  
> optional argument in the <node> entity so that specific hardware  
> can be targetted in moving and copying data.
>
> Comments, suggestions, etc.
>
>    Cheers,
>
>    Matthew