VOStore/VOSpace use cases

[Doug Tody]

Hi Matthew -

On Thu, 11 Aug 2005, Matthew Graham wrote:
> Hi Doug,
> Guy has mentioned to me that you had some specific VOStore/VOSpace use cases
> - are these already documented somewhere or could you provide a short
> description of them?

Probably this refers to my data access related use-cases.  The are three
primary cases:

    1) Service generates datasets and puts them in a local VOStore.
       Client picks up the images with a getData request via URL (this
       is the default and is what we do currently).

    2) Client operates a local VOStore.  Client issues a staging request
       to a remote service to generate datasets and deliver them to
       the Client's VOStore.  Service sends a message to the client as
       each dataset is delivered (or client gets the messages when it
       polls etc.).  In a typical scenario a client may issue many such
       asynchronous requests which execute concurrently.  Support for
       authorization is needed since the local VOStore is externally
       writeable, and to support access to proprietary data.

       This is the scenario I see for VO-Client functionality integrated
       into a desktop data analysis system.  The VO-Client implements
       the client side of the data access services including support for
       queries, asynchronous services, authentication/autorization, etc.
       Data is fetched and placed in a local VOStore in the local VO-Client
       daemon.  Data analysis software can then directly access the files
       as they would a local file, or we can put an object API in front
       of the data and the analysis code will see an object API and can
       completely ignore all the plumbing.

       In this use-case the client may also function as a service,
       generating new data products which it exposes to the VO via the
       local VOStore.

    3) As in 2), but the staging request specifies that data is to be
       delivered to a third VOStore, not co-located with either the
       controlling client or the service.  This permits general grid
       workflows.  The client "application" is generally controlled from
       one location, but may have components which execute at multiple
       locations.

Note that asynchronous services, VOStore, and security all need to be
interoperable to do these things.  Ideally we need some mechanism for
asynchronous messaging.  A polling-based mechanism would be adequate for
simple cases but does not scale well.

All of these use-cases emphasize dataflow and data caching rather than
permanent storage of data.  The only exception might be 2) where we might
want to keep the data around indefinitely.  However, it is still primarily
a network data cache mechanism.

Another major use case is a user filestore where data is maintained
indefinitely, possibly distributed to multiple locations.  This is more
the VOSpace type of scenario.

	- Doug