VOStore/VOSpace use cases
dtody at nrao.edu
Thu Aug 11 10:40:35 PDT 2005
Hi Matthew -
On Thu, 11 Aug 2005, Matthew Graham wrote:
> Hi Doug,
> Guy has mentioned to me that you had some specific VOStore/VOSpace use cases
> - are these already documented somewhere or could you provide a short
> description of them?
Probably this refers to my data access related use-cases. The are three
1) Service generates datasets and puts them in a local VOStore.
Client picks up the images with a getData request via URL (this
is the default and is what we do currently).
2) Client operates a local VOStore. Client issues a staging request
to a remote service to generate datasets and deliver them to
the Client's VOStore. Service sends a message to the client as
each dataset is delivered (or client gets the messages when it
polls etc.). In a typical scenario a client may issue many such
asynchronous requests which execute concurrently. Support for
authorization is needed since the local VOStore is externally
writeable, and to support access to proprietary data.
This is the scenario I see for VO-Client functionality integrated
into a desktop data analysis system. The VO-Client implements
the client side of the data access services including support for
queries, asynchronous services, authentication/autorization, etc.
Data is fetched and placed in a local VOStore in the local VO-Client
daemon. Data analysis software can then directly access the files
as they would a local file, or we can put an object API in front
of the data and the analysis code will see an object API and can
completely ignore all the plumbing.
In this use-case the client may also function as a service,
generating new data products which it exposes to the VO via the
3) As in 2), but the staging request specifies that data is to be
delivered to a third VOStore, not co-located with either the
controlling client or the service. This permits general grid
workflows. The client "application" is generally controlled from
one location, but may have components which execute at multiple
Note that asynchronous services, VOStore, and security all need to be
interoperable to do these things. Ideally we need some mechanism for
asynchronous messaging. A polling-based mechanism would be adequate for
simple cases but does not scale well.
All of these use-cases emphasize dataflow and data caching rather than
permanent storage of data. The only exception might be 2) where we might
want to keep the data around indefinitely. However, it is still primarily
a network data cache mechanism.
Another major use case is a user filestore where data is maintained
indefinitely, possibly distributed to multiple locations. This is more
the VOSpace type of scenario.
More information about the vospace