VoStore current status

[Guy Rixon]

On Thu, 28 Jul 2005, Dave Morris wrote:

> Matthew Graham wrote:
>
> > - "VoStore? services would provide the lower level storage of objects,
> > in a simple flat file space." This sounds like an implementation
> > detail and VoStore is just the API spec - whether the underlying
> > storage technology is a file system or a database does not matter, the
> > outward appearance, however, is as if it is a simple flat file space.
>
> Yep, text not clear enough, sorry.
> You are right, the specification applies to the outward appearance of
> the API only, not the implementation.
> Different implementations can store the data in files, database tables
> or anything else they want to.
> I have updated text to read
> "The VoSpace service(s) would provide the top level hierarchical view of
> the users data, VoStore services would provide a flat, non-hierarchical,
> view of the objects in each individual VoStore"

At some point we must work out what the hiearchy means when the storage is
relational. Can we address a DB table with an IVOID or only a DB?

> > - there is also the issue of how VoStore handles structured
> > (tabulated) data: one of the BigIdeas (TM) was that structured and
> > unstructured data objects were both first-level components and that as
> > a minimum, both would be stored as BLOBs but certain VoStore
> > implementations would have the ability to handle structured data
> > objects in a more intuitive manner, e.g. load them into a relational
> > db as tables.
>
> You are right .... However, I thought that this was to be handled in a
> separate API,  although it could be implemented as an extension of the
> VoStore service.
> Reason for this was to make VoStore the basic common API, which all
> store services must implement.
> Database based stores could provide additional methods for manipulating
> database tables - covered in a separate spec.

Definitely! Any relational stuff has to be via SkyNode et al., otherwise we'll
never finish VOStore.

However, we need to think a little about how the relational service presents
the data loaded into the store. Presumably it can't publish it in the registry
as the registry is too slow. Perhaps clients have to call the getRegistration
interface to get the details? Or should we be emitting table details in the
VOStore interface?

> > - the identifier of an object stored in a VoStore is: <IVOA Identifier
> > for VoStore>#<Identifier peculiar to this VoStore> - that way the
> > VoStore can be resolved by looking up its identifier in a registry.
>
> That works, if the object identifier is a unique identifier generated by
> the server not the client, which is how the current AstroGrid FileStore
> works.
> e.g. <IVOA Identifier for VoStore>#12366A-7563835
>
> However, some of the participants in the discussions leading upto Kyoto
> wanted the client to be able to access the data in a VoStore using human
> friendly names, e.g. '/mydata/results.vot',
> without going via a VoSpace service.
> e.g. <IVOA Identifier for VoStore>#/mydata/results.vot
>
> Which works as long as we only want one user account to accesses objects
> in the store.
> If we have more than one user account to access objects in the store, we
> need some prefix to distinguish between objects with the same name
> '/mydata/results.vot' for two different accounts 'john' and  'fred'.
> e.g. <IVOA Identifier for VoStore>#<identifier or prefix for
> john>/mydata/results.vot
> and <IVOA Identifier for VoStore>#<identifier or prefix for
> fred>/mydata/results.vot
>
> In AstroGrid, we use a FileManager (VoSpace) service to handle the
> mapping between the hierarchical human friendly names
> e.g. <community identifier for account>#/mydata/results.vot'
> and the physical storage identifiers
> e.g. <IVOA Identifier for VoStore>#12366A-7563835
>
> If the requirement is to be able to use human friendly names _without_
> using something like a VoSpace service to handle the mapping, then we
> need a different solution.

It's easier if we let the client pick the path part of the URI,
/mydata/results.vot in this example, but not the IVOID part. In fact, the
IVOID is presumabkly the IVOID of the store, so clients will never be able to
specify that part. The store could then impose a format like this:

  ivo://some.authority/resource-key-for-store#user-name#/mydata/results.vot

where the client chooses the part after the second hash.


Guy Rixon 				        gtr at ast.cam.ac.uk
Institute of Astronomy   	                Tel: +44-1223-337542
Madingley Road, Cambridge, UK, CB3 0HA		Fax: +44-1223-337523