Changes to VOSpace specification

[Dave Morris]

Paul Harrison wrote:

> On 27.11.2006, at 06:18, Dave Morris wrote:
>
>> Paul added some notes when he saw them in September, and since then I 
>> have re-evaluated some of the ideas in light of his comments.
>> So, the details in these documents are already out of date, but (I 
>> hope) the general idea is still sound.
>
> To summarize my objections
>
> * sometimes it is more convenient for both the client and the server 
> implementation to have a specific API for operations rather than 
> having to manipulate objects using a "general" api - though I do 
> recognize that there is some power and flexibility in the general api 
> approach cf everything is a file in unix...

Yep, I agree.
The existing VOSpace tools would give us a standard way of referring to 
the status object (their URI) and a quick way of checking their status 
by looking at the properties.
Changing the state _could_ be done by setting the node properties, but I 
agree it is stretching things.
It might be better to have a separate API with methods like completed(), 
failed() and cancel() .
It could still use the same URI identifiers, so you would pass the URI 
of the status node to the cancel() method.

> * I was not keen on the "control" objects polluting the namespace in 
> the sense that as illustrated in the document that they would appear 
> directly in listings

Yep, I agree.
Sorry, I haven't updated the document since we talked.
The status nodes should not appear alongside the data nodes.

> - I would much prefer that any such control objects attached to a data 
> node were only accessible by using the ? (query) portion of the vos 
> url - e.g. the full URL to obtain the transfer status for a data node 
> could be
>
> vos://org.test!vospace/container/node?transfer
>
> and a list of all pending transfers for the space itself could be 
> referred to by such a query on the root node.

Not sure about that one.
There are possibly better uses of the ? operator - and we can only use 
it once.

The functionality you describe could be handled by a separate 
ListStatusNodes() method.
You pass in the URI of the data node you are interested in, and it 
returns a list of the active transfers for that node.
We could extend it later to query for all the 
active|completed|failed|canceled transfers over a specific time range 
... giving us a history of the target data node.

It also means that it is up to the server implementation where it puts them.
They could be in a separate container, or even a separate space.
The ListStatusNodes() method would find them and return a single list of 
simple nodes or even just their URIs.

In the document I wasn't clear about where the status nodes were, 
because I hadn't figured that bit out yet.
If we added a ListStatusNodes() method the UI would be able to show them 
in a separate panel, possibly something triggered off the right-click menu.

>> Whatever callback mechanism we adopt, it will need some way to refer 
>> to the persistent state within the VOSpace server, that represents 
>> the state of the transfer and the individual protocol options within 
>> it. Representing these as VOSpace nodes means that we can use the 
>> existing "vos://..." URI scheme to refer to them, and the existing 
>> API to list, query and modify them.
>
>
> It might just be a question of teminology, but as I said the idea that 
> they are just "ordinary nodes" that appear in the container listings, 
> I do not like,

Yep, I agree, not in the same list as the data nodes.
Implementation dependent as to where the service actually puts them, 
just not in the same place as the normal data nodes that they refer to.

They could even be in a separate space altogether, one that just 
contains status nodes.
The two spaces could be within the same service implementation - just 
different ivo:// registration URIs for the root nodes.
The ListStatusNodes() method would just return vos://... URIs pointing 
to the status nodes in the 'other space'.

> however if they are accessible via the query part of the URL, I am 
> more amenable. However, although the existing api is ok for querying 
> and listing control objects, I am not so sure it is that suitable for 
> modifying them - after all, this whole discussion has flared up 
> because of the complexities of the data upload within VOSpace - whilst 
> these complexities are acceptable for uploading data objects (so that 
> we can take advantage of the special qualities of existing data 
> transfer protocols), an specialized api might be more suitable for 
> modifying control objects.

Yep - agree with that too.
It would be better to modify the status using a separate set of methods, 
complete(), fail() and cancel() etc.
The status node can display the current state as properties, but they 
would be read-only.
Attempting to modify a transfer status by tweaking one of the properties 
should probably throw PermissionDenied.

I think I agree with most of your comments, I just hadn't had time to 
update the doc yet.
I'll see if I can bring together some of the new ideas from various 
discussions over the last few months and write a new doc.

Thanks,
Dave