From patrick.dowler at nrc-cnrc.gc.ca Thu Aug 11 11:30:47 2011 From: patrick.dowler at nrc-cnrc.gc.ca (Patrick Dowler) Date: Thu, 11 Aug 2011 11:30:47 -0700 Subject: transfers (Re: VOSpace 2.0 updates) In-Reply-To: <00CB08BB-A5E8-49C2-8033-447D542EFBDE@cacr.caltech.edu> References: <00CB08BB-A5E8-49C2-8033-447D542EFBDE@cacr.caltech.edu> Message-ID: <201108111130.47942.patrick.dowler@nrc-cnrc.gc.ca> I am trying to iron out the details of the the async and sync transfer resources in our vospace implementation. On 2011-06-28 15:23:49 Matthew Graham wrote: > (3) /transferDetails is a standard endpoint for synchronous HTTP PUT Section 3.8 (REST bindings) has: /{properties} /{views} /{protocols} /{searches} /{nodes}/(node-path) /{transfers} /{transfers}/(job-id)/{results}/{transferDetails} I understand the { } to mean the actual name of the resource is not specified and will be discovered in the VOSI-capabilities document. However, if {transfers} is the async UWS job list, then {results} is not actually variable: it must be named "results" exactly. The {transferDetails} in the last line also says we are not mandating the name of the result: the caller will have to look for the (single?) result in there? Or should this have been "transferDetails" without the { }? If {transfers} is the UWS async resource, which one is the sync resource one POSTs a jobInfo to for the immediate response? The description of the above resources says the last one is the "transfer details for synchronous jobs". I don't see how that can be the case, or at least it is not ssufficient to define sync transfer negotiation. One could certainly implement it that way, but I think we still need a separate /{synctrans} to which we POST - and which could redirect to /{transfers}/(job-id)/{results}/{transferDetails} - because this resource does a few things differently from the UWS async resource: it executes immediately and it responds with a (via redirect or in the response) rather than responds with a redirect to /{transfers}/(job-id). So, am I missing something or do we need to specify separate async and sync transfer resources? I do like the idea that clients using the sync transfer can extract the job URL in a robust fashion as this lets the client get the server's error message if something fails later on... that implies that we specify the resource name/path after the job-id, or at least that it is a direct child of the job url, eg: POST /{synctrans} 303 /{synctrans}(job-id)/foo GET /{synctrans}(job-id)/foo read document ... if something goes wrong, strip off /foo and GET /{synctrans}(job-id) to see what the service thought went wrong. -- Patrick Dowler Tel/T?l: (250) 363-0044 Canadian Astronomy Data Centre National Research Council Canada 5071 West Saanich Road Victoria, BC V9E 2M7 Centre canadien de donnees astronomiques Conseil national de recherches Canada 5071, chemin West Saanich Victoria (C.-B.) V9E 2M7 From patrick.dowler at nrc-cnrc.gc.ca Thu Aug 11 12:01:56 2011 From: patrick.dowler at nrc-cnrc.gc.ca (Patrick Dowler) Date: Thu, 11 Aug 2011 12:01:56 -0700 Subject: moveNode and copyNode (Re: VOSpace 2.0 updates) In-Reply-To: <00CB08BB-A5E8-49C2-8033-447D542EFBDE@cacr.caltech.edu> References: <00CB08BB-A5E8-49C2-8033-447D542EFBDE@cacr.caltech.edu> Message-ID: <201108111201.56428.patrick.dowler@nrc-cnrc.gc.ca> We are just working through the internal move and copy functionality and I find it rather odd. 1. In the normal (external) use of transfers, the direction takes on one of four enumerated values. 2. For an internal move/copy, the direction field/element is re-used to contain the destination vos URI. 3. In an external copy initiated by the server (e.g. pushFromVoSpace) the destination is specified as an endpoint under the protocol. This would work for non-vospace external resources. Just to clarify, can I do a vospace-vospace transfer using a vos URI as the direction? The examples show only the non-vospace external usage with protocols and endpoints. If I have to do #3, that seems to imply that the client has to negotiate the transfer in one vospace to get the protocols and endpoints, and then use that to set up the transfer in the other vospace. If you could specify a remote vos URI, you would only talk to one vospace. Of course, that also means there are many more things that could go wrong :-) PS-It is kind of ugly in the code to reuse direction for an enum and a URI but I can live with it. Probably not worth changing the schema to add a separate destination URI. -- Patrick Dowler Tel/T?l: (250) 363-0044 Canadian Astronomy Data Centre National Research Council Canada 5071 West Saanich Road Victoria, BC V9E 2M7 Centre canadien de donnees astronomiques Conseil national de recherches Canada 5071, chemin West Saanich Victoria (C.-B.) V9E 2M7 From patrick.dowler at nrc-cnrc.gc.ca Thu Aug 25 10:05:14 2011 From: patrick.dowler at nrc-cnrc.gc.ca (Patrick Dowler) Date: Thu, 25 Aug 2011 10:05:14 -0700 Subject: transfer jobs Message-ID: <201108251005.14593.patrick.dowler@nrc-cnrc.gc.ca> The text in the latest WD still says "HTTP POST of a Job representation". I expect just an editorial issue as the examples are posting a document. (this is in 5.4.1.1, 5.4.2.1, etc). The text could be considered correct with a squishy definition of "job representation" but since the word "job" has specific meaning (UWS) that is explicitly used in the document I think this should be clarified, probably to "HTTP POST of a transfer representation". Also, in 5.4.1.2 (pushToVoSpace) the response from the alternate convenience method (the synchronous transfer) should be a 303 to a specific url (ending in {job-id}/results/transferDetails. Is this an example (of piggybacking on the async UWS impl) or is it specifically mandated that this be the endpoint? imo: Since one wants to be able to go back and get the job description (in case of error) I find the specific URL appealing from a client point of view... does it also imply that /sync/{job-id} must be a UWS job, with all the same child nodes and resources? In 5.4.3.1 (pullFromVoSpace) the text mentions the alternate convenience method (view=data) only and not a sync transfer negotiation as above. We have found the sync negotiation useful since a redirect from view=data does not allow for multiple protocol choices or protocol switching (eg you use https to authenticate while negotiating the transfer, but request a different protocol for the data transfer). It was (for us) trivial to have the sync transfer resource handle both directions. -- Patrick Dowler Tel/T?l: (250) 363-0044 Canadian Astronomy Data Centre National Research Council Canada 5071 West Saanich Road Victoria, BC V9E 2M7 Centre canadien de donnees astronomiques Conseil national de recherches Canada 5071, chemin West Saanich Victoria (C.-B.) V9E 2M7