From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From m.b.taylor at bristol.ac.uk Thu Jul 2 10:33:57 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Thu, 2 Jul 2009 18:33:57 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: On Thu, 2 Jul 2009, Robert Hanisch wrote: > On behalf of the Standing Committee on Standards and Processes, I have > posted responses to the comments that came in during the recent RFC. (And I > apologize that these responses were not posted in a more timely manner!) > Please see the RFC page for details. > > http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 > > In order to make sure that there is broad review and understanding of this > document, as it is the basis for all document promotion processes in the > IVOA, I am posting this overview to the stdproc and interop distribution > lists. Detailed discussion is best carried out on the stdproc list, but > that list has not been used much for a few years. Further comments can also > be posted on the RFC page above. > > Several comments on the RFC were posted regarding the need for interoperable > implementations and validators. The Committee agrees that some > strengthening of the language in the document on this matter is warranted. > In particular, item 4 concerning the promotion from Working Draft to > Proposed Recommendation currently reads: > > "4. Each feature of the Working Draft has been implemented. Preferably, > the Working Group should be able to demonstrate two interoperable > implementations of each feature, and validation tools should be available. > If the chair of the Working Group believes that broader review is critical, > the chair may advance the document to Proposed Recommendation even without > adequate implementation experience. In this case, the document status > section should indicate why the chair promoted the document directly to > Proposed Recommendation. A report describing the implementations or any > associated validation tools may be published as a Note, or may be documented > as part of the Request for Comments (see below)." > > Several people argued that having at least two interoperable implementations > and at least one validator service should be mandatory. As noted in the RFC > response, however, the Committee feels that making software implementations > mandatory is beyond the scope of the IVOA. One might argue that if one or > more VO projects invests the efforts into developing the standard itself, it > follows that they would put effort into implementations and validators. But > this could put a lot of pressure on schedule and resources of individual > projects, and could lead to major delays in the promotion process. > > I think we can remove the "Preferably" language above, which seems very > soft, and make the text a SHOULD in the canonical sense. Similarly the > "may" in the last sentence can become SHOULD. > > If we are to embrace the idea that implementations and validators are to > become mandatory, it would be important to hear from the project managers as > to whether or not they are willing to support this requirement. Speaking as > project manager of NVO I would have some reticence, especially now when we > have no resources to follow through. > > Please follow up with comments to stdproc at ivoa.net, and if you wish to > participate in this discussion make sure you are on that list. Bob, thanks for the response. I would like to take issue with it on two main grounds: firstly that I don't find the 'out of scope' argument convincing given what is clearly in scope for the IVOA, and secondly that I believe the effort saved by not requiring these things is in fact illusory. >From the response on the RFC page: "The problem we see with making such implementations mandatory is really one of scope and authority of the IVOA. The IVOA has no funding to support such activities, and thus the IVOA has no basis for demanding such things. If we were in position to fund such work, that would be one thing, but were are not; we rely on the goodwill and resources of the VO projects. -- BobHanisch on behalf of the Committee" I don't follow the argument that since the IVOA does not fund writing of implementations or validators, it cannot mandate them. Exactly the same could be said about writing the text of the standards documents themselves. Writing and reviewing the standards can require a great deal of time and effort from contributors, as I'm sure that you and many readers of this list know first hand, but the IVOA feels able to state requirements about these without providing any funding for the authorship or review process. "If we make interoperable implementations and a validator mandatory, how do we make sure that they get developed? Who validates the validator? We have seen already that even our most experienced software engineers have written validators with errors in them. -- BobHanisch on behalf of the Committee" We make sure that they get developed in the same way that we (in principle, at any rate) currently ensure the quality of the written standards: by public review from those IVOA members who feel sufficiently motivated to participate, and by witholding Recommendation status until agreement among those participants has been reached. Regarding validating the validators: obviously I'm not suggesting that associated software with a bug count which is provably zero is to be a requirement. As with the standards text, the authors make their best efforts in good faith, and if others scrutinising their work believe there are problems with it these are addressed by discussion. Yes this system is open to abuse (people claiming they have a validator when it doesn't do much validation), but we're mostly grown ups here. The measure of success of the IVOA is surely not how many standards it can churn out, it's more about the amount those standards are used, which in turn has to depend on the existence and quality of the implementations of those standards. Surely, a standard only becomes worth anything when at least one usable implementation exists. Here is the most important part of my argument: Postulate: If your goal is to reach an implementation, it is much easier (you can do it both faster and better; also, for the benefit of project managers, cheaper) to write the implementation and validation alongside the standard than it is to write the standard on paper, get it set in stone, and then start to write the software. The reason is that when you write it down on paper, there is a very good chance that you will write things which are contradictory or ambiguous or conflict with other established standards or are hard or impossible to implement. These are things you will stand a much much better chance of spotting if you are writing software alongside the text. The arguments for implementations and validators are similar but slightly different; I can elaborate on request. Based on my experience I believe that the postulate above is true: I would like to know whether the DocStd committee believe it is true, false, or just not relevant here. Finally, I should concede that the software requirement is clearly inapplicable in some cases - for instance I'm not suggesing that a validation suite be required for the DocStd document itself. So some concession about applicability should be included in the discussion of requirements. I'm happy to hear other people's views on this ... Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Thu Jul 2 10:57:04 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 13:57:04 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: Thanks Mark, for the (as usual) thoughtful comments. First, let me quote myself in the e-mail message I sent earlier today: "One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process." I agree completely with you that doing implementations in parallel with standards development is the best way to go, with the obvious exception that some of the IVOA standards, as you point out, are not really about software implementations. But making the implementations and the validator mandatory for promotion is where I, and Christophe and Francoise, feel it is necessary to pause. For example, if right now we were to enforce this requirement on TAP, with the resource constraints we have in several of the major projects, we would be forced to stop the promotion process. This would be an awful situation for IVOA, I think. We want prototypes to inform the standards development process, for sure, we SHOULD have them, but from a management perspective I think we have to avoid boxing ourselves into a corner. It is also a bit ambiguous just how one proves that implementations are interoperable. So far we have taken people's word, but IVOA has no means to independently test these assertions. I would probably agree fully with you if, say, IVOA were a dues-based membership organization, with its own financial resources to build validators, or test those developed by others. But I see a distinction, albeit a subtle one, between the projects' willingness to contribute to development of a standard and their ability to provide supporting software and validators on a timescale that does not jeopardize the promotion process. Finally, in any case, we have a revision system that is there to deal with problems and shortcomings found in earlier versions. If V1.0 of a standard ends up having problems -- unsupportable features, inconsistencies, etc. (and we have had this experience) -- then we fix the problems in V1.x or V2.0. So, as I said to begin with, I do believe we need to strengthen the language, but I am very reluctant to make things mandatory. Bob On 7/2/09 1:33 PM, "Mark Taylor" wrote: > On Thu, 2 Jul 2009, Robert Hanisch wrote: > >> On behalf of the Standing Committee on Standards and Processes, I have >> posted responses to the comments that came in during the recent RFC. (And I >> apologize that these responses were not posted in a more timely manner!) >> Please see the RFC page for details. >> >> http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >> >> In order to make sure that there is broad review and understanding of this >> document, as it is the basis for all document promotion processes in the >> IVOA, I am posting this overview to the stdproc and interop distribution >> lists. Detailed discussion is best carried out on the stdproc list, but >> that list has not been used much for a few years. Further comments can also >> be posted on the RFC page above. >> >> Several comments on the RFC were posted regarding the need for interoperable >> implementations and validators. The Committee agrees that some >> strengthening of the language in the document on this matter is warranted. >> In particular, item 4 concerning the promotion from Working Draft to >> Proposed Recommendation currently reads: >> >> "4. Each feature of the Working Draft has been implemented. Preferably, >> the Working Group should be able to demonstrate two interoperable >> implementations of each feature, and validation tools should be available. >> If the chair of the Working Group believes that broader review is critical, >> the chair may advance the document to Proposed Recommendation even without >> adequate implementation experience. In this case, the document status >> section should indicate why the chair promoted the document directly to >> Proposed Recommendation. A report describing the implementations or any >> associated validation tools may be published as a Note, or may be documented >> as part of the Request for Comments (see below)." >> >> Several people argued that having at least two interoperable implementations >> and at least one validator service should be mandatory. As noted in the RFC >> response, however, the Committee feels that making software implementations >> mandatory is beyond the scope of the IVOA. One might argue that if one or >> more VO projects invests the efforts into developing the standard itself, it >> follows that they would put effort into implementations and validators. But >> this could put a lot of pressure on schedule and resources of individual >> projects, and could lead to major delays in the promotion process. >> >> I think we can remove the "Preferably" language above, which seems very >> soft, and make the text a SHOULD in the canonical sense. Similarly the >> "may" in the last sentence can become SHOULD. >> >> If we are to embrace the idea that implementations and validators are to >> become mandatory, it would be important to hear from the project managers as >> to whether or not they are willing to support this requirement. Speaking as >> project manager of NVO I would have some reticence, especially now when we >> have no resources to follow through. >> >> Please follow up with comments to stdproc at ivoa.net, and if you wish to >> participate in this discussion make sure you are on that list. > > Bob, > > thanks for the response. I would like to take issue with it on two > main grounds: firstly that I don't find the 'out of scope' argument > convincing given what is clearly in scope for the IVOA, and secondly > that I believe the effort saved by not requiring these things is > in fact illusory. > > From the response on the RFC page: > > "The problem we see with making such implementations mandatory is > really one of scope and authority of the IVOA. The IVOA has no > funding to support such activities, and thus the IVOA has no basis > for demanding such things. If we were in position to fund such work, > that would be one thing, but were are not; we rely on the goodwill > and resources of the VO projects. > -- BobHanisch on behalf of the Committee" > > I don't follow the argument that since the IVOA does not fund writing > of implementations or validators, it cannot mandate them. > Exactly the same could be said about writing the text of the > standards documents themselves. Writing and reviewing the standards > can require a great deal of time and effort from contributors, > as I'm sure that you and many readers of this list know first hand, > but the IVOA feels able to state requirements about these without > providing any funding for the authorship or review process. > > "If we make interoperable implementations and a validator mandatory, > how do we make sure that they get developed? Who validates the > validator? We have seen already that even our most experienced software > engineers have written validators with errors in them. > -- BobHanisch on behalf of the Committee" > > We make sure that they get developed in the same way that we > (in principle, at any rate) currently ensure the quality of the written > standards: by public review from those IVOA members who feel > sufficiently motivated to participate, and by witholding Recommendation > status until agreement among those participants has been reached. > Regarding validating the validators: obviously I'm not suggesting > that associated software with a bug count which is provably zero is > to be a requirement. As with the standards text, the authors make > their best efforts in good faith, and if others scrutinising their > work believe there are problems with it these are addressed by > discussion. Yes this system is open to abuse (people claiming they > have a validator when it doesn't do much validation), but we're > mostly grown ups here. > > The measure of success of the IVOA is surely not how many standards > it can churn out, it's more about the amount those standards are used, > which in turn has to depend on the existence and quality of the > implementations of those standards. Surely, a standard only becomes > worth anything when at least one usable implementation exists. Here is > the most important part of my argument: > > Postulate: If your goal is to reach an implementation, it is much > easier (you can do it both faster and better; also, for the > benefit of project managers, cheaper) to write the > implementation and validation alongside the standard than it is > to write the standard on paper, get it set in stone, and then > start to write the software. > > The reason is that when you write it down on paper, there is a very > good chance that you will write things which are contradictory or > ambiguous or conflict with other established standards or are hard or > impossible to implement. These are things you will stand a much much > better chance of spotting if you are writing software alongside the > text. The arguments for implementations and validators are similar > but slightly different; I can elaborate on request. > > Based on my experience I believe that the postulate above is true: > I would like to know whether the DocStd committee believe it is > true, false, or just not relevant here. > > Finally, I should concede that the software requirement is clearly > inapplicable in some cases - for instance I'm not suggesing that > a validation suite be required for the DocStd document itself. > So some concession about applicability should be included in the > discussion of requirements. > > I'm happy to hear other people's views on this ... > > Mark From mjg at cacr.caltech.edu Thu Jul 2 12:19:23 2009 From: mjg at cacr.caltech.edu (Matthew Graham) Date: Thu, 2 Jul 2009 12:19:23 -0700 Subject: Versioning Message-ID: <833254ED-BA5D-4D10-8536-46DAFEBE4989@cacr.caltech.edu> Hi, This is a discussion that I started within my WG just to solicit opinion. Can the Standards WG please clarify: The response was posted to my comments on the RFC page: "The Committee agrees that the version numbering scheme is challenging when dealing with namespaces and WSDL, and leads to the conclusion that when IVOA standards describe web services or have associated XML schemas, with namespaces that when changed, cause software to break, then these changes must both be accompanied by an increment to the integer part of the document and the associated "supplementary" files. This would not affect most of the standards documents, and should not present any real logistical difficulty, as there are a sufficient number of integers available to support any number of revisions. " I explain: This has greatest impact for this working group (GWS but I am also cross-posting to Semantics) and essentially means that ALL (WD, PR, etc) versions of our specs with WSDL/XML/RDF documents (anything with a namespace) will only carry integer versions. So, for example, the progression of VOSpace 2.0 would actually proceed as: VOSpace 2 (first WD) VOSpace 3 (second WD) VOSpace 4 (third WD) VOSpace 5 (first PR) VOSpace 6 (second PR) VOSpace 7 (final PR) VOSpace 8 (REC) The next version would then VOSpace 9, etc. Is this a valid interpretation of the spec? The standards documents says: "The number to the left of the (first) decimal point starts with 0 for documents that are being discussed within a Working Group prior to publication for IVOA-wide review. The number increments to 1 for the first public version, and to 2, 3, ..., for subsequent versions that are not backward compatible and/or require substantial revisions to implementations." The definition of a Working Draft is: "A Working Draft is published at the discretion of a Working Group once the WG is satisfied that the document is sufficiently developed to merit broader exposure and feedback" So I think that a Working Draft fulfils the "IVOA-wide review" criterion. The alternate suggestion from Dave Morris and with much support is: If the integer rule only applied to something that has been accepted as a recommendation. VOSpace 2.0-yyymmdd (first WD) VOSpace 2.0-yyymmdd (second WD) VOSpace 2.0-yyymmdd (third WD) VOSpace 2.0-yyymmdd (first PR) VOSpace 2.0-yyymmdd (second PR) VOSpace 2.0-yyymmdd (final PR) VOSpace 2.0 (REC) VOSpace 2.1-yyymmdd (first WD) (minor text changes only) VOSpace 2.1-yyymmdd (final PR) (minor text changes only) VOSpace 2.1 (REC) VOSpace 3.0-yyymmdd (first WD) (changes to service behavior or xml schema) ... I think this needs to be clarified one way or the other. Cheers, Matthew From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From hanisch at stsci.edu Thu Jul 2 16:27:04 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 19:27:04 -0400 Subject: versioning and WSDL and namespaces, etc. Message-ID: Following a bit of side discussion with Matthew to make sure I understand the issue, I think we can resolve the problem if we consider "software breakage" in the sense of manual recoding. Changes to WSDL or XML namespaces typically require regeneration of bindings and recompilation of code, but this is all (supposed) to be more or less automatic. So, with some clarification to the text, the implication would be that the integer part of the version remains fixed -- it is notional and describes the intention -- and updates to WSDL, etc., coincide with increments to the part of the version number to the right of the decimal point. As for Paul's comment: What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. What the document says is ? The number to the left of the (first) decimal point starts with 0 for documents that are being discussed within a Working Group prior to publication for IVOA-wide review. The number increments to 1 for the first public version, and to 2, 3, ..., for subsequent versions that are not backward compatible and/or require substantial revisions to implementations. Thus, I do not see why there is confusion on this point. Bob From m.b.taylor at bristol.ac.uk Fri Jul 3 02:53:14 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Fri, 3 Jul 2009 10:53:14 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: Bob, On Thu, 2 Jul 2009, Robert Hanisch wrote: > First, let me quote myself in the e-mail message I sent earlier today: > > "One might argue that if one or more VO projects invests the efforts into > developing the standard itself, it follows that they would put effort into > implementations and validators. But this could put a lot of pressure on > schedule and resources of individual projects, and could lead to major > delays in the promotion process." > > I agree completely with you that doing implementations in parallel with > standards development is the best way to go, with the obvious exception that > some of the IVOA standards, as you point out, are not really about software > implementations. > > But making the implementations and the validator mandatory for promotion is > where I, and Christophe and Francoise, feel it is necessary to pause. For > example, if right now we were to enforce this requirement on TAP, with the > resource constraints we have in several of the major projects, we would be > forced to stop the promotion process. This would be an awful situation for > IVOA, I think. We want prototypes to inform the standards development > process, for sure, we SHOULD have them, but from a management perspective I > think we have to avoid boxing ourselves into a corner. I agree with you that requiring the associated software would slow down the promotion process. However, as I tried to make clear in my previous message, I don't believe that the date of publication of the Recommendation is what we should be focussing on. When a document reaches Recommendation status, yes it looks good on Gantt charts, but all it means is we have a bit of paper to wave in the air. Nothing useful has been achieved until effective use of that standard is made, which in most cases requires software to be written. I believe that attaining that second goal will actually be accelerated rather than decelerated if the software is written in tandem with the text (I tried to get you to agree or disagree with this statement in my last message, but I still don't know your opinion). Doing them in parallel will also lead to a better standard. In any case: if you can't find anyone who is prepared to implement the standard, there's probably a good reason; either it's too hard to do, or it's not something that people need. In either case it's no great achievement to have published it. > It is also a bit ambiguous just how one proves that implementations are > interoperable. So far we have taken people's word, but IVOA has no means to > independently test these assertions. Agreed it's ambiguous, and rigorous enforcement is not really feasible. I'm prepared to rely on the good will of the authors/implementors, supplemented by whatever scrutiny other IVOA members are prepared to provide on an ad-hoc basis. > I would probably agree fully with you if, say, IVOA were a dues-based > membership organization, with its own financial resources to build > validators, or test those developed by others. But I see a distinction, > albeit a subtle one, between the projects' willingness to contribute to > development of a standard and their ability to provide supporting software > and validators on a timescale that does not jeopardize the promotion > process. Again - I believe that the timescale of the promotion process itself is a red herring. That given, I don't see the subtle difference here (management is neither my enthusiasm nor my expertise, so I'm quite prepared to believe there are issues over my head here) - could you elaborate a bit? But I'll say this, I think it is somewhat dangerous for the system to encourage or allow organisations to shove their oar in when it comes to writing requirements, but not when it comes to resourcing their implementation. > Finally, in any case, we have a revision system that is there to deal with > problems and shortcomings found in earlier versions. If V1.0 of a standard > ends up having problems -- unsupportable features, inconsistencies, etc. > (and we have had this experience) -- then we fix the problems in V1.x or > V2.0. My understanding of the basic intention for the versioning system is that version X+delta is version X with the additions or changes gained from working with version X for a while, as well as enhancements which weren't feasible (e.g. from lack of resources) on the timescale of version X. While it can also be used for patching up things which were broken or ill-thought-out in the first place, it's not very well designed for that purpose, if only because of the very long timescales typically involved. > (and we have had this experience) -- then we fix the problems in V1.x or and we'd have had the experience a lot less if software was written in tandem with the original versions! Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Fri Jul 3 05:35:07 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Fri, 03 Jul 2009 08:35:07 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: On 7/3/09 5:53 AM, "Mark Taylor" wrote: > Bob, > > On Thu, 2 Jul 2009, Robert Hanisch wrote: > >> First, let me quote myself in the e-mail message I sent earlier today: >> >> "One might argue that if one or more VO projects invests the efforts into >> developing the standard itself, it follows that they would put effort into >> implementations and validators. But this could put a lot of pressure on >> schedule and resources of individual projects, and could lead to major >> delays in the promotion process." >> >> I agree completely with you that doing implementations in parallel with >> standards development is the best way to go, with the obvious exception that >> some of the IVOA standards, as you point out, are not really about software >> implementations. >> >> But making the implementations and the validator mandatory for promotion is >> where I, and Christophe and Francoise, feel it is necessary to pause. For >> example, if right now we were to enforce this requirement on TAP, with the >> resource constraints we have in several of the major projects, we would be >> forced to stop the promotion process. This would be an awful situation for >> IVOA, I think. We want prototypes to inform the standards development >> process, for sure, we SHOULD have them, but from a management perspective I >> think we have to avoid boxing ourselves into a corner. > > I agree with you that requiring the associated software would slow down > the promotion process. However, as I tried to make clear in my previous > message, I don't believe that the date of publication of the > Recommendation is what we should be focussing on. > When a document reaches Recommendation status, yes it looks good on > Gantt charts, but all it means is we have a bit of paper to wave > in the air. Nothing useful has been achieved until effective use of > that standard is made, which in most cases requires software to be > written. I believe that attaining that second goal will actually > be accelerated rather than decelerated if the software is written > in tandem with the text (I tried to get you to agree or disagree with > this statement in my last message, but I still don't know your opinion). > Doing them in parallel will also lead to a better standard. > > In any case: if you can't find anyone who is prepared to implement > the standard, there's probably a good reason; either it's too hard > to do, or it's not something that people need. In either case > it's no great achievement to have published it. I disagree with you on this point, and perhaps that is the crux of the issue. Many organizations will not even look at an IVOA standard until it has been finalized. They say "why should I bother implementing something that is just going to change." Plus, like it or not, just reaching agreement on the standard IS a major milestone for our development projects. I don't know what is faster overall ... you are probably right, that when the standard is accompanied by two or more implementations, it is more likely to be correct and implementable by others outside of the core development team. But we also all work to external expectations (from our funding agencies), and I have heard many times now comments from people at data centers saying they are not going to code up an interface because it is still a WD. Please keep in mind that many of the implementors of IVOA standards are not part of the VO development teams. >> It is also a bit ambiguous just how one proves that implementations are >> interoperable. So far we have taken people's word, but IVOA has no means to >> independently test these assertions. > > Agreed it's ambiguous, and rigorous enforcement is not really feasible. > I'm prepared to rely on the good will of the authors/implementors, > supplemented by whatever scrutiny other IVOA members are prepared > to provide on an ad-hoc basis. We have no choice at the moment! >> I would probably agree fully with you if, say, IVOA were a dues-based >> membership organization, with its own financial resources to build >> validators, or test those developed by others. But I see a distinction, >> albeit a subtle one, between the projects' willingness to contribute to >> development of a standard and their ability to provide supporting software >> and validators on a timescale that does not jeopardize the promotion >> process. > > Again - I believe that the timescale of the promotion process itself > is a red herring. That given, I don't see the subtle difference here > (management is neither my enthusiasm nor my expertise, so I'm quite > prepared to believe there are issues over my head here) - could you > elaborate a bit? As above. Agreement on these standards is a major milestone. Read any of the NVO Quarterly Reports to NSF (all are on the NVO web page, in the document collection) and you can see how important these are in terms of deliverables. > But I'll say this, I think it is somewhat dangerous for the system > to encourage or allow organisations to shove their oar in when it > comes to writing requirements, but not when it comes to resourcing > their implementation. This is not unique to the IVOA process! Everyone is eager to tell you what needs to be done, as long as someone else actually does it! >> Finally, in any case, we have a revision system that is there to deal with >> problems and shortcomings found in earlier versions. If V1.0 of a standard >> ends up having problems -- unsupportable features, inconsistencies, etc. >> (and we have had this experience) -- then we fix the problems in V1.x or >> V2.0. > > My understanding of the basic intention for the versioning system > is that version X+delta is version X with the additions or changes > gained from working with version X for a while, as well as > enhancements which weren't feasible (e.g. from lack of resources) > on the timescale of version X. While it can also be used for > patching up things which were broken or ill-thought-out in the > first place, it's not very well designed for that purpose, if only > because of the very long timescales typically involved. Which will be longer if we make interoperable implementations and validators mandatory at every step of the process. >> (and we have had this experience) -- then we fix the problems in V1.x or > > and we'd have had the experience a lot less if software was written > in tandem with the original versions! > > Mark Bob From m.b.taylor at bristol.ac.uk Fri Jul 3 06:51:10 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Fri, 3 Jul 2009 14:51:10 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: On Fri, 3 Jul 2009, Robert Hanisch wrote: > >Bob wrote: > >> But making the implementations and the validator mandatory for promotion is > >> where I, and Christophe and Francoise, feel it is necessary to pause. For > >> example, if right now we were to enforce this requirement on TAP, with the > >> resource constraints we have in several of the major projects, we would be > >> forced to stop the promotion process. This would be an awful situation for > >> IVOA, I think. We want prototypes to inform the standards development > >> process, for sure, we SHOULD have them, but from a management perspective I > >> think we have to avoid boxing ourselves into a corner. > > > Mark wrote: > > I agree with you that requiring the associated software would slow down > > the promotion process. However, as I tried to make clear in my previous > > message, I don't believe that the date of publication of the > > Recommendation is what we should be focussing on. > > When a document reaches Recommendation status, yes it looks good on > > Gantt charts, but all it means is we have a bit of paper to wave > > in the air. Nothing useful has been achieved until effective use of > > that standard is made, which in most cases requires software to be > > written. I believe that attaining that second goal will actually > > be accelerated rather than decelerated if the software is written > > in tandem with the text (I tried to get you to agree or disagree with > > this statement in my last message, but I still don't know your opinion). > > Doing them in parallel will also lead to a better standard. > > > > In any case: if you can't find anyone who is prepared to implement > > the standard, there's probably a good reason; either it's too hard > > to do, or it's not something that people need. In either case > > it's no great achievement to have published it. > I disagree with you on this point, and perhaps that is the crux of the > issue. Many organizations will not even look at an IVOA standard until it > has been finalized. They say "why should I bother implementing something > that is just going to change." Plus, like it or not, just reaching > agreement on the standard IS a major milestone for our development projects. > I don't know what is faster overall ... you are probably right, that when > the standard is accompanied by two or more implementations, it is more > likely to be correct and implementable by others outside of the core > development team. But we also all work to external expectations (from our > funding agencies), and I have heard many times now comments from people at > data centers saying they are not going to code up an interface because it is > still a WD. Please keep in mind that many of the implementors of IVOA > standards are not part of the VO development teams. That's a fair point as stated. However, my impression (not backed up by evidence, and possibly incorrect - data welcomed) is that the first implementations for the standards that we're talking about are in fact by VO/IVOA insiders. Non-IVOA data centers in practice don't wait until the REC acceptance date and then start implementing; they wait until it looks like it's going to be a standard which is actually being used, or going to get used, by other people and then start implementing. This latter only happens after somebody else (most likely an IVOA insider) has provided an implementation of some kind. > >> I would probably agree fully with you if, say, IVOA were a dues-based > >> membership organization, with its own financial resources to build > >> validators, or test those developed by others. But I see a distinction, > >> albeit a subtle one, between the projects' willingness to contribute to > >> development of a standard and their ability to provide supporting software > >> and validators on a timescale that does not jeopardize the promotion > >> process. > > > > Again - I believe that the timescale of the promotion process itself > > is a red herring. That given, I don't see the subtle difference here > > (management is neither my enthusiasm nor my expertise, so I'm quite > > prepared to believe there are issues over my head here) - could you > > elaborate a bit? > As above. Agreement on these standards is a major milestone. Read any of > the NVO Quarterly Reports to NSF (all are on the NVO web page, in the > document collection) and you can see how important these are in terms of > deliverables. To some extent, this is what I was goading you to say. I do concede that there are political considerations here which will affect what decisions we take. When I said above that nothing useful is achieved by the standard publication per se, I should really have said nothing *technically* useful has been achieved - I admit that there may be some (maybe, a lot of) political advantage. If we're going to determine the standards process on that basis however, I'd like to have it out in the open amongst ourselves rather than to pretend that the StdDoc text is motivated purely by the considerations of technical excellence and technical pragmatism. If we admit that, maybe there's another way forward: declare an intermediate document stage of "accepted in principle, no major changes expected, but revisions possible on the basis of implementation experience". This stage could be sold as the one that counts to the funding agencies. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Fri Jul 3 15:43:49 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Fri, 03 Jul 2009 18:43:49 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: > That's a fair point as stated. However, my impression (not backed up > by evidence, and possibly incorrect - data welcomed) is that the first > implementations for the standards that we're talking about are in > fact by VO/IVOA insiders. Non-IVOA data centers in practice don't > wait until the REC acceptance date and then start implementing; > they wait until it looks like it's going to be a standard which > is actually being used, or going to get used, by other people > and then start implementing. This latter only happens after > somebody else (most likely an IVOA insider) has provided an > implementation of some kind. I don't have very complete information on this -- mostly anecdotal evidence. Most developers like to start with something already existing; others eschew what's been done and start from scratch. >>>> I would probably agree fully with you if, say, IVOA were a dues-based >>>> membership organization, with its own financial resources to build >>>> validators, or test those developed by others. But I see a distinction, >>>> albeit a subtle one, between the projects' willingness to contribute to >>>> development of a standard and their ability to provide supporting software >>>> and validators on a timescale that does not jeopardize the promotion >>>> process. >>> >>> Again - I believe that the timescale of the promotion process itself >>> is a red herring. That given, I don't see the subtle difference here >>> (management is neither my enthusiasm nor my expertise, so I'm quite >>> prepared to believe there are issues over my head here) - could you >>> elaborate a bit? >> As above. Agreement on these standards is a major milestone. Read any of >> the NVO Quarterly Reports to NSF (all are on the NVO web page, in the >> document collection) and you can see how important these are in terms of >> deliverables. > > To some extent, this is what I was goading you to say. I do concede > that there are political considerations here which will affect what > decisions we take. When I said above that nothing useful is achieved > by the standard publication per se, I should really have said nothing > *technically* useful has been achieved - I admit that there may be > some (maybe, a lot of) political advantage. If we're going to determine > the standards process on that basis however, I'd like to have > it out in the open amongst ourselves rather than to pretend that the > StdDoc text is motivated purely by the considerations of technical > excellence and technical pragmatism. If we admit that, maybe there's > another way forward: declare an intermediate document stage of > "accepted in principle, no major changes expected, but revisions > possible on the basis of implementation experience". This stage > could be sold as the one that counts to the funding agencies. It was understood from the beginning of IVOA that standards would evolve and that first versions would likely need to change. I found something I wrote back in March 2004, concerning the registry standards, that I think remains relevant today: We need to reach agreements quickly, build to them, and iterate to improve them. We need to be willing to build and revise, or even in some cases throw away and start again. We have set up a standards process that recognizes change, and is intended to be responsive to change. I do not believe we can afford to set 2-3 year schedules for reaching consensus. If it takes this long, the community upon which we will utimately depend for support will see us as purveyors of snake-oil, and will blow us off. - - - - - All this said, I think we will update the document to make the implementations and validator strongly/highly recommended, with some reason needing to be provided if these have not been done in order to move the standard ahead. I still think we do not want to go with mandatory, for the reasons explained in the past several messages. Off now for our independence holiday (which started today, actually). cheers, Bob From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From hanisch at stsci.edu Thu Jul 2 08:08:06 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 11:08:06 -0400 Subject: IVOA Document Standards RFC V1.2 Message-ID: On behalf of the Standing Committee on Standards and Processes, I have posted responses to the comments that came in during the recent RFC. (And I apologize that these responses were not posted in a more timely manner!) Please see the RFC page for details. http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 In order to make sure that there is broad review and understanding of this document, as it is the basis for all document promotion processes in the IVOA, I am posting this overview to the stdproc and interop distribution lists. Detailed discussion is best carried out on the stdproc list, but that list has not been used much for a few years. Further comments can also be posted on the RFC page above. Several comments on the RFC were posted regarding the need for interoperable implementations and validators. The Committee agrees that some strengthening of the language in the document on this matter is warranted. In particular, item 4 concerning the promotion from Working Draft to Proposed Recommendation currently reads: "4. Each feature of the Working Draft has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature, and validation tools should be available. If the chair of the Working Group believes that broader review is critical, the chair may advance the document to Proposed Recommendation even without adequate implementation experience. In this case, the document status section should indicate why the chair promoted the document directly to Proposed Recommendation. A report describing the implementations or any associated validation tools may be published as a Note, or may be documented as part of the Request for Comments (see below)." Several people argued that having at least two interoperable implementations and at least one validator service should be mandatory. As noted in the RFC response, however, the Committee feels that making software implementations mandatory is beyond the scope of the IVOA. One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process. I think we can remove the "Preferably" language above, which seems very soft, and make the text a SHOULD in the canonical sense. Similarly the "may" in the last sentence can become SHOULD. If we are to embrace the idea that implementations and validators are to become mandatory, it would be important to hear from the project managers as to whether or not they are willing to support this requirement. Speaking as project manager of NVO I would have some reticence, especially now when we have no resources to follow through. Please follow up with comments to stdproc at ivoa.net, and if you wish to participate in this discussion make sure you are on that list. thanks, Bob From m.b.taylor at bristol.ac.uk Thu Jul 2 10:33:57 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Thu, 2 Jul 2009 18:33:57 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: On Thu, 2 Jul 2009, Robert Hanisch wrote: > On behalf of the Standing Committee on Standards and Processes, I have > posted responses to the comments that came in during the recent RFC. (And I > apologize that these responses were not posted in a more timely manner!) > Please see the RFC page for details. > > http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 > > In order to make sure that there is broad review and understanding of this > document, as it is the basis for all document promotion processes in the > IVOA, I am posting this overview to the stdproc and interop distribution > lists. Detailed discussion is best carried out on the stdproc list, but > that list has not been used much for a few years. Further comments can also > be posted on the RFC page above. > > Several comments on the RFC were posted regarding the need for interoperable > implementations and validators. The Committee agrees that some > strengthening of the language in the document on this matter is warranted. > In particular, item 4 concerning the promotion from Working Draft to > Proposed Recommendation currently reads: > > "4. Each feature of the Working Draft has been implemented. Preferably, > the Working Group should be able to demonstrate two interoperable > implementations of each feature, and validation tools should be available. > If the chair of the Working Group believes that broader review is critical, > the chair may advance the document to Proposed Recommendation even without > adequate implementation experience. In this case, the document status > section should indicate why the chair promoted the document directly to > Proposed Recommendation. A report describing the implementations or any > associated validation tools may be published as a Note, or may be documented > as part of the Request for Comments (see below)." > > Several people argued that having at least two interoperable implementations > and at least one validator service should be mandatory. As noted in the RFC > response, however, the Committee feels that making software implementations > mandatory is beyond the scope of the IVOA. One might argue that if one or > more VO projects invests the efforts into developing the standard itself, it > follows that they would put effort into implementations and validators. But > this could put a lot of pressure on schedule and resources of individual > projects, and could lead to major delays in the promotion process. > > I think we can remove the "Preferably" language above, which seems very > soft, and make the text a SHOULD in the canonical sense. Similarly the > "may" in the last sentence can become SHOULD. > > If we are to embrace the idea that implementations and validators are to > become mandatory, it would be important to hear from the project managers as > to whether or not they are willing to support this requirement. Speaking as > project manager of NVO I would have some reticence, especially now when we > have no resources to follow through. > > Please follow up with comments to stdproc at ivoa.net, and if you wish to > participate in this discussion make sure you are on that list. Bob, thanks for the response. I would like to take issue with it on two main grounds: firstly that I don't find the 'out of scope' argument convincing given what is clearly in scope for the IVOA, and secondly that I believe the effort saved by not requiring these things is in fact illusory. >From the response on the RFC page: "The problem we see with making such implementations mandatory is really one of scope and authority of the IVOA. The IVOA has no funding to support such activities, and thus the IVOA has no basis for demanding such things. If we were in position to fund such work, that would be one thing, but were are not; we rely on the goodwill and resources of the VO projects. -- BobHanisch on behalf of the Committee" I don't follow the argument that since the IVOA does not fund writing of implementations or validators, it cannot mandate them. Exactly the same could be said about writing the text of the standards documents themselves. Writing and reviewing the standards can require a great deal of time and effort from contributors, as I'm sure that you and many readers of this list know first hand, but the IVOA feels able to state requirements about these without providing any funding for the authorship or review process. "If we make interoperable implementations and a validator mandatory, how do we make sure that they get developed? Who validates the validator? We have seen already that even our most experienced software engineers have written validators with errors in them. -- BobHanisch on behalf of the Committee" We make sure that they get developed in the same way that we (in principle, at any rate) currently ensure the quality of the written standards: by public review from those IVOA members who feel sufficiently motivated to participate, and by witholding Recommendation status until agreement among those participants has been reached. Regarding validating the validators: obviously I'm not suggesting that associated software with a bug count which is provably zero is to be a requirement. As with the standards text, the authors make their best efforts in good faith, and if others scrutinising their work believe there are problems with it these are addressed by discussion. Yes this system is open to abuse (people claiming they have a validator when it doesn't do much validation), but we're mostly grown ups here. The measure of success of the IVOA is surely not how many standards it can churn out, it's more about the amount those standards are used, which in turn has to depend on the existence and quality of the implementations of those standards. Surely, a standard only becomes worth anything when at least one usable implementation exists. Here is the most important part of my argument: Postulate: If your goal is to reach an implementation, it is much easier (you can do it both faster and better; also, for the benefit of project managers, cheaper) to write the implementation and validation alongside the standard than it is to write the standard on paper, get it set in stone, and then start to write the software. The reason is that when you write it down on paper, there is a very good chance that you will write things which are contradictory or ambiguous or conflict with other established standards or are hard or impossible to implement. These are things you will stand a much much better chance of spotting if you are writing software alongside the text. The arguments for implementations and validators are similar but slightly different; I can elaborate on request. Based on my experience I believe that the postulate above is true: I would like to know whether the DocStd committee believe it is true, false, or just not relevant here. Finally, I should concede that the software requirement is clearly inapplicable in some cases - for instance I'm not suggesing that a validation suite be required for the DocStd document itself. So some concession about applicability should be included in the discussion of requirements. I'm happy to hear other people's views on this ... Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Thu Jul 2 10:57:04 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 13:57:04 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: Thanks Mark, for the (as usual) thoughtful comments. First, let me quote myself in the e-mail message I sent earlier today: "One might argue that if one or more VO projects invests the efforts into developing the standard itself, it follows that they would put effort into implementations and validators. But this could put a lot of pressure on schedule and resources of individual projects, and could lead to major delays in the promotion process." I agree completely with you that doing implementations in parallel with standards development is the best way to go, with the obvious exception that some of the IVOA standards, as you point out, are not really about software implementations. But making the implementations and the validator mandatory for promotion is where I, and Christophe and Francoise, feel it is necessary to pause. For example, if right now we were to enforce this requirement on TAP, with the resource constraints we have in several of the major projects, we would be forced to stop the promotion process. This would be an awful situation for IVOA, I think. We want prototypes to inform the standards development process, for sure, we SHOULD have them, but from a management perspective I think we have to avoid boxing ourselves into a corner. It is also a bit ambiguous just how one proves that implementations are interoperable. So far we have taken people's word, but IVOA has no means to independently test these assertions. I would probably agree fully with you if, say, IVOA were a dues-based membership organization, with its own financial resources to build validators, or test those developed by others. But I see a distinction, albeit a subtle one, between the projects' willingness to contribute to development of a standard and their ability to provide supporting software and validators on a timescale that does not jeopardize the promotion process. Finally, in any case, we have a revision system that is there to deal with problems and shortcomings found in earlier versions. If V1.0 of a standard ends up having problems -- unsupportable features, inconsistencies, etc. (and we have had this experience) -- then we fix the problems in V1.x or V2.0. So, as I said to begin with, I do believe we need to strengthen the language, but I am very reluctant to make things mandatory. Bob On 7/2/09 1:33 PM, "Mark Taylor" wrote: > On Thu, 2 Jul 2009, Robert Hanisch wrote: > >> On behalf of the Standing Committee on Standards and Processes, I have >> posted responses to the comments that came in during the recent RFC. (And I >> apologize that these responses were not posted in a more timely manner!) >> Please see the RFC page for details. >> >> http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >> >> In order to make sure that there is broad review and understanding of this >> document, as it is the basis for all document promotion processes in the >> IVOA, I am posting this overview to the stdproc and interop distribution >> lists. Detailed discussion is best carried out on the stdproc list, but >> that list has not been used much for a few years. Further comments can also >> be posted on the RFC page above. >> >> Several comments on the RFC were posted regarding the need for interoperable >> implementations and validators. The Committee agrees that some >> strengthening of the language in the document on this matter is warranted. >> In particular, item 4 concerning the promotion from Working Draft to >> Proposed Recommendation currently reads: >> >> "4. Each feature of the Working Draft has been implemented. Preferably, >> the Working Group should be able to demonstrate two interoperable >> implementations of each feature, and validation tools should be available. >> If the chair of the Working Group believes that broader review is critical, >> the chair may advance the document to Proposed Recommendation even without >> adequate implementation experience. In this case, the document status >> section should indicate why the chair promoted the document directly to >> Proposed Recommendation. A report describing the implementations or any >> associated validation tools may be published as a Note, or may be documented >> as part of the Request for Comments (see below)." >> >> Several people argued that having at least two interoperable implementations >> and at least one validator service should be mandatory. As noted in the RFC >> response, however, the Committee feels that making software implementations >> mandatory is beyond the scope of the IVOA. One might argue that if one or >> more VO projects invests the efforts into developing the standard itself, it >> follows that they would put effort into implementations and validators. But >> this could put a lot of pressure on schedule and resources of individual >> projects, and could lead to major delays in the promotion process. >> >> I think we can remove the "Preferably" language above, which seems very >> soft, and make the text a SHOULD in the canonical sense. Similarly the >> "may" in the last sentence can become SHOULD. >> >> If we are to embrace the idea that implementations and validators are to >> become mandatory, it would be important to hear from the project managers as >> to whether or not they are willing to support this requirement. Speaking as >> project manager of NVO I would have some reticence, especially now when we >> have no resources to follow through. >> >> Please follow up with comments to stdproc at ivoa.net, and if you wish to >> participate in this discussion make sure you are on that list. > > Bob, > > thanks for the response. I would like to take issue with it on two > main grounds: firstly that I don't find the 'out of scope' argument > convincing given what is clearly in scope for the IVOA, and secondly > that I believe the effort saved by not requiring these things is > in fact illusory. > > From the response on the RFC page: > > "The problem we see with making such implementations mandatory is > really one of scope and authority of the IVOA. The IVOA has no > funding to support such activities, and thus the IVOA has no basis > for demanding such things. If we were in position to fund such work, > that would be one thing, but were are not; we rely on the goodwill > and resources of the VO projects. > -- BobHanisch on behalf of the Committee" > > I don't follow the argument that since the IVOA does not fund writing > of implementations or validators, it cannot mandate them. > Exactly the same could be said about writing the text of the > standards documents themselves. Writing and reviewing the standards > can require a great deal of time and effort from contributors, > as I'm sure that you and many readers of this list know first hand, > but the IVOA feels able to state requirements about these without > providing any funding for the authorship or review process. > > "If we make interoperable implementations and a validator mandatory, > how do we make sure that they get developed? Who validates the > validator? We have seen already that even our most experienced software > engineers have written validators with errors in them. > -- BobHanisch on behalf of the Committee" > > We make sure that they get developed in the same way that we > (in principle, at any rate) currently ensure the quality of the written > standards: by public review from those IVOA members who feel > sufficiently motivated to participate, and by witholding Recommendation > status until agreement among those participants has been reached. > Regarding validating the validators: obviously I'm not suggesting > that associated software with a bug count which is provably zero is > to be a requirement. As with the standards text, the authors make > their best efforts in good faith, and if others scrutinising their > work believe there are problems with it these are addressed by > discussion. Yes this system is open to abuse (people claiming they > have a validator when it doesn't do much validation), but we're > mostly grown ups here. > > The measure of success of the IVOA is surely not how many standards > it can churn out, it's more about the amount those standards are used, > which in turn has to depend on the existence and quality of the > implementations of those standards. Surely, a standard only becomes > worth anything when at least one usable implementation exists. Here is > the most important part of my argument: > > Postulate: If your goal is to reach an implementation, it is much > easier (you can do it both faster and better; also, for the > benefit of project managers, cheaper) to write the > implementation and validation alongside the standard than it is > to write the standard on paper, get it set in stone, and then > start to write the software. > > The reason is that when you write it down on paper, there is a very > good chance that you will write things which are contradictory or > ambiguous or conflict with other established standards or are hard or > impossible to implement. These are things you will stand a much much > better chance of spotting if you are writing software alongside the > text. The arguments for implementations and validators are similar > but slightly different; I can elaborate on request. > > Based on my experience I believe that the postulate above is true: > I would like to know whether the DocStd committee believe it is > true, false, or just not relevant here. > > Finally, I should concede that the software requirement is clearly > inapplicable in some cases - for instance I'm not suggesing that > a validation suite be required for the DocStd document itself. > So some concession about applicability should be included in the > discussion of requirements. > > I'm happy to hear other people's views on this ... > > Mark From mjg at cacr.caltech.edu Thu Jul 2 12:19:23 2009 From: mjg at cacr.caltech.edu (Matthew Graham) Date: Thu, 2 Jul 2009 12:19:23 -0700 Subject: Versioning Message-ID: <833254ED-BA5D-4D10-8536-46DAFEBE4989@cacr.caltech.edu> Hi, This is a discussion that I started within my WG just to solicit opinion. Can the Standards WG please clarify: The response was posted to my comments on the RFC page: "The Committee agrees that the version numbering scheme is challenging when dealing with namespaces and WSDL, and leads to the conclusion that when IVOA standards describe web services or have associated XML schemas, with namespaces that when changed, cause software to break, then these changes must both be accompanied by an increment to the integer part of the document and the associated "supplementary" files. This would not affect most of the standards documents, and should not present any real logistical difficulty, as there are a sufficient number of integers available to support any number of revisions. " I explain: This has greatest impact for this working group (GWS but I am also cross-posting to Semantics) and essentially means that ALL (WD, PR, etc) versions of our specs with WSDL/XML/RDF documents (anything with a namespace) will only carry integer versions. So, for example, the progression of VOSpace 2.0 would actually proceed as: VOSpace 2 (first WD) VOSpace 3 (second WD) VOSpace 4 (third WD) VOSpace 5 (first PR) VOSpace 6 (second PR) VOSpace 7 (final PR) VOSpace 8 (REC) The next version would then VOSpace 9, etc. Is this a valid interpretation of the spec? The standards documents says: "The number to the left of the (first) decimal point starts with 0 for documents that are being discussed within a Working Group prior to publication for IVOA-wide review. The number increments to 1 for the first public version, and to 2, 3, ..., for subsequent versions that are not backward compatible and/or require substantial revisions to implementations." The definition of a Working Draft is: "A Working Draft is published at the discretion of a Working Group once the WG is satisfied that the document is sufficiently developed to merit broader exposure and feedback" So I think that a Working Draft fulfils the "IVOA-wide review" criterion. The alternate suggestion from Dave Morris and with much support is: If the integer rule only applied to something that has been accepted as a recommendation. VOSpace 2.0-yyymmdd (first WD) VOSpace 2.0-yyymmdd (second WD) VOSpace 2.0-yyymmdd (third WD) VOSpace 2.0-yyymmdd (first PR) VOSpace 2.0-yyymmdd (second PR) VOSpace 2.0-yyymmdd (final PR) VOSpace 2.0 (REC) VOSpace 2.1-yyymmdd (first WD) (minor text changes only) VOSpace 2.1-yyymmdd (final PR) (minor text changes only) VOSpace 2.1 (REC) VOSpace 3.0-yyymmdd (first WD) (changes to service behavior or xml schema) ... I think this needs to be clarified one way or the other. Cheers, Matthew From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From paul.harrison at manchester.ac.uk Thu Jul 2 12:48:01 2009 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Thu, 2 Jul 2009 20:48:01 +0100 Subject: Versioning In-Reply-To: <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> Message-ID: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Hi All, During the discussion that took place on this before the latest Standards document the consensus was that for a document name of the form WD-VOSpace-2.0-yyyymmdd * That the 2.0 was the version number of the "protocol/standard" * yyyymmdd was effectively the version number of the document What was not explicitly thought about is the progression from a version 1.0 to a version 2.0 of a protocol, - in this case the document numbering can only start and stay with 2.0 from first WD to REC - even though there might be substantial (protocol/interface) changes between the drafts (which would make the programmer in us want to change the version number). Only after reaching the REC status does the "2.0" number have to change. What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. Under this numbering scheme, what is currently VOTable 1.2 should really be VOTable 2.0 in my opinion, as it causes VOTable 1.1 xml to be invalid, even though the changes in the written document look quite minimal. However, I am not sure who will police this sort of issue..... Cheers, Paul. On 2009-07 -02, at 19:07, Matthew Graham wrote: > Hi Doug, > > The standards documents says: > > "The number to the left of the (first) decimal point starts with 0 > for documents that are being discussed within a Working Group prior > to publication for IVOA-wide review. The number increments to 1 for > the first public version, and to 2, 3, ..., for subsequent versions > that are not backward compatible and/or require substantial > revisions to implementations." > > The definition of a Working Draft is: > > "A Working Draft is published at the discretion of a Working Group > once the WG is satisfied that the document is sufficiently developed > to merit broader exposure and feedback" > > So I think that a Working Draft fulfils the "IVOA-wide review" > criterion. > > If the integer progression is only a prerequisite for REC versions > then the standards document needs to be amended to say this. > > Cheers, > > Matthew > > > On Jul 2, 2009, at 10:52 AM, Doug Tody wrote: > >> If this were true I agree it would be crazy. But is a WD or PR a >> "standard"? I should think this would only apply to established >> standards that are already deployed. That is, to recommendations. >> Perhaps the documents and standards folks should clarify. >> >> Also, I don't think this is just limited to GWS; any DAL or DM >> standard >> for example might also have to deal with this issue. >> >> - Doug >> >> >> On Thu, 2 Jul 2009, Matthew Graham wrote: >> >>> Hi, >>> >>> You might be aware that the Document Standards v1.2 PR has just >>> completed its RFC (http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/DocStdRFC2 >>> ). I raised concerns about the proposed versioning scheme: >>> >>> "The document now states that there is an integer increment in the >>> version number in the case where subsequent versions are not >>> backward compatible. " >>> >>> and the response that has been posted is: >>> >>> "The Committee agrees that the version numbering scheme is >>> challenging when dealing with namespaces and WSDL, and leads to >>> the conclusion that when IVOA standards describe web services or >>> have associated XML schemas, with namespaces that when changed, >>> cause software to break, then these changes must both be >>> accompanied by an increment to the integer part of the document >>> and the associated "supplementary" files. This would not affect >>> most of the standards documents, and should not present any real >>> logistical difficulty, as there are a sufficient number of >>> integers available to support any number of revisions. " >>> >>> This has greatest impact for this working group (GWS but I am also >>> cross-posting to Semantics) and essentially means that ALL (WD, >>> PR, etc) versions of our specs with WSDL/XML/RDF documents >>> (anything with a namespace) will only carry integer versions. >>> >>> So, for example, the progression of VOSpace 2.0 would actually >>> proceed as: >>> >>> VOSpace 2 (first WD) >>> VOSpace 3 (second WD) >>> VOSpace 4 (third WD) >>> VOSpace 5 (first PR) >>> VOSpace 6 (second PR) >>> VOSpace 7 (final PR) >>> VOSpace 8 (REC) >>> >>> The next version would then VOSpace 9, etc. >>> >>> Although this is very much a procedural issue, I just wanted to >>> flag it so that everyone is aware and happy with it before I >>> approve. >>> >>> Cheers, >>> >>> Matthew >>> >> > From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From seaman at noao.edu Thu Jul 2 13:14:26 2009 From: seaman at noao.edu (Rob Seaman) Date: Thu, 2 Jul 2009 13:14:26 -0700 Subject: Versioning In-Reply-To: <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> References: <4137BEF6-99BC-4E8D-9B26-B001C9A604E3@cacr.caltech.edu> <7DCCB579-8289-4AF2-8EDE-C0EAA5A080F7@cacr.caltech.edu> <7E91E847-3F8F-44F1-A59D-FB4588819819@manchester.ac.uk> Message-ID: On Jul 2, 2009, at 12:48 PM, Paul Harrison wrote: > During the discussion that took place on this before the latest > Standards document the consensus was that for a document name of the > form > > WD-VOSpace-2.0-yyyymmdd > > * That the 2.0 was the version number of the "protocol/standard" > * yyyymmdd was effectively the version number of the document > > What was not explicitly thought about is the progression from a > version 1.0 to a version 2.0 of a protocol, - in this case the > document numbering can only start and stay with 2.0 from first WD to > REC - even though there might be substantial (protocol/interface) > changes between the drafts This distinction between version of the standard and version of the document maps well onto the process used with VOEvent. I have about 50 sequential (and some parallel) versions of the document that eventually turned into v1.0 of the standard. The first few versions were named in a rather ad hoc manner until it became clear that one has to say just these two things with the nomenclature from the very first draft: - What standard am I working on? - What version of the document is this? Otherwise you find yourself inadvertently incrementing the version of the standard, when what you are really doing is just revising a draft description of the same final product. It would be silly to require the version of the standard to be incremented with every notion thrashed out by a working group while reaching consensus. Which is to say that assigning a version number to a new standard is something that occurs at the beginning of the development/revision process, not just as a final swat on its tush. Incrementing by integers has the main advantage of making progress in the IVOA appear to be occurring ten times as rapidly. (It's also more- or-less the industry norm.) Rob From hanisch at stsci.edu Thu Jul 2 16:27:04 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Thu, 02 Jul 2009 19:27:04 -0400 Subject: versioning and WSDL and namespaces, etc. Message-ID: Following a bit of side discussion with Matthew to make sure I understand the issue, I think we can resolve the problem if we consider "software breakage" in the sense of manual recoding. Changes to WSDL or XML namespaces typically require regeneration of bindings and recompilation of code, but this is all (supposed) to be more or less automatic. So, with some clarification to the text, the implication would be that the integer part of the version remains fixed -- it is notional and describes the intention -- and updates to WSDL, etc., coincide with increments to the part of the version number to the right of the decimal point. As for Paul's comment: What should be removed from the Standards document is the idea of a public WD that has a 0.x number, as this is logically inconsistent with the new scheme - the first public WD of a new protocol/standard should have a 1.0 number - the new numbering scheme was intended to remove this ambiguity, but I agree that the Standards document still does not make it clear. What the document says is ? The number to the left of the (first) decimal point starts with 0 for documents that are being discussed within a Working Group prior to publication for IVOA-wide review. The number increments to 1 for the first public version, and to 2, 3, ..., for subsequent versions that are not backward compatible and/or require substantial revisions to implementations. Thus, I do not see why there is confusion on this point. Bob From m.b.taylor at bristol.ac.uk Fri Jul 3 02:53:14 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Fri, 3 Jul 2009 10:53:14 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: Bob, On Thu, 2 Jul 2009, Robert Hanisch wrote: > First, let me quote myself in the e-mail message I sent earlier today: > > "One might argue that if one or more VO projects invests the efforts into > developing the standard itself, it follows that they would put effort into > implementations and validators. But this could put a lot of pressure on > schedule and resources of individual projects, and could lead to major > delays in the promotion process." > > I agree completely with you that doing implementations in parallel with > standards development is the best way to go, with the obvious exception that > some of the IVOA standards, as you point out, are not really about software > implementations. > > But making the implementations and the validator mandatory for promotion is > where I, and Christophe and Francoise, feel it is necessary to pause. For > example, if right now we were to enforce this requirement on TAP, with the > resource constraints we have in several of the major projects, we would be > forced to stop the promotion process. This would be an awful situation for > IVOA, I think. We want prototypes to inform the standards development > process, for sure, we SHOULD have them, but from a management perspective I > think we have to avoid boxing ourselves into a corner. I agree with you that requiring the associated software would slow down the promotion process. However, as I tried to make clear in my previous message, I don't believe that the date of publication of the Recommendation is what we should be focussing on. When a document reaches Recommendation status, yes it looks good on Gantt charts, but all it means is we have a bit of paper to wave in the air. Nothing useful has been achieved until effective use of that standard is made, which in most cases requires software to be written. I believe that attaining that second goal will actually be accelerated rather than decelerated if the software is written in tandem with the text (I tried to get you to agree or disagree with this statement in my last message, but I still don't know your opinion). Doing them in parallel will also lead to a better standard. In any case: if you can't find anyone who is prepared to implement the standard, there's probably a good reason; either it's too hard to do, or it's not something that people need. In either case it's no great achievement to have published it. > It is also a bit ambiguous just how one proves that implementations are > interoperable. So far we have taken people's word, but IVOA has no means to > independently test these assertions. Agreed it's ambiguous, and rigorous enforcement is not really feasible. I'm prepared to rely on the good will of the authors/implementors, supplemented by whatever scrutiny other IVOA members are prepared to provide on an ad-hoc basis. > I would probably agree fully with you if, say, IVOA were a dues-based > membership organization, with its own financial resources to build > validators, or test those developed by others. But I see a distinction, > albeit a subtle one, between the projects' willingness to contribute to > development of a standard and their ability to provide supporting software > and validators on a timescale that does not jeopardize the promotion > process. Again - I believe that the timescale of the promotion process itself is a red herring. That given, I don't see the subtle difference here (management is neither my enthusiasm nor my expertise, so I'm quite prepared to believe there are issues over my head here) - could you elaborate a bit? But I'll say this, I think it is somewhat dangerous for the system to encourage or allow organisations to shove their oar in when it comes to writing requirements, but not when it comes to resourcing their implementation. > Finally, in any case, we have a revision system that is there to deal with > problems and shortcomings found in earlier versions. If V1.0 of a standard > ends up having problems -- unsupportable features, inconsistencies, etc. > (and we have had this experience) -- then we fix the problems in V1.x or > V2.0. My understanding of the basic intention for the versioning system is that version X+delta is version X with the additions or changes gained from working with version X for a while, as well as enhancements which weren't feasible (e.g. from lack of resources) on the timescale of version X. While it can also be used for patching up things which were broken or ill-thought-out in the first place, it's not very well designed for that purpose, if only because of the very long timescales typically involved. > (and we have had this experience) -- then we fix the problems in V1.x or and we'd have had the experience a lot less if software was written in tandem with the original versions! Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Fri Jul 3 05:35:07 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Fri, 03 Jul 2009 08:35:07 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: On 7/3/09 5:53 AM, "Mark Taylor" wrote: > Bob, > > On Thu, 2 Jul 2009, Robert Hanisch wrote: > >> First, let me quote myself in the e-mail message I sent earlier today: >> >> "One might argue that if one or more VO projects invests the efforts into >> developing the standard itself, it follows that they would put effort into >> implementations and validators. But this could put a lot of pressure on >> schedule and resources of individual projects, and could lead to major >> delays in the promotion process." >> >> I agree completely with you that doing implementations in parallel with >> standards development is the best way to go, with the obvious exception that >> some of the IVOA standards, as you point out, are not really about software >> implementations. >> >> But making the implementations and the validator mandatory for promotion is >> where I, and Christophe and Francoise, feel it is necessary to pause. For >> example, if right now we were to enforce this requirement on TAP, with the >> resource constraints we have in several of the major projects, we would be >> forced to stop the promotion process. This would be an awful situation for >> IVOA, I think. We want prototypes to inform the standards development >> process, for sure, we SHOULD have them, but from a management perspective I >> think we have to avoid boxing ourselves into a corner. > > I agree with you that requiring the associated software would slow down > the promotion process. However, as I tried to make clear in my previous > message, I don't believe that the date of publication of the > Recommendation is what we should be focussing on. > When a document reaches Recommendation status, yes it looks good on > Gantt charts, but all it means is we have a bit of paper to wave > in the air. Nothing useful has been achieved until effective use of > that standard is made, which in most cases requires software to be > written. I believe that attaining that second goal will actually > be accelerated rather than decelerated if the software is written > in tandem with the text (I tried to get you to agree or disagree with > this statement in my last message, but I still don't know your opinion). > Doing them in parallel will also lead to a better standard. > > In any case: if you can't find anyone who is prepared to implement > the standard, there's probably a good reason; either it's too hard > to do, or it's not something that people need. In either case > it's no great achievement to have published it. I disagree with you on this point, and perhaps that is the crux of the issue. Many organizations will not even look at an IVOA standard until it has been finalized. They say "why should I bother implementing something that is just going to change." Plus, like it or not, just reaching agreement on the standard IS a major milestone for our development projects. I don't know what is faster overall ... you are probably right, that when the standard is accompanied by two or more implementations, it is more likely to be correct and implementable by others outside of the core development team. But we also all work to external expectations (from our funding agencies), and I have heard many times now comments from people at data centers saying they are not going to code up an interface because it is still a WD. Please keep in mind that many of the implementors of IVOA standards are not part of the VO development teams. >> It is also a bit ambiguous just how one proves that implementations are >> interoperable. So far we have taken people's word, but IVOA has no means to >> independently test these assertions. > > Agreed it's ambiguous, and rigorous enforcement is not really feasible. > I'm prepared to rely on the good will of the authors/implementors, > supplemented by whatever scrutiny other IVOA members are prepared > to provide on an ad-hoc basis. We have no choice at the moment! >> I would probably agree fully with you if, say, IVOA were a dues-based >> membership organization, with its own financial resources to build >> validators, or test those developed by others. But I see a distinction, >> albeit a subtle one, between the projects' willingness to contribute to >> development of a standard and their ability to provide supporting software >> and validators on a timescale that does not jeopardize the promotion >> process. > > Again - I believe that the timescale of the promotion process itself > is a red herring. That given, I don't see the subtle difference here > (management is neither my enthusiasm nor my expertise, so I'm quite > prepared to believe there are issues over my head here) - could you > elaborate a bit? As above. Agreement on these standards is a major milestone. Read any of the NVO Quarterly Reports to NSF (all are on the NVO web page, in the document collection) and you can see how important these are in terms of deliverables. > But I'll say this, I think it is somewhat dangerous for the system > to encourage or allow organisations to shove their oar in when it > comes to writing requirements, but not when it comes to resourcing > their implementation. This is not unique to the IVOA process! Everyone is eager to tell you what needs to be done, as long as someone else actually does it! >> Finally, in any case, we have a revision system that is there to deal with >> problems and shortcomings found in earlier versions. If V1.0 of a standard >> ends up having problems -- unsupportable features, inconsistencies, etc. >> (and we have had this experience) -- then we fix the problems in V1.x or >> V2.0. > > My understanding of the basic intention for the versioning system > is that version X+delta is version X with the additions or changes > gained from working with version X for a while, as well as > enhancements which weren't feasible (e.g. from lack of resources) > on the timescale of version X. While it can also be used for > patching up things which were broken or ill-thought-out in the > first place, it's not very well designed for that purpose, if only > because of the very long timescales typically involved. Which will be longer if we make interoperable implementations and validators mandatory at every step of the process. >> (and we have had this experience) -- then we fix the problems in V1.x or > > and we'd have had the experience a lot less if software was written > in tandem with the original versions! > > Mark Bob From m.b.taylor at bristol.ac.uk Fri Jul 3 06:51:10 2009 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Fri, 3 Jul 2009 14:51:10 +0100 (BST) Subject: IVOA Document Standards RFC V1.2 In-Reply-To: References: Message-ID: On Fri, 3 Jul 2009, Robert Hanisch wrote: > >Bob wrote: > >> But making the implementations and the validator mandatory for promotion is > >> where I, and Christophe and Francoise, feel it is necessary to pause. For > >> example, if right now we were to enforce this requirement on TAP, with the > >> resource constraints we have in several of the major projects, we would be > >> forced to stop the promotion process. This would be an awful situation for > >> IVOA, I think. We want prototypes to inform the standards development > >> process, for sure, we SHOULD have them, but from a management perspective I > >> think we have to avoid boxing ourselves into a corner. > > > Mark wrote: > > I agree with you that requiring the associated software would slow down > > the promotion process. However, as I tried to make clear in my previous > > message, I don't believe that the date of publication of the > > Recommendation is what we should be focussing on. > > When a document reaches Recommendation status, yes it looks good on > > Gantt charts, but all it means is we have a bit of paper to wave > > in the air. Nothing useful has been achieved until effective use of > > that standard is made, which in most cases requires software to be > > written. I believe that attaining that second goal will actually > > be accelerated rather than decelerated if the software is written > > in tandem with the text (I tried to get you to agree or disagree with > > this statement in my last message, but I still don't know your opinion). > > Doing them in parallel will also lead to a better standard. > > > > In any case: if you can't find anyone who is prepared to implement > > the standard, there's probably a good reason; either it's too hard > > to do, or it's not something that people need. In either case > > it's no great achievement to have published it. > I disagree with you on this point, and perhaps that is the crux of the > issue. Many organizations will not even look at an IVOA standard until it > has been finalized. They say "why should I bother implementing something > that is just going to change." Plus, like it or not, just reaching > agreement on the standard IS a major milestone for our development projects. > I don't know what is faster overall ... you are probably right, that when > the standard is accompanied by two or more implementations, it is more > likely to be correct and implementable by others outside of the core > development team. But we also all work to external expectations (from our > funding agencies), and I have heard many times now comments from people at > data centers saying they are not going to code up an interface because it is > still a WD. Please keep in mind that many of the implementors of IVOA > standards are not part of the VO development teams. That's a fair point as stated. However, my impression (not backed up by evidence, and possibly incorrect - data welcomed) is that the first implementations for the standards that we're talking about are in fact by VO/IVOA insiders. Non-IVOA data centers in practice don't wait until the REC acceptance date and then start implementing; they wait until it looks like it's going to be a standard which is actually being used, or going to get used, by other people and then start implementing. This latter only happens after somebody else (most likely an IVOA insider) has provided an implementation of some kind. > >> I would probably agree fully with you if, say, IVOA were a dues-based > >> membership organization, with its own financial resources to build > >> validators, or test those developed by others. But I see a distinction, > >> albeit a subtle one, between the projects' willingness to contribute to > >> development of a standard and their ability to provide supporting software > >> and validators on a timescale that does not jeopardize the promotion > >> process. > > > > Again - I believe that the timescale of the promotion process itself > > is a red herring. That given, I don't see the subtle difference here > > (management is neither my enthusiasm nor my expertise, so I'm quite > > prepared to believe there are issues over my head here) - could you > > elaborate a bit? > As above. Agreement on these standards is a major milestone. Read any of > the NVO Quarterly Reports to NSF (all are on the NVO web page, in the > document collection) and you can see how important these are in terms of > deliverables. To some extent, this is what I was goading you to say. I do concede that there are political considerations here which will affect what decisions we take. When I said above that nothing useful is achieved by the standard publication per se, I should really have said nothing *technically* useful has been achieved - I admit that there may be some (maybe, a lot of) political advantage. If we're going to determine the standards process on that basis however, I'd like to have it out in the open amongst ourselves rather than to pretend that the StdDoc text is motivated purely by the considerations of technical excellence and technical pragmatism. If we admit that, maybe there's another way forward: declare an intermediate document stage of "accepted in principle, no major changes expected, but revisions possible on the basis of implementation experience". This stage could be sold as the one that counts to the funding agencies. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ From hanisch at stsci.edu Fri Jul 3 15:43:49 2009 From: hanisch at stsci.edu (Robert Hanisch) Date: Fri, 03 Jul 2009 18:43:49 -0400 Subject: IVOA Document Standards RFC V1.2 In-Reply-To: Message-ID: > That's a fair point as stated. However, my impression (not backed up > by evidence, and possibly incorrect - data welcomed) is that the first > implementations for the standards that we're talking about are in > fact by VO/IVOA insiders. Non-IVOA data centers in practice don't > wait until the REC acceptance date and then start implementing; > they wait until it looks like it's going to be a standard which > is actually being used, or going to get used, by other people > and then start implementing. This latter only happens after > somebody else (most likely an IVOA insider) has provided an > implementation of some kind. I don't have very complete information on this -- mostly anecdotal evidence. Most developers like to start with something already existing; others eschew what's been done and start from scratch. >>>> I would probably agree fully with you if, say, IVOA were a dues-based >>>> membership organization, with its own financial resources to build >>>> validators, or test those developed by others. But I see a distinction, >>>> albeit a subtle one, between the projects' willingness to contribute to >>>> development of a standard and their ability to provide supporting software >>>> and validators on a timescale that does not jeopardize the promotion >>>> process. >>> >>> Again - I believe that the timescale of the promotion process itself >>> is a red herring. That given, I don't see the subtle difference here >>> (management is neither my enthusiasm nor my expertise, so I'm quite >>> prepared to believe there are issues over my head here) - could you >>> elaborate a bit? >> As above. Agreement on these standards is a major milestone. Read any of >> the NVO Quarterly Reports to NSF (all are on the NVO web page, in the >> document collection) and you can see how important these are in terms of >> deliverables. > > To some extent, this is what I was goading you to say. I do concede > that there are political considerations here which will affect what > decisions we take. When I said above that nothing useful is achieved > by the standard publication per se, I should really have said nothing > *technically* useful has been achieved - I admit that there may be > some (maybe, a lot of) political advantage. If we're going to determine > the standards process on that basis however, I'd like to have > it out in the open amongst ourselves rather than to pretend that the > StdDoc text is motivated purely by the considerations of technical > excellence and technical pragmatism. If we admit that, maybe there's > another way forward: declare an intermediate document stage of > "accepted in principle, no major changes expected, but revisions > possible on the basis of implementation experience". This stage > could be sold as the one that counts to the funding agencies. It was understood from the beginning of IVOA that standards would evolve and that first versions would likely need to change. I found something I wrote back in March 2004, concerning the registry standards, that I think remains relevant today: We need to reach agreements quickly, build to them, and iterate to improve them. We need to be willing to build and revise, or even in some cases throw away and start again. We have set up a standards process that recognizes change, and is intended to be responsive to change. I do not believe we can afford to set 2-3 year schedules for reaching consensus. If it takes this long, the community upon which we will utimately depend for support will see us as purveyors of snake-oil, and will blow us off. - - - - - All this said, I think we will update the document to make the implementations and validator strongly/highly recommended, with some reason needing to be provided if these have not been done in order to move the standard ahead. I still think we do not want to go with mandatory, for the reasons explained in the past several messages. Off now for our independence holiday (which started today, actually). cheers, Bob