From amicol.ivoa at googlemail.com  Thu Dec 15 05:42:50 2011
From: amicol.ivoa at googlemail.com (Alberto Micol)
Date: Thu, 15 Dec 2011 14:42:50 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
Message-ID: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>


Dear All,

On 2 Sep 2011, at 01:40, Douglas Tody wrote:

> This would be a fine solution, considering only STC; however it is
> perfectly legal for a DM to be based upon other models (in a way  
> defined
> by the new model such as Char or SIA) and define new, more
> application-specific Utypes.  There is an especially strong argument  
> for
> this for essential elements of the new model so that we have a well
> defined and self consistent model.  "Falling through" to reference
> arbitrary elements of another model, with its own externally defined
> namespace, is fine for extension but could significantly expand the
> complexity of the new model as a client application might then need to
> fully understand multiple models to be able to deal with the data.

There are two kinds of clients: it seems to me that Doug here above
refers to client applications that deal with a specific data model,
while what I'm going to describe is the case of a generic client.

Let me consider a well known client application (Aladin), and four quite
common use cases:

- As a user, I want to take the VOTable generated by a ObsTAP service
and display the contained footprints.

- As a user, I want to take the VOTable generated by a SIAP service
and display the contained footprints.

- As a user, I want to take the VOTable generated by a CharDM  
compliant service,
and display the contained footprints.

- As a user, I want to take the VOTable generated by an archive query  
form
and display the therein contained footprints


What is the magic bit that will allow Aladin to recognise a given  
FIELD as a footprint?

It cannot be the UTYPE, if every data model uses a different UTYPE for  
a footprint.

It cannot be the FIELD name, given that only one of the three services  
above (ObsTAP)
guarantees the name to always be "s_region".

Could it be a new UCD?
(Btw, when is that a client should look for a UCD, and when for a  
UTYPE, to know what to do?)

The only logical possibilities I see, with different levels of  
technical or political difficulty to implement,
are:

1.- to standardise the footprint UTYPE across all data models
2.- to standardise the FIELD name of a footprint field, a la ObsTAP
3.- to create a new UCD for a footprint
4.- to use xtype="adql:Region" (is that acceptable? xtype is really  
meant to represent the storage units)


Would you suggest other ways, or the "good enough" way is already  
listed above?

Note: In the years we have had the same discussion again and again,  
this is just
the first time it happens for footprints. It would be nice to solve  
the originating problem
not just the footprint problem.

Alberto

PS: I would love to suggest a 5th possibility, that is, to create a  
new FIELD attribute, called stdname,
    which hosts an IVOA standard name for all those quantities that  
are really common across
    all data models, e.g. "footprint", "telescope", "instrument", etc.,
    as in <FIELD name="myFootprint" stdname="footprint"  
utype="u.name.it" xtype="adql:Region">,
    but I won't,
    unless asked.
;-)


>  In
> the case here, Char defines strong and consistent concepts for the
> measurement axes, Coverage, Support, etc. which are important to the
> Char model, and merely uses STC-S to represent the datum in question.
>
> Regarding UCDs for representing footprints in SIA-1, this can only be
> done by extension and there is no standard way to do it at present.   
> We
> really need SIAV2 to address this sort of concern.
>
> 	- Doug
>
>
> On Thu, 1 Sep 2011, Arnold Rots wrote:
>
>> Here is what, in my view, is the correct utype for a footprint  
>> region:
>>
>>    stc:ObservationLocation.AstroCoordArea.Region
>>
>> Pretty simple; though one might quibble whether it should be:
>>
>>    stc:ObsDataLocation.ObservationLocation.AstroCoordArea.Region
>>
>>
>> - Arnold
>>
>>
>> Francois Bonnarel wrote:
>> [ Charset ISO-8859-1 unsupported, converting... ]
>>> Hi Thomas,
>>>    To complete my answer to Markus
>>>     we could have something like
>>>
>>>    <FIELD name="fov" unit="deg" xtype="STC-s" datatype="char"
>>> arraysize="*" utype="sia:Char.SpatialAxis.Coverage.Support.Area" >
>>>
>>>       I think something like this can be proposed (if not yet the  
>>> case)
>>> in SIA2...
>>>       this will be consistent with ObsTap... and could also be  
>>> included
>>> in the next version of SSA et al.
>>>
>>>       For current SIA services, maybe we could have a short
>>> recommendation or note to propose using
>>> the same as an additional field in the SIA1 Query response
>>> Cheers
>>> Fran?ois
>>>> To illustrate my request, here is is how the STC-S FIELD is  
>>>> described
>>>> in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a  
>>>>> forecoming
>>>>> revision of the SIAP document, so that clients can easily find out
>>>>> which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be  
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming  
>>>>>>> more
>>>>>>> and more widespread, and we would like to support this feature  
>>>>>>> in
>>>>>>> Aladin.
>>>>>>> I would like to know if there is a standard (or de facto  
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region  
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> To illustrate my request, here is is how the STC-S FIELD is  
>>>> described
>>>> in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a  
>>>>> forecoming
>>>>> revision of the SIAP document, so that clients can easily find out
>>>>> which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be  
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming  
>>>>>>> more
>>>>>>> and more widespread, and we would like to support this feature  
>>>>>>> in
>>>>>>> Aladin.
>>>>>>> I would like to know if there is a standard (or de facto  
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region  
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> Thomas Boch a ?crit :
>>>> To illustrate my request, here is is how the STC-S FIELD is  
>>>> described
>>>> in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a  
>>>>> forecoming
>>>>> revision of the SIAP document, so that clients can easily find out
>>>>> which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be  
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming  
>>>>>>> more
>>>>>>> and more widespread, and we would like to support this feature  
>>>>>>> in
>>>>>>> Aladin.
>>>>>>> I would like to know if there is a standard (or de facto  
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region  
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> = 
>>> ====================================================================
>>> Fran?ois   Bonnarel           Observatoire Astronomique de  
>>> Strasbourg
>>> CDS (Centre de donn?es        11, rue de l'Universit?
>>> astronomiques de Strasbourg)  F--67000 Strasbourg (France)
>>>
>>> Tel: +33-(0)3 68 85 24 11     WWW: http://cdsweb.u-strasbg.fr/people/fb.html
>>> Fax: +33-(0)3 68 85 24 25     E-mail: francois.bonnarel at astro.unistra.fr
>>> ---------------------------------------------------------------------
>>>
>> --------------------------------------------------------------------------
>> Arnold H. Rots                                Chandra X-ray Science  
>> Center
>> Smithsonian Astrophysical Observatory                tel:  +1 617  
>> 496 7701
>> 60 Garden Street, MS 67                              fax:  +1 617  
>> 495 7356
>> Cambridge, MA 02138                             arots at head.cfa.harvard.edu
>> USA                                     http://hea-www.harvard.edu/~arots/
>> --------------------------------------------------------------------------
>>


From msdemlei at ari.uni-heidelberg.de  Thu Dec 15 08:59:29 2011
From: msdemlei at ari.uni-heidelberg.de (Markus Demleitner)
Date: Thu, 15 Dec 2011 17:59:29 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
Message-ID: <20111215165929.GA1457@ari.uni-heidelberg.de>

Hi all,

On Thu, Dec 15, 2011 at 02:42:50PM +0100, Alberto Micol wrote:
> There are two kinds of clients: it seems to me that Doug here above
> refers to client applications that deal with a specific data model,
> while what I'm going to describe is the case of a generic client.
D'accord -- by the way, even on the server side, not having three
different representations for the same thing is nice once you're
speaking more than one protocol or you're delivering VOTables to,
say, HTML forms and still want to define your STCs.

> What is the magic bit that will allow Aladin to recognise a given
> FIELD as a footprint?
> 
> It cannot be the UTYPE, if every data model uses a different UTYPE
> for a footprint.
Yes it can, and it should be.  Sorry if I'm getting on your nerves,
but let me again promote the group/fieldref mechanism.

Using it, a single FIELD can fill as many roles as you want; that is,
you can define its role in char *and* whatever fancy data model you
want to support in addition.  In particular, you can have the
generic designation as a footprint, using, say, the char model:

<GROUP utype="char:SpatialAxis">
  <FIELDref utype="char:SpatialAxis.Coverage.Support.Area"
    ref="footprint"/>
</GROUP>

(in reality, I'd say let's agree on obscore types for this kind of
thing, but char would be somewhat more logical) and *at the same
time* use that column to fill some role in another data model, say:

<GROUP utype="stc:CatalogEntryLocation">
  <FIELDref utype="stc:AstroCoordArea.Circle" ref="footprint"/>
</GROUP>

So, both clients that understand char and clients that understand
stc can make sense of that field within what those models give them.
If we just said "if you have an observation, please include a group
linking your data to the obscore model to the extent possible" and
"please include a group defining your coordinate systems whenever
you're talking about coordinates" in some appropriate place (the
VOTable spec?), IMHO we'd have made quite a step towards relatively
simple but useful metadata for a large proportion of the astromonical
data out there.

Sorry to repeat that again -- but I still feel we're trying to solve
something here that's already solved ever since FIELDref and
PARAMref were introduced to VOTable.

Cheers,

      Markus


From amicol.ivoa at googlemail.com  Thu Dec 15 09:28:59 2011
From: amicol.ivoa at googlemail.com (Alberto Micol)
Date: Thu, 15 Dec 2011 18:28:59 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <20111215165929.GA1457@ari.uni-heidelberg.de>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<20111215165929.GA1457@ari.uni-heidelberg.de>
Message-ID: <824543C2-E236-4A2A-93B8-B3E9513BA9C3@googlemail.com>


> Hi all,
>
> On Thu, Dec 15, 2011 at 02:42:50PM +0100, Alberto Micol wrote:
>> There are two kinds of clients: it seems to me that Doug here above
>> refers to client applications that deal with a specific data model,
>> while what I'm going to describe is the case of a generic client.
> D'accord -- by the way, even on the server side, not having three
> different representations for the same thing is nice once you're
> speaking more than one protocol or you're delivering VOTables to,
> say, HTML forms and still want to define your STCs.

Yes!

>
>> What is the magic bit that will allow Aladin to recognise a given
>> FIELD as a footprint?
>>
>> It cannot be the UTYPE, if every data model uses a different UTYPE
>> for a footprint.
> Yes it can, and it should be.  Sorry if I'm getting on your nerves,
> but let me again promote the group/fieldref mechanism.
>

No, you are not getting onto my nerves, I hope you will say the same  
after
reading this email...

Yes, what you proposed and propose is a possible solution, but
it is not elegant, nor efficient.
Instead of unifying concepts, you propose to multiply FIELDrefs.

One would have to potentially introduce a FIELDref as many times as  
there are data models.

And that should happen for all the fields that are potentially in common
among the selected data models.

Quite a useless and inefficient multiplication of FIELDrefs.
And a multiplication that is bound to grow in the future when
e.g. SIAP2 will be out, and then will come the SED thingy,
and so on and so forth.

Basically my super-duper archive service will have to know about all  
possible models,
and will have to change every time a new model comes out.

My point is that a footprint is a footprint,
no matter which protocol is used. The concept "footprint" is super  
partes.

And definitively I want my service to also be super partes
to reduce maintenance, and because simpler is better.

Alberto



On 15 Dec 2011, at 17:59, Markus Demleitner wrote:


> Using it, a single FIELD can fill as many roles as you want; that is,
> you can define its role in char *and* whatever fancy data model you
> want to support in addition.  In particular, you can have the
> generic designation as a footprint, using, say, the char model:
>
> <GROUP utype="char:SpatialAxis">
>  <FIELDref utype="char:SpatialAxis.Coverage.Support.Area"
>    ref="footprint"/>
> </GROUP>
>
> (in reality, I'd say let's agree on obscore types for this kind of
> thing, but char would be somewhat more logical) and *at the same
> time* use that column to fill some role in another data model, say:
>
> <GROUP utype="stc:CatalogEntryLocation">
>  <FIELDref utype="stc:AstroCoordArea.Circle" ref="footprint"/>
> </GROUP>
>
> So, both clients that understand char and clients that understand
> stc can make sense of that field within what those models give them.
> If we just said "if you have an observation, please include a group
> linking your data to the obscore model to the extent possible" and
> "please include a group defining your coordinate systems whenever
> you're talking about coordinates" in some appropriate place (the
> VOTable spec?), IMHO we'd have made quite a step towards relatively
> simple but useful metadata for a large proportion of the astromonical
> data out there.
>
> Sorry to repeat that again -- but I still feel we're trying to solve
> something here that's already solved ever since FIELDref and
> PARAMref were introduced to VOTable.
>
> Cheers,
>
>      Markus
>


From francois.bonnarel at astro.unistra.fr  Thu Dec 15 13:10:56 2011
From: francois.bonnarel at astro.unistra.fr (=?ISO-8859-1?Q?Fran=E7ois_Bonnarel?=)
Date: Thu, 15 Dec 2011 22:10:56 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
Message-ID: <4EEA6260.8050109@astro.unistra.fr>

Hi Alberto, all

Le 15/12/2011 14:42, Alberto Micol a ?crit :
>
> Dear All,
>
> On 2 Sep 2011, at 01:40, Douglas Tody wrote:
>
>> This would be a fine solution, considering only STC; however it is
>> perfectly legal for a DM to be based upon other models (in a way defined
>> by the new model such as Char or SIA) and define new, more
>> application-specific Utypes.  There is an especially strong argument for
>> this for essential elements of the new model so that we have a well
>> defined and self consistent model.  "Falling through" to reference
>> arbitrary elements of another model, with its own externally defined
>> namespace, is fine for extension but could significantly expand the
>> complexity of the new model as a client application might then need to
>> fully understand multiple models to be able to deal with the data.
>
> There are two kinds of clients: it seems to me that Doug here above
> refers to client applications that deal with a specific data model,
> while what I'm going to describe is the case of a generic client.
>
> Let me consider a well known client application (Aladin), and four quite
> common use cases:
>
> - As a user, I want to take the VOTable generated by a ObsTAP service 
> enerated by a CharDM compliant service
> and display the contained footprints.
>
> - As a user, I want to take the VOTable generated by a SIAP service
> and display the contained footprints.
Well I think the utype for footprint in ObsTAp , ie 
"obscore:Char.SpatialAxis.Coverage.Support.Area" will be 
"sia:Char.SpatialAxis.Coverage.Support.Area" in SIA2...
same classes and attribute in different namespaces ...

>
> - As a user, I want to take the VOTable generated by a CharDM 
> compliant service,
> and display the contained footprints.
>
A VOTABLE generated by a CharDM compliant service should have 
"Char.SpatialAxis.Coverage.Support.Area" as a name space for a 
footprint. Actually the Obstap utype is  a char utype in that case no ?
> - As a user, I want to take the VOTable generated by an archive query 
> form
> and display the therein contained footprints
>
>
> What is the magic bit that will allow Aladin to recognise a given 
> FIELD as a footprint?
>
> It cannot be the UTYPE, if every data model uses a different UTYPE for 
> a footprint.
>
In that case IVOA model should not use different utypes. See current 
discussion on utypes ....
> It cannot be the FIELD name, given that only one of the three services 
> above (ObsTAP)
> guarantees the name to always be "s_region".
>
yes
> Could it be a new UCD?
> (Btw, when is that a client should look for a UCD, and when for a 
> UTYPE, to know what to do?)
>
I dont' think so UCD is too fuzzy in general...
> The only logical possibilities I see, with different levels of 
> technical or political difficulty to implement,
> are:
>
> 1.- to standardise the footprint UTYPE across all data models
Yes, basically that's the direction we are going to.
>
> 2.- to standardise the FIELD name of a footprint field, a la ObsTAP
two much constraint on non-ObsTAP services
>
> 3.- to create a new UCD for a footprint
> 4.- to use xtype="adql:Region" (is that acceptable? xtype is really 
> meant to represent the storage units)
>
>
> Would you suggest other ways, or the "good enough" way is already 
> listed above?
>
> Note: In the years we have had the same discussion again and again, 
> this is just
> the first time it happens for footprints. It would be nice to solve 
> the originating problem
> not just the footprint problem.
Yes, see the utype current restart and finalizing effort (started in Pune)
>
> Alberto
Regards
Fran?ois
>
> PS: I would love to suggest a 5th possibility, that is, to create a 
> new FIELD attribute, called stdname,
>    which hosts an IVOA standard name for all those quantities that are 
> really common across
>    all data models, e.g. "footprint", "telescope", "instrument", etc.,
>    as in <FIELD name="myFootprint" stdname="footprint" 
> utype="u.name.it" xtype="adql:Region">,
>    but I won't,
>    unless asked.
> ;-)
>
>
>>  In
>> the case here, Char defines strong and consistent concepts for the
>> measurement axes, Coverage, Support, etc. which are important to the
>> Char model, and merely uses STC-S to represent the datum in question.
>>
>> Regarding UCDs for representing footprints in SIA-1, this can only be
>> done by extension and there is no standard way to do it at present.  We
>> really need SIAV2 to address this sort of concern.
>>
>>     - Doug
>>
>>
>> On Thu, 1 Sep 2011, Arnold Rots wrote:
>>
>>> Here is what, in my view, is the correct utype for a footprint region:
>>>
>>>    stc:ObservationLocation.AstroCoordArea.Region
>>>
>>> Pretty simple; though one might quibble whether it should be:
>>>
>>>    stc:ObsDataLocation.ObservationLocation.AstroCoordArea.Region
>>>
>>>
>>> - Arnold
>>>
>>>
>>> Francois Bonnarel wrote:
>>> [ Charset ISO-8859-1 unsupported, converting... ]
>>>> Hi Thomas,
>>>>    To complete my answer to Markus
>>>>     we could have something like
>>>>
>>>> <FIELD name="fov" unit="deg" xtype="STC-s" datatype="char"
>>>> arraysize="*" utype="sia:Char.SpatialAxis.Coverage.Support.Area" >
>>>>
>>>>       I think something like this can be proposed (if not yet the 
>>>> case)
>>>> in SIA2...
>>>>       this will be consistent with ObsTap... and could also be 
>>>> included
>>>> in the next version of SSA et al.
>>>>
>>>>       For current SIA services, maybe we could have a short
>>>> recommendation or note to propose using
>>>> the same as an additional field in the SIA1 Query response
>>>> Cheers
>>>> Fran?ois
>>>>> To illustrate my request, here is is how the STC-S FIELD is described
>>>>> in the query response of three different SIAP services :
>>>>>
>>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>>
>>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>>> ucd="phys.area;obs.field" unit="deg">
>>>>>
>>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS" datatype="char"
>>>>> arraysize="*">
>>>>>
>>>>> 3 different services, 3 different ways to express the same thing.
>>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>>> very helpful for client consuming these services.
>>>>> Cheers,
>>>>>
>>>>> Thomas
>>>>>
>>>>> Thomas Boch wrote:
>>>>>> Fran?ois,
>>>>>>
>>>>>> I wish this is something that could be standardized in a forecoming
>>>>>> revision of the SIAP document, so that clients can easily find out
>>>>>> which FIELD support the STC-S description.
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>> Fran?ois Bonnarel wrote:
>>>>>>> Hi Thomas,
>>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>>   I guess eventually for sia we will have something like
>>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>>   although the problem of the name space is still to be 
>>>>>>> discussed...
>>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>>> Cheers
>>>>>>> Fran?ois
>>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Embedding a STC region in the SIAP query response is becoming more
>>>>>>>> and more widespread, and we would like to support this feature in
>>>>>>>> Aladin.
>>>>>>>> I would like to know if there is a standard (or de facto standard)
>>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>>> description.
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> To illustrate my request, here is is how the STC-S FIELD is described
>>>>> in the query response of three different SIAP services :
>>>>>
>>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>>
>>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>>> ucd="phys.area;obs.field" unit="deg">
>>>>>
>>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS" datatype="char"
>>>>> arraysize="*">
>>>>>
>>>>> 3 different services, 3 different ways to express the same thing.
>>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>>> very helpful for client consuming these services.
>>>>> Cheers,
>>>>>
>>>>> Thomas
>>>>>
>>>>> Thomas Boch wrote:
>>>>>> Fran?ois,
>>>>>>
>>>>>> I wish this is something that could be standardized in a forecoming
>>>>>> revision of the SIAP document, so that clients can easily find out
>>>>>> which FIELD support the STC-S description.
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>> Fran?ois Bonnarel wrote:
>>>>>>> Hi Thomas,
>>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>>   I guess eventually for sia we will have something like
>>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>>   although the problem of the name space is still to be 
>>>>>>> discussed...
>>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>>> Cheers
>>>>>>> Fran?ois
>>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Embedding a STC region in the SIAP query response is becoming more
>>>>>>>> and more widespread, and we would like to support this feature in
>>>>>>>> Aladin.
>>>>>>>> I would like to know if there is a standard (or de facto standard)
>>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>>> description.
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>> Thomas Boch a ?crit :
>>>>> To illustrate my request, here is is how the STC-S FIELD is described
>>>>> in the query response of three different SIAP services :
>>>>>
>>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>>
>>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>>> ucd="phys.area;obs.field" unit="deg">
>>>>>
>>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS" datatype="char"
>>>>> arraysize="*">
>>>>>
>>>>> 3 different services, 3 different ways to express the same thing.
>>>>> Having a unique unambiguous way to detect the STC-S field would be
>>>>> very helpful for client consuming these services.
>>>>> Cheers,
>>>>>
>>>>> Thomas
>>>>>
>>>>> Thomas Boch wrote:
>>>>>> Fran?ois,
>>>>>>
>>>>>> I wish this is something that could be standardized in a forecoming
>>>>>> revision of the SIAP document, so that clients can easily find out
>>>>>> which FIELD support the STC-S description.
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>> Fran?ois Bonnarel wrote:
>>>>>>> Hi Thomas,
>>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>>   I guess eventually for sia we will have something like
>>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>>   although the problem of the name space is still to be 
>>>>>>> discussed...
>>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>>> Cheers
>>>>>>> Fran?ois
>>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Embedding a STC region in the SIAP query response is becoming more
>>>>>>>> and more widespread, and we would like to support this feature in
>>>>>>>> Aladin.
>>>>>>>> I would like to know if there is a standard (or de facto standard)
>>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>>> description.
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> -- 
>>>> =====================================================================
>>>> Fran?ois   Bonnarel           Observatoire Astronomique de Strasbourg
>>>> CDS (Centre de donn?es        11, rue de l'Universit?
>>>> astronomiques de Strasbourg)  F--67000 Strasbourg (France)
>>>>
>>>> Tel: +33-(0)3 68 85 24 11     WWW: 
>>>> http://cdsweb.u-strasbg.fr/people/fb.html
>>>> Fax: +33-(0)3 68 85 24 25     E-mail: 
>>>> francois.bonnarel at astro.unistra.fr
>>>> ---------------------------------------------------------------------
>>>>
>>> -------------------------------------------------------------------------- 
>>>
>>> Arnold H. Rots                                Chandra X-ray Science 
>>> Center
>>> Smithsonian Astrophysical Observatory                tel:  +1 617 
>>> 496 7701
>>> 60 Garden Street, MS 67                              fax:  +1 617 
>>> 495 7356
>>> Cambridge, MA 02138                             
>>> arots at head.cfa.harvard.edu
>>> USA                                     
>>> http://hea-www.harvard.edu/~arots/
>>> -------------------------------------------------------------------------- 
>>>
>>>
>


From msdemlei at ari.uni-heidelberg.de  Fri Dec 16 01:16:16 2011
From: msdemlei at ari.uni-heidelberg.de (Markus Demleitner)
Date: Fri, 16 Dec 2011 10:16:16 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <824543C2-E236-4A2A-93B8-B3E9513BA9C3@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<20111215165929.GA1457@ari.uni-heidelberg.de>
	<824543C2-E236-4A2A-93B8-B3E9513BA9C3@googlemail.com>
Message-ID: <20111216091616.GA17896@ari.uni-heidelberg.de>

Dear list,

On Thu, Dec 15, 2011 at 06:28:59PM +0100, Alberto Micol wrote:
> >>What is the magic bit that will allow Aladin to recognise a given
> >>FIELD as a footprint?
> >>
> >>It cannot be the UTYPE, if every data model uses a different UTYPE
> >>for a footprint.
> >Yes it can, and it should be.  Sorry if I'm getting on your nerves,
> >but let me again promote the group/fieldref mechanism.
> >
> No, you are not getting onto my nerves, I hope you will say the same
Phew!  Thanks...

> Yes, what you proposed and propose is a possible solution, but
> it is not elegant, nor efficient.
> Instead of unifying concepts, you propose to multiply FIELDrefs.
> 
> One would have to potentially introduce a FIELDref as many times as
> there are data models.
Frankly, I don't think that's surprising: If you want to "support a
data model", you will have to link whatever data you deliver with
this data model.  If what you're delivering really contains instances
of seven data models, then you'll have seven such linkages.

I wouldn't be worried too much about efficiency -- we're only talking
about a few kilobytes, for all such linkages combined.  Doing the
mapping can be messier, but the effort to do that probably doesn't
change by orders of magnitude for different techniques of declaring
the pairs of data model elements and instance values.

As to elegance -- well, that's a different issue.  I sure don't
claim I'd build such a system if I could start from scratch....

> And that should happen for all the fields that are potentially in common
> among the selected data models.
There could be a solution to this, but it's probably too intrusive to
prescribe at this point.  Still, since the topic comes up I guess
there's no harm mentioning my little group references-as-utype values
scheme; the idea would be to write stuff like

<GROUP id="fp1" utype="stc:AstroCoordArea">    
  <PARAM name="href" datatype="char" arraysize="*"
     utype="stc:AstroCoordSystem.href"
     value="ivo://STClib/CoordSys#TT-ICRS-TOPO"/>
   <PARAM name="URI" datatype="char" arraysize="*"
     utype="stc:DataModel.URI"
     value="http://www.ivoa.net/xml/STC/stc-v1.30.xsd"/>
  <FIELDref ref="region" utype="stc:AstroCoordArea.Circle"/>
</GROUP>

to define, say, an area.  This could be picked up by anything
that understands STC; such a client would know there's an area, it
would work out the coordinate system, etc.

To actually say it's the footprint of an observation, you'd have,
say, an obscore group:

<GROUP utype="obscore:char.SpatialAxis">
  <PARAM name="u" datatype="char" arraysize="*"
    utype="obscore:char.spatialaxis.coverage"
    value="#fp1"/>
</GROUP>

(I'm not completely sold to the idea of using the #-syntax to say
"this value is a reference", but since it looks nicely familiar, I
think it's a start).

This would save you repeating all the information on, say, coordinate
systems, the STC DM used (and possibly much more) in the obscore
group.  I'd like that, but it makes parsing these things harder to
clients just wanting to handle obscore.  So, again it's a tradeoff
whether we want to design to make things simple for generic clients
or for consumers of specific data models.

If people think this whole referencing thing is worthwhile, *now* is
the time to say so, since the utypes document is being worked on, and
that would be the place to specify it.

> Basically my super-duper archive service will have to know about
> all possible models, and will have to change every time a new model
> comes out.
...*if* you want to support that data model.  But that's to be
expected, isn't it?

> My point is that a footprint is a footprint,
> no matter which protocol is used. The concept "footprint" is super
> partes.
> 
> And definitively I want my service to also be super partes
> to reduce maintenance, and because simpler is better.
Definitely.  And that's why I suggest to just say: If what you have
is remotely like an observation, include a group mapping it to
obscore types.  Same thing for photometric fields, STC fields,
whatever.

Generic clients would use that, and if fancier, more expressive data
models come along, servers and clients that *choose* to support them
can do so without clobbering their obscore, phot, whatever
declarations.  Thus the generic clients and servers will just
continue to work.  Wouldn't that be nice?

Cheers,

           Markus


From norman at astro.gla.ac.uk  Fri Dec 16 02:14:23 2011
From: norman at astro.gla.ac.uk (Norman Gray)
Date: Fri, 16 Dec 2011 10:14:23 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
Message-ID: <0CEBBDDB-4837-4ED7-8ACC-3FBFD1E7778B@astro.gla.ac.uk>


Greetings, all.

On 2011 Dec 15, at 13:42, Alberto Micol wrote:

> On 2 Sep 2011, at 01:40, Douglas Tody wrote:
> 
>> This would be a fine solution, considering only STC; however it is
>> perfectly legal for a DM to be based upon other models (in a way
>> defined
>> by the new model such as Char or SIA) and define new, more
>> application-specific Utypes.  There is an especially strong argument
>> for
>> this for essential elements of the new model so that we have a well
>> defined and self consistent model.  "Falling through" to reference
>> arbitrary elements of another model, with its own externally defined
>> namespace, is fine for extension but could significantly expand the
>> complexity of the new model as a client application might then need to
>> fully understand multiple models to be able to deal with the data.
> 
> There are two kinds of clients: it seems to me that Doug here above
> refers to client applications that deal with a specific data model,
> while what I'm going to describe is the case of a generic client.

[use-cases snipped]

> What is the magic bit that will allow Aladin to recognise a given
> FIELD as a footprint?
> 
> It cannot be the UTYPE, if every data model uses a different UTYPE for
> a footprint.

It can be, and I think it should be (I'm being a broken record, here...)

To pick up Markus's example, you want to indicate that a particular FIELD is a char:SpatialAxis.Coverage.Support.Area, without excluding clients who only understand another data model.  They can understand stc:AstroCoordArea.Circle, but not the char ones.

Markus describes a way in which a VOTable can indicate that a particular FIELD can be associated with more than one utype.  However, if I'm understanding Markus's example correctly, that depends on the author_of_that_VOTable making the multiple associations, and they could (a) be mistaken, or (b) forget one.

(This situation becomes more extreme if some data provider wants or needs to extend the notion of footprint to some more precise notion, and wants to be able to distribute data using that data model.)

What would be preferable is for the author_of_the_utype to say that ...Support.Area is a ...AstroCoordArea.Circle.  The advantages of this are:

  * The group which created the utype can be authoritative about what this utype means.

  * This doesn't rely on the authors of VOTables remembering to include all of the other data models that a FIELD can be regarded as referring to.

Such a mechanism is technically feasible, and straightforward to implement and use.  If you do the equivalent of 

    % curl char:SpatialAxis.Coverage.Support.Area
    char:SpatialAxis.Coverage.Support.Area is a stc:AstroCoordArea.Circle
    %

or 

    % curl http://example.org/resolver?char:SpatialAxis.Coverage.Support.Area
    stc:AstroCoordArea.Circle
    %

...then any type of client which sees the char:SpatialAxis.Coverage.Support.Area can know that it can treat this as a stc:AstroCoordArea.Circle with the blessing of the char authors.

This doesn't have to be done dynamically (shudder), but could be cached aggressively, or done at 'compile' time.

This mechanism requires minimal standardisation, requires no major complication to clients, but provides significant interoperability.

This mechanism is worked out in precise detail in <http://www.ivoa.net/Documents/Notes/UTypes/utype-uri-20070302.html#utype-rdf>.  I did implement the resolver service that document describes, and found no problems.

Markus again:

> The only logical possibilities I see, with different levels of
> technical or political difficulty to implement,
> are:
> 
> 1.- to standardise the footprint UTYPE across all data models
> 2.- to standardise the FIELD name of a footprint field, a la ObsTAP
> 3.- to create a new UCD for a footprint
> 4.- to use xtype="adql:Region" (is that acceptable? xtype is really
> meant to represent the storage units)

The fifth possibility is to get a UType to itself explain what it is.  That requires no syntactical changes, no new standardised utypes, no standarised FIELD names, and no new UCDs.

If you as a client are asked to do something with a Utype you don't recognise, you look it up, and are told "char:SpatialAxis.Coverage.Support.Area is like stc:AstroCoordArea.Circle".  You don't have to ask again.

All the best,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK


From amicol.ivoa at googlemail.com  Fri Dec 16 03:36:26 2011
From: amicol.ivoa at googlemail.com (Alberto Micol)
Date: Fri, 16 Dec 2011 12:36:26 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <0CEBBDDB-4837-4ED7-8ACC-3FBFD1E7778B@astro.gla.ac.uk>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<0CEBBDDB-4837-4ED7-8ACC-3FBFD1E7778B@astro.gla.ac.uk>
Message-ID: <CCBB4D16-1B48-40DB-A9FA-A636E88E6752@googlemail.com>


Dear Norman,

Little note: I think you meant not stc:AstroCoordArea.Circle
but stc:ObservationLocation.AstroCoordArea.Region instead.
At least, I read your email with this in mind.

Presumably, your utype resolver service will return multiple answers,
not just one:  Shouldn't it return all the possible associations?

I think so, unless you tell me that there is, for any concept,
one UTYPE that is better than any other one, that is meant to be
the progenitor, and defined to be the "standard" utype for that concept.

If you really meant that one and only one utype, the best one, is  
returned,
well, then we would need a standardisation process for this, isn't it?

And wouldn't that be equivalent to the (tiring) standardisation
process already happening, as mentioned by Francois Bonnarel,
whereby a new data model does not invent new utypes, but
reuse them?

Thanks,
Alberto


On 16 Dec 2011, at 11:14, Norman Gray wrote:

>
> Greetings, all.
>
> On 2011 Dec 15, at 13:42, Alberto Micol wrote:
>
>> On 2 Sep 2011, at 01:40, Douglas Tody wrote:
>>
>>> This would be a fine solution, considering only STC; however it is
>>> perfectly legal for a DM to be based upon other models (in a way
>>> defined
>>> by the new model such as Char or SIA) and define new, more
>>> application-specific Utypes.  There is an especially strong argument
>>> for
>>> this for essential elements of the new model so that we have a well
>>> defined and self consistent model.  "Falling through" to reference
>>> arbitrary elements of another model, with its own externally defined
>>> namespace, is fine for extension but could significantly expand the
>>> complexity of the new model as a client application might then  
>>> need to
>>> fully understand multiple models to be able to deal with the data.
>>
>> There are two kinds of clients: it seems to me that Doug here above
>> refers to client applications that deal with a specific data model,
>> while what I'm going to describe is the case of a generic client.
>
> [use-cases snipped]
>
>> What is the magic bit that will allow Aladin to recognise a given
>> FIELD as a footprint?
>>
>> It cannot be the UTYPE, if every data model uses a different UTYPE  
>> for
>> a footprint.
>
> It can be, and I think it should be (I'm being a broken record,  
> here...)
>
> To pick up Markus's example, you want to indicate that a particular  
> FIELD is a char:SpatialAxis.Coverage.Support.Area, without excluding  
> clients who only understand another data model.  They can understand  
> stc:AstroCoordArea.Circle, but not the char ones.
>
> Markus describes a way in which a VOTable can indicate that a  
> particular FIELD can be associated with more than one utype.   
> However, if I'm understanding Markus's example correctly, that  
> depends on the author_of_that_VOTable making the multiple  
> associations, and they could (a) be mistaken, or (b) forget one.
>
> (This situation becomes more extreme if some data provider wants or  
> needs to extend the notion of footprint to some more precise notion,  
> and wants to be able to distribute data using that data model.)
>
> What would be preferable is for the author_of_the_utype to say  
> that ...Support.Area is a ...AstroCoordArea.Circle.  The advantages  
> of this are:
>
>  * The group which created the utype can be authoritative about what  
> this utype means.
>
>  * This doesn't rely on the authors of VOTables remembering to  
> include all of the other data models that a FIELD can be regarded as  
> referring to.
>
> Such a mechanism is technically feasible, and straightforward to  
> implement and use.  If you do the equivalent of
>
>    % curl char:SpatialAxis.Coverage.Support.Area
>    char:SpatialAxis.Coverage.Support.Area is a  
> stc:AstroCoordArea.Circle
>    %
>
> or
>
>    % curl http://example.org/resolver?char:SpatialAxis.Coverage.Support.Area
>    stc:AstroCoordArea.Circle
>    %
>
> ...then any type of client which sees the  
> char:SpatialAxis.Coverage.Support.Area can know that it can treat  
> this as a stc:AstroCoordArea.Circle with the blessing of the char  
> authors.
>
> This doesn't have to be done dynamically (shudder), but could be  
> cached aggressively, or done at 'compile' time.
>
> This mechanism requires minimal standardisation, requires no major  
> complication to clients, but provides significant interoperability.
>
> This mechanism is worked out in precise detail in <http://www.ivoa.net/Documents/Notes/UTypes/utype-uri-20070302.html#utype-rdf 
> >.  I did implement the resolver service that document describes,  
> and found no problems.
>
> Markus again:
>
>> The only logical possibilities I see, with different levels of
>> technical or political difficulty to implement,
>> are:
>>
>> 1.- to standardise the footprint UTYPE across all data models
>> 2.- to standardise the FIELD name of a footprint field, a la ObsTAP
>> 3.- to create a new UCD for a footprint
>> 4.- to use xtype="adql:Region" (is that acceptable? xtype is really
>> meant to represent the storage units)
>
> The fifth possibility is to get a UType to itself explain what it  
> is.  That requires no syntactical changes, no new standardised  
> utypes, no standarised FIELD names, and no new UCDs.
>
> If you as a client are asked to do something with a Utype you don't  
> recognise, you look it up, and are told  
> "char:SpatialAxis.Coverage.Support.Area is like  
> stc:AstroCoordArea.Circle".  You don't have to ask again.
>
> All the best,
>
> Norman
>
>
> -- 
> Norman Gray  :  http://nxg.me.uk
> SUPA School of Physics and Astronomy, University of Glasgow, UK
>


From arots at head.cfa.harvard.edu  Fri Dec 16 07:18:11 2011
From: arots at head.cfa.harvard.edu (Arnold Rots)
Date: Fri, 16 Dec 2011 10:18:11 -0500 (EST)
Subject: utype for STC region in SIAP query response
In-Reply-To: <CCBB4D16-1B48-40DB-A9FA-A636E88E6752@googlemail.com>
Message-ID: <201112161518.pBGFIBIF026784@xebec.cfa.harvard.edu>

I think I want to get back into this conversation.

The summary of my argument is:
- IVOA has a standard for describing regions: STC's Region
- That is precisely the kind of beast that a footprint is
- The STC-S serialization of STC Regions is eminently suited to describe
footprints since it is a (fairly) simple string that includes all
relevant information and provides a complete description in most
circumstances (and I don't intend to get into the exceptions here)

Yes, Alberto is correct, it should be:
     stc:ObservationLocation.AstroCoordArea.Region

Although an earlier post from Markus provided the more genreal
version for a VOTable context:

     <GROUP utype="stc:CatalogEntryLocation">
	 <FIELDref utype="stc:AstroCoordArea.Region" ref="footprint"/>
     </GROUP>

The AstroCoordArea.Region could be a companent of a
CatalogEntryLocation, an ObservationLocations, etc.
These parent elements tell one what the footprint pertains to.

One would definitely not want to make it AstroCoordArea.Circle, since
the Circle specification would be provided in the STC-S value.

This scheme is simple, unambiguous, and based on an existing standard.
The STC-S strings are being used for footprint specifications in
various applications and have turned out to provide a mechanism that
is perfectly adequate for this task, as it has teh flexibility to
describe any footprint one can think of.


The remaining question might be whether one needs to identify a
particular instance of a Region as a footprint. I would argue that, in
a way, every region is a footprint of some kind. If it is a member of
an ObservationLocation, it is the footprint of an observation; if it
is a member of a CatalogEntryLocation, it is a catalog footprint.
One might reinforce the point with the FIELD's name, but that does not
change the nature of the item as defined by the utype.

Cheers,

  - Arnold


Alberto Micol wrote:
> 
> Dear Norman,
> 
> Little note: I think you meant not stc:AstroCoordArea.Circle
> but stc:ObservationLocation.AstroCoordArea.Region instead.
> At least, I read your email with this in mind.
> 
> Presumably, your utype resolver service will return multiple answers,
> not just one:  Shouldn't it return all the possible associations?
> 
> I think so, unless you tell me that there is, for any concept,
> one UTYPE that is better than any other one, that is meant to be
> the progenitor, and defined to be the "standard" utype for that concept.
> 
> If you really meant that one and only one utype, the best one, is  
> returned,
> well, then we would need a standardisation process for this, isn't it?
> 
> And wouldn't that be equivalent to the (tiring) standardisation
> process already happening, as mentioned by Francois Bonnarel,
> whereby a new data model does not invent new utypes, but
> reuse them?
> 
> Thanks,
> Alberto
> 
> 
> On 16 Dec 2011, at 11:14, Norman Gray wrote:
> 
> >
> > Greetings, all.
> >
> > On 2011 Dec 15, at 13:42, Alberto Micol wrote:
> >
> >> On 2 Sep 2011, at 01:40, Douglas Tody wrote:
> >>
> >>> This would be a fine solution, considering only STC; however it is
> >>> perfectly legal for a DM to be based upon other models (in a way
> >>> defined
> >>> by the new model such as Char or SIA) and define new, more
> >>> application-specific Utypes.  There is an especially strong argument
> >>> for
> >>> this for essential elements of the new model so that we have a well
> >>> defined and self consistent model.  "Falling through" to reference
> >>> arbitrary elements of another model, with its own externally defined
> >>> namespace, is fine for extension but could significantly expand the
> >>> complexity of the new model as a client application might then  
> >>> need to
> >>> fully understand multiple models to be able to deal with the data.
> >>
> >> There are two kinds of clients: it seems to me that Doug here above
> >> refers to client applications that deal with a specific data model,
> >> while what I'm going to describe is the case of a generic client.
> >
> > [use-cases snipped]
> >
> >> What is the magic bit that will allow Aladin to recognise a given
> >> FIELD as a footprint?
> >>
> >> It cannot be the UTYPE, if every data model uses a different UTYPE  
> >> for
> >> a footprint.
> >
> > It can be, and I think it should be (I'm being a broken record,  
> > here...)
> >
> > To pick up Markus's example, you want to indicate that a particular  
> > FIELD is a char:SpatialAxis.Coverage.Support.Area, without excluding  
> > clients who only understand another data model.  They can understand  
> > stc:AstroCoordArea.Circle, but not the char ones.
> >
> > Markus describes a way in which a VOTable can indicate that a  
> > particular FIELD can be associated with more than one utype.   
> > However, if I'm understanding Markus's example correctly, that  
> > depends on the author_of_that_VOTable making the multiple  
> > associations, and they could (a) be mistaken, or (b) forget one.
> >
> > (This situation becomes more extreme if some data provider wants or  
> > needs to extend the notion of footprint to some more precise notion,  
> > and wants to be able to distribute data using that data model.)
> >
> > What would be preferable is for the author_of_the_utype to say  
> > that ...Support.Area is a ...AstroCoordArea.Circle.  The advantages  
> > of this are:
> >
> >  * The group which created the utype can be authoritative about what  
> > this utype means.
> >
> >  * This doesn't rely on the authors of VOTables remembering to  
> > include all of the other data models that a FIELD can be regarded as  
> > referring to.
> >
> > Such a mechanism is technically feasible, and straightforward to  
> > implement and use.  If you do the equivalent of
> >
> >    % curl char:SpatialAxis.Coverage.Support.Area
> >    char:SpatialAxis.Coverage.Support.Area is a  
> > stc:AstroCoordArea.Circle
> >    %
> >
> > or
> >
> >    % curl http://example.org/resolver?char:SpatialAxis.Coverage.Support.Area
> >    stc:AstroCoordArea.Circle
> >    %
> >
> > ...then any type of client which sees the  
> > char:SpatialAxis.Coverage.Support.Area can know that it can treat  
> > this as a stc:AstroCoordArea.Circle with the blessing of the char  
> > authors.
> >
> > This doesn't have to be done dynamically (shudder), but could be  
> > cached aggressively, or done at 'compile' time.
> >
> > This mechanism requires minimal standardisation, requires no major  
> > complication to clients, but provides significant interoperability.
> >
> > This mechanism is worked out in precise detail in <http://www.ivoa.net/Documents/Notes/UTypes/utype-uri-20070302.html#utype-rdf 
> > >.  I did implement the resolver service that document describes,  
> > and found no problems.
> >
> > Markus again:
> >
> >> The only logical possibilities I see, with different levels of
> >> technical or political difficulty to implement,
> >> are:
> >>
> >> 1.- to standardise the footprint UTYPE across all data models
> >> 2.- to standardise the FIELD name of a footprint field, a la ObsTAP
> >> 3.- to create a new UCD for a footprint
> >> 4.- to use xtype="adql:Region" (is that acceptable? xtype is really
> >> meant to represent the storage units)
> >
> > The fifth possibility is to get a UType to itself explain what it  
> > is.  That requires no syntactical changes, no new standardised  
> > utypes, no standarised FIELD names, and no new UCDs.
> >
> > If you as a client are asked to do something with a Utype you don't  
> > recognise, you look it up, and are told  
> > "char:SpatialAxis.Coverage.Support.Area is like  
> > stc:AstroCoordArea.Circle".  You don't have to ask again.
> >
> > All the best,
> >
> > Norman
> >
> >
> > -- 
> > Norman Gray  :  http://nxg.me.uk
> > SUPA School of Physics and Astronomy, University of Glasgow, UK
> >
> 
--------------------------------------------------------------------------
Arnold H. Rots                                Chandra X-ray Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 67                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots at head.cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------

From norman at astro.gla.ac.uk  Fri Dec 16 07:39:55 2011
From: norman at astro.gla.ac.uk (Norman Gray)
Date: Fri, 16 Dec 2011 15:39:55 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <CCBB4D16-1B48-40DB-A9FA-A636E88E6752@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<0CEBBDDB-4837-4ED7-8ACC-3FBFD1E7778B@astro.gla.ac.uk>
	<CCBB4D16-1B48-40DB-A9FA-A636E88E6752@googlemail.com>
Message-ID: <23752511-9ACE-43EF-9466-3F5A33EE2E1D@astro.gla.ac.uk>


Alberto, hello.

On 2011 Dec 16, at 11:36, Alberto Micol wrote:

> Little note: I think you meant not stc:AstroCoordArea.Circle
> but stc:ObservationLocation.AstroCoordArea.Region instead.
> At least, I read your email with this in mind.

Quite possibly -- I confess to not having all of STC comfortably in my head.

> Presumably, your utype resolver service will return multiple answers,
> not just one:  Shouldn't it return all the possible associations?

The resolver I wrote did return all the things which were 'more general' than the utype it was asked to explain.  One can be more precise than this, but the resolver was intended to act as a bare-bones help to a client-in-a-hurry.

> I think so, unless you tell me that there is, for any concept,
> one UTYPE that is better than any other one, that is meant to be
> the progenitor, and defined to be the "standard" utype for that concept.

No, there's no implicit idea that there's one utype which is better than all others in a particular context.

The paradigmatic case is where someone (or some application) wants to use a utype such as ...AstroCoordArea.Region _but_ it's not quite right.  For example the Char notion of 'footprint' isn't just a _region_, but is instead a region where there is coverage, to which the Char data model gives the more precise name char:SpatialAxis.Coverage.Support.Area.

Any application which knows what a ...Coverage.Support.Area is, will presumably be familiar with the IVOA's blessed Char semantics for this.  A more generic application which doesn't have to care about footprints, but which _is_ able to display regions, will presumably know what an ...AstroCoordArea.Region is, and so can be useful here, as long as it can be told that the ...Coverage.Support.Area is more or less the same thing.

One can imagine other relationships between utypes; the resolver I mentioned is intended to help with the simplest and commonest case of 'more specific than'.

> If you really meant that one and only one utype, the best one, is  
> returned,
> well, then we would need a standardisation process for this, isn't it?

There would be no standardisation required, beyond the requirement that, if IVOA data models (ie utype sets) want to play well with this mechanism, they document their relationship with other data models using the mechanism described in <http://www.ivoa.net/Documents/Notes/UTypes/utype-uri-20070302.html> (which means putting onto the web a file in an already-standardised syntax).  Those relationships could very naturally be included in the existing definitions of those data models.

> And wouldn't that be equivalent to the (tiring) standardisation
> process already happening, as mentioned by Francois Bonnarel,
> whereby a new data model does not invent new utypes, but
> reuse them?

In this picture, a data model consists of a set of coherent concepts (such as in the Char model).  Often, some of those concepts will be the same as, or similar to, concepts in other data models.  This can be handled either by importing (which can become complicated), or by description: "newmodel:A is the same as popularmodel:X, but newmodel:B is a refinement of concept popularmodel:Y".

If a client already knows how to deal with popularmodel:X and popularmodel:Y, then it can already deal with newmodel:A and newmodel:B.  Interoperability is achieved!

This sounds exotic, but it's not.  The way of doing this is already standardised and already deployed all over the web.

Best wishes,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK


From norman at astro.gla.ac.uk  Fri Dec 16 07:51:07 2011
From: norman at astro.gla.ac.uk (Norman Gray)
Date: Fri, 16 Dec 2011 15:51:07 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <201112161518.pBGFIBIF026784@xebec.cfa.harvard.edu>
References: <201112161518.pBGFIBIF026784@xebec.cfa.harvard.edu>
Message-ID: <18E21116-BB60-4070-9522-CDCB5518F78F@astro.gla.ac.uk>


Arnold and all, hello.

On 2011 Dec 16, at 15:18, Arnold Rots wrote:

> Yes, Alberto is correct, it should be:
>     stc:ObservationLocation.AstroCoordArea.Region

Righto!

> The remaining question might be whether one needs to identify a
> particular instance of a Region as a footprint. I would argue that, in
> a way, every region is a footprint of some kind. If it is a member of
> an ObservationLocation, it is the footprint of an observation; if it
> is a member of a CatalogEntryLocation, it is a catalog footprint.
> One might reinforce the point with the FIELD's name, but that does not
> change the nature of the item as defined by the utype.

Seen from the point of view of geometry on the sky, yes, a footprint is just a Region.

However one might want to produce a VOtable in which one includes both the region of sky which was in the field of view of a satellite during one orbit, and the region of sky in which the satellite actually took data -- they might be different for any number of reasons.  From the point of view of STC, these are both just Regions.  From the point of view of the client, however, these are very different things, and it would be natural and helpful to this client if the data were described in a data model which distinguished the notion of 'footprint' (meaning the area of sky where there are observations) from whatever term you'd use for the other region.

This is the point: an application that plots sky regions has a natural data model which consists of only Regions (something like STC); an application which is searching for datasets which cover a particular point on the sky has a _different_ natural data model (something more like Char).  Neither is _the_ correct dataset, but the data can easily conform to both.

Best wishes,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK


From jfay at microsoft.com  Fri Dec 16 11:54:18 2011
From: jfay at microsoft.com (Jonathan Fay)
Date: Fri, 16 Dec 2011 19:54:18 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
Message-ID: <5AC8AF9D933B124A95DFD9E1F029816108764400@TK5EX14MBXC241.redmond.corp.microsoft.com>

Dear Thomas, Alberto and All,

I have been following this thread from afar since September, I too have an interest in see the expanded use a regions to allow visualization of footprints. WWT already supports footprints for HLA (using ucd as "phys.area;obs.field" and stc-s as content), but I had hoped a more wide spread standard might evolve since it helps improve the user experience when they can identify, select and preview images by their coverage in a more intuitive and less ambiguous way. Even HLA defines two *different* types of footprints in their SIA responses.

I am aware that discussions regarding footprints have been going on for years and even now there are a few competing proposals right now, but since the interest and activity has suddenly jumped in this subject it would be really worthwhile to converge on a consensus. All this standards discussion will mean little if we can't translate it into useful features and get it in to the hands of real users.

Converging on a de-facto standard that we could all implement in interoperable viewers and publish as a IVOA note might get some traction in the short term.

Even if this is not a single one-size-fits-all standard, but a collection of  recommended ways of expressing footprints currently in common use so that users could see them across a wide range of software.

I know if this can happen I would definitely want to add support for it immediately on both WWT clients.

Jonathan Fay


-----Original Message-----
From: dal-bounces at ivoa.net [mailto:dal-bounces at ivoa.net] On Behalf Of Alberto Micol
Sent: Thursday, December 15, 2011 5:43 AM
To: Douglas Tody
Cc: dal at ivoa.net
Subject: Re: utype for STC region in SIAP query response


Dear All,

On 2 Sep 2011, at 01:40, Douglas Tody wrote:

> This would be a fine solution, considering only STC; however it is 
> perfectly legal for a DM to be based upon other models (in a way 
> defined by the new model such as Char or SIA) and define new, more 
> application-specific Utypes.  There is an especially strong argument 
> for this for essential elements of the new model so that we have a 
> well defined and self consistent model.  "Falling through" to 
> reference arbitrary elements of another model, with its own externally 
> defined namespace, is fine for extension but could significantly 
> expand the complexity of the new model as a client application might 
> then need to fully understand multiple models to be able to deal with 
> the data.

There are two kinds of clients: it seems to me that Doug here above refers to client applications that deal with a specific data model, while what I'm going to describe is the case of a generic client.

Let me consider a well known client application (Aladin), and four quite common use cases:

- As a user, I want to take the VOTable generated by a ObsTAP service and display the contained footprints.

- As a user, I want to take the VOTable generated by a SIAP service and display the contained footprints.

- As a user, I want to take the VOTable generated by a CharDM compliant service, and display the contained footprints.

- As a user, I want to take the VOTable generated by an archive query form and display the therein contained footprints


What is the magic bit that will allow Aladin to recognise a given FIELD as a footprint?

It cannot be the UTYPE, if every data model uses a different UTYPE for a footprint.

It cannot be the FIELD name, given that only one of the three services above (ObsTAP) guarantees the name to always be "s_region".

Could it be a new UCD?
(Btw, when is that a client should look for a UCD, and when for a UTYPE, to know what to do?)

The only logical possibilities I see, with different levels of technical or political difficulty to implement,
are:

1.- to standardise the footprint UTYPE across all data models
2.- to standardise the FIELD name of a footprint field, a la ObsTAP
3.- to create a new UCD for a footprint
4.- to use xtype="adql:Region" (is that acceptable? xtype is really meant to represent the storage units)


Would you suggest other ways, or the "good enough" way is already listed above?

Note: In the years we have had the same discussion again and again, this is just the first time it happens for footprints. It would be nice to solve the originating problem not just the footprint problem.

Alberto

PS: I would love to suggest a 5th possibility, that is, to create a new FIELD attribute, called stdname,
    which hosts an IVOA standard name for all those quantities that are really common across
    all data models, e.g. "footprint", "telescope", "instrument", etc.,
    as in <FIELD name="myFootprint" stdname="footprint"  
utype="u.name.it" xtype="adql:Region">,
    but I won't,
    unless asked.
;-)


>  In
> the case here, Char defines strong and consistent concepts for the 
> measurement axes, Coverage, Support, etc. which are important to the 
> Char model, and merely uses STC-S to represent the datum in question.
>
> Regarding UCDs for representing footprints in SIA-1, this can only be
> done by extension and there is no standard way to do it at present.   
> We
> really need SIAV2 to address this sort of concern.
>
> 	- Doug
>
>
> On Thu, 1 Sep 2011, Arnold Rots wrote:
>
>> Here is what, in my view, is the correct utype for a footprint
>> region:
>>
>>    stc:ObservationLocation.AstroCoordArea.Region
>>
>> Pretty simple; though one might quibble whether it should be:
>>
>>    stc:ObsDataLocation.ObservationLocation.AstroCoordArea.Region
>>
>>
>> - Arnold
>>
>>
>> Francois Bonnarel wrote:
>> [ Charset ISO-8859-1 unsupported, converting... ]
>>> Hi Thomas,
>>>    To complete my answer to Markus
>>>     we could have something like
>>>
>>>    <FIELD name="fov" unit="deg" xtype="STC-s" datatype="char"
>>> arraysize="*" utype="sia:Char.SpatialAxis.Coverage.Support.Area" >
>>>
>>>       I think something like this can be proposed (if not yet the
>>> case)
>>> in SIA2...
>>>       this will be consistent with ObsTap... and could also be 
>>> included in the next version of SSA et al.
>>>
>>>       For current SIA services, maybe we could have a short 
>>> recommendation or note to propose using the same as an additional 
>>> field in the SIA1 Query response Cheers Fran?ois
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> Thomas Boch a ?crit :
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> =
>>> ====================================================================
>>> Fran?ois   Bonnarel           Observatoire Astronomique de  
>>> Strasbourg
>>> CDS (Centre de donn?es        11, rue de l'Universit?
>>> astronomiques de Strasbourg)  F--67000 Strasbourg (France)
>>>
>>> Tel: +33-(0)3 68 85 24 11     WWW: http://cdsweb.u-strasbg.fr/people/fb.html
>>> Fax: +33-(0)3 68 85 24 25     E-mail: francois.bonnarel at astro.unistra.fr
>>> --------------------------------------------------------------------
>>> -
>>>
>> --------------------------------------------------------------------------
>> Arnold H. Rots                                Chandra X-ray Science  
>> Center
>> Smithsonian Astrophysical Observatory                tel:  +1 617  
>> 496 7701
>> 60 Garden Street, MS 67                              fax:  +1 617  
>> 495 7356
>> Cambridge, MA 02138                             arots at head.cfa.harvard.edu
>> USA                                     http://hea-www.harvard.edu/~arots/
>> ---------------------------------------------------------------------
>> -----
>>




From jfay at microsoft.com  Fri Dec 16 11:58:13 2011
From: jfay at microsoft.com (Jonathan Fay)
Date: Fri, 16 Dec 2011 19:58:13 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <5AC8AF9D933B124A95DFD9E1F029816108764400@TK5EX14MBXC241.redmond.corp.microsoft.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<5AC8AF9D933B124A95DFD9E1F029816108764400@TK5EX14MBXC241.redmond.corp.microsoft.com>
Message-ID: <5AC8AF9D933B124A95DFD9E1F029816108764477@TK5EX14MBXC241.redmond.corp.microsoft.com>

As another thought. Maybe those of us who will be at AAS in Austin can get together and discuss this in person and share collective thoughts back to the list.

Jonathan

-----Original Message-----
From: dal-bounces at ivoa.net [mailto:dal-bounces at ivoa.net] On Behalf Of Jonathan Fay
Sent: Friday, December 16, 2011 11:54 AM
To: Alberto Micol; Douglas Tody; Thomas Boch (boch at vizir.u-strasbg.fr)
Cc: Andrew Connolly; Christa Anderson; Alyssa Goodman; dal at ivoa.net
Subject: RE: utype for STC region in SIAP query response

Dear Thomas, Alberto and All,

I have been following this thread from afar since September, I too have an interest in see the expanded use a regions to allow visualization of footprints. WWT already supports footprints for HLA (using ucd as "phys.area;obs.field" and stc-s as content), but I had hoped a more wide spread standard might evolve since it helps improve the user experience when they can identify, select and preview images by their coverage in a more intuitive and less ambiguous way. Even HLA defines two *different* types of footprints in their SIA responses.

I am aware that discussions regarding footprints have been going on for years and even now there are a few competing proposals right now, but since the interest and activity has suddenly jumped in this subject it would be really worthwhile to converge on a consensus. All this standards discussion will mean little if we can't translate it into useful features and get it in to the hands of real users.

Converging on a de-facto standard that we could all implement in interoperable viewers and publish as a IVOA note might get some traction in the short term.

Even if this is not a single one-size-fits-all standard, but a collection of  recommended ways of expressing footprints currently in common use so that users could see them across a wide range of software.

I know if this can happen I would definitely want to add support for it immediately on both WWT clients.

Jonathan Fay


-----Original Message-----
From: dal-bounces at ivoa.net [mailto:dal-bounces at ivoa.net] On Behalf Of Alberto Micol
Sent: Thursday, December 15, 2011 5:43 AM
To: Douglas Tody
Cc: dal at ivoa.net
Subject: Re: utype for STC region in SIAP query response


Dear All,

On 2 Sep 2011, at 01:40, Douglas Tody wrote:

> This would be a fine solution, considering only STC; however it is 
> perfectly legal for a DM to be based upon other models (in a way 
> defined by the new model such as Char or SIA) and define new, more 
> application-specific Utypes.  There is an especially strong argument 
> for this for essential elements of the new model so that we have a 
> well defined and self consistent model.  "Falling through" to 
> reference arbitrary elements of another model, with its own externally 
> defined namespace, is fine for extension but could significantly 
> expand the complexity of the new model as a client application might 
> then need to fully understand multiple models to be able to deal with 
> the data.

There are two kinds of clients: it seems to me that Doug here above refers to client applications that deal with a specific data model, while what I'm going to describe is the case of a generic client.

Let me consider a well known client application (Aladin), and four quite common use cases:

- As a user, I want to take the VOTable generated by a ObsTAP service and display the contained footprints.

- As a user, I want to take the VOTable generated by a SIAP service and display the contained footprints.

- As a user, I want to take the VOTable generated by a CharDM compliant service, and display the contained footprints.

- As a user, I want to take the VOTable generated by an archive query form and display the therein contained footprints


What is the magic bit that will allow Aladin to recognise a given FIELD as a footprint?

It cannot be the UTYPE, if every data model uses a different UTYPE for a footprint.

It cannot be the FIELD name, given that only one of the three services above (ObsTAP) guarantees the name to always be "s_region".

Could it be a new UCD?
(Btw, when is that a client should look for a UCD, and when for a UTYPE, to know what to do?)

The only logical possibilities I see, with different levels of technical or political difficulty to implement,
are:

1.- to standardise the footprint UTYPE across all data models
2.- to standardise the FIELD name of a footprint field, a la ObsTAP
3.- to create a new UCD for a footprint
4.- to use xtype="adql:Region" (is that acceptable? xtype is really meant to represent the storage units)


Would you suggest other ways, or the "good enough" way is already listed above?

Note: In the years we have had the same discussion again and again, this is just the first time it happens for footprints. It would be nice to solve the originating problem not just the footprint problem.

Alberto

PS: I would love to suggest a 5th possibility, that is, to create a new FIELD attribute, called stdname,
    which hosts an IVOA standard name for all those quantities that are really common across
    all data models, e.g. "footprint", "telescope", "instrument", etc.,
    as in <FIELD name="myFootprint" stdname="footprint"  
utype="u.name.it" xtype="adql:Region">,
    but I won't,
    unless asked.
;-)


>  In
> the case here, Char defines strong and consistent concepts for the 
> measurement axes, Coverage, Support, etc. which are important to the 
> Char model, and merely uses STC-S to represent the datum in question.
>
> Regarding UCDs for representing footprints in SIA-1, this can only be
> done by extension and there is no standard way to do it at present.   
> We
> really need SIAV2 to address this sort of concern.
>
> 	- Doug
>
>
> On Thu, 1 Sep 2011, Arnold Rots wrote:
>
>> Here is what, in my view, is the correct utype for a footprint
>> region:
>>
>>    stc:ObservationLocation.AstroCoordArea.Region
>>
>> Pretty simple; though one might quibble whether it should be:
>>
>>    stc:ObsDataLocation.ObservationLocation.AstroCoordArea.Region
>>
>>
>> - Arnold
>>
>>
>> Francois Bonnarel wrote:
>> [ Charset ISO-8859-1 unsupported, converting... ]
>>> Hi Thomas,
>>>    To complete my answer to Markus
>>>     we could have something like
>>>
>>>    <FIELD name="fov" unit="deg" xtype="STC-s" datatype="char"
>>> arraysize="*" utype="sia:Char.SpatialAxis.Coverage.Support.Area" >
>>>
>>>       I think something like this can be proposed (if not yet the
>>> case)
>>> in SIA2...
>>>       this will be consistent with ObsTap... and could also be 
>>> included in the next version of SSA et al.
>>>
>>>       For current SIA services, maybe we could have a short 
>>> recommendation or note to propose using the same as an additional 
>>> field in the SIA1 Query response Cheers Fran?ois
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> Thomas Boch a ?crit :
>>>> To illustrate my request, here is is how the STC-S FIELD is 
>>>> described in the query response of three different SIAP services :
>>>>
>>>> Service 1 : <FIELD name="position_bounds" unit="deg"
>>>> xtype="adql:REGION" datatype="char" arraysize="*">
>>>>
>>>> Service 2 : <FIELD name="stcs" datatype="char" arraysize="*"
>>>> ucd="phys.area;obs.field" unit="deg">
>>>>
>>>> Service 3 : <FIELD ID="regionSTCS" name="regionSTCS"  
>>>> datatype="char"
>>>> arraysize="*">
>>>>
>>>> 3 different services, 3 different ways to express the same thing.
>>>> Having a unique unambiguous way to detect the STC-S field would be 
>>>> very helpful for client consuming these services.
>>>> Cheers,
>>>>
>>>> Thomas
>>>>
>>>> Thomas Boch wrote:
>>>>> Fran?ois,
>>>>>
>>>>> I wish this is something that could be standardized in a 
>>>>> forecoming revision of the SIAP document, so that clients can 
>>>>> easily find out which FIELD support the STC-S description.
>>>>>
>>>>> Thomas
>>>>>
>>>>> Fran?ois Bonnarel wrote:
>>>>>> Hi Thomas,
>>>>>>   Obstap has obs:Char.SpatialAxis.Coverage.Support.Area
>>>>>> for this . We have something similar in the SIA2 prototypes
>>>>>>   I guess eventually for sia we will have something like
>>>>>>            sia:Char.SpatialAxis.Coverage.Support.Area
>>>>>>   although the problem of the name space is still to be 
>>>>>> discussed...
>>>>>> By the way you can write this field as an STC-S feature ...
>>>>>> Cheers
>>>>>> Fran?ois
>>>>>> Le 29/08/2011 12:02, Thomas Boch a ?crit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Embedding a STC region in the SIAP query response is becoming 
>>>>>>> more and more widespread, and we would like to support this 
>>>>>>> feature in Aladin.
>>>>>>> I would like to know if there is a standard (or de facto
>>>>>>> standard)
>>>>>>> utype to characterize the FIELD holding the STC region 
>>>>>>> description.
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> =
>>> ====================================================================
>>> Fran?ois   Bonnarel           Observatoire Astronomique de  
>>> Strasbourg
>>> CDS (Centre de donn?es        11, rue de l'Universit?
>>> astronomiques de Strasbourg)  F--67000 Strasbourg (France)
>>>
>>> Tel: +33-(0)3 68 85 24 11     WWW: http://cdsweb.u-strasbg.fr/people/fb.html
>>> Fax: +33-(0)3 68 85 24 25     E-mail: francois.bonnarel at astro.unistra.fr
>>> --------------------------------------------------------------------
>>> -
>>>
>> --------------------------------------------------------------------------
>> Arnold H. Rots                                Chandra X-ray Science  
>> Center
>> Smithsonian Astrophysical Observatory                tel:  +1 617  
>> 496 7701
>> 60 Garden Street, MS 67                              fax:  +1 617  
>> 495 7356
>> Cambridge, MA 02138                             arots at head.cfa.harvard.edu
>> USA                                     http://hea-www.harvard.edu/~arots/
>> ---------------------------------------------------------------------
>> -----
>>






From dtody at nrao.edu  Sun Dec 18 17:33:39 2011
From: dtody at nrao.edu (Douglas Tody)
Date: Sun, 18 Dec 2011 18:33:39 -0700 (MST)
Subject: utype for STC region in SIAP query response
In-Reply-To: <AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
Message-ID: <alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>

On Thu, 15 Dec 2011, Alberto Micol wrote:

> On 2 Sep 2011, at 01:40, Douglas Tody wrote:
>
>> This would be a fine solution, considering only STC; however it is
>> perfectly legal for a DM to be based upon other models (in a way defined
>> by the new model such as Char or SIA) and define new, more
>> application-specific Utypes.  There is an especially strong argument for
>> this for essential elements of the new model so that we have a well
>> defined and self consistent model.  "Falling through" to reference
>> arbitrary elements of another model, with its own externally defined
>> namespace, is fine for extension but could significantly expand the
>> complexity of the new model as a client application might then need to
>> fully understand multiple models to be able to deal with the data.
>
> There are two kinds of clients: it seems to me that Doug here above
> refers to client applications that deal with a specific data model,
> while what I'm going to describe is the case of a generic client.
    [...]
>
> What is the magic bit that will allow Aladin to recognise a given FIELD as a 
> footprint?
>
> It cannot be the UTYPE, if every data model uses a different UTYPE for a 
> footprint.

Well this is the key point; yes we can standardize the core model and
the UTYPE used for this purpose and they do not need to be different
despite being included in different models.  Yes the namespace (dataset
class or context) may be different for different classes of data, but
the UTYPE, minus the namespace prefix defining the context where it
used, can be the same and can be defined to meant the same thing.  This
is how UTYPEs have historically been used up to now, and we can
formalize this further in the UTYPE spec (hopefully without
over-complicating the mechanism).

So as Francois mentions we already have a UTYPE for the overall dataset
footprint:

     Char.SpatialAxis.Coverage.Support.Area

This is currently what is used in Obscore, SSA, and Spectrum, and soon
also in SpectralDM, SED, TimeSeries, SIAV2, etc.  So it already provides
a consistent way to specify a simple footprint, or very nearly so.
Obviously we do want this to be consistent across all these models,
along with many other things.

I suggest that this is sufficient for simple "bounding box" type
footprints.  As I mentioned in the original email quoted above, it is
also possible with our current UTYPE mechanisms to pass-through bits of
external data models such as STC (note STC is also already the basis for
Char.SpatialAxis.Coverage.Support.Area although usage is somewhat
restricted to keep the compexity manageable).  This could possibly be
used as an extension mechanism to define more complex footprints if
desired; if so we might want to separate it out as a separate footprint
object as it is starting to get complicated.  Nonetheless the simpler
representation is already in use, should be provided if nothing else,
and can be standardized across all our dataset classes.

 	- Doug

From m.b.taylor at bristol.ac.uk  Mon Dec 19 01:55:35 2011
From: m.b.taylor at bristol.ac.uk (Mark Taylor)
Date: Mon, 19 Dec 2011 09:55:35 +0000 (GMT)
Subject: utype for STC region in SIAP query response
In-Reply-To: <alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
Message-ID: <alpine.LRH.2.00.1112190953260.26405@andromeda.star.bris.ac.uk>

On Sun, 18 Dec 2011, Douglas Tody wrote:

> On Thu, 15 Dec 2011, Alberto Micol wrote:
> 
> > On 2 Sep 2011, at 01:40, Douglas Tody wrote:
> > 
> > > This would be a fine solution, considering only STC; however it is
> > > perfectly legal for a DM to be based upon other models (in a way defined
> > > by the new model such as Char or SIA) and define new, more
> > > application-specific Utypes.  There is an especially strong argument for
> > > this for essential elements of the new model so that we have a well
> > > defined and self consistent model.  "Falling through" to reference
> > > arbitrary elements of another model, with its own externally defined
> > > namespace, is fine for extension but could significantly expand the
> > > complexity of the new model as a client application might then need to
> > > fully understand multiple models to be able to deal with the data.
> > 
> > There are two kinds of clients: it seems to me that Doug here above
> > refers to client applications that deal with a specific data model,
> > while what I'm going to describe is the case of a generic client.
>    [...]
> > 
> > What is the magic bit that will allow Aladin to recognise a given FIELD as a
> > footprint?
> > 
> > It cannot be the UTYPE, if every data model uses a different UTYPE for a
> > footprint.
> 
> Well this is the key point; yes we can standardize the core model and
> the UTYPE used for this purpose and they do not need to be different
> despite being included in different models.  Yes the namespace (dataset
> class or context) may be different for different classes of data, but
> the UTYPE, minus the namespace prefix defining the context where it
> used, can be the same and can be defined to meant the same thing.  This
> is how UTYPEs have historically been used up to now, and we can
> formalize this further in the UTYPE spec (hopefully without
> over-complicating the mechanism).
> 
> So as Francois mentions we already have a UTYPE for the overall dataset
> footprint:
> 
>     Char.SpatialAxis.Coverage.Support.Area
> 
> This is currently what is used in Obscore, SSA, and Spectrum, and soon
> also in SpectralDM, SED, TimeSeries, SIAV2, etc.  So it already provides
> a consistent way to specify a simple footprint, or very nearly so.
> Obviously we do want this to be consistent across all these models,
> along with many other things.

This option seems to me to abuse the way that namespaces are intended
to be used, and lacks the flexibility and power of both Markus's and
Norman's suggestions.  It looks eminently comprehensible and easy to use
from both provider and consumer points of view though.  Gets my vote.

Mark

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

From norman at astro.gla.ac.uk  Mon Dec 19 04:18:21 2011
From: norman at astro.gla.ac.uk (Norman Gray)
Date: Mon, 19 Dec 2011 12:18:21 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
Message-ID: <43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>


Doug, hello.

On 2011 Dec 19, at 01:33, Douglas Tody wrote:

>> It cannot be the UTYPE, if every data model uses a different UTYPE for a 
>> footprint.
> 
> Well this is the key point; yes we can standardize the core model and
> the UTYPE used for this purpose and they do not need to be different
> despite being included in different models.  Yes the namespace (dataset
> class or context) may be different for different classes of data, but
> the UTYPE, minus the namespace prefix defining the context where it
> used, can be the same and can be defined to meant the same thing.  This
> is how UTYPEs have historically been used up to now, and we can
> formalize this further in the UTYPE spec (hopefully without
> over-complicating the mechanism).

What would this achieve?

Say an application comes across a UTYPE

    foo:SpatialAxis.Coverage.Support.Area,

and suppose that the 'foo' namespace was standardised after the application itself was written, or since it was last updated.  Should that application treat this as a footprint?

I presume the answer is 'no'.  In that case, the application is stuck -- it has no idea what to do with this UTYPE, and cannot have any idea, until the application author reads the relevant standard and encodes it in an update of the application, which is then released and distributed.

If the answer is 'yes', because 'SpatialAxis.Coverage.Support.Area' is expected to mean the same thing everywhere, then it is clear that there is no point in having namespaces, and people should stop talking about them.

Best wishes,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK


From francois.bonnarel at astro.unistra.fr  Mon Dec 19 14:43:46 2011
From: francois.bonnarel at astro.unistra.fr (=?ISO-8859-1?Q?Fran=E7ois_Bonnarel?=)
Date: Mon, 19 Dec 2011 23:43:46 +0100
Subject: utype for STC region in SIAP query response
In-Reply-To: <43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
	<43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>
Message-ID: <4EEFBE22.2030709@astro.unistra.fr>

Dear all,
      This discussion is really interesting but I think we should avoid 
being to theoretic and generic and do as we didn't have any material yet.

      a ) As far as I understand  the problem adressed by Alberto (and 
previously by Thomas ) is : "do we have a single utype for OBSERVATION 
footprints?"
and this whatever DAL protocol we are using: Obstap, SIA2, SSA, and 
maybe SED and Time series (as mentionned by Doug and Norman)... In that 
case the utype explicitly defined  by the Obscore/Obstap recommendation 
"obscore:Char.SpatialAxis.Coverage.Support.Area" is the most generic and 
should be reused in all the more precise and derived models of the 
specific protocols such as SIA2, SSA, etc ... Check the first SIA2 draft 
there 
http://www.ivoa.net/internal/IVOA/SiaInterface/WD-SIAP-2.0-20091104.pdf 
and you will discover the SAME utype for the same context allready more 
than 2 years ago...
       b ) stc is both a model and a structure definition ...for 
coordinates, regions in general and coordinate systems. Several possible 
structures (stc-X, stc-S) are defined for the same model attributes...
The full char/obs model defines Char.SpatialAxis.Coverage.Support.Area 
as an stc:AstroCoordArea structure... In this very specific case the use 
of an STC-S structure in the VOTABLE FIELD identified by the correct 
Obs/Char utype will be qualified by the   xtype="Stc-S" attribute of the 
FIELD and this will allow to INFER accuratly  the stc utype if 
needed...  All this is possible there because we are not talking of 
something generic but of using stc structures in the context of 
observation footprints...
       c ) The problem     of other kind of footprints, for example 
catalog or survey footprints  may require other combinations of stc 
structures or utypes with other models such as a catalog data model if 
it  exists some day. STC is generic for geometric regions and can 
require combination with other models for adding semantics as suggested 
by Norman... The actual combination mode is use-case dependant . 
References to STC-X structures may be possible or needed sometimes.
      d )  The solutions proposed by Norman and Markus are very generic 
and maybe really usefull in the case of other combinations of models 
than the one adressed by Thoams and Alberto . I have in mind 
Observation/provenance and simulation data model, or characterisation 
and simulation data models... BUt in the case of the obtsap and simple 
DAL  protocols,  I think the utype/xtype combination provides all what 
we need and is unambiguous...
Cheers
Fran?ois

Le 19/12/2011 13:18, Norman Gray a ?crit :
> Doug, hello.
>
> On 2011 Dec 19, at 01:33, Douglas Tody wrote:
>
>>> It cannot be the UTYPE, if every data model uses a different UTYPE for a
>>> footprint.
>> Well this is the key point; yes we can standardize the core model and
>> the UTYPE used for this purpose and they do not need to be different
>> despite being included in different models.  Yes the namespace (dataset
>> class or context) may be different for different classes of data, but
>> the UTYPE, minus the namespace prefix defining the context where it
>> used, can be the same and can be defined to meant the same thing.  This
>> is how UTYPEs have historically been used up to now, and we can
>> formalize this further in the UTYPE spec (hopefully without
>> over-complicating the mechanism).
> What would this achieve?
>
> Say an application comes across a UTYPE
>
>      foo:SpatialAxis.Coverage.Support.Area,
>
> and suppose that the 'foo' namespace was standardised after the application itself was written, or since it was last updated.  Should that application treat this as a footprint?
>
> I presume the answer is 'no'.  In that case, the application is stuck -- it has no idea what to do with this UTYPE, and cannot have any idea, until the application author reads the relevant standard and encodes it in an update of the application, which is then released and distributed.
>
> If the answer is 'yes', because 'SpatialAxis.Coverage.Support.Area' is expected to mean the same thing everywhere, then it is clear that there is no point in having namespaces, and people should stop talking about them.
>
> Best wishes,
>
> Norman
>
>



From dtody at nrao.edu  Mon Dec 19 17:35:27 2011
From: dtody at nrao.edu (Douglas Tody)
Date: Mon, 19 Dec 2011 18:35:27 -0700 (MST)
Subject: utype for STC region in SIAP query response
In-Reply-To: <43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
	<43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>
Message-ID: <alpine.OSX.1.00.1112191540420.295@colorado.tuc.noao.edu>

Hi Norman -

I hope the holidays are treating you well.

On Mon, 19 Dec 2011, Norman Gray wrote:
> On 2011 Dec 19, at 01:33, Douglas Tody wrote:
>
>>> It cannot be the UTYPE, if every data model uses a different UTYPE for a
>>> footprint.
>>
>> Well this is the key point; yes we can standardize the core model and
>> the UTYPE used for this purpose and they do not need to be different
>> despite being included in different models.  Yes the namespace (dataset
>> class or context) may be different for different classes of data, but
>> the UTYPE, minus the namespace prefix defining the context where it
>> used, can be the same and can be defined to meant the same thing.  This
>> is how UTYPEs have historically been used up to now, and we can
>> formalize this further in the UTYPE spec (hopefully without
>> over-complicating the mechanism).
>
> What would this achieve?
>
> Say an application comes across a UTYPE
>
>    foo:SpatialAxis.Coverage.Support.Area,
>

> and suppose that the 'foo' namespace was standardised after the
> application itself was written, or since it was last updated.  Should
> that application treat this as a footprint?

In the first place lets not forget that the problem we need to solve
here is fairly limited.  VO only requires half a dozen or so interfaces
for the major classes of data (at least for observation-based data).  We
have nearly all of them now and they are already fairly consistent.  We
can eventually make them fully consistent in terms of the standard
metadata and submodels (Char etc.) included in each object class.  It
does not have to be all that complicated and the mechanism should not be
allowed to get too complicated, at least not for the standard core
metadata.

> I presume the answer is 'no'.  In that case, the application is stuck --
> it has no idea what to do with this UTYPE, and cannot have any idea,
> until the application author reads the relevant standard and encodes it
> in an update of the application, which is then released and distributed.
>
> If the answer is 'yes', because 'SpatialAxis.Coverage.Support.Area' is
> expected to mean the same thing everywhere, then it is clear that there
> is no point in having namespaces, and people should stop talking about
> them.

Neither of these options is quite true - Utypes from submodels can be
standardized and reused in larger models, but we still need namespaces
to describe and understand entire new dataset classes and their
relationships.

Namespaces (as well as more formal rules for how to reuse existing
standard data models in new models) can provide a solution to this
problem.  A data model namespace, at least within the DAL interfaces,
corresponds to an object class like Spectrum, SSA query response,
Observation/Dataset, etc.  Usually a new class extends an existing one
inheriting the bulk of its attributes, and this relationship can be
specified as part of the computer readable namespace metadata.

Hence if a new dataset class appears which an application is not aware
of, but it extends an existing well known class, then the application
can assume that any standard Utypes defined by the parent class are
inherited by the new class if the Utype name does not change (assuming
we define that as a composition rule).  So long as the application
understands this relationship, represented in a computer readable form
by the runtime namespace definition, it will already understand much
about the new dataset class.  Of course any new semantics (object
attributes or Utypes, modified semantics for same, or larger constructs
relating to the new object) will not be knowable without fully
understanding the new class of data, but the generic stuff will be well
defined.

So for example Spectrum extends "Spectral" (the new Spectral DM).  Then
later we add TimeSeries which also extends Spectral.  An application
which understands Spectral already understands much about TimeSeries, in
particular anything specific to the underlying generic model.
Ultimately we can extend this to the underlying Observation/Dataset
model (which is the basis for all the DAL interfaces) once the data
models eventually stabilize and converge.

In theory we could instead compose an object instance by instantiating
external submodels like Char, Photometry, STC, Target, etc. in
combination with some custom object metadata.  But this is difficult and
probably overkill - too complex.  It is simpler to incorporate subsets
of well defined versions of these models into a unified model such as
ObsCore, SpectralDM, or whatever, otherwise we have death by version
confusion and model bloat (including stuff not needed by the specific
use case).

This larger problem is really a DM architecture issue as we discussed in
Pune.  Formalizing the Utype mechanism is part of it.  What we have
already works fairly well but it would be good to clarify the conceptual
model at the basis for all this.

 	- Doug


From norman at astro.gla.ac.uk  Thu Dec 22 08:04:22 2011
From: norman at astro.gla.ac.uk (Norman Gray)
Date: Thu, 22 Dec 2011 16:04:22 +0000
Subject: utype for STC region in SIAP query response
In-Reply-To: <alpine.OSX.1.00.1112191540420.295@colorado.tuc.noao.edu>
References: <201109011804.p81I4f9Q025872@xebec.cfa.harvard.edu>
	<alpine.OSX.1.00.1109011725230.385@colorado.tuc.noao.edu>
	<AD7735D8-635B-4844-A7FA-7C627A84F135@googlemail.com>
	<alpine.OSX.1.00.1112181810120.295@colorado.tuc.noao.edu>
	<43753CA0-C146-4059-B074-CB013FA5A59E@astro.gla.ac.uk>
	<alpine.OSX.1.00.1112191540420.295@colorado.tuc.noao.edu>
Message-ID: <846CF5F8-72D3-4F52-A4D2-19180295D9A6@astro.gla.ac.uk>


Doug, hello.

On 2011 Dec 20, at 01:35, Douglas Tody wrote:

> I hope the holidays are treating you well.

Here's hoping; and to you, too.  I'm finishing up for the New Year break this afternoon, so I'll have to bow out of this discussion with this message.

I don't really have a horse in this race, so I participate in the discussion with some diffidence (possibly more diffidence than may be obvious...).  My remarks below are rather general, therefore, and are intended to express my general uneasiness with the overall shape of the DM proposals.  Sans horse, I can't reasonably go much further than that.

>> and suppose that the 'foo' namespace was standardised after the
>> application itself was written, or since it was last updated.  Should
>> that application treat this as a footprint?
> 
> In the first place lets not forget that the problem we need to solve
> here is fairly limited.  VO only requires half a dozen or so interfaces
> for the major classes of data (at least for observation-based data).  We
> have nearly all of them now and they are already fairly consistent.  We
> can eventually make them fully consistent in terms of the standard
> metadata and submodels (Char etc.) included in each object class.  It
> does not have to be all that complicated and the mechanism should not be
> allowed to get too complicated, at least not for the standard core
> metadata.

The argument from practicality is a strong one, and comes down to judgements about the relative weight of different downsides.

Having said that, I'm looking at the draft UTYPE specification at <http://www.ivoa.net/internal/IVOA/Utypes/WD-Utypes-0.4-20091107.pdf> and can see UML (therefore object oriented notions of inheritance), XML Schemas, some quasi-XPath, a partial conflation of data modelling and interface design, a novel conceptual and syntactical framework for composing names which are derived from UML models, minus an associated semantics, but plus a sketched abbreviation syntax.  I broadly understand why each of those things is there, historically, but the end result is _scarily_ complicated.

If the things I've suggested in the past sound half as complicated as the UTYPE draft, then I've been explaining them very badly.

Perhaps this doesn't matter.  If everyone in the IVOA knows roughly what all the UTYPEs mean, and uses them by spotting substrings in @utype attributes, then this may end up stable in the long term.  A logically similar mechanism long worked for FITS, though FITS was hampered by with shorter names, and a more ad hoc approach to generating and documenting them.  The informality does however limit what you can do with the terms in the future, to roughly what you can do with them now.

The other thing is...

> A data model namespace, at least within the DAL interfaces,
> corresponds to an object class...

> Usually a new class extends an existing one...

> ...but it extends an existing well known class

> In theory we could instead compose an object instance by instantiating
> external submodels like Char, Photometry, STC, Target, etc.

The discussion round UTYPEs is often in terms of object-oriented design (for example in the repeated implicit and explicit reference to UML).  But interface design is very different from data modelling: one is about the entry points to a library -- meaning predictability of behaviour -- and the other is about the static structure of data -- meaning predictability and interchangeability of meaning.

This is not angels-on-pins territory.  There is clearly some overlap between the two activities, since an interface implies something about the structure of the domain, and a data model suggests a natural way to get access to the data, but the two activities are very different.

A Java interface definition (object modelling) makes some guarantees about the input and output to functions, and what functions are available, and (as it turns out) can help robustness by keeping some bits of information inaccessible; but it has next to nothing to say about data or meaning. An XML Schema definition (data modelling) is pretty obviously about shared meaning, but the only link with object-orientiation is that both frameworks happen to use the word 'inheritance' (nothing I say should imply that I think that XSchema is a good data modelling language).  The fact that there are libraries which generate Java classes from XSchemas tends to blur the distinction, but (to me) just explains why these generated libraries are less use than one might expect.

In this light, the appearance of object-modelling terminology and techniques in the IVOA's _data_ modelling group is slightly puzzling.

----

But that's quite enough from me (more than enough, I'm sure I heard someone mutter...).  Have a good break, everyone who's taking one, and have a good New Year when it comes.

All the best,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK


From arots at head.cfa.harvard.edu  Thu Dec 22 08:33:14 2011
From: arots at head.cfa.harvard.edu (Arnold Rots)
Date: Thu, 22 Dec 2011 11:33:14 -0500 (EST)
Subject: utype for STC region in SIAP query response
In-Reply-To: <846CF5F8-72D3-4F52-A4D2-19180295D9A6@astro.gla.ac.uk>
Message-ID: <201112221633.pBMGXEhc007829@xebec.cfa.harvard.edu>

Norman Gray wrote:
> 
> Doug, hello.
> 
> On 2011 Dec 20, at 01:35, Douglas Tody wrote:
> 
> > I hope the holidays are treating you well.
> 
> Here's hoping; and to you, too.  

And the same to all of you from me.

> ...
> The argument from practicality is a strong one, and comes down to
> judgements about the relative weight of different downsides. 
> 
> Having said that, I'm looking at the draft UTYPE specification at
> <http://www.ivoa.net/internal/IVOA/Utypes/WD-Utypes-0.4-20091107.pdf>
> and can see UML (therefore object oriented notions of inheritance),
> XML Schemas, some quasi-XPath, a partial conflation of data
> modelling and interface design, a novel conceptual and syntactical
> framework for composing names which are derived from UML models,
> minus an associated semantics, but plus a sketched abbreviation
> syntax.  I broadly understand why each of those things is there,
> historically, but the end result is _scarily_ complicated. 

At the risk of sounding like a broken record, that's why I keep
suggesting the simple generic utype stc:AstroCoordArea.Region,
or in context something like:

     <GROUP utype="stc:CatalogEntryLocation">
         <FIELDref utype="stc:AstroCoordArea.Region" ref="footprint"/>
     </GROUP>

It's independent of any other models that may be hanging around, yet
unambiguous.

  - Arnold

> 
> ...
> But that's quite enough from me (more than enough, I'm sure I heard
> someone mutter...).  Have a good break, everyone who's taking one,
> and have a good New Year when it comes. 
> 
> All the best,
> 
> Norman
> 
> 
> -- 
> Norman Gray  :  http://nxg.me.uk
> SUPA School of Physics and Astronomy, University of Glasgow, UK
> 
> 
--------------------------------------------------------------------------
Arnold H. Rots                                Chandra X-ray Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 67                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots at head.cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------