Dear Francois,
this new approach, where you include different tables in the same RESOURCE results, is closer to what we propose.
Although we like it more than your previous proposal, we have been looking at the pros and cons of your approach vs ours, and these are the conclusions we have drawn. We think there is no-single solution for this issue, but also believe it is good to post these ideas to the whole community for discussion. Here they go:
In your example, a table "points" to a structure which is outside itself (another table). This is to avoid modification of the DTD to allow tables inside tables. All the tables appear, therefore, at the same level.
This means that the external tables do have to have an identifier to be able to do the joins among tables.
This would be an exact mapping of how an RDBMS works: independent tables joined through identifiers.
The problem with this approach is that it leaves the whole of the interpretation of the structure to the client.
As the service is hiding the structure (it does not have any structure by itself, only pointers and identifiers to be able to do joins), the client will have to re-interpret it. In the case of a couple of observations, this could be done fast. In the case of hundreds of observations, in your approach the client would have to go through the whole results for each and every observation to figure out which exposures belong to it (and later, for each and every exposure to find the sources, etc.).
As the results given in your way do not contain any specific ordering, the display client will have to make this ordering by itself. This reordering would be done _without_ indexes (in the database terminology sense) and therefore would be a very un-efficient task. In our case, however, indexes are implicit in the structure itself, as "substructures" (as exposures, sources, etc.) are already inserted below a specific "superstructure" (like observation) and therefore the client only travels the structures once.
In our proposal, therefore, the join issues are handled at the server level, and the structure is already unveiled to the client, which would only have to display it in one go.
Another "artifact" of your proposal is the use of those "dummy" placeholders:
<TD>Exposure_Table</TD>
These ones appear due to the fact that VOTable is imposing to have an entry per FIELD, and thus the <FIELD ID=Exposure_Table>, which in our case goes _inside_ the TABLE, has to have a dummy <TD> in your case as it points to an outside table.
This might look like cosmetics thing, but in our view it reveals an inconsistency between the VOTable real requirements and the way we would use the VOTables in your example. Of course, this problem would be solved by modifying somehow the DTD.
As a last example we were trying to use to clarify our own ideas, we were identifying the structure we wanted to include in the SIAP to a file system directory structure. In our approach, we send the directory structure directly to the client, allowing it to display it the way it wants. In your approach, a very simple directory structure would be completely flat, with each subdirectory containing a pointer to the directory where it hangs from, allowing to "join" directories and subdirectories. Instead of giving the structure directly to the client, you give it the tools to create the structure, but then it is the client which has to do the whole process of "structurization". Although a valid approach, we believe that the time spent by the client in doing this should be spent by the server, which in the end is doing an asynchronous work with the rest of the servers being requested.
Anyway, we believe this approach is much close to our idea of how things should work, and we think we are working towards a positive evolution for SIAP results.
Best regards,
P&J
--
Pedro Osuna Alcalaya
Software Engineer
VILSPA Archive Development Team
e-mail: Pedro.Osuna-at-esa.int
Tel + 34 91 8131314
European Space Agency
VILLAFRANCA Satellites Tracking Station
P.O. Box 50727
E-28080 Villafranca del Castillo
MADRID - SPAIN
On Mon, 2004-04-19 at 19:57, Francois Bonnarel wrote:
> Dear Jesus and Pedro,
>
> As I annouced in my mail last week-end I took your example
> and modified it a little bit by introducing in your peculiar fields
> "EXposure_Table" "Source_Table" and "Energy_Band_Table" a "VOTable
> ref" to a corresponding Table ID which one can find further in the
> VOTable document.
> In addition I created a "DIS:LowerLevelTable" pseudo UCD (DIS for
> discussion) which may be not necessary because the occurence of the
> reference may be sufficient to identify such kind of fields.
> I added the upper level Table Identifier field (ObsId) in the Exposure
> one in order to joint the Observation and Exposure Tables.
> Something similar would have to be done in Source and Energy_Band
> Tables which I was to lazy to complete!
> If I didn't misunderstood your example I Think this is a serialization
> of your data model at page 2, using some of the rules we described with
> Mireille in our "Data Model serialization in VOTable" note.
> This is done without modifying the VOTable DTD, which I think is reasonable
> for Boston, because there has been a very long process of discussion for
> version 1.1.
>
> Cheers
> François
>
>
>
> <?xml version = "1.0" ?>
> <!DOCTYPE VOTABLE (View Source for full doctype...)>
> <VOTABLE version = "1.0">
> <RESOURCE type="results">
> <DESCRIPTION> XMM-Newton Simple Image Protocol (SIAP) Service</DESCRIPTION>
> <INFO name="QUERY_STATUS" value="OK" />
> <TABLE ID="Observation">
> <FIELD ID="ObsId" ucd="OBS_ID" datatype="char" arraysize="*" />
> <FIELD ID="Target_Name" ucd="OBS_ID" datatype="char" arraysize="*" />
> <FIELD ID="Start_Time" ucd="VOX:OBS_START_TIME" datatype="char" arraysize="*" />
> <FIELD ID="End_Time" ucd="VOX:OBS_END_TIME" datatype="char" arraysize="*" />
> <FIELD ID="On_Time" ucd="VOX:OBS_DURATION" datatype="int" />
> <FIELD ID="RA" ucd="POS_EQ_RA_MAIN" datatype="char" arraysize="*" />
> <FIELD ID="DEC" ucd="POS_EQ_DEC_MAIN" datatype="char" arraysize="*" />
> <FIELD ID="FOV" ucd="VOX:Field_Of_View" datatype="char" arraysize="*" />
> <FIELD ID="FORMAT" ucd="VOX:Image_Format" datatype="char" arraysize="*" />
> <FIELD ID="Reference" ucd="DATA_LINK" datatype="char" arraysize="*" />
> <FIELD ID="Exposure_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Exposure" />
> <DATA>
> <TABLEDATA>
> <TR>
> <TD>0102640101</TD>
> <TD>XMM EPIC Target M33_1 </TD>
> <TD>2000-08-04 05:16:00.0 </TD>
> <TD>2000-08-04 10:27:12.0 </TD>
> <TD>18672</TD>
> <TD>23.458305</TD>
> <TD>30.66414</TD>
> <TD>0.72x0.72</TD>
> <TD>image/fits</TD>
> <TD>
> <![CDATA[http://xsa.vilspa.esa.es:8080/alo/jsp?obsno=0102640101&name=OIMAGE&level=PPS&extension=FTZ&protocol=HTTP]]>
> </TD>
> <TD>Exposure_Table</TD>
> </TR>
> </TABLEDATA>
> </DATA>
> </TABLE>
> <TABLE ID="Exposure">
> <FIELD ref="ObsId" />
> <FIELD ID="ExposureNumber" ucd="EXP_ID" datatype="char" arraysize="*" />
> <FIELD ID="Image_Name" ucd="VOX:Image_Title" datatype="char" arraysize="*" />
> <FIELD ID="Instrument" ucd="VOX:INST_ID" datatype="char" arraysize="*" />
> <FIELD ref="Start_Time" />
> <FIELD ref="End_Time" />
> <FIELD ref="On_Time" />
> <FIELD ref="RA" />
> <FIELD ref="DEC" />
> <FIELD ref="FOV" />
> <FIELD ref="FORMAT" />
> <FIELD ref="Reference" />
> <FIELD ID="Source_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Source" />
> <FIELD ID="Energy_Band_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Energy_Band" />
> <DATA>
> <DATA>
> <TABLEDATA>
> <TR>
> <TD>0102640101</TD>
> <TD>S001</TD>
> <TD>XMM EPIC EPN Image. Target : M31Core . Exposure S001 </TD>
> <TD>2000-08-04 06:06:43.0 </TD>
> <TD>2000-08-04 09:51:52.0 </TD>
> <TD>13509</TD>
> <TD>23.458305</TD>
> <TD>30.66414</TD>
> <TD>0.72x0.72</TD>
> <TD>image/fits</TD>
> <TD>
> <![CDATA[http://xsa.vilspa.esa.es:8080/alo/jsp?obsno=0102640101&&instname=PN&datasubsetno=8&name=IMAGE&level=PPS&extension=FTZ&protocol=HTTP]]>
> </TD>
> <TD>Source_Table</TD>
> <TD>Energy_Band_Table</TD>
> </TR>
> </TABLEDATA>
> </DATA>
> </TABLE>
> <TABLE ID = "Source">
> .......
> </TABLE>
> <TABLE ID = "Energy_Band">
> ......
> </TABLE>
> SAUVONS LA RECHERCHE : <http://recherche-en-danger.apinc.org/>
>
> =====================================================================
> Francois Bonnarel Observatoire Astronomique de Strasbourg
> CDS (Centre de donnees 11, rue de l'Universite
> astronomiques de Strasbourg) F--67000 Strasbourg (France)
>
> Tel: +33-(0)3 90 24 24 11 WWW: http://cdsweb.u-strasbg.fr/people/fb.html
> Fax: +33-(0)3 90 24 24 25 E-mail: bonnarel-at-astro.u-strasbg.fr
> ---------------------------------------------------------------------
--
Pedro Osuna Alcalaya
Software Engineer
VILSPA Archive Development Team
e-mail: Pedro.Osuna-at-esa.int
Tel + 34 91 8131314
European Space Agency
VILLAFRANCA Satellites Tracking Station
P.O. Box 50727
E-28080 Villafranca del Castillo
MADRID - SPAIN
Received on 2004-04-20Z13:39:48