Collaboration on Source Catalogue DM, ADQL and SkyNodes

[Pedro Osuna]

Hi Mark,

wonderful news. I do want to chat about it with you, and I suggest that
we do so from now on out of the general list to not annoy people
unnecessarily...

This is the type of collaboration I was seeking when sending a very
early -maybe even premature- mail, and at first glance your libraries
might fit easily in our scheme.

I'll send you a separate mail and look forward to a wide collaboration.

Thanks.
Cheers,
P.




On Wed, 2005-12-21 at 12:02 +0000, Mark Taylor wrote:
> On Wed, 21 Dec 2005, Pedro Osuna wrote:
> 
> > Hi Maria,
> > 
> > 
> > > - How does Catalogue Data Model used look like, especially what is the
> > >   common set of attributes and the associated metadata.
> > 
> > The point is in the (Source) Catalogue Data Model, with emphasis in the
> > "Source" part. This one is the one I showed on behalf of the Catalogue
> > DM subgroup at our last interop meeting here at ESAC. I attach a pdf
> > with the initial proposal, but please use it only for temporal
> > reference, as the whole document will be changed (according to
> > requirements from Jonathan after the interop meeting).
> > 
> > > - What are the plans about registration? Will these nodes (Basic?) be
> > >   registered and therefore accessible through Open SkyQuery? How many?
> > 
> > yes, they will. How many, I don't know. In Strasbourg, Inaki and
> > Aurelien worked on a couple of them, Tycho-2 and USNOB, but the CDS
> > colleagues will work on more.... Francois will answer to this question
> > at some point I presume.
> > 
> > > - The cross-match functionallity. Are you planning on a 2 catalog
> > >cross-match or n-catalog cross-match? How does the cross-match function
> > >look like?
> > 
> > n-catalogue cross-match is what we are trying to get at; it will be a
> > client based cross-match, and therefore the cross-match function will be
> > designed and run at the client side (i.e., servers do not need to worry
> > about implementing one specific cross-match or the other). 
> > At the current status, the client sends an ADQL to the server to discern
> > which type of cross-match it can do with it (whether only positional,
> > positional with errors, etc.), and takes the corresponding action.
> 
> Pedro,
> 
> my STILTS package contains a standalone crossmatcher suitable for
> client-side operation which might be of use in implementing the
> crossmatching functionality you need.  It is highly flexible 
> (can perform matches in N-dimensional space, e.g. sky position only
> or sky position plus proximity in one or more flux values),
> suitable for matches up to around a million rows or so (could be
> extended with some work), and fairly efficient (a recent test 
> matching 600,000 vs 700,000 row tables resulting in 2000 matches 
> took about 90 sec).  It does not understand ADQL so it's not going
> to be a complete solution to your problem, but it may be of use
> as a back end matching engine.  The tool currently only 
> matches two tables, but I'm planning to provide N-table matching
> fairly soon.  STILTS provides this functionality from a command-line
> tool (tmatch2), but a public java API is also available for 
> programs that want access to it within a JVM.
> 
> I'm not sure how you're planning to implement your matching, so
> this may not suit your purposes, but I'm happy to discuss it
> further if you think it might be of interest.
> 
> For more details see:
> 
>    http://www.starlink.ac.uk/stilts/sun256/tmatch2.html
> 
> Mark
> 
-- 
Pedro Osuna Alcalaya

ESA 
Science Archives System Engineer
Science Archive Team
European Space Astronomy Centre
(ESAC/ESA)
e-mail: Pedro.Osuna at esa.int
Tel + 34 91 8131314
---------------------------------                                                                                
European Space Astronomy Centre
European Space Agency
P.O. Box 50727
E-28080 Villafranca del Castillo
MADRID - SPAIN