thakar at skysrv.pha.jhu.edu
Wed Dec 7 15:26:33 PST 2005
There are two different logging services that we are talking about here,
- site logging
- VO logging
The first is the logging service at a given site that may or may not
provide a query interface for users to view the logged data. This logs
all the services at a given site (e.g. STScI).
The second is the VO-wide log harvesting that collects the logs from all
the VO services in a VO domain (e.g. NVO).
Which one do you mean by "centralized logging service"?
The logging interface in the support interfaces draft only deals with
the minimum that any site or service needs to do to get its log data
harvested to the VO logging VOStore.
It's fine if each site has a central logging service that has a log
harvesting interface hanging off it that the VO log harvester can use to
retrieve log data from it periodically. Each individual service does
not need to support a log harvesting interface.
In the current proposal the VO log harvester initiates the xfer of log
data by giving the site a range of dates and a VOStore identifier. The
site then pushes the log data to the VO harvester store.
This way if there are any changes to the VO harvester store location or
authentication, all the VO sites need not be updated.
Anyway, more at the tech-WG telecon tomorrow.
On Wed, 7 Dec 2005, KevinBenson wrote:
> Yes this is sounding quite good. I like the idea of a more centralized
> logging service where by other services can push the data to the central
> logging service.
> Now for the central logging service I don't think in this first version
> it would need a "push" model, I would prefer to let clients/apps come
> and get the data.
> A possible suggestion is "OAI" for obtaining the data from centralized
> logging services. Deals with dates such as "from" or "until". One of
> the IVOA components "registry" currently uses it. And also deals with
> large amounts of data with the notion of a "resumptionToken" allowing a
> client to page through the results. And finally there are already
> several OAI tools to help the implementation.
> Doug Tody wrote:
> > One way this could work:
> > o Each site operates a logging service which is distinct from
> > other
> > service instances (for data access, computation, etc.).
> > o Services send lightweight messages to the local, centralized
> > logging service whenever anything happens. This helps keeps the
> > services simple and stateless.
> > o The logging service has the ability to both push logging data to
> > subscribers, as well as respond to queries for logging data.
> > o In push or publish/subscribe mode, it would be relatively
> > easy to
> > buffer data and send a message containing logging data only
> > periodically. This would avoid the problems of managing very
> > large amounts of logging data, but would still be efficient due
> > to the buffering.
> > o The logging service could also expose a query interface for
> > a client to directly access logging data, e.g., by a range of
> > dates, everything since a certain date, or some other method.
> > Since it is a separate service this does not complicate other
> > service interfaces.
> > o If security is an issue it can easily be dealt with by the
> > logging
> > service. This is a somewhat different case than for normal
> > data access/analysis services, as only grid infrastructure needs
> > to access logging information.
> > It would also be possible for simple services which are not heavily used
> > (small sites) to send small amounts of logging information to a remote
> > logging service, to avoid the need for small sites to support a logging
> > service locally. - Doug
Aniruddha R. Thakar, Research Scientist
Center for Astrophysical Sciences, JHU
3701 San Martin Drive, Baltimore MD 21218-2695
410-516-4850, Fax: 410-516-5096
thakar at jhu.edu, http://www.sdss.jhu.edu/~thakar
Public opinion is a weak tyrant compared with our own private opinion.
More information about the grid