How do we get metadata into the registry network?
rplante at ncsa.uiuc.edu
Fri Jun 25 10:12:12 PDT 2004
On Fri, 25 Jun 2004, Clive Page wrote:
> One notable omission from the Registry interfaces document dated June 16th
> is any way of loading metadata from the primary sources, which are the
> files or databases in each data centre.
> Where to store metadata?
The reason you haven't found much on this topic is perhaps because it is
not an issue we need to standardize on. As you alluded to, many centers
will need to automate the exposure of their metadata by hooking the
harvesting interface into their local metadata management systems. How
this will be done will vary widely. Certainly, it will help if we can
provide general tools that assist with this.
This is how it's been approached in the NVO:
o For the data provider that has only a few resources (e.g. <10) to
register, we have built registration forms at our various
registries that providers can fill out. I encourage you to try out
the page at NCSA:
With this form, a provider uses a public publishing registry to
expose their resources; that is, the records are store at the remote
o For the provider that has a few more resources (e.g. 10-100), they
may prefer to manage their records locally. For this set, we have
begun building deployable registries that have the harvesting
interface built in. Examples include NCSA's VORegistry-in-a-Box
Caltech's Carnivore (http://mercury.cacr.caltech.edu:8080/carnivore/)
o For the provider with many resources that have sophisticated
metadata management systems (like Vizier), they will want to plug the
harvesting interface directly into database. The examples from the
NVO prototype are HEASARC and CDS. These chose to build the
harvesting interface themselves (leveraging, I believe, existing OAI
> So - who is going to develop these tools?
As illustrated from the example above, development is shared between
providers and VO developers. If we want see broad participation from
providers, the cost must be low; thus, we must recognize:
1. Incremental levels of compliance--how much providers register, how
much info they provide, and what standard services they deploy--
makes it easier for providers build up participation gradually as
they can afford.
2. The more tools we can give providers to get over the difficult bits,
the more compliance they can afford.
I think both these are in evidence in the above example.
More information about the registry