From roy at caltech.edu Tue Jan 3 13:54:49 2012 From: roy at caltech.edu (Roy Williams) Date: Tue, 03 Jan 2012 13:54:49 -0800 Subject: VOEvent at AAS next week? Message-ID: <4F037929.10407@caltech.edu> Dear VOEventers For those going to the AAS in Austin Texas next week, perhaps we can meet up to discuss astronomical events, their representation, transport, information infrastructure, and so on? Any suggestions for relevant talks and sessions? Roy Williams --- Caltech LIGO roy at caltech.edu 626 395 3670 From mjg at cacr.caltech.edu Tue Jan 3 14:02:14 2012 From: mjg at cacr.caltech.edu (Matthew Graham) Date: Tue, 3 Jan 2012 14:02:14 -0800 Subject: VOEvent at AAS next week? In-Reply-To: <4F037929.10407@caltech.edu> References: <4F037929.10407@caltech.edu> Message-ID: <211D1BB4-F5B5-4AC0-80D8-C1054327E6CE@cacr.caltech.edu> Hi, It might be easiest to suggest a time/venue and just see who turns up? Cheers, Matthew On Jan 3, 2012, at 1:54 PM, Roy Williams wrote: > Dear VOEventers > > For those going to the AAS in Austin Texas next week, perhaps we can meet up to discuss astronomical events, their representation, transport, information infrastructure, and so on? Any suggestions for relevant talks and sessions? > > Roy Williams > > --- > Caltech LIGO > roy at caltech.edu > 626 395 3670 > From Hessman at Astro.physik.Uni-Goettingen.DE Tue Jan 3 14:09:41 2012 From: Hessman at Astro.physik.Uni-Goettingen.DE (Frederic V. Hessman) Date: Tue, 3 Jan 2012 23:09:41 +0100 Subject: VOEvent at AAS next week? In-Reply-To: <4F037929.10407@caltech.edu> References: <4F037929.10407@caltech.edu> Message-ID: <00DD14CE-5F29-4A7F-85DA-152D8FFEDE9A@Astro.physik.Uni-Goettingen.DE> I'm in the process of moving within G?ttingen and go to McDonald Observatory at the end of the month, so I unfortunately missed the Austin AAS meeting. May I suggest you all meet for BBQ and a local beer at the "County Line" and then tell me how wonderful it was....... ;-( Rick On 3 Jan 2012, at 22:54, Roy Williams wrote: > Dear VOEventers > > For those going to the AAS in Austin Texas next week, perhaps we can meet up to discuss astronomical events, their representation, transport, information infrastructure, and so on? Any suggestions for relevant talks and sessions? > > Roy Williams > > --- > Caltech LIGO > roy at caltech.edu > 626 395 3670 From seaman at noao.edu Tue Jan 3 14:17:43 2012 From: seaman at noao.edu (Rob Seaman) Date: Tue, 3 Jan 2012 15:17:43 -0700 Subject: VOEvent at AAS next week? In-Reply-To: <211D1BB4-F5B5-4AC0-80D8-C1054327E6CE@cacr.caltech.edu> References: <4F037929.10407@caltech.edu> <211D1BB4-F5B5-4AC0-80D8-C1054327E6CE@cacr.caltech.edu> Message-ID: <35A120B3-9087-46BE-A9E3-44944A11AE42@noao.edu> Roy might join me in advising Matthew that the former chairs of the WG have not noted that "easiest" is a hallmark of getting things done around here. Let's rephrase the action items: 1) Speak up if you will be at the Austin meeting! 2) What is the best day for folks to meet? 3) Don't forget the Skype. I will not "turn up". I will go out of the way to attend remotely if arrangements are made. Still just time enough to sneak in an abstract for the SPIE. Amsterdam is not Austin, but some of us will try to find a way to deal with the angst. Rob -- On Jan 3, 2012, at 3:02 PM, Matthew Graham wrote: > Hi, > > It might be easiest to suggest a time/venue and just see who turns up? > > Cheers, > > Matthew > > > On Jan 3, 2012, at 1:54 PM, Roy Williams wrote: > >> Dear VOEventers >> >> For those going to the AAS in Austin Texas next week, perhaps we can meet up to discuss astronomical events, their representation, transport, information infrastructure, and so on? Any suggestions for relevant talks and sessions? >> >> Roy Williams From scott at milkyway.gsfc.nasa.gov Tue Jan 3 14:29:43 2012 From: scott at milkyway.gsfc.nasa.gov (Scott Barthelmy) Date: Tue, 3 Jan 2012 17:29:43 -0500 Subject: VOEvent at AAS next week? In-Reply-To: <4F037929.10407@caltech.edu> References: <4F037929.10407@caltech.edu> Message-ID: <20120103222943.GF18837@gris1.gsfc.nasa.gov> I won't/can't be going to Austin (Gov travel restrictions). Sigh, --scott > Dear VOEventers > > For those going to the AAS in Austin Texas next week, perhaps we can > meet up to discuss astronomical events, their representation, transport, > information infrastructure, and so on? Any suggestions for relevant > talks and sessions? > > Roy Williams > > --- > Caltech LIGO > roy at caltech.edu > 626 395 3670 From rdenny at dc3.com Tue Jan 3 17:46:34 2012 From: rdenny at dc3.com (Bob Denny) Date: Tue, 03 Jan 2012 18:46:34 -0700 Subject: VOEvent at AAS next week? In-Reply-To: <11633443.34289.1325628190747.JavaMail.root@m11> References: <11633443.34289.1325628190747.JavaMail.root@m11> Message-ID: <4F03AF7A.40608@dc3.com> Can't attend for time and money reasons. I wish I could, it has been years since I attended a AAS meeting and I suspect it might be good for my business. Just too soon after the holidays mainly. I have a TON of traffic to catch up on. PS: I wrote two separate responses to the recent transport discussion, and elected to send neither. Maybe I'll look at it again in a few days and see if I can restrain myself :-) -- Bob You said: > Dear VOEventers > > For those going to the AAS in Austin Texas next week, perhaps we can > meet up to discuss astronomical events, their representation, transport, > information infrastructure, and so on? Any suggestions for relevant > talks and sessions? > > Roy Williams > > --- > Caltech LIGO > roy at caltech.edu > 626 395 3670 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From petr at iaa.es Tue Jan 3 23:49:35 2012 From: petr at iaa.es (Petr =?ISO-8859-1?Q?Kub=E1nek?=) Date: Wed, 04 Jan 2012 08:49:35 +0100 Subject: VOEvent at AAS next week? In-Reply-To: <4F03AF7A.40608@dc3.com> References: <11633443.34289.1325628190747.JavaMail.root@m11> <4F03AF7A.40608@dc3.com> Message-ID: <1325663375.2934.3.camel@theta> Hi, > PS: I wrote two separate responses to the recent transport discussion, > and elected to send neither. Maybe I'll look at it again in a few days > and see if I can restrain myself :-) On similar tone - I even did not have time to read the full discussion, as I have real telescopes observing (and crashing) on real mountains. But I will try to at least read it.. Petr From mjg at cacr.caltech.edu Wed Jan 4 00:11:20 2012 From: mjg at cacr.caltech.edu (Matthew Graham) Date: Wed, 4 Jan 2012 00:11:20 -0800 Subject: Transport discussion In-Reply-To: <1325663375.2934.3.camel@theta> References: <11633443.34289.1325628190747.JavaMail.root@m11> <4F03AF7A.40608@dc3.com> <1325663375.2934.3.camel@theta> Message-ID: <09743517-7D55-453B-8336-F768DD5534D9@cacr.caltech.edu> Hi, I was very pleased to see the level of active discussion on the list recently about transport protocols. It was interesting to hear about people's experiments with new mechanisms and I think it is important that we continue to track such technologies. However, I would like to reiterate the position that the IVOA VOEvent WG does not nor intends to mandate any particular transport mechanism. We are happy, however, to define new protocols that may be of use to the community (such as TCPV specified in the IVOA Note on Transport Protocols (http://www.ivoa.net/Documents/Notes/VOEventTransport/)) or to provide guidelines on how existing (third-party) technologies should be used by the VOEvent community (such as Jabber/XMPP). It is along these latter lines that we intend to "standardize" the transport infrastructure this year (there is prior art in such standardization of security and web service technologies within the IVOA) and I look forward to the participation of many of you in this process. Please continue to use this list to discuss and report on transportation issues (and other VOEvent-related matters). Cheers, Matthew (VOEvent WG Chair) -------------- next part -------------- An HTML attachment was scrubbed... URL: From swinbank at transientskp.org Thu Jan 5 09:11:04 2012 From: swinbank at transientskp.org (John Swinbank) Date: Thu, 5 Jan 2012 18:11:04 +0100 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> Message-ID: Hello, On 27 Dec 2011, at 21:22, Mike Fitzpatrick wrote: > The prospect of many thousands of parallel vTCP connections is very appealing > either ... > > Remember too, for some events to be useful (e.g. GRB followup) they > cannot be delivered 12 hours later, so the problem (at least a major use-case) > has to also solve the question of how to deliver 10K (or is it 50K, or 100K?) events > in a matter of *minutes*. I don't think the bar napkin on which vTCP was designed > was big enough to consider that case thoroughly 8-) Just for amusement, I wrote a simple test to see how this might work. My results & the code that generated them are at . Broadly speaking, maintaining a rate of 2e6 events in 12 hours (~50 events/second), or even 50K events in 5 minutes (167 events/second), was pretty doable. My simple VOEvent generator ran out of steam trying to send 100K events in 5 minutes (333 events/second), though. I certainly don't suggest that's a comprehensive test of the protocol ? the aim of the exercise was simply my personal education ? but perhaps the code is of interest to others. Cheers, John From roy at caltech.edu Thu Jan 5 09:28:55 2012 From: roy at caltech.edu (Roy Williams) Date: Thu, 05 Jan 2012 09:28:55 -0800 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> Message-ID: <4F05DDD7.5020604@caltech.edu> Thanks for this work, John, I think after all the talking, you are the first to actually run some tests! But ...... When I fetch the paper mail from the mailbox in front of my house, I have no difficulty carrying the quantity of paper. Not so easy is the task of separating out unwanted mail, connecting messages with their previous context, and deciding how to respond to what is left. On the publishing side, I have no problem carrying 100 blank envelopes (or even 1000!), but it would take me a long time to write 100 letters. Therefore let us not buy super trucks that can carry up to 1000 kg of envelopes! Because the rate-limiting step is not the bulk of the messages, but rather the evaluation of those messages. We could investigate ways to farm out event annotation and decision into remote ('cloud') servers. We could bundle events to get more efficiency in the evaluation. We could split streams to multiple servers. We could have a sequence of decision filters of increasing sophistication. There are a lot of dimensions to this problem. Roy On 1/5/12 9:11 AM, John Swinbank wrote: > Hello, > > On 27 Dec 2011, at 21:22, Mike Fitzpatrick wrote: > >> The prospect of many thousands of parallel vTCP connections is very >> appealing either ... >> >> Remember too, for some events to be useful (e.g. GRB followup) >> they cannot be delivered 12 hours later, so the problem (at least a >> major use-case) has to also solve the question of how to deliver >> 10K (or is it 50K, or 100K?) events in a matter of *minutes*. I >> don't think the bar napkin on which vTCP was designed was big >> enough to consider that case thoroughly 8-) > > Just for amusement, I wrote a simple test to see how this might work. > My results& the code that generated them are > at. > > Broadly speaking, maintaining a rate of 2e6 events in 12 hours (~50 > events/second), or even 50K events in 5 minutes (167 events/second), > was pretty doable. My simple VOEvent generator ran out of steam > trying to send 100K events in 5 minutes (333 events/second), though. > > I certainly don't suggest that's a comprehensive test of the protocol > ? the aim of the exercise was simply my personal education ? but > perhaps the code is of interest to others. > > Cheers, > > John -- --- Caltech LIGO roy at caltech.edu 626 395 3670 From aa at astro.ex.ac.uk Thu Jan 5 10:46:34 2012 From: aa at astro.ex.ac.uk (Alasdair Allan) Date: Thu, 5 Jan 2012 18:46:34 +0000 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: <4F05DDD7.5020604@caltech.edu> References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> <4F05DDD7.5020604@caltech.edu> Message-ID: > Thanks for this work, John, I think after all the talking, you are the first to actually run some tests! But ...... As I've repeatedly mentioned before, I ran similar simulated network tests several years ago. Al. From seaman at noao.edu Thu Jan 5 16:02:41 2012 From: seaman at noao.edu (Rob Seaman) Date: Thu, 5 Jan 2012 17:02:41 -0700 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> <4F05DDD7.5020604@caltech.edu> Message-ID: Alasdair Allan wrote: >> Thanks for this work, John, I think after all the talking, you are the first to actually run some tests! But ...... > > As I've repeatedly mentioned before, I ran similar simulated network tests several years ago. We welcome your renewed participation! :-) Roy Williams continued: >> When I fetch the paper mail from the mailbox in front of my house, I have no difficulty carrying the quantity of paper. Not so easy is the task of separating out unwanted mail, connecting messages with their previous context, and deciding how to respond to what is left. On the publishing side, I have no problem carrying 100 blank envelopes (or even 1000!), but it would take me a long time to write 100 letters. Therefore let us not buy super trucks that can carry up to 1000 kg of envelopes! Because the rate-limiting step is not the bulk of the messages, but rather the evaluation of those messages. This is the "Miracle on 34th Street" effect: "Hey Lou, how many Santa Claus letters we got at the dead-letter office?" Lou: "Bags and bags all over the joint. And there's more coming in every day!" System integration is in the edge cases. Rob From swinbank at transientskp.org Fri Jan 6 03:31:37 2012 From: swinbank at transientskp.org (John Swinbank) Date: Fri, 6 Jan 2012 12:31:37 +0100 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: <4F05DDD7.5020604@caltech.edu> References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> <4F05DDD7.5020604@caltech.edu> Message-ID: Hi Roy, I think there are (at least) two issues here, both of which are relevant. Of course you're correct that given some millions of events (or letters), extracting the relevant information is challenging ? indeed, I'm sure that's far more challenging than simply transporting the envelopes about. It's surely something that this WG will have to devote some considerable thought to. However, I don't think we can avoid the issue of bulk transport. I assume that the LSST folks (for example) have done their homework, and really will produce a couple of million events which are worthy of distribution per night. They will then face the problem of shifting them down a high-latency, low-bandwidth line. They still need a "super truck" capable of that, even if it's only to carry letters to a sorting office. Cheers, John On 5 Jan 2012, at 18:28, Roy Williams wrote: > Thanks for this work, John, I think after all the talking, you are the first to actually run some tests! But ...... > > When I fetch the paper mail from the mailbox in front of my house, I have no difficulty carrying the quantity of paper. Not so easy is the task of separating out unwanted mail, connecting messages with their previous context, and deciding how to respond to what is left. On the publishing side, I have no problem carrying 100 blank envelopes (or even 1000!), but it would take me a long time to write 100 letters. Therefore let us not buy super trucks that can carry up to 1000 kg of envelopes! Because the rate-limiting step is not the bulk of the messages, but rather the evaluation of those messages. > > We could investigate ways to farm out event annotation and decision into remote ('cloud') servers. We could bundle events to get more efficiency in the evaluation. We could split streams to multiple servers. We could have a sequence of decision filters of increasing sophistication. There are a lot of dimensions to this problem. > > Roy > > > On 1/5/12 9:11 AM, John Swinbank wrote: >> Hello, >> >> On 27 Dec 2011, at 21:22, Mike Fitzpatrick wrote: >> >>> The prospect of many thousands of parallel vTCP connections is very >>> appealing either ... >>> >>> Remember too, for some events to be useful (e.g. GRB followup) >>> they cannot be delivered 12 hours later, so the problem (at least a >>> major use-case) has to also solve the question of how to deliver >>> 10K (or is it 50K, or 100K?) events in a matter of *minutes*. I >>> don't think the bar napkin on which vTCP was designed was big >>> enough to consider that case thoroughly 8-) >> >> Just for amusement, I wrote a simple test to see how this might work. >> My results& the code that generated them are >> at. >> >> Broadly speaking, maintaining a rate of 2e6 events in 12 hours (~50 >> events/second), or even 50K events in 5 minutes (167 events/second), >> was pretty doable. My simple VOEvent generator ran out of steam >> trying to send 100K events in 5 minutes (333 events/second), though. >> >> I certainly don't suggest that's a comprehensive test of the protocol >> ? the aim of the exercise was simply my personal education ? but >> perhaps the code is of interest to others. >> >> Cheers, >> >> John > > -- > --- > Caltech LIGO > roy at caltech.edu > 626 395 3670 From shaw at noao.edu Fri Jan 6 07:48:34 2012 From: shaw at noao.edu (Dick Shaw) Date: Fri, 06 Jan 2012 08:48:34 -0700 Subject: 0MQ: The Intelligent Transport Layer In-Reply-To: References: <16827241.1348.1324966650909.JavaMail.root@web04> <1E2D3C9D-5BC7-4517-88F8-6C362C1BD9BA@transientskp.org> <8F9FC8BB-26DF-491C-987B-156A9948932A@transientskp.org> <4F05DDD7.5020604@caltech.edu> Message-ID: On Fri, 6 Jan 2012 12:31:37 +0100 John Swinbank wrote: > However, I don't think we can avoid the issue of bulk transport. I assume >that the LSST folks (for example) have done their homework, and really will >produce a couple of million events which are worthy of distribution per >night. They will then face the problem of shifting them down a high-latency, >low-bandwidth line. They still need a "super truck" capable of that, even if >it's only to carry letters to a sorting office. We LSST folks have been told by our community that they want access to every notification, and that the LSST project should not be the judge of what events are "worthy." Tagging events with hints about the nature of the source, etc. is ok, but not pre-filtering. Also, building the "truck" is not within the current LSST scope. This is not to say that the transport mechanisms are of no concern--quite the contrary. I think there is a need for a system of brokers, filters, classifiers, and information aggregators to help the community select what is most interesting from a science perspective, and to distill the information in a way that is both manageable and timely for the end-user. Put another way, usage patterns matter a lot when designing the transport capabilities of the system. Cheers, Dick #--- Dr. Richard A. Shaw, Scientist National Optical Astronomy Observatory 950 N. Cherry Avenue Tucson, AZ 85719 USA 520-318-8398 http://www.noao.edu/noao/staff/shaw/ From swinbank at transientskp.org Tue Jan 24 05:18:38 2012 From: swinbank at transientskp.org (John Swinbank) Date: Tue, 24 Jan 2012 14:18:38 +0100 Subject: Signing events Message-ID: <54641DEC-0BD3-4791-8D71-C532669DA031@transientskp.org> Dear all, Forgive me ignorance, but I'm trying to get up to speed on previous discussions about signing VOEvents. I've seen Bob Denny's proposal at and Steve Allen's publication (Astron. Nach., 329, 298-300, 2008). Both of those are a few years old now, though, and I'm wondering if either of them have gained community support/acceptance or if they state of the art has moved on to something different. I've had quick browse through the mailing list archives, but can't seen anything very conclusive. Any thoughts or references on current best practice? Thanks! John From seaman at noao.edu Tue Jan 24 14:46:12 2012 From: seaman at noao.edu (Rob Seaman) Date: Tue, 24 Jan 2012 15:46:12 -0700 Subject: Signing events In-Reply-To: <54641DEC-0BD3-4791-8D71-C532669DA031@transientskp.org> References: <54641DEC-0BD3-4791-8D71-C532669DA031@transientskp.org> Message-ID: <0673FF74-CD1D-4F5A-9C8E-9E94280AB1F5@noao.edu> Hi John, I believe both remain valid points of view. What the WG had decided - at least for v2.0 - was not to include explicit hooks for the signatures in the schema itself, but this shouldn't strongly impact any of the options. Few of the many and varied transient alert projects have had strong immediate signing requirements. Do you have a specific project need here? Rob -- On Jan 24, 2012, at 6:18 AM, John Swinbank wrote: > Dear all, > > Forgive me ignorance, but I'm trying to get up to speed on previous discussions about signing VOEvents. I've seen Bob Denny's proposal at and Steve Allen's publication (Astron. Nach., 329, 298-300, 2008). Both of those are a few years old now, though, and I'm wondering if either of them have gained community support/acceptance or if they state of the art has moved on to something different. > > I've had quick browse through the mailing list archives, but can't seen anything very conclusive. Any thoughts or references on current best practice? > > Thanks! > > John From rdenny at dc3.com Tue Jan 24 15:47:04 2012 From: rdenny at dc3.com (Bob Denny) Date: Tue, 24 Jan 2012 16:47:04 -0700 Subject: Signing events In-Reply-To: <9170204.29958.1327411392084.JavaMail.root@m11> References: <9170204.29958.1327411392084.JavaMail.root@m11> Message-ID: <4F1F42F8.8080902@dc3.com> My work is far more than a "point of view"... The Dakota broker/sender/receiver package (cross platform, http://www.ivoa.net/Documents/Notes/DakotaBroker/) fully supports digisig access-authentication and integrity checking per the VOEvent Transport 1.0 IVOA Note (http://www.ivoa.net/Documents/Notes/VOEventTransport/20090805/NOTE-VOEventTransport-1.1-20090805.html). Dakota is a production package for Mac, Linux and Windows (http://voevent.dc3.com/). Dakota has command line tools for sending, receiving, and checking VOEvent messages. It also has a full-up broker that will run as a daemon on any of the above three operating systems. We operate a broker here (voevent.dc3.com) and can arrange to receive signed events from you and validate the signature, as well as allow publishing based on the presence of your signature. I would just need your GPG public key. Unfortunately, due to the fragmented nature of VOEvent message transport, the only people who will receive your events are the others tied to my broker. Right now that's AAVSO Net and several amateur and low-visibility university observatories that are getting CBAT and NASA/Swift feeds. -- Bob Rob S Said: > Hi John, > > I believe both remain valid points of view. What the WG had decided - at least for v2.0 - was not to include explicit hooks for the signatures in the schema itself, but this shouldn't strongly impact any of the options. Few of the many and varied transient alert projects have had strong immediate signing requirements. Do you have a specific project need here? > > Rob John Said: > Dear all, > > Forgive me ignorance, but I'm trying to get up to speed on previous discussions about signing VOEvents. I've seen Bob Denny's proposal at and Steve Allen's publication (Astron. Nach., 329, 298-300, 2008). Both of those are a few years old now, though, and I'm wondering if either of them have gained community support/acceptance or if they state of the art has moved on to something different. > > I've had quick browse through the mailing list archives, but can't seen anything very conclusive. Any thoughts or references on current best practice? > > Thanks! > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seaman at noao.edu Tue Jan 24 16:10:13 2012 From: seaman at noao.edu (Rob Seaman) Date: Tue, 24 Jan 2012 17:10:13 -0700 Subject: Signing events In-Reply-To: <4F1F42F8.8080902@dc3.com> References: <9170204.29958.1327411392084.JavaMail.root@m11> <4F1F42F8.8080902@dc3.com> Message-ID: Bob says: > My work is far more than a "point of view"? Indeed! I didn't mean to suggest otherwise. Rather, I wanted to understand the context of John's interest. > Dakota has command line tools for sending, receiving, and checking VOEvent messages. It also has a full-up broker that will run as a daemon on any of the above three operating systems. And I was mentioning Dakota as a good option to somebody in my office earlier today. The poster from "Eventful Universe" is just down the hallway. > We operate a broker here (voevent.dc3.com) and can arrange to receive signed events from you and validate the signature, as well as allow publishing based on the presence of your signature. To confirm the viability of the signing concept, Steve also demonstrated his version at the NVO Summer School in 2005. I don't believe scalability has been aggressively evaluated for either paradigm. > Unfortunately, due to the fragmented nature of VOEvent message transport, the only people who will receive your events are the others tied to my broker. Right now that's AAVSO Net and several amateur and low-visibility university observatories that are getting CBAT and NASA/Swift feeds. In fact, Aaron Price of AAVSO was on that same team. Regarding transport, I suggest that 2012 will be a good year to start to move forward with this again. There will be a transient-response session at the LSST all-hands meeting this Summer. Hotwired III seems most likely for Spring 2013, but that implies that we should move various initiatives forward in 2012 to prime the pumps. As a beginning, can we get more events funneling into DC3, and more users or brokers receiving its message streams? Rob From seaman at noao.edu Tue Jan 24 19:32:57 2012 From: seaman at noao.edu (Rob Seaman) Date: Tue, 24 Jan 2012 20:32:57 -0700 Subject: Fwd: NOAO Survey Letters of Intent Due 15 Feb 2012 References: <201201250302.q0P32Qop064440@scope.tuc.noao.edu> Message-ID: <780F9037-1466-48C2-8102-774E2F91F826@noao.edu> FYI: Begin forwarded message: > Subject: NOAO Survey Letters of Intent Due 15 Feb 2012 > Date: January 24, 2012 8:02:26 PM MST > > NOAO SURVEY PROGRAM LETTERS OF INTENT DUE 15 February 2012 > > The NOAO Survey Program will be accepting proposals for new surveys > to start in the 2012B and 2013A semesters. This program supports > large observational projects using the Gemini, KPNO, and CTIO > telescopes that allow the identification of complete, well-defined > samples that can yield both conclusions based on statistical analysis > of the survey data itself and also provide important subsets for more > detailed observations with larger telescopes. In addition, surveys > are expected to provide coherent datasets that will be useful for > other researchers. > > Investigators must submit letters of intent to propose for the NOAO > Survey Program to surveys at noao.edu by February 15, 2012 to be > eligible to propose for an NOAO Survey Program commencing in the > 2012B/2013A semester. The deadline for receiving completed Survey > proposals is March 29, 2012 at 11:59pm. > > Survey proposals are generally restricted to instruments that have > been commissioned and are well-characterized for scientific use, due > to the significant time investment required for Survey programs. The > newest instruments at Gemini South, the multi-object near-infrared > spectrograph Flamingos-2 and the GEMS Multi-Conjugate Adaptive Optics > system and GSAOI camera, will not be available for this round of > Surveys. The CTIO 4-m Dark Energy Camera (DECam) will also not be > available for Surveys. Please also note that commissioning of DECam, > as well as early operation of the Dark Energy Survey and DECam > community programs, is likely to highly restrict the availability of > ISPI and Hydra-CTIO for Surveys. > > For more information, go to http://www.noao.edu/gateway/surveys/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From swinbank at transientskp.org Wed Jan 25 00:36:24 2012 From: swinbank at transientskp.org (John Swinbank) Date: Wed, 25 Jan 2012 09:36:24 +0100 Subject: Signing events In-Reply-To: <0673FF74-CD1D-4F5A-9C8E-9E94280AB1F5@noao.edu> References: <54641DEC-0BD3-4791-8D71-C532669DA031@transientskp.org> <0673FF74-CD1D-4F5A-9C8E-9E94280AB1F5@noao.edu> Message-ID: <1B1F427A-CABC-42F5-B4FB-B9AD12687C79@transientskp.org> Hello, Thanks Rob & Bob for your comments! As LOFAR gets nearer to implementing automatic response to TOOs, we're starting to consider the real nuts & bolts of the situation. One of the obvious questions is how can we be certain that the events we're responding to are real alerts from the facilities we're interested in, and not malicious or accidental forgeries. I'm not sure how likely forged events are, but it seems to be a possibility we should at least consider, given the potential disruption & expense they could cause. Both of the solutions mooted seem broadly technically viable (although I have a couple of areas of concern about both), but it's disappointing to see no widespread community support yet. I guess that's something we need to work on? Cheers, John On 24 Jan 2012, at 23:46, Rob Seaman wrote: > Hi John, > > I believe both remain valid points of view. What the WG had decided - at least for v2.0 - was not to include explicit hooks for the signatures in the schema itself, but this shouldn't strongly impact any of the options. Few of the many and varied transient alert projects have had strong immediate signing requirements. Do you have a specific project need here? > > Rob > -- > > On Jan 24, 2012, at 6:18 AM, John Swinbank wrote: > >> Dear all, >> >> Forgive me ignorance, (And my typing!) >> but I'm trying to get up to speed on previous discussions about signing VOEvents. I've seen Bob Denny's proposal at and Steve Allen's publication (Astron. Nach., 329, 298-300, 2008). Both of those are a few years old now, though, and I'm wondering if either of them have gained community support/acceptance or if they state of the art has moved on to something different. >> >> I've had quick browse through the mailing list archives, but can't seen anything very conclusive. Any thoughts or references on current best practice? >> >> Thanks! >> >> John > From roy at caltech.edu Wed Jan 25 08:33:03 2012 From: roy at caltech.edu (Roy Williams) Date: Wed, 25 Jan 2012 08:33:03 -0800 Subject: GCN-XML and Dakota Message-ID: <4F202EBF.4010809@caltech.edu> Dear VOEventers I am trying to understand the difference between GCN-XML [1][2] and Dakota [3][4] as transport protocols for VOEvents. -- Each sends out VOEvents, although Dakota can also receive events. -- The GCN seems to emulate the "broker to subscriber" part of Dakota (section 3.2), not the other parts. -- Each uses the uncommon idea that the receiver of the message initiates the connection (usually the sender connects). -- Dakota uses "iamalive" messages whereas GCN uses "imalive" messages (note spelling). It would be great to unify these two seemingly equivalent transport mechanisms. One has the events and 20 years of trust, the other has the open source implementation. Your wisdom welcome! Roy [1] http://gcn.gsfc.nasa.gov/tech_describe.html#tc17 [2] http://gcn.gsfc.nasa.gov/xml_sock_demo.c [3] http://www.ivoa.net/Documents/Notes/VOEventTransport/ [4] http://voevent.dc3.com/dakota/index.html --- Caltech LIGO roy at caltech.edu 626 395 3670 From aa at astro.ex.ac.uk Wed Jan 25 08:37:21 2012 From: aa at astro.ex.ac.uk (Alasdair Allan) Date: Wed, 25 Jan 2012 16:37:21 +0000 Subject: GCN-XML and Dakota In-Reply-To: <4F202EBF.4010809@caltech.edu> References: <4F202EBF.4010809@caltech.edu> Message-ID: <72236653-C943-45DD-85A9-1F8796444580@astro.ex.ac.uk> > -- Each uses the uncommon idea that the receiver of the message initiates the connection (usually the sender connects). The vTCP paradigm is that the received makes a client connection to the broker's server. Messages are pushed by the broker down these connection. I'm not entirely sure where you get the idea that this isn't the case? 3.2 Broker to Subscriber When a subscriber wants to receive VOEvent message traffic, it opens a TCP connection to a broker. This connection is kept open continuously. When the broker receives a message, it relays a copy of the VOEvent message to all of the connected subscribers. Thus, a subscriber must continuously listen on the TCP connection and be prepared to receive VOEvent messages at any time, even when it is busy processing a previously received VOEvent message. When a subscriber receives a VOEvent message from its broker, it sends back a response. The response from the subscriber is a Transport message. This isn't "uncommon" that's how the majority of network protocols work, a client makes a connection to a server which it wants to get information from. That's how the WWW works after all? Al. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roy at caltech.edu Wed Jan 25 09:04:26 2012 From: roy at caltech.edu (Roy Williams) Date: Wed, 25 Jan 2012 09:04:26 -0800 Subject: GCN-XML and Dakota In-Reply-To: <72236653-C943-45DD-85A9-1F8796444580@astro.ex.ac.uk> References: <4F202EBF.4010809@caltech.edu> <72236653-C943-45DD-85A9-1F8796444580@astro.ex.ac.uk> Message-ID: <4F20361A.4050400@caltech.edu> Alasdair With email, for example, the sender makes the socket connection, sends the message, then disconnects. Whereas here we have the receiver makes the connection, waits for a while, then the message comes and the receiver continues listening. Whether we say one is common or uncommon is I suppose beside the point. Perhaps the real difference is short versus long-lived connection? Roy On 1/25/12 8:37 AM, Alasdair Allan wrote: > >> -- Each uses the uncommon idea that the receiver of the message >> initiates the connection (usually the sender connects). > > The vTCP paradigm is that the received makes a client connection to the > broker's server. Messages are pushed by the broker down these > connection. I'm not entirely sure where you get the idea that this isn't > the case? > > 3.2Broker to Subscriber > When a subscriber wants to receive VOEvent message traffic, it opens a > TCP connection to a broker. This connection is kept open continuously. > When the broker receives a message, it relays a copy of the VOEvent > message to all of the connected subscribers. Thus, a subscriber must > continuously listen on the TCP connection and be prepared to receive > VOEvent messages at any time, _even when it is busy processing a > previously received VOEvent message_. When a subscriber receives a > VOEvent message from its broker, it sends back a response. The response > from the subscriber is a Transport message. > > > This isn't "uncommon" that's how the majority of network protocols work, > a client makes a connection to a server which it wants to get > information from. That's how the WWW works after all? > > Al. > -- --- Caltech LIGO roy at caltech.edu 626 395 3670 From seaman at noao.edu Wed Jan 25 10:25:57 2012 From: seaman at noao.edu (Rob Seaman) Date: Wed, 25 Jan 2012 11:25:57 -0700 Subject: GCN-XML and Dakota In-Reply-To: <4F202EBF.4010809@caltech.edu> References: <4F202EBF.4010809@caltech.edu> Message-ID: We seem to be drifting off the point. Roy said: > It would be great to unify these two seemingly equivalent transport mechanisms. One has the events and 20 years of trust, the other has the open source implementation. This seems like a great idea - especially the greatness-of-heart for originating from a Jabberian. Perhaps Scott and Bob can comment? > -- Dakota uses "iamalive" messages whereas GCN uses "imalive" messages (note spelling). If there is a difference perhaps we can pick one or the other. I don't think we need wait for VOEvent v3.0 to address this issue :-) Rob From aa at astro.ex.ac.uk Wed Jan 25 10:57:22 2012 From: aa at astro.ex.ac.uk (Alasdair Allan) Date: Wed, 25 Jan 2012 18:57:22 +0000 Subject: GCN-XML and Dakota In-Reply-To: References: <4F202EBF.4010809@caltech.edu> Message-ID: >> It would be great to unify these two seemingly equivalent transport mechanisms. One has the events and 20 years of trust, the other has the open source implementation. > > This seems like a great idea - especially the greatness-of-heart for originating from a Jabberian. Perhaps Scott and Bob can comment? My understanding is that both implement the vTCP note that Bob and I wrote? Am I wrong here? >> -- Dakota uses "iamalive" messages whereas GCN uses "imalive" messages (note spelling). > > If there is a difference perhaps we can pick one or the other. I don't think we need wait for VOEvent v3.0 to address this issue :-) I think that's a bug, not a feature? Al. From swinbank at transientskp.org Thu Jan 26 08:01:44 2012 From: swinbank at transientskp.org (John Swinbank) Date: Thu, 26 Jan 2012 17:01:44 +0100 Subject: GCN-XML and Dakota In-Reply-To: <4F202EBF.4010809@caltech.edu> References: <4F202EBF.4010809@caltech.edu> Message-ID: <821F32BD-960C-4ACB-8851-D7DD1FB7F20B@transientskp.org> Hi Roy, On 25 Jan 2012, at 17:33, Roy Williams wrote: > -- Each uses the uncommon idea that the receiver of the message initiates the connection (usually the sender connects). Regardless of whether we regard that as "uncommon" or not, I'm not actually sure that it's correct. What you describe is certainly true or the vTCP protocol. However, my understanding is that the GCN system is the reverse. To quote (from the comments in xml_sock_demo.c): > To do this GCN needs to be the "client" end. If your "server" end is running, > the GCN "client" will connect to it and send the various XML_packets (imalives > and burst types). GCN is able to do this because they maintain a "sites.cfg" file which lists all the subscribers to connect to. In the vTCP case, such a file is unnecessary: modulo administrative restrictions, any subscriber can connect and start receiving events with no additional configuration at the sender. By the way: > I am trying to understand the difference between GCN-XML [1][2] and Dakota [3][4] Although the Dakota Tools make for an excellent reference implementation for the vTCP system, I don't think there's anything Dakota specific about it. Certainly, the spec is self contained and easy to implement without reference to Dakota. Cheers, John From aa at astro.ex.ac.uk Thu Jan 26 08:07:05 2012 From: aa at astro.ex.ac.uk (Alasdair Allan) Date: Thu, 26 Jan 2012 16:07:05 +0000 Subject: GCN-XML and Dakota In-Reply-To: <821F32BD-960C-4ACB-8851-D7DD1FB7F20B@transientskp.org> References: <4F202EBF.4010809@caltech.edu> <821F32BD-960C-4ACB-8851-D7DD1FB7F20B@transientskp.org> Message-ID: <9F50813C-A803-4338-ACEB-878C57A4E0A9@astro.ex.ac.uk> >> I am trying to understand the difference between GCN-XML [1][2] and Dakota [3][4] > > Although the Dakota Tools make for an excellent reference implementation for the vTCP system, I don't think there's anything Dakota specific about it. Certainly, the spec is self contained and easy to implement without reference to Dakota. There have in fact been at least five independent implementations of the spec, Dakota is only one of them. Al. From aa at astro.ex.ac.uk Thu Jan 26 08:10:23 2012 From: aa at astro.ex.ac.uk (Alasdair Allan) Date: Thu, 26 Jan 2012 16:10:23 +0000 Subject: GCN-XML and Dakota In-Reply-To: <821F32BD-960C-4ACB-8851-D7DD1FB7F20B@transientskp.org> References: <4F202EBF.4010809@caltech.edu> <821F32BD-960C-4ACB-8851-D7DD1FB7F20B@transientskp.org> Message-ID: <133B1547-DCE6-48E7-8218-A99433BB64C0@astro.ex.ac.uk> >> -- Each uses the uncommon idea that the receiver of the message initiates the connection (usually the sender connects). > > Regardless of whether we regard that as "uncommon" or not, I'm not actually sure that it's correct. > > What you describe is certainly true or the vTCP protocol. However, my understanding is that the GCN system is the reverse. To quote (from the comments in xml_sock_demo.c): This was certainly true of the 40-byte packet system. I haven't looked at the XML system... Al. From seaman at noao.edu Mon Jan 30 06:25:00 2012 From: seaman at noao.edu (Rob Seaman) Date: Mon, 30 Jan 2012 07:25:00 -0700 Subject: Time series in IVOA? References: <10622_1327918050_4F266BE2_10622_158264_3_4F266BF0.9070503@esa.int> Message-ID: <9A5AF1AF-4E55-48E8-951B-D676B555F9DD@noao.edu> Howdy, The IVOA Roadmap for 2012 is now available: > From: Christophe Arviset > Subject: The IVOA in 2011: Technical Assessment and Roadmap for 2012 > Date: January 30, 2012 3:07:44 AM MST > To: interop at ivoa.net > > Dear all > > Following many inputs from Working Groups and Interest Groups chairs and vice-chairs, "The IVOA in 2011: Technical Assessment and Roadmap for 2012" has been endorsed by the IVOA Exec and is now available at: > > http://www.ivoa.net/Documents/Notes/IVOATechRoadmap2011/index.html > > Many thanks to all the people who have participated to this document ! VOEvent is mentioned a couple of dozen times. We should be eager to see the registry work move forward, for instance. I'm skeptical that cone search in a time window really addresses the science use cases for SEAP, however. On the other hand, time series are mentioned just three times. Massively backpedaling on the schedule (2012 is for use cases when 2010/2011 was previously a target for the final product): > 3.3.7 Other DAL standards > ... > Time Series and Event access protocols require that data model work be closer to completion before the DAL WG can assess whether existing standards can be used or new standards are required. Specifically, we expect to use TAP for discovery (along the same lines as the ObsCoreDM is used) and will try to use other DAL standards (e.g. DataLink and PQL) to satisfy other use cases. We will be collecting use cases in 2012. and similarly underwhelming: > 3.4.1 Relationship between the various Data Models > ... > o A potential TimeSerieDM still to be defined and very subjunctively: > 3.8.2 Other VOEvent standards > Following the early discussions which have taken place within the VOEvent WG about a possible Simple Light Curve Access Protocol (SLiCAP), it now appears that once the time series data model is defined, light curves could be retrieved through a suitable DAL interface, e.g. TAP, hence there would be no need to defined a specific access protocol for light curves. In the absence of an undefined potential TimeSeries DM, thoughts on conveying such data sets pragmatically? References to external SimpleTimeSeries? Embedded table with interim convention for representing some useful subset of same? Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjg at cacr.caltech.edu Mon Jan 30 10:21:56 2012 From: mjg at cacr.caltech.edu (Matthew Graham) Date: Mon, 30 Jan 2012 10:21:56 -0800 Subject: Time series in IVOA? In-Reply-To: <9A5AF1AF-4E55-48E8-951B-D676B555F9DD@noao.edu> References: <10622_1327918050_4F266BE2_10622_158264_3_4F266BF0.9070503@esa.int> <9A5AF1AF-4E55-48E8-951B-D676B555F9DD@noao.edu> Message-ID: Hi Rob, On Jan 30, 2012, at 6:25 AM, Rob Seaman wrote: > Howdy, > > The IVOA Roadmap for 2012 is now available: > >> From: Christophe Arviset >> Subject: The IVOA in 2011: Technical Assessment and Roadmap for 2012 >> Date: January 30, 2012 3:07:44 AM MST >> To: interop at ivoa.net >> >> Dear all >> >> Following many inputs from Working Groups and Interest Groups chairs and vice-chairs, "The IVOA in 2011: Technical Assessment and Roadmap for 2012" has been endorsed by the IVOA Exec and is now available at: >> >> http://www.ivoa.net/Documents/Notes/IVOATechRoadmap2011/index.html >> >> Many thanks to all the people who have participated to this document ! > > VOEvent is mentioned a couple of dozen times. We should be eager to see the registry work move forward, for instance. I'm skeptical that cone search in a time window really addresses the science use cases for SEAP, however. Our community solicitation exercise last summer revealed that most are just happy with cone search + time window and don't want a full query against the VOEvent data model. > On the other hand, time series are mentioned just three times. Massively backpedaling on the schedule (2012 is for use cases when 2010/2011 was previously a target for the final product): > >> 3.3.7 Other DAL standards >> ... >> Time Series and Event access protocols require that data model work be closer to completion before the DAL WG can assess whether existing standards can be used or new standards are required. Specifically, we expect to use TAP for discovery (along the same lines as the ObsCoreDM is used) and will try to use other DAL standards (e.g. DataLink and PQL) to satisfy other use cases. We will be collecting use cases in 2012. > > and similarly underwhelming: > >> 3.4.1 Relationship between the various Data Models >> ... >> o A potential TimeSerieDM still to be defined > > and very subjunctively: > >> 3.8.2 Other VOEvent standards >> Following the early discussions which have taken place within the VOEvent WG about a possible Simple Light Curve Access Protocol (SLiCAP), it now appears that once the time series data model is defined, light curves could be retrieved through a suitable DAL interface, e.g. TAP, hence there would be no need to defined a specific access protocol for light curves. > > In the absence of an undefined potential TimeSeries DM, thoughts on conveying such data sets pragmatically? References to external SimpleTimeSeries? Embedded table with interim convention for representing some useful subset of same? As you aware, IVOA schedules are not set in stone (nor are they contractually binding) and sometimes things need to be changed to reflect strategy changes. This is what happened with TimeSeries last year (in Naples). TimeSeries is part of the suite of DMs that inherit from Spectral 2.0 DM, which is actively worked on by the DM group. Once this is stable, a TimeSeries spec will appear. We do have a need for it beyond including time series in VOEvents - the VAO time series work would really benefit from it. VOEvent 2.0 allows the inclusion of tabular data and this is the mechanism we should be promoting if you want to bloat your VOEvent packet with light curves. Cheers, Matthew -------------- next part -------------- An HTML attachment was scrubbed... URL: