I am impressed by the RDF backend of OpenGuides, and the goal of making the information reusable on the Semantic Web.
I am giving a talk next month at AjaxWorld about Semantic Mashups using RDF (which, for full disclosure, will feature my company's product, TopBraid Composer).
Part of the story will be, "but where will you find all that data in RDF?"
OpenGuides is a very good answer to that question.
I want to use some RDF data from OpenGuides in my demo (with attribution, of course). But I am having trouble coaxing RDF data from the OpenGuide website in a useful way.
Here's what I would like to do. I see that in the OpenGuide Boston, there is geocode information for all the MBTA train stations. For instance, a page like http://boston.openguides.org/?id=Northeastern_Station;format=rdf gives me the geocode location for the Northeastern University stop.
I would like to find a page that has, say, all the Green Line stations, with their geocodes, all in one file (URL). Similarly for Redline, etc. I can make a pretty interesting mashup page from such information. I can do the same with all the single-station pages, but that will seem awfully tedious when I show it as part of a demo.
Is there a way to get the RDF exported in this way? Is this something that the OpenGuides folks would be interested in allowing me to do?
Thanks,
Dean Allemang Chief Technology Consultant TopQuadrant Inc.
On Tue, 6 Feb 2007, Dean Allemang wrote:
Here's what I would like to do. I see that in the OpenGuide Boston, there is geocode information for all the MBTA train stations. For instance, a page like http://boston.openguides.org/?id=Northeastern_Station;format=rdf gives me the geocode location for the Northeastern University stop.
I would like to find a page that has, say, all the Green Line stations, with their geocodes, all in one file (URL). Similarly for Redline, etc. I can make a pretty interesting mashup page from such information. I can do the same with all the single-station pages, but that will seem awfully tedious when I show it as part of a demo.
http://oxford.openguides.org/wiki/?action=index;index_type=category;index_va...
for example.
Bob Walker wrote:
On Tue, 6 Feb 2007, Dean Allemang wrote:
Here's what I would like to do. I see that in the OpenGuide Boston, there is geocode information for all the MBTA train stations. For instance, a page like http://boston.openguides.org/?id=Northeastern_Station;format=rdf gives me the geocode location for the Northeastern University stop.
I would like to find a page that has, say, all the Green Line stations, with their geocodes, all in one file (URL). Similarly for Redline, etc. I can make a pretty interesting mashup page from such information. I can do the same with all the single-station pages, but that will seem awfully tedious when I show it as part of a demo.
http://oxford.openguides.org/wiki/?action=index;index_type=category;index_va...
for example.
Thanks! That was a quick reply! I was able to modify your example to get something close to what I wanted.
The best I can do is to find the list of all the T Stops (index_value=T%20Stops;), but I can't find out from the RDF which stops are red line stations etc. Is that information available in RDF?
Thanks,
Dean
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
The best I can do is to find the list of all the T Stops (index_value=T%20Stops;), but I can't find out from the RDF which stops are red line stations etc. Is that information available in RDF?
No, because that data hasn't been added to the guide. If you add all the red line stations to something like "Category Red Line" then you can see that in RDF too.
See for example: http://london.randomness.org.uk/wiki.cgi?Category_Jubilee_Line http://london.randomness.org.uk/wiki.cgi?action=index;index_type=category;in...
Kake
Kake L Pugh wrote:
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
The best I can do is to find the list of all the T Stops (index_value=T%20Stops;), but I can't find out from the RDF which stops are red line stations etc. Is that information available in RDF?
No, because that data hasn't been added to the guide. If you add all the red line stations to something like "Category Red Line" then you can see that in RDF too.
See for example: http://london.randomness.org.uk/wiki.cgi?Category_Jubilee_Line http://london.randomness.org.uk/wiki.cgi?action=index;index_type=category;in...
Kake
I guess I will become a contributor to OpenGuides sooner than I thought :)
I will read through the contributor pages and figure out how to add that information. I used to live in Boston, so I think I can remember which ones are which.
I now live in Los Angeles, so I might want to start a losangeles.openguides.org. What is involved in doing such a thing?
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
I now live in Los Angeles, so I might want to start a losangeles.openguides.org. What is involved in doing such a thing?
You either need to install and host the software yourself, or persuade someone else to do so. You also need several other people who're committed to adding stuff to your guide, or it'll just go out of date and die off.
Kake
On Wed, Feb 07, 2007 at 09:55:16AM +0000, Kake L Pugh wrote:
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
I now live in Los Angeles, so I might want to start a losangeles.openguides.org. What is involved in doing such a thing?
You either need to install and host the software yourself, or persuade someone else to do so. You also need several other people who're committed to adding stuff to your guide, or it'll just go out of date and die off.
http://dev.openguides.org/wiki/OnlineDocumentation
has some useful information about this (although it's not particularly complete yet)
Dominic.
Kake L Pugh wrote:
No, because that data hasn't been added to the guide. If you add all the red line stations to something like "Category Red Line" then you can see that in RDF too.
See for example: http://london.randomness.org.uk/wiki.cgi?Category_Jubilee_Line http://london.randomness.org.uk/wiki.cgi?action=index;index_type=category;in...
Kake
Okay, I've started doing this. It isn't too difficult, so I should finish in another hour or so.
I accidentally created a category whose name didn't begin with the word "Category ", unlike all the other categories. I can't seem to delete it. Can someone help me out and clean that up?
Also, while the forms interface is fairly easy to use, it would be even easier if I could just do a bulk RDF/XML import. Is there a way to do that?
Thanks,
Dean
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
I accidentally created a category whose name didn't begin with the word "Category ", unlike all the other categories. I can't seem to delete it. Can someone help me out and clean that up?
You need someone with the admin password to delete pages - but you can make a redirect yourself from "MBTA Red Line" to "Category MBTA Red Line" using the syntax "#REDIRECT [[Category MBTA Red Line" on the former. Probably best to remove all categories/locales/other info from redirected pages, too, to avoid it coming up in searches.
Also, for future reference, you don't have to create the categories yourself - just add something to a page as a category, and if the category doesn't exist then it'll be auto-created.
Also, while the forms interface is fairly easy to use, it would be even easier if I could just do a bulk RDF/XML import. Is there a way to do that?
I think crschmidt might have done something like this at some point?
Kake
On Wed 07 Feb 2007, Kake L Pugh kake@earth.li wrote:
[...] using the syntax "#REDIRECT [[Category MBTA Red Line" [...]
This should of course be #REDIRECT [[Category MBTA Red Line]]
Kake
On Wed, Feb 07, 2007 at 09:53:40AM +0000, Kake L Pugh wrote:
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
I accidentally created a category whose name didn't begin with the word "Category ", unlike all the other categories. I can't seem to delete it. Can someone help me out and clean that up?
You need someone with the admin password to delete pages - but you can make a redirect yourself from "MBTA Red Line" to "Category MBTA Red Line" using the syntax "#REDIRECT [[Category MBTA Red Line" on the former. Probably best to remove all categories/locales/other info from redirected pages, too, to avoid it coming up in searches.
Also, for future reference, you don't have to create the categories yourself - just add something to a page as a category, and if the category doesn't exist then it'll be auto-created.
Also, while the forms interface is fairly easy to use, it would be even easier if I could just do a bulk RDF/XML import. Is there a way to do that?
I think crschmidt might have done something like this at some point?
AFAIK, it's built into the application. Just go to http://anyguide.openguides.org/?action=index;format=rdf
However, it's unlikely to be valid RDF, because of XML encoding issues.
Once you have this page, you can use it as a scutterplan to get the rest of the data.
Getting the data out takes 30 seconds on the Open Guide to Boston, and returns 5.4MB of data. The file, after being downloaded and XML errors corrected, is available from http://boston.openguides.org/all.xml .
(I don't recommend opening that file in Firefox, for what it's worth.)
Of course, if you really want all the data, the best way to get it is to set up your own guide instance with the database dump, available from the http://boston.openguides.org/?Database_Dumps page, rather than pounding away at the server to get all the pages out.
Regards,
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
Also, while the forms interface is fairly easy to use, it would be even easier if I could just do a bulk RDF/XML import. Is there a way to do that?
On Wed 07 Feb 2007, Christopher Schmidt crschmidt@crschmidt.net wrote:
AFAIK, it's built into the application. Just go to http://anyguide.openguides.org/?action=index;format=rdf
I think Dean's talking about putting information _in_, rather than getting it out.
Kake
On Wed, Feb 07, 2007 at 02:40:02PM +0000, Kake L Pugh wrote:
On Tue 06 Feb 2007, Dean Allemang dallemang@topquadrant.com wrote:
Also, while the forms interface is fairly easy to use, it would be even easier if I could just do a bulk RDF/XML import. Is there a way to do that?
On Wed 07 Feb 2007, Christopher Schmidt crschmidt@crschmidt.net wrote:
AFAIK, it's built into the application. Just go to http://anyguide.openguides.org/?action=index;format=rdf
I think Dean's talking about putting information _in_, rather than getting it out.
Ah, so he is! My reading eyes are rather broken.
I've never done RDF import. I wrote some tools which automated the process of import, but they just used standard HTTP post. My interest in OpenGuides is entirely unrelated to its RDF output, largely because I think that RDF is an unusable technology at this point.
If you can write RDF/XML, you can likely write a parser for your data, and once you can write a parser, you can automate the input of data. If you want help doing that, a sample of the data you wish to input would be the first step towards assisting in creating tools that will help pull the data in.
Regards,
On Wed, 7 Feb 2007, Christopher Schmidt wrote:
I've never done RDF import. I wrote some tools which automated the process of import, but they just used standard HTTP post. My interest in OpenGuides is entirely unrelated to its RDF output, largely because I think that RDF is an unusable technology at this point.
Hopefully not getting too off topic here, but I'm curious why RDF is unusable for OpenGuides / in general? (From reading some web pages, one sort of gets the impression that RDF is the second coming. :)
Justin
On Thu, Feb 08, 2007 at 01:39:34PM -0800, justin+openguides@dicatek.com wrote:
On Wed, 7 Feb 2007, Christopher Schmidt wrote:
I've never done RDF import. I wrote some tools which automated the process of import, but they just used standard HTTP post. My interest in OpenGuides is entirely unrelated to its RDF output, largely because I think that RDF is an unusable technology at this point.
Hopefully not getting too off topic here, but I'm curious why RDF is unusable for OpenGuides / in general? (From reading some web pages, one sort of gets the impression that RDF is the second coming. :)
http://crschmidt.net/blog/archives/85/ sums up my position nicely, as far as I remember. I wrote it close to two years ago but most of my opinions haven't changed, though I would add the severe lack of a "Getting Things Done" attitude is probably what has harmed the semantic web the most. OpenGuides is a counter to that, and there are exceptions to the rule (Dan Brickley, and the FOAF crowd in general, chief among them), but for the most part, the semantic web is more tied up in semantic wankery than actually achieving anything with the tools that they've created.
I can expand more, but this is probably not the best place for it, and I don't participate in semweb community precisely *because* of all the reasons that make me complain, so anyone who wants to can follow up off the list with me :)
Regards,
On Wed, 7 Feb 2007, Christopher Schmidt wrote:
However, it's unlikely to be valid RDF, because of XML encoding issues.
I thought we'd got the xml encoding stuff sorted at the hackathon?
That said, I only remember touching the xml encoding on the atom and rss feeds, not the rdf feeds, so that bit might still need some tweaking. Can anyone confirm or deny if we did fix the xml stuff for rdf too?
Nick
On Wed, Feb 07, 2007 at 04:15:21PM +0000, Nick Burch wrote:
On Wed, 7 Feb 2007, Christopher Schmidt wrote:
However, it's unlikely to be valid RDF, because of XML encoding issues.
I thought we'd got the xml encoding stuff sorted at the hackathon?
That said, I only remember touching the xml encoding on the atom and rss feeds, not the rdf feeds, so that bit might still need some tweaking. Can anyone confirm or deny if we did fix the xml stuff for rdf too?
What OpenGuides as a project has done and what the Open Guide to Boston has done are totally different. Because of the large number of code changes that I have made[1] that are not part of the OpenGuides code, I'm running on the same code that I originally wrote the Google Maps support into. Since I don't know enough perl to set up a testing instance of OpenGuides that I can merge my changes against (combined with the database schema changes), I've accepted the fact that I have no upgrade path, and have chosen to stick with the current setup rather than spend time changing it.
So, any development within the past year is not available from boston.openguides.org.
[1] http://dev.openguides.org/ticket/11 http://dev.openguides.org/ticket/100 http://dev.openguides.org/ticket/101 http://dev.openguides.org/ticket/99
Regards,
This thread is confusing me, but not in a bad way.
I have tried several variants of the URL to extract data from OpenGuides in RDF. All of them produce valid RDF/XML (at least, valid enough to pass the test at http://www.w3.org/RDF/Validator, and more importantly for my project, to be parsed by Jena).
As far as I can tell, it ain't broke, so there's no need to fix it.
Dean
Christopher Schmidt wrote:
On Wed, Feb 07, 2007 at 04:15:21PM +0000, Nick Burch wrote:
On Wed, 7 Feb 2007, Christopher Schmidt wrote:
However, it's unlikely to be valid RDF, because of XML encoding issues.
I thought we'd got the xml encoding stuff sorted at the hackathon?
That said, I only remember touching the xml encoding on the atom and rss feeds, not the rdf feeds, so that bit might still need some tweaking. Can anyone confirm or deny if we did fix the xml stuff for rdf too?
What OpenGuides as a project has done and what the Open Guide to Boston has done are totally different. Because of the large number of code changes that I have made[1] that are not part of the OpenGuides code, I'm running on the same code that I originally wrote the Google Maps support into. Since I don't know enough perl to set up a testing instance of OpenGuides that I can merge my changes against (combined with the database schema changes), I've accepted the fact that I have no upgrade path, and have chosen to stick with the current setup rather than spend time changing it.
So, any development within the past year is not available from boston.openguides.org.
[1] http://dev.openguides.org/ticket/11 http://dev.openguides.org/ticket/100 http://dev.openguides.org/ticket/101 http://dev.openguides.org/ticket/99
Regards,
On Thu, Feb 15, 2007 at 08:47:10AM -0800, Dean Allemang wrote:
This thread is confusing me, but not in a bad way.
I have tried several variants of the URL to extract data from OpenGuides in RDF. All of them produce valid RDF/XML (at least, valid enough to pass the test at http://www.w3.org/RDF/Validator, and more importantly for my project, to be parsed by Jena).
As far as I can tell, it ain't broke, so there's no need to fix it.
You're absolutely right in the current version of the software.
I was speaking specifically about the guide that was under discussion at the start of the thread, the Open Guide to Boston.
http://boston.openguides.org/?action=index;index_type=category;index_value=F...
That's an example of malformed XML (and therefore invalid RDF) being served by that guide.
But no other guide will serve malformed XML: Just mine :)
Regards,
Kake L Pugh wrote:
Also, for future reference, you don't have to create the categories yourself - just add something to a page as a category, and if the category doesn't exist then it'll be auto-created.
I'm glad I waited to create the categories for the blue, orange and green lines. It will be simpler to just use the names.
Thanks for your patience and help.
Dean
that's fantastic! I didn't know we could do that! (Mark, the fluffy social guy at miltonkeynes.openguides.org)
http://miltonkeynes.openguides.org/?action=index;index_type=category;index_v...
:-)
Incidently - thanks again everybody who helped me and Tom with the paper we put into WikiSym this summer[1]. Just to let you know I've had a proposal for a book chapter accepted for an academic publication: "Augmented public spaces: articulating the physical and electronic city" and I'll be drawing from the WikiSym paper, so I'll be waving the Open Guide flag again! Please get in contact if you'd like to know more.
all the best Mark
[1] http://kmi.open.ac.uk/people/mark/papers/gaved_wiki06.pdf
Mark Gaved Knowledge Media Institute The Open University Walton Hall Milton Keynes, UK MK7 6AA
http://kmi.open.ac.uk/people/mark
-----Original Message----- From: openguides-dev-bounces@lists.openguides.org on behalf of Bob Walker Sent: Tue 2/6/2007 10:45 PM To: OpenGuides software developers' list Subject: Re: [OGDev] Getting OpenGuide info in RDF On Tue, 6 Feb 2007, Dean Allemang wrote:
Here's what I would like to do. [snip]
http://oxford.openguides.org/wiki/?action=index;index_type=category;index_va... for example.
openguides-dev@lists.openguides.org
-
Bob Walker -
Christopher Schmidt -
Dean Allemang -
Dominic Hargreaves -
justin+openguides＠dicatek.com -
Kake L Pugh -
M.B.Gaved -
Nick Burch