OpenGuides-Dev February 2006

openguides-dev@lists.openguides.org

3 participants
6 discussions

[OpenGuides-Dev] OG performance
by Paul Makepeace 20 Jun '07

20 Jun '07

Hi, I don't follow OG development but as host operator a few things are coming up. The machine's load is gradually climbing over time and some of that is OG. Despite a relatively low hit rate on OG it is consuming quite a bit of resource. If OG started taking off it would take the machine down. First up: index.cgi requires 0.35s to perform a `perl -c` syntax check. Any thoughts on putting OG on a mod_perl server? I have mod_perl running here of course and we'd need to coordinate some apache.conf stuff. Second: the supersearch.cgi gulps down CPU, often for seconds at a time. It is a frequent resident of `top` output. This isn't really acceptable. I'm going to request this feature be turned off unless an effective optimisation plan or some other way to reduce its impact here is constructed pretty soon. Sorry about this but it's encroaching on others. Third: I wonder if there's some way to instruct robots not to spider parts of your wiki. This ought to speak for itself: $ grep crawl /var/log/apache/london.openguides.org-access.log | grep 'action=edit' | wc -l 8242 $ Finally: I posted about a DoS and was wondering what the status of a solution was. http://openguides.org/mail/openguides-dev/2004-October/000542.html Cheers, Paul (any overbearing tone unintentional ;-) -- Paul Makepeace .............................. http://paulm.com/inchoate/ "If my elbow was straight, then I'll show oyu mine!" -- http://paulm.com/toys/surrealism/

10 22

Custom category/locale prefixes
by Dominic Hargreaves 02 Mar '06

02 Mar '06

Some work's been done on this - see http://dev.openguides.org/ticket/81 - and I'm just looking for comments on this change really. PerfDave, do can you explain you reason for wanting this? I think it'll help me understand the value of having it if so. One thing would be for translation purposes, I suppose? Cheers, Dominic. -- Dominic Hargreaves | http://www.larted.org.uk/~dom/ PGP key 5178E2A5 from the.earth.li (keyserver,web,email)

2 3

Open Guide to Atlanta
by Christopher Schmidt 28 Feb '06

28 Feb '06

I recently received a request from someone looking to set up the Open Guide to Atlanta. I spent an hour last night doing the steps that are needed to set up a new guide on Athena (the boston.og.org webserver): * Create web directories with appropriate permissions (5 minutes -- scripted) * Copy over Boston web directory. This ended up including a couple things that I didn't mean for it to, like extra files I had accidentally stuck in the directory. Clean those up (10 minutes) * Grab new gmaps key. (5 minutes) * Add a temporary domain forward from atlanta.crschmidt.net -- then realize that meant I had to get a new gmaps key. D'oh. (10 minutes) * Modify all the "base" lat and long pairs to use ones that worked for Atlanta, rather than ones that worked for Boston. Note that this is probably not required if I ever get around to updating my templates and configuration to the latest code, since I'm assuming Dom made these configuration params. This took about 10 minutes. * Remove all the Boston-specific stuff - T lines (both special icons and the polylines), town boundary drawing, etc. This took the longest -- about 15 minutes -- but now that it's done, I can simply copy out the Atlanta templates instead of the Boston ones. * Copied over and set up configuration files for stats (a la http://boston.openguides.org/stats/) and webstats (a la http://boston.openguides.org/webstats/). 10 minutes. * Copied over and set up scripts for loading places from atlanta.zami.com, added 43 places to the guide, checked it out, called it done. Total time: 1 hour. Total time next time around: Probably closer to 30 minutes. hex, when you get back, if you could please set up atlanta.openguides.org to point to 65.110.51.60 or as a CNAME to Boston, I'd appreciate it. Next step: Setting up a couple more Guides, that people can claim from me later if they want. I'll do the hosting and admin until someone else comes along who wants to admin the site -- then they can either keep it on my server, or they can have the data and files and move elsewhere. The ones that I plan to set up at the moment are: * Chicago * Seattle * Portland Because I know people who know those areas well enough to add some content. I would also be happy to set up guides for any of the places listed on http://zami.com/ -- because I have a spider which can crawl the data out of there a bit more easily than I'd be able to get it elsewhere otherwise. If anyone has any requests for these places, I'd love to hear them, and set them up. (Note that crawling this data in the US is considered legal -- at least for the time being. "Collection" protection only comes into play when there is some level of creativity involved in the data -- which yellow page listings are not. The original court case regarding this actually used white pages as an example, which is exactly the data I'm copying at the moment.) Also, thanks to rich_gibson of Geocoder.us, Athena now has non-restricted access to Geocoder.us. Thanks a million for that, Rich! Geocoder.us offers a ton of great commercial services, so if you're in the US and wanting to do location lookups, he's a good person to talk to. This is long and rambly. I'm going to stop talking, and get on to working on setting these things up. -- Christopher Schmidt Web Developer

3 3

Improving Searching in OpenGuides
by Christopher Schmidt 26 Feb '06

26 Feb '06

Per a request on the Milton Keynes guide, I took up the matter of looking at making searches include more than just the text of the entry, but instead also listing the metadata in the nodes. I successfully completed this in the following manner: * Modify the rename.pl in the dist to be more along the lines of: http://crschmidt.net/openguides/reindex.pl You can see here that this indexes phone address city postcode website fax summary, as well as categories and locales. * Tested this to make sure the results worked. Number searching seems broken, but other than that it's in good shape. * Head in to try and make this change to the underlying guide code. * Realize that searching is basically hardcoded into write_node, with easy way around it. resign self to this fact and decide that double-indexing is the way to proceed. * Write a CGI::Wiki Plugin. Bang head against wall until realizing you need to set @ISA yourself. * Write reindexer similar to reindex.pl above. * Find "write_node" in Openguides.pm. Place Reindexer plugin above write_node (see code snippet at end of email). * Test it out. Seems to work okay. * Upload Indexer.pm code to http://crschmidt.net/openguides/Indexer.pm * Post to mailing list. I'm not sure if anyone else will find this interesting, but I certainly did, so I figured I'd share the knowledge in case anyone else out there wants it. my $indexer = CGI::Wiki::Plugin::Indexer->new; $wiki->register_plugin( plugin => $indexer ); $indexer->reindex($node, $content, \%metadata); my $written = $wiki->write_node($node, $content, $checksum, \%metadata ); -- Christopher Schmidt Web Developer

1 0

Running OpenGuides under mod_perl
by Christopher Schmidt 21 Feb '06

21 Feb '06

Currently, when running OpenGuides under mod_perl, redirects do not work, because the Apache $r object associated with the CGI object do not exist. (You get an error message: Can't call method "send_cgi_header" on an undefined value at (eval 64) line 62.) This happens both after an edit, and when following redirects, and possibly in other cases as well. After some investigation last night, I tracked the source of this down: currently, OpenGuides redirects use a class method of CGI: CGI->redirect($redirectparams); However, this does not work. It seems that class methods in this case never initialize the Apache request variable that is needed in order to send_cgi_header. The solution for this, is to replace CGI->redirect calls with: my $q = new CGI; $q->redirect($redirectparams); I have too little knowledge to be able to speak to whether this is a fault within CGI or a fault within OpenGuides usage of it. However, this change has allowed me to successfully use mod_perl on the Open Guide to Boston in testing, and this fix or a similar fix should probably be integrated into OpenGuides in either case, since upgrading the system CGI.pm is a difficult task. The current debian packages for stable include mod_perl 1.999.021, which is not compatible with CGI.pm 3.16, and upgrading mod_perl requires a full removal due to an API rename (as described at http://perl.apache.org/docs/2.0/rename.html). The code that needs to be changed is in OpenGuides.pm, in two functions: sub redirect_to_node sub find_within_distance A patch which makes these changes is attached. -- Christopher Schmidt Web Developer

2 1

[jo@frot.org: [Openstreetmap] Please Help Petition for Public Access to Geodata in Europe]
by Dominic Hargreaves 18 Feb '06

18 Feb '06

I know Jo's on here anyway, but this has ramifications for the OpenGuides project so it's worth considering. Cheers, Dominic. ----- Forwarded message from Jo Walsh <jo(a)frot.org> ----- Date: Tue, 14 Feb 2006 18:14:36 -0800 From: Jo Walsh <jo(a)frot.org> To: openstreetmap(a)vr.ucl.ac.uk User-Agent: Mutt/1.4.1i Subject: [Openstreetmap] Please Help Petition for Public Access to Geodata in Europe X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.0.3 dear openstreetmap list, Please sign the petition: http://petition.publicgeodata.org/ Yesterday the INSPIRE Directive on European "spatial data infrastructure" began its second reading in the European Parliament. Within a few weeks or months it may entrench a policy of charging European citizens to access public information they've already paid to collect. Members of the European Parliament think this an obscure and dull technical issue; but free public access to geographic information is essential for economic innovation and civic co-operation. Benjamin Henrion and I are starting the above petition to MEPs to reject the current draft of the INSPIRE directive. It's all rather last-minute; we may have as little as a month before the Directive gets through its second reading; so it's important we try and get as many signatures as we can, as soon as we can, and get the word out. People writing open source software, people who create a lot of GPS information as a byproduct of being a hiker or cycle courier, people who love cartography and just want to improve the representation of the world around them, all over Europe, are getting together through http://openstreetmap.org/ to create and share geographic information. New data exchange standards, at both ad-hoc and industry level, allow for more of the creation of maps and description of the environment to be carried out by people who know the location well; map data, census information and other essential data about how people and places interact, which is used to manage civic society. INSPIRE ignores all these trends. It is a narrow view that was formed without a full survey of what's being done with spatial information - especially with GPS and GSM positioning based media, new kinds of transport logistics systems, and the upcoming GALILEO satellite network. GALILEO will guarantee a reasonable quality signal for free to all citizens. A Directive about access to state-collected public geographic information, should guarantee freedom of access to data. If you are in Europe, please help us raise awareness among MEPs that this is a much bigger issue than it seems, and shouldn't be rubber-stamped! Help us by signing http://petition.publicgeodata.org/ . Please tell other interest groups that you're a part of - non-geo, or even non-geek - and ask other European friends to help us by signing the petition. Please feel free to forward this mail, or bits of it, onwards. http://publicgeodata.org/ is a wiki for collaboration on research into the issue, into some of the history and context around this legislation, also providing a quick rough guide for focusing energy when contacting your MEPs about this regressive, short-termist law. If INSPIRE passes its second reading with this wording, all our in-car navigation systems, mobile phones, local search services, will become more expensive and less useful; INSPIRE threatens to set back economic innovation in location aware technology in Europe by many years. -jo _______________________________________________ Openstreetmap mailing list Openstreetmap(a)vr.ucl.ac.uk http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap ----- End forwarded message ----- -- Dominic Hargreaves | http://www.larted.org.uk/~dom/ PGP key 5178E2A5 from the.earth.li (keyserver,web,email)

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

OpenGuides-Dev February 2006