One of the recurring themes with using OpenGuides is that it doesn't do a particularly good or consistent job of allowing people to use charsets other than iso-8859-1. I've made a page on the wiki:
http://openguides.org/dev/?node=Character%20Sets
which hopefully sets out what problems there are and how to solve them.
Please hack on that wiki page with additions and corrections, and/or make comments on list. Once we have a clear plan of how to proceed we'll need someone to implement it, too :) If you are regularly dealing with non iso-8859-1 charsets at the moment, then your input is especially valuable here.
Cheers,
Dominic.
On Dec 12, 2004, at 23:27, Dominic Hargreaves wrote:
need someone to implement it, too :) If you are regularly dealing with non iso-8859-1 charsets at the moment, then your input is especially valuable here.
THE PAIN. THE HORRIBLE, HORRIBLE PAIN.
CGI::Wiki (recent versions therof) isn't fully supportive of all charsets, but I don't care, because the only charset worth caring about is utf-8, and it does that. You'll need perl 5.8 upwards, though. It shouldn't be a lot of work over that to make OG do it - DBI tends to be the real annoyance with these things.
Patches against the current release ok?
tom
On Mon, Dec 13, 2004 at 01:02:24AM +0000, Tom Insam wrote:
CGI::Wiki (recent versions therof) isn't fully supportive of all charsets, but I don't care, because the only charset worth caring about is utf-8, and it does that.
I assume you are taking about mixing charsets in the same database? We *do* need to support different charsets in different databases - it is not okay to mandate utf8 everywhere at the moment, IMO.
If the latter, we would presumably need to patch CGI::Wiki (I was expecting a handful of fixes needed there anyway, to tell the database what charset to use and so on).
You'll need perl 5.8 upwards, though. It shouldn't be a lot of work over that to make OG do it - DBI tends to be the real annoyance with these things.
*nod*. We do need to try and keep upgrade paths stable and not lock people with older systems out completely.
Patches against the current release ok?
That would be fantastic!
Cheers,
Dominic.
On Mon 13 Dec 2004, Dominic Hargreaves dom@earth.li wrote:
*nod*. We do need to try and keep upgrade paths stable and not lock people with older systems out completely.
Whatever you decide to do, OpenGuides _must_ still work on 5.6.1, because I've got a guide running on it (on the.earth.li), and I use it as well as the kakemirror to test out new features before I commit them.
CGI::Wiki::Kwiki doesn't install happily on 5.6.1, and that's held me up from doing much hacking on it; I don't want the same to happen to OpenGuides.
Kake
On Mon, Dec 13, 2004 at 07:10:19AM +0000, Kake L Pugh wrote:
Whatever you decide to do, OpenGuides _must_ still work on 5.6.1, because I've got a guide running on it (on the.earth.li), and I use it as well as the kakemirror to test out new features before I commit them.
Oxford, Nottingham and Chester at least, as well.
Dominic.
On Dec 13, 2004, at 1:10, Dominic Hargreaves wrote:
On Mon, Dec 13, 2004 at 01:02:24AM +0000, Tom Insam wrote:
CGI::Wiki (recent versions therof) isn't fully supportive of all charsets, but I don't care, because the only charset worth caring about is utf-8, and it does that.
I assume you are taking about mixing charsets in the same database? We *do* need to support different charsets in different databases - it is not okay to mandate utf8 everywhere at the moment, IMO.
Why not?
If the latter, we would presumably need to patch CGI::Wiki (I was expecting a handful of fixes needed there anyway, to tell the database what charset to use and so on).
Well, assuming you want 'utf-8' and 'lots of single-byte encodings', it'll probably work fine now. But yes, explicit support would be better.
You'll need perl 5.8 upwards, though. It shouldn't be a lot of work over that to make OG do it - DBI tends to be the real annoyance with these things.
*nod*. We do need to try and keep upgrade paths stable and not lock people with older systems out completely.
They get latin-1 and live with it, then.
On Dec 13, 2004, at 1:10, Dominic Hargreaves wrote:
On Mon, Dec 13, 2004 at 01:02:24AM +0000, Tom Insam wrote:
CGI::Wiki (recent versions therof) isn't fully supportive of all charsets, but I don't care, because the only charset worth caring about is utf-8, and it does that.
I assume you are taking about mixing charsets in the same database? We *do* need to support different charsets in different databases - it is not okay to mandate utf8 everywhere at the moment, IMO.
It's ocurred to me that you might mean 'we have to allow latin-1 as well' here, which I agree with. but I think offering a choice of latin-1 or utf-8 is reasonable. Having said that, I think the code will support any charset you ask of it..
If the latter, we would presumably need to patch CGI::Wiki (I was expecting a handful of fixes needed there anyway, to tell the database what charset to use and so on).
0.62 looks like it's got most of my UTF-8 work - page names in Recent Changes are still handled badly, that's the only major bug I know of in there.
Patches against the current release ok?
That would be fantastic!
Right then. I'm working on this in my svn repository because I hate CVS. So far I have utf8 page contents and names, usernames, other preferences, and metadata, I think.
Resources:
svn repository - http://dev.jerakeen.org/svn/tomi/Projects/OpenGuides working copy - http://dev.jerakeen.org/og_utf8/wiki.cgi?Home (slow, that's just the colo) patch - http://dev.jerakeen.org/files/2004/12/13/utf8.diff
Known bugs:
Without my SVN CGI::Wiki, you'll have broken page names in the recent changes view. This is only for people trying this themselves - the copy linked above is fine. the diff view is broken: http://dev.jerakeen.org/og_utf8/wiki.cgi? id=Home&version=1&diffversion=2 - but then, it's broken in the real thing too. I haven't tried this on 5.6 - I can't get OG to build on 5.6 here. Anyone want to try it? I'm not expecting utf-8 under 5.6, I'm just expecting it to not break.
Anything else anyone can find, I'd love to know about.
tom
On Mon, Dec 13, 2004 at 10:13:15PM +0000, Tom Insam wrote:
svn repository - http://dev.jerakeen.org/svn/tomi/Projects/OpenGuides working copy - http://dev.jerakeen.org/og_utf8/wiki.cgi?Home (slow, that's just the colo) patch - http://dev.jerakeen.org/files/2004/12/13/utf8.diff
Known bugs:
Without my SVN CGI::Wiki, you'll have broken page names in the recent changes view. This is only for people trying this themselves - the copy linked above is fine.
We'll need to have your CGI::Wiki changes integrated before we can integrate the OpenGuides changes, I think; otherwise there'll be too much pain. Is work on this under way?
the diff view is broken: http://dev.jerakeen.org/og_utf8/wiki.cgi? id=Home&version=1&diffversion=2 - but then, it's broken in the real thing too.
I assume this is a case of not making things worse - ie diffs on non-unicode data are ok?
I haven't tried this on 5.6 - I can't get OG to build on 5.6 here. Anyone want to try it? I'm not expecting utf-8 under 5.6, I'm just expecting it to not break.
This is on my list of things to test.
Cheers,
Dominic.
On Mon, Dec 13, 2004 at 10:13:15PM +0000, Tom Insam wrote:
Without my SVN CGI::Wiki, you'll have broken page names in the recent changes view.
On Sun 02 Jan 2005, Dominic Hargreaves dom@earth.li wrote:
We'll need to have your CGI::Wiki changes integrated before we can integrate the OpenGuides changes, I think; otherwise there'll be too much pain. Is work on this under way?
Tom's sent me a patch, but the test he sent with it doesn't actually test the bug, and while amending the test I hit the snag that if we really want to have proper Unicode support then we're going to be at least doubling the test running time. And _then_ while looking into that I realised that the previous patch Tom sent (which has been applied) was a badly placed band-aid on a broken leg. The wrong leg. That needs ripping out, and a proper solution applying.
Fixing this properly is going to take a long time. However! Given that OpenGuides already has a load of slapped-together shonky fixes, one more won't hurt. Can someone point me to the guide pages that are suffering from this problem, please?
Kake
openguides-dev@lists.openguides.org
-
Dominic Hargreaves -
Kake L Pugh -
Tom Insam