On Fri, Dec 02, 2005 at 11:16:22PM +0000, Bob Walker wrote:
On Fri, 2 Dec 2005, Christopher Schmidt wrote:
I'm already seeing some slowdowns on Boston in some pages, so I may be looking to add some caching to different aspects of the page - specifically, when loading ?action=index;format=map (which is still one of the more common page requests I see), which loads *everything* in the DB, it's slow. So, I think that if we're going to go for the whizbang nature here, loading all the data may be more likely, and SQLite is not going to be scaling "enough".
indexes.
Perhaps I'm misunderstanding you here, but my MySQL tables already *define* indexes on the name column. What more could be done?
The problem is that it's selecting 700 nodes, and all the metadata for them, which is just a lot of frickin data. (The latest database dump is 1.95 meg.)
Skip the pain. MySQL isn't that big of a requirement. SQLite may be more painful. Let's not hurt OG by encouraging using a possibly non-scaling database product which users would then later have to move from to serve their users. The "Slo-penguides" joke has already been made even with the "real" databases :)
indexes. also the fact that it loads half of cpan.
The loading half of CPAN happens on every page. The loading 1.95MB of data and building the associated structures inside the request (attaching all the metadata and so on) only happens on the index page.
Home node for boston: 3 second load time, including network. ?action=index node for boston: 13 second load time, including network.
Part of that is the network - about 3 seconds, for the index node, it looks like. However, the other 7 seconds of difference are pure load time.
hm. Dumping the whole database using mysqldump only takes .3 seconds, so that's probably not our blocker. However, how long would it take to load all the data with mysqlite? Anyone got a mysql->sqlite conversion script I could use, so I could test that?
which is really noticable if you dont have a powerful machine.
I'm not. It's 2.0GHz box, and nothing else on it runs slow. and as I said, the guide itself only runs slow when it has to load a lot of data.
I'm not saying that the speed isn't related to other aspects, but at least some of this will start to be limited by the database once it gets bigger.
-- Chris