Excessive Load Times and Blank Pages with Large Nav

Permalink
I am working on a site with over 500 pages. The top navigation on this site lists all pages (with the exception of blog postings). The issue I am having is that occasionally a page will load slowly (15+ seconds) or will not display at all (no 404, just an empty DOM).
All pages have the top nav (all pages), a breadcrumb nav, a nav showing all pages below the second level, and a nav showing all "featured" pages.

I cannot make any of these navs static, as the client is constantly adding and deleting pages (I actually expect the site to grow quite a bit), so I need a solution that will allow all these navs to remain up-to-date for the user.

leinteractive
 
leinteractive replied on at Permalink Reply
leinteractive
Here is what I have tried so far...

I have installed MISER, but the issue remained. MISER has since been removed.

I switch to APC caching, but the issue remained, so I switched back to the built in (Zend) caching.

Updated concrete > core > models > permission > category.php based on this post:http://www.concrete5.org/community/forums/installation/time-to-firs...

Updated concrete > core > models > collection_version.php to reduce db queries based on this:https://github.com/concrete5/concrete5/pull/737...

Updated the concrete > core > models > collection_types.php based on this:https://github.com/concrete5/concrete5/pull/741...

Modified the concrete > core > models > permission > access > model.php based on this post:http://www.concrete5.org/developers/bugs/5-6-0-2/page-load-time-inc... This modification reduced the “select paID, paIsInUse from PermissionAccess where paID = ?” query from 233 to 15 per page load (based on dbbbb from Remo, Mklyhttp://www.concrete5.org/developers/bugs/5-6-0-2/page-load-time-inc...

Updated concrete > core > models > page.php based on this:https://github.com/concrete5/concrete5/pull/739...

Updated concrete > core > models > block.php, concrete > core > models > collection.php, concrete > core > models > collection_version.php based on this post:http://www.concrete5.org/developers/bugs/5-6-0-2/page-load-time-inc...

The server has compression enabled and I have added future expire dates via the .htaccess file.
<ifModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 1 seconds"
  ExpiresByType text/html "access plus 1 seconds"
  ExpiresByType image/gif "access plus 2592000 seconds"
  ExpiresByType image/jpeg "access plus 2592000 seconds"
  ExpiresByType image/png "access plus 2592000 seconds"
  ExpiresByType text/css "access plus 604800 seconds"
  ExpiresByType text/javascript "access plus 216000 seconds"
  ExpiresByType application/x-javascript "access plus 216000 seconds"
</ifModule>


The site is hosted on a virtual dedicated server at Rackspace, who assure me that I have more than enough resources. I have several other sites on this server that all perform admirably.
leinteractive replied on at Permalink Reply 1 Attachment
leinteractive
The attached document shows the results of dbbb on a specific page (Note: I modified the code to run for a guest user). With dbbb off and the above changes, I get the following load times with a loaded cache.

Test 1: 11.375
Test 2: 11.415
Test 3: 7.251
Test 4: 11.393
Test 5: 12.373

I need to get the pages to load in one to two seconds. Any ideas on what more I can do to improve performance?
leinteractive replied on at Permalink Reply 1 Attachment
leinteractive
Here are my cache settings.

Basic Cache: On
Overrides Cache: On
Full Page Caching: On - If blocks allow it.
Full Page Cache Rebuild: Automatic

My environment information is attached.
admin replied on at Permalink Best Answer Reply
You may already know this, but (if you're using v5.6.0.2 or earlier) playing with your Cache settings in the C5 UI might help you. Some people experienced faster load times with the caching turned off rather than on.

That said, you may try updating to v5.6.1 which just released yesterday. It's roughly 33% faster on the sites I've tested the beta with.

Overall though, it sounds like you're between a rock and a hard place. Could you put some of those nav's in a Stack as straight HTML and paste the stack into a global area? This would still allow the client to update it without having to actually build the nav on every page load (which is almost certainly what's crushing you). A URL may help troubleshoot things as well.
leinteractive replied on at Permalink Reply
leinteractive
Thanks for the reply, but unfortunately, the nav is too large for the client to edit as html.

I am actually working on a job that will export the nav at night (and on Event > on_page_update) to an XML file and creating blocks to read this XML file for all the navs.

What I don't understand is that the cache should be handling this, so what I am doing should be unnecessary. I just don't see any other options.
leinteractive replied on at Permalink Reply
leinteractive
@adamjohnson: v5.6.1 has saved me! Initial page views are down from ~15s to ~5s. Repeat (cached) views are down to under a second on some pages. I am normally slow to adapt new versions, but I am definatly making an exception in this case. Thank you!