Time to First Byte (TTFB) Problems - Database Issues???

Permalink 2 users found helpful
I'm so confused, my head is about to explode!

I've been using C5 for quite awhile with no problems, but recently I've experienced significant lags in loading my pages. First, here's my set-up:

Site: http://www.crazychris.ca/
Host: Hostpapa (dedicated IP)
C5 Version: 5.6.0.2
PHP Version: 5.3.16

To illustrate the problem, here are some links to tests run on WebPagetest.org:

http://www.webpagetest.org/result/121101_WN_EG9/...
http://www.webpagetest.org/result/121101_2X_41K/...

Besides the TTFB lags, I'm also occasionally getting "508 Resource Limit Reached" errors.

I talked to Hostpapa and they say that their Apache server is functioning properly. I have checked all my databases (in CPanel) and they're all okay. I implemented the Cloudflare CDN which has improved loading content after the TTFB, but hasn't done anything to help TTFB. I have wandered all over the C5 forums looking for a fix and have found much tips regarding speeding up loading content (e.g. installing Miser, etc.) and have tried many of these tips but, again, the TTFB remains very high.

Can anyone give me any pointers on how to get the TTFB down? Is this a database issue (i.e. slow queries), problems in C5, or what?

Thanks SO much for any help/advice offered!

 
chrison999 replied on at Permalink Reply 1 Attachment
As an addendum, I also implemented Pingdom which is showing response times all over the map, from 3,673 ms 29,973 ms with 303 time-outs out of 1,015 pings. Attached is the file (CVS format) for anyone who wants to take a look.

Thanks, again, for any help/advice offered!
mkly replied on at Permalink Reply
mkly
If this is a new problem, meaning its been performing well in the past, I would try clearing and then disabling you site cache to see if you are running into garbage collection issues with the cache.

I would also check to see how many page versions you have. Running the remove old page versions jobs could help you out as well.
chrison999 replied on at Permalink Reply
Thanks for the reply.

Caching is off. I couldn't find the remove old versions in the dashboard (even though the php file is in the /concrete/jobs directory!) so I used the site map to remove all old versions. I also deleted all files/directories in the /files/cache directory. However, the site is still unacceptably slow. Here's another test run:

http://www.webpagetest.org/result/121102_M5_5DE/...

Any other thoughts?

Thanks, again, for any help you can provide.
chrison999 replied on at Permalink Reply
UPDATE:

I have done a bit of experimenting. First, I created a static html page (test.html) with only "This is a test page" for content. No images, css, nothing. This page loaded *very* fast (395 ms) as seen by this test report:

http://www.webpagetest.org/result/121102_JA_CQ2/...

Then, I installed a new C5 site with only the sample data so now we're doing MySQL queries. This site loaded reasonably quickly (TTFB of 8,467 ms for first view and 5,396 ms for repeat view). Here's the test report on that one:

http://www.webpagetest.org/result/121102_9M_CK2/...

Finally, I re-ran the test on the main page for my main site. The load was *terribly* slow (TTFB of 23,818 ms for first view and 20,772 for repeat view). Here's that report:

http://www.webpagetest.org/result/121102_4E_CQW/...

Obviously, what's slowing everything down appears to be the MySQL queries. My first question is whether the TTFB for the test site (8,467 ms) is "reasonable" given it only contains the sample data? If not, then where should I start looking for bottlenecks?

If the TTFB for the test site *are* reasonable, then is my problem with MySQL (e.g. a trashed index or something) or perhaps one (or more) of my C5 plug-ins?

Finally, are there any "tools" I can use to watch what's happening in-between the first connection and TTFB to maybe diagnose what's going on?

Thanks, again, for any help/advice provided.
Remo replied on at Permalink Reply
Remo
Well, comparing a static HTML file to a full blown PHP and database based CMS doesn't work well. Sure, an HTML is faster but it doesn't do anything, it doesn't process input parameters, it doesn't sanitize input, it doesn't request from a database, it doesn't create a session, it doesn't check permissions..

395ms for a static HTML page is very slow I'd say! A PHP CMS always does several times as much as a static HTML page.
chrison999 replied on at Permalink Reply
First, thanks for the reply!

Of course a simple, raw html page is going to load super-fast compared to a full-blown CMS running under PHP and MySQL! The reason I ran that little test was to see if anyone felt the server is running slow even *without* the back-end overhead of PHY and MySQL.

You seem to have answered that question by saying 395 ms is "very slow for a static html page." If that's the case, then perhaps I need to *first* worry about diagnosing and fixing a slow server *before* I start worrying about the databases and CMS applications?

If 395 ms is "very slow" for a simple static html page, then what specifically could be causing that slowness? What should I be asking my hosting company's tech support people to look for?

Second, do you think the TTFB for the test site (fresh C5 install with only the sample date) is "reasonable" or is that slower than should be expected too?

Thanks for any help you can provide.
Remo replied on at Permalink Reply
Remo
It's hard to say but from my experience it was mostly a slow filesystem. I usually try to make sure that I get the first byte within less than 500ms. But only if the server is on the same continent.. I don't think that 8 seconds is reasonable!
Remo replied on at Permalink Reply
Remo
This is a site I'm trying to make faster:

http://www.webpagetest.org/result/121102_MJ_DGK/1/details/...

The home page contains several blocks and even fetches data form another server with every request!
mkly replied on at Permalink Reply
mkly
I would contact your host and see if you database is localhost or on a different server in their network. People typically see much better results with a database on localhost.
pvernaglia replied on at Permalink Reply
pvernaglia
Those really slow first byte problems are usually a hosting issue, too many sites being served from one server. If you are on a Cpanel server you may have an option in your control panel called 'Optimize Website' go in there and enable compress all content.

In your config/site.php see if it is better when you add:
define('CACHE_FRONTEND_OPTIONS', serialize(array('automatic_cleaning_factor' => 0)));
Try adding this in  your .htaccess file in the directory where C5 is installed:
<IfModule mod_deflate.c>
# compress text, html, javascript, css, xml:
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript
</IfModule>
chrison999 replied on at Permalink Reply
Thanks, guys, for the further replies.

Remo:

My site's server is located in Toronto, Canada, and the tests were run from Chantilly, Virginia, so the everything is on the same continent. Those results for your site are pretty impressive. Also, I loaded your site from here in Canada it it came up quickly.

mkly:

MyPHPAdmin shows this for MySQL:

Server: Localhost via UNIX socket
Server version: 5.5.27-cll
Protocol version: 10
User: xxxxxxx@localhost
MySQL charset: UTF-8 Unicode (utf8)


It appears that the databases are on localhost, or should I still ask?

pvernaglia:

I added "define('CACHE_FRONTEND_OPTIONS', serialize(array('automatic_cleaning_factor' => 0)));" to the site.php file yesterday and it made some difference in load times *after* TTFB but didn't seem to effect TTFB itself.

The lines for compressing output were also added to .htaccess yesterday. Again, this seemed to make some difference in loading content after TTFB but didn't seem to effect TTFB itself.

I don't see any "Optimize Website" option in CPanel. (SIGH!) Is there a way to do this from myPHPAdmin (or some other way?)

Thanks, again, for any suggestions/advice provided!
Remo replied on at Permalink Reply
Remo
Unix sockets on localhost should be good. I'd say that this server has too many customers on it.. While they can still say apache works, it doesn't mean it works fast enough.
chrison999 replied on at Permalink Best Answer Reply
Does everyone have their crying towels out? (wink!) It's time for another update...

Hostpapa tech support confirms that C5 and the databases are on the same server. As well, they INSIST there are no issues with the server in terms of load or speed. They keep telling me, "You should optimize the databases... that should fix your problems." I have created a cron job to run mysqlcheck -Ao twice daily and the reports coming back from that are saying everything is optimized.

To recap, here's what I have already done:

1. Optimized the databases (see above)
2. Caching is as follows: Basic Cache = OFF, Overrides Cache = OFF, Full Page Caching = OFF
3. define('CACHE_FRONTEND_OPTIONS', serialize(array('automatic_cleaning_factor' => 0))); has been added to root/config/site.php
4. AddOutputFilterByType DEFLATE to enable compression has been added to .htaccess in the site's root directory
5. CDN (CloudFlare) has been enabled and is working

Does anyone have ANY further suggestions on how I can get this puppy functioning properly? I'd move to another host, but I've prepaid Hostpapa to 2014 (BIG mistake!) and personal finances prevent me from switching right now (after all, it IS a recession folks!).

Again, a HUGE THANKS to all who have provided tips and suggestions! And THANKS for any further help provided!

Regards,

Chris
chrison999 replied on at Permalink Reply
Opps! Forgot to post a link to a recent test using webpagetest.org:

http://www.webpagetest.org/result/121106_KC_G22/...

Pretty awful, eh?!?! (SIGH!)

Chris
Remo replied on at Permalink Reply
Remo
Well, what's the definition of an issue. The server runs, but how does it run? We could also insist that concrete5 doesn't have any issues, it runs on hundreds on servers without a problem..

I'd bet that I could improve the performance of your site if they'd give me full root access. I'd do this by either killing all other customers on that server or by changing the configuration. You can ask them if they are willing to take the bet, after all it is a recession ;-)

concrete5 certainly needs a lot of resources, and just saying that you should optimize your database sounds a bit blunt to me.

But anyways.. If you check this discussion, you'll see a number of patches I created that reduce the number of SQL queries which in case the database is the bottleneck, should improve things:http://www.concrete5.org/developers/bugs/5-6-0-2/page-load-time-inc... But always make a backup before you apply such changes, they aren't well tested yet! Bleeding edge!

If possible, can you please try to install an add-on? I've explained it here:http://www.concrete5.org/developers/bugs/5-6-0-2/page-load-time-inc... Post the two screenshots like in the other discussion and we'll see how well the database responds.
chrison999 replied on at Permalink Reply 3 Attachments
Not too long ago, I was a manager in-charge of a tech support department for a fairly large organization (500+ employees). I'm well aware of some of the "games" tech support plays to deflect "blame" for problems away from themselves in order to manage workload. I'm also well aware that "the server has no issues" doesn't necessarily mean that but, instead, can mean "we really don't want to take a lot of time helping you with your problem." So, yes, telling me to optimize my database was blunt but there ain't much I can do about that. (SIGH!)

That said, I'll bet you COULD improve things if you were given root access to the server. However, I won't bother asking for that as I'm pretty sure I know what the answer will be. :-))) Anyway...

Before I get into mucking with C5 by installing patches, I downloaded the add-on to document database response. Attached are 3 screenshots of the add-on's output. Hopefully, that will give you a better clue of what's going on. If, after viewing the screenshots, you think I should go ahead and install the patches, let me know.

Btw, Remo, I ***REALLY*** appreciate all the help you've given me! Although I'm fairly technically savvy, I'm *not* a programmer, so much of this has my head spinning! THANKS again!

Regards,

Chris
Remo replied on at Permalink Reply
Remo
Well, it's not really helpful for you, but there's one good thing about these servers/configurations: They tell us where we should improve concrete5.

In the second screenshot you can see that a single query is executed 1679 and consumes a total of 5.54 seconds. You can also see that the runtime for a single execution varies between 0.000091 and 0.097039. While those numbers are still very small, it also shows you that it's not a very constant number. Unfortunately the add-on of mkly which I extended doesn't show the standard deviation, I might add that one day. It might be a buffer issue on the MySQL server. If there's not enough memory available for the database, it has get remove data from the memory rather quickly and reload it again.

The first query has been improved a lot by a change Andrew made a while ago. Unfortunately it's not in the official stable version yet. But if you don't mind to fiddle around with some files, check this page:https://github.com/concrete5/concrete5/commit/14d03d48f2a5c3a8bd1571... and try to make sure concrete/core/models/permission/access/model.php looks like shown on github.

The patches I've created aren't related to the attributes, only to the collection, pages and collectionversions. I doubt that they will change a lot on your site. I'd start with the link above and see if it changes anything..
Remo replied on at Permalink Reply
Remo
There's another thing I'd like to explain. This query:

select cPointerID from Pages where cID = ? LIMIT 1


tells me, that your site has to load 115 page object for a single page. In concrete5 a page is represented as an object which has to be loaded using several methods. Loading a page always take a bit of time. It's usually not an issue but if you're server isn't the fasted, you should probably try to avoid loading that many page objects.

How can that be done? Basically check the number of links you have in your page. A drop down navigation is pretty bad for this as concrete5 has to print a lot of sub items when the page is rendered. The same goes for the date nav block in the right sidebar. Most links are hidden but they are still rendered in the HTML.

If possible, try to use a navigation where you have to display less pages in autonav.. I'm aware that this isn't always an option, I'm just stating a general rule which could cause a bad performance..
chrison999 replied on at Permalink Reply 2 Attachments
I made the changes to /public_html/updates/concrete5.6.0.2/concrete/core/models/permission/category.php and /public_html/updates/concrete5.6.0.2/concrete/core/models/permission/access/model.php as specified in the github link. That does seem to improve the querying somewhat (far fewer suspicious and expensive queries, and noticeably better times). I'm attaching 2 screenshots if you wish to take a look.

However, the webpagetest.org results are no better than before:

http://www.webpagetest.org/result/121106_0Y_JG4/...

So, from an "internal" viewpoint it seems things improved, but from an "external" viewpoint it doesn't seem to be much better.
Remo replied on at Permalink Reply 2 Attachments
Remo
I guess them main cause slowing down your site are the JavaScript navigations. The main nav on top as well as the date nav in the right sidebar and the sub nav in the left sidebar contain a lot of pages.

If possible, I would recommend that you'd use something without JavaScript that only displays the related subpages and not every level.

I've attache two screenshots showing your all places where you load several pages. Three out of four are based on JavaScript and therefore contain hidden HTML elements that have to be rendered on page load as well.
chrison999 replied on at Permalink Reply 3 Attachments
For the main page, if I get rid of the drop-down nav at the top, and the date nav and slideshow in the right sidebar, performance increases dramatically:

http://www.webpagetest.org/result/121106_6G_M0W/...

There's still a few suspicious and expensive queries (see attached screenshots) but the times have improved greatly too.

I can easily get rid of the date nav and gallery in the right sidebar. Unfortunately, I *do* need some sort of navigation on the site (DUH!). But as soon as I add the autonav performance degrades again:

http://www.webpagetest.org/result/121106_S5_MM3/...

Hmmm.... I'm going to have to give this some thought!
Cahueya replied on at Permalink Reply
Why don't you just replace the C5 autonav with a hardcoded HTML-Nav, if this is so helpful to performance?

I got myself a root server after many switches to different virtual hosts without very often no possibility to fiddle around the server.

But still, I am bloody jealous of what Remo did to that swiss travel site. :-)

Great Job!

btw. Chris: If this was my site, I would try setting it up new with another version of C5, I am using 5.5.2.1 usually, as I have the best experience with it and all the bug reports of 5.6 keep me away from it until it is really fixed.

All the best!
chrison999 replied on at Permalink Reply
Thanks for the reply!

Replacing the auto nav with a hardcoded html nav kind of defeats the purpose of a CMS, doesn't it? I mean, then I'd have to manually edit the html nav every time I added a page or changed something else.

Just as a test, I rolled back C5 to version 5.5.2.1 (by changing config/site.php) and it didn't make any difference in performance, at least as far as I could see.
Cahueya replied on at Permalink Reply
Hey there,

well, depends. I found, that I rareley add new pages to the "top hierarchy", from my point of view. the most useful aspect of a CMS is content-editing and easy integration of add-ons.

Or try to establish the nav with a Stack that is using Textlinks - just to see if this is helping performance.

Hope you can fix this :(
Remo replied on at Permalink Reply
Remo
Well, I'm not suggesting that you remove the navigation ;-) But if you have more levels on your site, less sub-pages per page and only show the relevant sub pages in autonav, you'll get a much faster site.

I'm fairly confident that I can improve the loading process of a page object (on a site without APC) by at least 30% but the change I've started to make is a bit complicated and I'm not finished yet and will fly to Sweden tomorrow.

About that travel site, we also use a JavaScript nav which renders a lot of pages into the HTML output, even if not all of them by default. But I have to tell you that I've built a completely new autonav block to achieve this. The performance was okay but the original autonav block consumed about 1.5 seconds of the total loading time, and the new about 0.2 seconds. It's a lot of work and doesn't support all the features you have in autonav.
mhawke replied on at Permalink Reply
mhawke
Lots of good advice here from learned c5 gurus. I'll just suggest that IMHO, Overrides Cache should be 'on' unless you're adding/removing override files. In theory, this 'should' reduce the work your server has to do before it can render a page. I have a site on a $3.50/month shared host and it has a TTFB of between 1-2 seconds. Not excellent but pretty good for $3.50.

What does everyone think about trying to increase the memory_limit in the php.ini. Has that helped anyone in the past? I know 5.6 generally needs more memory than 5.5 but I'm not sure it solves TTFB issues.

I've also heard that installing a lot of packages & blocks can slow down a site down irregardless of whether those blocks are actually instantiated on pages.

Just more to think about.
chrison999 replied on at Permalink Reply
@mhawke:

I don't see anything in the C5 dashboard about override caching. Where is that?

Also, I haven't been able to find php.ini anywhere in my site's file structure, nor is there anywhere in CPanel to edit that (at least that I could find). Any suggestions where I might look for that?

Finally, I *do* have a lot of add-ons installed that I'm not using so tomorrow I'm going to do a clean-up and see if that has any effect.

Thanks for the further suggestions! Btw, I'd die and go to heaven if my TTFB times were the same as yours! :)

Regards,

Chris
mhawke replied on at Permalink Reply
mhawke
The overrides cache is in 'Dashboard > Cache and Speed Settings > Overrides Cache'. You posted above that it has been turned off.

Prior to 5.6, every time concrete5 opened any file, it had to search through all the folders to see if the developer has overridden any of the files. This created a lot of unnecessary overhead when 99% of the time there is no override file to be found. Version 5.6 added the ability to remember which files have been overridden and so it doesn't have to perform this fruitless search all the time.

This overrides cache does cause problems for developers though because when we do override a file, C5 will not see it unless we remember to turn off the overrides cache.

On the php issue... the host has a default php.ini file that it's using but sometimes they let you override that configuration by adding your own php.ini file to your root folder. There is a free add-on in the marketplace that shows you what php.ini file is being used along with a gazzillion other settings. You can get the add-on here:

http://www.concrete5.org/marketplace/addons/php-info/...

You can also find out the basic php settings from 'Dashboard > System and Settings > Environment'. Scroll down and see how much memory is being allocated for your site to run php scripts. It needs to be 64M or higher for 5.6.
chrison999 replied on at Permalink Reply
Based on a comment by Cahueya (above) about possible "bugs" with 5.6.0.2 I rolled back to 5.5.2.1 to see if that made a difference (it didn't!) and had forgotten I had done that. I have rolled forward to 5.6.0.2 again and, indeed, Override Caching is back. I have set it to ON but I don't see much of a difference in performance.

I installed that add-on (Thanks!) and found php.ini and memory is 64M. There's a comment directly above that parameter saying Hostpapa will cancel your service if you increase memory above that, so I guess I won't touch it. :) Btw, are there any other php.ini parameters that are critical to C5 that I should check?

Today, I'm going to go through all of the add-ons and themes and get rid of those that I'm not using. Will post later the results of that on performance.

Thanks for the additional comments & tips!

Chris
chrison999 replied on at Permalink Reply 2 Attachments
Okay, it took awhile but I have now deleted all unused add-ons from my site. While I was at it, I removed any themes non-stock themes I wasn't using.

The MySQL queries have improved quite a bit (see attachments) but, unfortunately, TTFB is still terrible:

http://www.webpagetest.org/result/121107_CX_HBV/...

Comments? Other suggestions?

Thanks, again, for any help!

Regards,

Chris
mhawke replied on at Permalink Reply 1 Attachment
mhawke
I ran your site through this service which is similar to the webpagetest you've been running except that it tries to measure the load that's on the server.

http://www.unshit.com

Not a very classy name but the final test it does is to put a load on the server and measure how fast it can respond. Your site is so slow that this test times out.

I have attached a quick 'single page' that I just put together to try to test the capacity of the server. It ain't pretty but it spits out some numbers that might be helpful. To install it, download it and rename it 'mySQL_benchmark.php' and upload it to your '[root]/single_pages' folder. Then visit 'Dashboard > Single Pages' and add 'mySQL_benchmark' (without the .php). This will add this page to the main navigation so you will want to move it somewhere more obscure.

The numbers it spits out are for comparison purposes only. These tests are just grabbed from other sources on the net and stuffed onto this page. If you can run this page and try to cut'n paste the results here, I can compare them to my numbers.
chrison999 replied on at Permalink Reply 2 Attachments
I tried running unshit.com on my page, too, and got "Error!" 3 times.

Thanks for that benchmark file. I installed it. (Btw, you forgot to mention having to edit the MySQL login parameters! LOL!) Attached are 2 screenshots of the results. I haven't a *clue* what any of that means, so I'm looking forward to hearing y our interpretation and (hopefully!) suggestions.

Thanks, again...

Regards,

Chris
chrison999 replied on at Permalink Reply 2 Attachments
This morning, I sent a lengthy and polite-but-firm e-mail to Hostpapa tech support. I stated that I have been working hard for several days to optimize my site, but the performance issues persist. I stated that C5 runs on hundreds (thousands?) of servers around the world and that having TTFB times of 30 to 50 *seconds* is NOT "normal." I then stated that IMHO there MUST be something wrong on the server end, even though they have repeatedly said for days "there are no issues on the server."

I then asked 3 questions:

1. Is my account being "throttled" in any way because of resource utilization, particularly memory usage?

2. Is it possible to increase my memory allocation to MySQL higher than what it currently is?

3. Is it possible to move my site to another server that has less traffic, better performance, or more resources?

I closed by stating that a reply of "it's not the server" would be unacceptable, and that I'd *really* like some answers. I then thanked them for their time and effort.

Just now, I received a one line reply from Hostpapa tech support:

"We have adjusted the server. Your issue should now be resolved."

I have NO idea what they did, but the TTFB ratings HAVE improved significantly:

http://www.webpagetest.org/result/121108_48_3MA/...
http://www.webpagetest.org/result/121108_2Q_3MQ/...
http://www.webpagetest.org/result/121108_48_3MA/...

As well, the MySQL query times have also significantly improved (see attached screenshots).

My question, now, is whether the above results are "good enough" or is there more I can do to speed things up? I already see that I need to convert my JPG files to PNG format and some other things, but I'm wondering if there's some more things I can do with "tweaking" the MySQL queries to optimize them a bit more?

Finally, let me give another HUGE thank-you to everyone who's helped me out these past few days! I have learned a lot about how C5 works, and I'm VERY grateful for all of the suggestions, pointers to helpful add-ons, and other assistance!

If anyone has any further suggestions I'm all ears!

Thanks, again...

Regards,

Chris

P.S. It would have been REALLY nice if they had "adjusted the server" when I first submitted the ticket! (WINK!)
mhawke replied on at Permalink Reply
mhawke
Despite your great strides, I still wouldn't be happy with a 5 second TTFB and a 9 second render time mainly because I know how long I personally stay on slow sites.

I noticed that your server is 'nginx' rather than Apache. Nginx is a relative newcomer to the server world and although it is promising, I'm not sure it's the best choice right now to host concrete5. Can you ask your host if you can be moved to an Apache server?
chrison999 replied on at Permalink Reply 1 Attachment
Thanks for the reply, mhawke!

I'm "somewhat pleased" with whatever they did last night as, now, we're at least "in the ballpark" of what the performance should be. People won't "stay on a slow site" but it's even *worse* if the site isn't even loading, eh?!? That's been reflected in my site's "hit" stats... they've been pretty much down to zero since this mess started!

However, as you said, these TTFB and render times are still "slow" by today's standards. That's why I asked for further suggestions... IMHO, There still work to be done! I'm not sure what I can get the TTFB and render times down to, but I'm pretty sure they could be a bit better than this.

Uhhh... where did you see the info that I'm not on an Apache server??? CPanel is showing that I'm on an Apache 2.2.22 server (see attached screenshot). I've never heard of nginx before!

Thanks for pointing that out! I'll look into it!

Regards,

Chris

EDIT: Never mind! I see, now, where it's saying nginx... that's CloudFlare, the CDN I'm using. I'll turn-off CloudFlare and see if that makes a difference. Again, thanks for pointing that out!
Remo replied on at Permalink Reply
Remo
You actually use both, nginx and Apache. That's because your providers works with Cloudflare which uses nginx.. I doubt that this causes any issues!
chrison999 replied on at Permalink Reply
Yeah, I just figured that out. Hostpapa = Apache but CloudFlare = nginx.

Just as a test, I'm going to temporarily disable CloudFlare and see if that makes any difference.

Thanks!

Chris
chrison999 replied on at Permalink Reply
Quick question, Remy... I just turned-off CloudFlare and re-ran the webpagetest.org test but it's still showing the site going through CloudFlare. Am I to assume that I need to wait awhile to run the tests so that turning-off CloudFlare takes effect (e.g. maybe DNS needs to be updated and propagated)???

Thanks!

Chris
Remo replied on at Permalink Reply
Remo
I'm not very experienced with Cloudfare but I'm pretty sure it has to change the DNS settings. After all nginx is mostly a reverse proxy which would perfectly fit in to such a configuration.
chrison999 replied on at Permalink Reply
I have no idea what a reverse proxy server is, but I'll wait awhile and then retry the tests so that any DNS changes propagate.

Thanks!

Chris
chrison999 replied on at Permalink Reply
Also, if you've had a chance to look at the screenshots for Dbbbb_debug I posted last night, are those more "normal" or what should be expected? Any further comments/suggestions on "tweaking" the site so those numbers drop further?

Thanks, again...

Chris
Remo replied on at Permalink Reply
Remo
I still think that my servers could handle these queries quite a bit faster but you still have a lot of queries. If I were you, I'd still replace the drop down nav!
chrison999 replied on at Permalink Reply
I took a look at that yesterday. In order to get rid of the drop-down nav at the top, I'm going to have to re-do the parent pages of a couple of the sections of the site. That will require a bit of effort on my part, but I'm definitely considering that. I'm also doing a lot of thinking about how to accomplish that.

Also, per your suggestions about reducing the number of queries, for those pages that have a drop-down menu both at the top and in the left sidebar, I'm going to get rid of one or the other (probably the header nav?) but I've got to decide which is the best way to go.

I'm also probably going to drop the date nav on the right sidebar of the main page, as you suggested. If possible, though, I'd like to replace it with something that at least displays the most recent blog post (or a summary of it and link to it). Would you (or anyone) know of an add-on that does that? Or, would doing that merely replace one query with another one? Any suggestions on this?

Finally, you also pointed out the gallery pages as requiring a lot of queries. I'm using Deluxe Image Gallery (1.1.6) because of some of its features (drag-and-drop arranging of photos, html in captions and pop-up descriptions, etc.). I'm hesitant to get rid of that add-on but think I might be able to speed things up by reducing the number of images being displayed, right? I'm going to contact the add-on's developer and ask him if there are ways to optimize the add-on. He's been *fantastic* in providing quick and helpful support, so maybe he has some ideas on how to speed things up?

In short, I definitely going to try to implement the suggestions you made. It's just a matter of figuring out the best way of doing it.

Thanks, again, for all your help!

Regards,

Chris
chrison999 replied on at Permalink Reply
Quick update:

I removed CloudFlare and TTFB stayed about the same (around 4-5 seconds) but render times went way up (from our 8-9 seconds to 14+ seconds). So, I put CloudFlare back, at least for now. I may try this again later, if I'm able to further speed-up the site.

Thanks for the suggestion, mhawke!

Chris
Cahueya replied on at Permalink Reply
What did you do? I just tried again and your site loaded in about 5 seconds.

http://www.webpagetest.org/result/121112_AY_B5T/1/details/...
chrison999 replied on at Permalink Reply
After days of hearing from the host (Hostpapa) that "there's nothing wrong with the server" I got an e-mail from them saying "we have adjusted the server." Lo and behold, the TTFB and render times dropped dramatically! I have *no* idea what they did, though.

That page you tested has very little content, so I have been testing a page that has more content to get a truer picture of site performance. What's interesting is that if I run several back-to-back tests, the first test has fairly high results but the second test is much better. Here's an example of back-to-back tests I just ran:

http://www.webpagetest.org/result/121113_5H_G3W/...
http://www.webpagetest.org/result/121113_Y8_G4B/...

As you can see, the TTFB/render times in the first test are around 7.9 and 11.8 seconds, respectively. However, a second test ran just 1 minute later gives TTFB/render times of 3.5 and 5.2 seconds, respectively.

Why??? And, perhaps even more important, are these results what I should be expecting?

I went back to Hostpapa and asked them if they could do anything else to make things a bit faster and got a somewhat snarky reply saying if I want to "have server resources all to myself" I should move to a dedicated server. I'm *not* asking to "hog" server resources... all I'd like is to squeeze a much speed out of the server as possible, given that I'm on a shared server.

So, again, I'm wondering if the results I'm getting are about as good as it gets, or can more be done? Again, I'm still working on optimizing C5 and the content as much as possible, but I'm wondering if more can be done on the host side too?

Thanks, again, to anyone who has any suggestions/tips to offer.

Regards,

Chris
Remo replied on at Permalink Reply
Remo
Could you please post the autonav block settings you're using? (Edit autonav block and take a screenshot).

I do have one more idea that might be causing some issues.
Cahueya replied on at Permalink Reply
So whenever your custom autonav is available as a piece of code to be hardcoded into a theme, please let me know :) I highly trust in your opinion if something is useful code, as I have used a and learned a lot of the things you wrote. Thanks!
chrison999 replied on at Permalink Reply
Huh??? I'm not using a custom auto nav that I created. However, I am using the Accordian Menu add-on for the auto nav in the left sidebar.

As for your learning a lot from this thread... me too! :) If you've learned stuff from me then, "You're welcome!" :)

Regards,

Chris
Remo replied on at Permalink Reply
Remo
I think this was address to me (:
chrison999 replied on at Permalink Reply
LOL! Well, excuse me for "stealing" your thank you!

I, too, have learned a lot from you, so "Thanks again!" from me too!

Regards,

Chris
chrison999 replied on at Permalink Reply 1 Attachment
Here's a screenshot of how both the header autonav and left sidebar autonav are currently configured. Btw, the header autonav uses the "drop down menu" custom template and the left sidebar autonav uses the "accordion menu orange tidy" custom template, in case that is helpful?
Remo replied on at Permalink Reply
Remo
In the last drop down, can you try to select "Display a custom amount" and enter 1 in the level field that should appear. I think it shouldn't change the output, if it does, try 2 and if it still doesn't show the same output, revert back and tell me that my idea didn't work ;-)
chrison999 replied on at Permalink Reply 2 Attachments
I changed sub-levels to display to a custom amount of "1" and it didn't change the output. However, I ran a couple more tests on that page and it didn't seem to impact performance that much:

http://www.webpagetest.org/result/121113_1V_GSD/...

Btw, I have attached a screenshot of the queries in case you want to take a look.
Cahueya replied on at Permalink Reply
Yeah, make him blush :)

Or, so our testserver has a lookup time of 1.6sec, with a TTFB of 0.409sec, time to play around :)
mhawke replied on at Permalink Reply
mhawke
There have been several posts regarding very long TTFB times for the first time you hit the site and then faster times after that. I also had this problem with 40 seconds the first time and 5 seconds for subsequent refreshes (with my browser caching turned off). I bugged my host until I was finally elevated to someone who knew what they were doing and the problem miraculously went away.

I think it's common for the first level of tech support to be instructed to give us the brush-off because in all likelihood the casual user will accept their BS and put up with it. They may even be paid a bonus for faster customer 'throughput' so it's no wonder they don't want to spend the time required to fix the problem. I no longer put up with it and demand I be escalated immediately by telling the first person to pass me to 'Dave' or 'Frank' or whoever it was that took my problem seriously and had the power and knowledge to actually fix things.
thecaen replied on at Permalink Reply
It would be nice to know what the host does b/c i have a TOFB issue as well.
http://thecaen.ca/index.php - This week I have had to shut my site down with a page maintenance index.html page because it became too slow.http://thecaen.ca

I was one stormweb with C5 simple script, TOFB and speed resulted in conflict with the host, did a full reinstall on a new shared server that only has 56 sites on it and run by a friend of a friend (he boosted all my bandwidth memory limits above normal). The TOFB was an issue, but much improved from the previous, but as the amount of content grew TOFB (10 problog posts a day x 3months ) became crippling.

So if it is not other sites slowing me down, what can be done?

yes, i have unused add-ons and themes, too many apis (two are lazy load), but is that 10 stories a day and 2xday homepage updates just too much over time for a shared host environment? Should I just run it on a VPS - which i cannot afford? Should dynamic sites with lots of updates run a VPS?

Ultimately, the problog posts with limited addons on the story pages had a worse TOFB than the arguably heavy loaded homepage?

a few days ago, oUtside support built the site on a his hostgator shared server (no Images) and the TOFB is much less of an issue -http://codesolvent.com/projects/thecaen/index.php...

so if this a server configuration issue, than what is the magique?
IgalZe replied on at Permalink Reply
IgalZe
" I implemented the Cloudflare CDN which has improved loading content after the TTFB, but hasn't done anything to help TTFB."

CDNs, by default, will not have a positive effect on TTFB - unless they are able to cache HTML (which is often dynamic and thus marked max-age:0 in headers). In fact, 9 out of 10, will actually add latency by adding another "hop" between the browser and origin server.

This limitation can be overcome with an intelligent caching algorithms that will override the default header directives, which ensuring caching.

Reference:
http://www.incapsula.com/the-incapsula-blog/item/809-using-cdn-to-i...