Using CDN increases page (download) size

digitalmuse

I've moved my images to a CDN (Cloudfront) to speed up my site.

But it turned out that moving my images to a CDN caused my site to load much slower because it was downloading over twice the amount of data. My home page, for example, which only has 11 images jumped from 3.37MB to 8.55MB.

In looking at the source, I noticed that it's no longer loading images from /bfi_thumb - which I believe is why the file sizes are so much larger.

A little more info: I uploaded all of the images (including those in /bfi_thumb) to Amazon S3 and I have CloudFront CDN pull from S3.

I'm using WP Rocket to replace image hostnames to point to the CDN. According to their docs, it rewrites URLs for all assets (I'm only having it do this for images) that are added using WordPress functions (e.g., post thumbnails etc):
http://docs.wp-rocket.me/article/42-using-wp-rocket-with-a-cdn

Any idea how I can resolve?



Thanks!
Abby

Air

Hello:-)

If this plugin really only swaps names and takes real files to CDN there should not be such problem. From what you say it looks like it does something else to files, like asking for sources of images in different way.

We have CDN service active for Photon theme on our kinsta server and it works fine, doing proper job. I don't have any experience with that plugin that you are using. I think best answer what is happening you will get from this plugin support.

Could you post link to that site where this happens?

With kind regards.

Air

I have been on your site and I see that CDN is now deactivated.

Can you post me links to same image? One for non CDN source and one for CDN source?

digitalmuse

I activated a CDN on my staging server:
http://digitalmuse.site/davidbeifeld.photography/

As you can see - it's pulling the images from the /wp-content/uploads/... folder. e.g., :
http://cdn.davidbeifeld.photography/davidbeifeld.photography/wp-content/uploads/2015/10/1310214434-Czech-Rebublic-Cesky-Krumlov.jpg

Instead of the bfi_thumb files, which do exist on the CDN - e.g., :
http://cdn.davidbeifeld.photography/davidbeifeld.photography/wp-content/uploads/bfi_thumb/1310214434-Czech-Rebublic-Cesky-Krumlov-mgz3mi4jqxlx53sdy2tm499m1hcjzxc1n7f7uhdmc2.jpg


Without the CDN rewrite rule, it pulls the images from the bfi_thumb folder.
For example, in production (http://davidbeifeld.photography/) it gets it from:
http://davidbeifeld.photography/wp-content/uploads/bfi_thumb/1310214434-Czech-Rebublic-Cesky-Krumlov-mgz3mi4jqxlx53sdy2tm499m1hcjzxc1n7f7uhdmc2.jpg



Any ideas? I'd love to learn more about how you set up a CDN service for Photon? I'm confused as to:
a) how I'd generate all of the bfi_thumb images so that I can upload them to the CDN
b) how to tell Photon to get the bfi_thumb version of the images from the CDN
c) do you use a plugin to rewrite the URLs or do you use PHP that maybe you could share?


Thanks!
Abby

Air

As I said before, plugin you choose have to be doing something else beside only redirecting files to CDN.

digitalmuse said: a) how I'd generate all of the bfi_thumb images so that I can upload them to the CDN

They are generated when server asks for them, and if they don't exist they are created on the fly.

digitalmuse said: b) how to tell Photon to get the bfi_thumb version of the images from the CDN

Theme already is doing it, but your plugin overwrites it. You see, BFI THumb is adding filter to default Wordpress "Image Manager" so when WP asks for image with certain parameter, then it is generated by BFI.

In your case you are using plugin that is acting on code to change links to CDN. I assume it works this way :-)

CDN configuration was done for us by our hosting company. I think they used for it W3 Total cache plugin. It only replaces apollo13.kinsta.com with apollo13.cdn.kinsta.com in all sources, so it does not mess with generating images.

Hope it helps somehow.

PS. I have attached our CDN configuration form that plugin.

With kind regards.

digitalmuse

I think the problem is that Photon is not able to write to my CDN and so the bfi_thumb() function fails because it can't write to the directory and so defaults to returning the original, full size image -- even though I've already uploaded all of the \bfi_thumb images to the CDN (so there's no need for it to write to them).

I believe W3 Total Cache DOES allow you to write to the CDN and so that's why it's working for you.

I'm asking WP Rocket about it but do you know anyway to make this work so that:
a) the \bfi_thumb images are all generated up front so I can upload them to the CDN, and
b) bfi_thumb() function works (returns the \bfi_thumb version of the image) even if the \bfi_thumb directory isn't writeable

Thanks!
Abby

Air

I still think it is connected to the way WP Rocket is "attacking" image action of Wordpress

Air said: You see, BFI THumb is adding filter to default Wordpress "Image Manager" so when WP asks for image with certain parameter, then it is generated by BFI.

When server asks about image BFI checks does it have already in cache(bfi_thumb dir) and if not it creates it. It can't create such image when directory is not writable.

BUT !
This not how CDN works, according to my knowledge. Files should be created as they used to be on your server, and then "sucked" to CDN when page asks for them. And that is why I assume that issue is on WP filters level.

I am not expert in that field, so I can be very wrong:-)

With kind regards.

digitalmuse

Okay - I switched to W3 Total Cache and that does work *IF* I've already uploaded the bfi_thumb.

However, I'm using origin push to Amazon S3 and then my CDN pulls from S3. That means the files need to somehow get uploaded from my server to S3.

With W3 Total Cache, when I add a new photo to the WordPress Media Library, W3 Total Cache automatically uploads both that photo plus the thumbnails that WordPress creates it.

But the /bfi_thumb files are created on the fly by Photon, so I don't think there's any way to make W3 Total Cache recognize when a new one is added.

Can you tell me how I might be able to automatically create all the /bfi_thumb files when I add a new image so that I can upload them myself?


Thanks!
Abby

ps - the caching plugin I was using before was this one: http://wp-rocket.me/

Air

digitalmuse said: Can you tell me how I might be able to automatically create all the /bfi_thumb files when I add a new image so that I can upload them myself?

Like you have said, they are created on the fly, so it is not possible to do it beforehand.

What about theme images, like social icons - are they uploaded to CDN server beforehand?

I ask cause it is not how it should work. Media, JS, CSS, should be uploaded to CDN when server is asked about them. Or maybe our hosting provider did some script that does that for us. However even when we had CDN on https://www.maxcdn.com/ it worked same way - resource was "sucked" to CDN on first ask for such resource and then served from there.

With kind regards.

digitalmuse

I'm still learning all of this myself, but here's a little background from what I've learned:

There's different ways to do CDNs. One way is Origin Pull - which it sounds like you're using. With Origin Pull, when someone requests a file from your CDN, if it doesn't exist on the CDN, the CDN pulls it off of your server.

That doesn't always make sense as a good way to go though. For example, I have so many files on my site (over 700 images and that's actually going to get much larger as I have way more photos to add. But even with 700 photos, that equates to 2800 files when you count three bfi_thumb versions for each - which I'm not even sure are all of the possible sizes?). And the site isn't high traffic. So - when someone requests any particular image from the CDN, there's a decent chance it will have expired (no longer be on the CDN). In this case, the CDN then has to get it off my server... at that point, I might as well skip the CDN altogether because it's just grabbing things off my server. So, another option (which you'll see in the W3 Total Cache options - see attached image) is Origin Push -- meaning you tell W3 Total Cache to PUSH your files up.

I do Origin Push to Amazon S3 (their storage server). Then if it's expired on Amazon's CloudFront CDN - Amazon just pulls it from their S3 server - which is way faster then going back to my server.

--

So then, to answer your question: yes, I upload everything to the server before hand. As soon as I set W3 Total Cache up as Origin Push, it has me pick which files I want on the CDN and then it uploads everything for me. W3 Total Cache then watches my WordPress and whenever I add a new file or upload a new media file, it automatically pushes that to Amazon.

The problem, if I understand correctly (?), is that the way bfi_thumb's are created is outside of how WordPress typically creates thumbnails (e.g., it automatically creates 150x150, 300xwhatever, 1024xwhatever when you upload a new media file). So W3TotalCache isn't able to hook into that in order to upload it to Amazon.

What I've done so far is I've already got 3 bfi_thumb images for every photo in my /bfi_thumb directory (from people hitting the website) and so I uploaded those to Amazon.

But I don't even know - is it possible that Photon would create more sizes then 3/image?

If there's even a way to load the whole site with the different device types/screen sizes to force it to generate them (e.g., load from desktop, phone & tablet) then I could at least do that ahead of time whenever I add new photos.

But right now I don't understand how it works well enough to even know if I've been able to generate all possible /bfi_thumbs so that my site doesn't break.

Does that make sense? Any suggestions?

Thanks!
Abby

Air

digitalmuse said: I'm still learning all of this myself

I think you already know more then me :-)

digitalmuse said: One way is Origin Pull - which it sounds like you're using

I think so too.

digitalmuse said: And the site isn't high traffic.

What is the point of CDN then? Any other then learning things ?:-) Some systems only make sense in particular conditions, as they often introduce much more complications.

digitalmuse said: Does that make sense?

Yes it does :-) You don't need to run your site on other devices, desktop is enough.

We use bfi_thumb only cause of easier work for user when he changes column width in bricks mode. This gives us control over reasonable size and quality and user don't have rebuild all thumbnails each time he/she changes size of column in bricks mode. This also enables different column width per each album.

As far I as remember there are 3 place:
-Albums list
-Album in bricks mode
-Blog list in bricks mode

So depending on usage of particular image it can have 1 to 3 bfi_thumb sizes. BUT each album can have different setting for column width. However if you use each image in only one album then it shouldn't worry you. If hover you have 20 different albums with different column widths each, and in each you use same image, then you can have 20 bfi thumbs for it :-)

To clarify column width is dependent on bricks columns, bricks margin and Max width of bricks content.

If you would make script or run some plugin that crawls over whole your page visiting each album, albums list and blogs list then you would have all sizes of images. I am just not sure if lazy load of images wouldn't break it. However I think that when server returns link to particular bfi thumb, then it is already created.

Hope that helps :-)

With kind regards.

digitalmuse

The point of the CDN is to speed it up for the users. Just because it's not high traffic doesn't mean I don't want a good experience for my users And it's got hundreds of high res images so a CDN becomes so important to performance.

And okay, cool - I always load every page and make sure every image loads as a standard test before I deploy anyway so that's perfect - I'll just do that and tell W3 Total Cache to push the images up to Amazon.

Thanks for all of your help (once again!)
Abby

Air

digitalmuse said: The point of the CDN is to speed it up for the users

I believe :-) But I just wonder if it will be noticeable:-)

Good luck with your page!

With kind regards.