Need Help

Permalink Browser Info Environment
Hi,
Hope you're well.

We have migrated our website to new server. After migration, I had to reinstall this addon again on website, it was not there so.

After re-installing it we have lost all records of 1000 images that have been watermarked. In actually images have watermarks, but in WatermarkNN list, it shows only backup with red button.

I am attaching screenshot.
Take a look and let me know if we can do anything to solve this issue.

To get record back, I think I will have to go through all images assign watermark again, because rescanning all images with current watermark function gives internal server error.

We are still on production server, so we will again migrate website to live server. So I'm in fear that if I do all watermarks again and then move site to new server, I will lose all records again.

Thanks
Suraj

Type: Pre-Sale
Status: Resolved
srjahir32
View Replies:
emirii replied on at Permalink Reply
emirii
Hi Suraj,

I don't see your attachment on the post, but there are a few checks in place to make sure that the migration was successful:

1. Do the images in Concrete5 have the "watermark_n_n" attribute on them checked?
2. Was the Concrete5 "files" directory copied to the new server?

Maybe try to re-attach the image you were trying to attach? Did you use the Concrete5 backup/restore feature and copy all the application directory contents to production server?

Have no fear, we can try several different things before resorting to do all that work over again!
srjahir32 replied on at Permalink Reply 2 Attachments
srjahir32
Hi,
Yes Image folder were copied to new server.
Sorry, I forgot to attach file.
Here I've attached it and yes watermake_n_n attribute is assgined.
emirii replied on at Permalink Reply
emirii
OK! I think the images might just need to be re-scanned.. I will have to test.

I would like to create a Queueable Job for C5 for my package, that way you can run a batch job in the background to rescan all the images using a C5 job. Hopefully this will allow you to rescan all images. The rescan will hopefully add the watermark option to all the images again.

Please give me tonight and possibly tomorrow night to work on it. Thank you.
srjahir32 replied on at Permalink Reply
srjahir32
Thank you emirii,
:)
emirii replied on at Permalink Reply
emirii
Hello,

Please update to watermark version 1.1.0!

It will install a new job called "Rescan Watermarked Images", you can find the job under "yoursite.com/dashboard/system/optimization/jobs". It will re-apply the current watermark settings to all currently watermarked images. I think this should fix your issue. Concrete5 stores these "scanned" images in the cache directory I believe, which is sometimes hard deleted. This should help!

I tested it with 300 images and it ran no problems, it took around 8 minutes to complete. For 1,000 images I would give it maybe 30 minutes to an hour and see if it completes.

If you run into any issues please post back so I can take a look!
srjahir32 replied on at Permalink Reply 1 Attachment
srjahir32
HI emirii,
Its working correctly but it scanes only 25 images and process is completed after that.
So how can I re-scan other images except first 25.

attached screenshot.
Thanks
srjahir32 replied on at Permalink Reply
srjahir32
Hi Emirii,
One more thing. Will this happen again, if we move from production to live server ?
and If yes, what steps should be taken to overcome it ?

Thanks
emirii replied on at Permalink Reply
emirii
Hi there, sorry for the late reply, been a very busy few days.

I have been looking at the issue but I can't seem to reproduce it. Is the staging server live where I may log in to the dashboard and see?

As for sending to production, I'm not too sure.

It depends a lot on how you migrate, are displaying the images etc. If you are displaying with pre-made thumbnail presets, then all the thumbnails are located in "application/files/thumbnails/" which should also be transferred.

I would see if your images are being loaded under this directory, rather than the cache. If your images are loading from the cache, then that would probably be one issue.

I hope to get the job working for you, but I would have to see the issue happening on your end as I can't reproduce it.. Maybe it would give me some insight to answer the question above better also.

Thank you!
srjahir32 replied on at Permalink Reply
srjahir32
Hi emirii,
Thanks !
I've sent you message on concrete5 profile. I mean pm you.
Can you take a look at it.

Thanks
emirii replied on at Permalink Reply
emirii
Thank you for the information. I was able to log in and I did a quick search in the File Manager for Watermarked attributed images. Only 29 images showed as having the Watermark attribute on them.. When trying out the Job, I see that 29 were being processed, so that would be working correctly. The 29 images all had the watermark on their thumbnails.

Now the odd part, is I was looking through some of the normal images as well, and a lot of images do have the watermark also (probably from the transfer) but somehow the watermark attribute was removed or reset.

As for the transfer, did you transfer the entire Concrete5 directory install in tact and also export/import the database? Did you uninstall and reinstall any plugins? I can only think of a few things off the top of my head why Concrete5 would not save the file attributes...

Good news is the backups are in place in the application/files/watermark_n_n directory, I can write you another simple job that checks to see if a WatermarkNN backup is available for a particular file, if it is, then mark the watermark attribute to true again. It would save you a lot of work I think.

Give me this weekend for it!

Thank you.
emirii replied on at Permalink Reply
emirii
Ok, please update to version 1.1.1

I added new job "Reapply watermark attribute on backups". This will search the files and look to see if a WatermarkNN backup exists. If a WatermarkNN backup exists, it will mark the watermark attribute to true. Then you can use the other job and rescan all the watermarks, and you should be good to go.

It might catch a few images that were watermarked, then removed, but hopefully those are far and few between, it might be better than re-selecting all the watermarked images again.

Thanks, let me know if it works out for you.
srjahir32 replied on at Permalink Reply 1 Attachment
srjahir32
Hi Emirry, You last solution with crone job worked successfully. Thank you so much, you saved us lots of time.

But the crone job you created before keeps running and never ends. I tried to reschedule it but it never stops . I'm attaching screenshot. please take a look.
srjahir32 replied on at Permalink Reply
srjahir32
HI emirii, Did you got chance to look at issue with previous crone job ? It never gets ending.
emirii replied on at Permalink Reply
emirii
Hi, I'm so sorry, for some reason I did not receive the last notification...

C5 doesn't have very good error handling when it comes to the jobs, so the job here probably encountered some sort of error (most likely in our case out of memory or something similar). The good news is the job isn't running anymore in the background.

It is easy to fix so that the job isn't "spinning" anymore and can be re-runned if necessary, we just have to tell C5 that the job was completed. You will have to modify the database unfortunately:

Find C5 "Jobs" table, and in column "jStatus" it should say "ENABLED". If any of the jobs you aren't currently running don't say "ENABLED", it is safe to change the text value to "ENABLED".

That should clear it up! Sorry for the late reply.
srjahir32 replied on at Permalink Reply
srjahir32
HI Emirri,
Thanks you so much for all help.

It appears the issue is not so much as a bug, but its instead a combination of Concrete5's weak error handling and inadequate memory on our server?

So I am guessing we would make this feature request to Concrete5 involving more robust error handling for Cron Jobs?

I would be willing to submit this feature request to Concrete5, But I believe that your request would be better received since you're a concrete5 add-on developer with good reputation here.

Would Please do for us ? Let me know If I can do anything for you to do this.
Thanks
srjahir32 replied on at Permalink Reply
srjahir32
Hi Emirri, I know you've been busy. Apology for that.
Did you check my last reply.
Thanks
emirii replied on at Permalink Reply
emirii
Hello!

I'm sorry again, I could have swore I replied...

I believe the C5 team is quite aware of this downfall as it has been a problem since C5.6 as well.. So I'm not sure what good a bug report would do. It took them nearly 5 months to get around to one I submitted before...

Did editing the database fix the issue for you at least with the spinning icon?

Thanks!

concrete5 Environment Information

# concrete5 Version
Core Version - 5.7.5.8
Version Installed - 5.7.5.8
Database Version - 20160412000000

# concrete5 Packages
Add and Remove Classes and IDs (0.9.5), Fruitful (1.2), Honest Websites Back To Top (0.9.4), HTML5 Audio Player Basic (2.0.5), Responsive Embed (1.0.0), Share Me (0.9.1), Spacer (0.9.3), Thumbnail Generator (0.9.0.6), Touch Gallery (2.1.5), WatermarkNN (1.0.1).

# concrete5 Overrides
blocks/image/view.php, blocks/image, js/redactor.js, languages/da_DK/LC_MESSAGES/messages.mo, languages/da_DK/LC_MESSAGES, languages/da_DK, languages/de_DE/LC_MESSAGES/messages.mo, languages/de_DE/LC_MESSAGES, languages/de_DE, languages/el_GR/LC_MESSAGES/messages.mo, languages/el_GR/LC_MESSAGES, languages/el_GR, languages/fi_FI/LC_MESSAGES/messages.mo, languages/fi_FI/LC_MESSAGES, languages/fi_FI, languages/fr_FR/LC_MESSAGES/messages.mo, languages/fr_FR/LC_MESSAGES, languages/fr_FR, languages/it_IT/LC_MESSAGES/messages.mo, languages/it_IT/LC_MESSAGES, languages/it_IT, languages/ja_JP/LC_MESSAGES/messages.mo, languages/ja_JP/LC_MESSAGES, languages/ja_JP, languages/nl_NL/LC_MESSAGES/messages.mo, languages/nl_NL/LC_MESSAGES, languages/nl_NL, languages/pl_PL/LC_MESSAGES/messages.mo, languages/pl_PL/LC_MESSAGES, languages/pl_PL, languages/pt_BR/LC_MESSAGES/messages.mo, languages/pt_BR/LC_MESSAGES, languages/pt_BR, languages/ru_RU/LC_MESSAGES/messages.mo, languages/ru_RU/LC_MESSAGES, languages/ru_RU, languages/sl_SI/LC_MESSAGES/messages.mo, languages/sl_SI/LC_MESSAGES, languages/sl_SI, languages/sv_SE/LC_MESSAGES/messages.mo, languages/sv_SE/LC_MESSAGES, languages/sv_SE, languages/tr_TR/LC_MESSAGES/messages.mo, languages/tr_TR/LC_MESSAGES, languages/tr_TR, src/Retina/RetinaImage.php, src/Retina/RetinaPicture.php, src/Retina, blocks/image/view.php, blocks/image, js/redactor.js, languages/da_DK/LC_MESSAGES/messages.mo, languages/da_DK/LC_MESSAGES, languages/da_DK, languages/de_DE/LC_MESSAGES/messages.mo, languages/de_DE/LC_MESSAGES, languages/de_DE, languages/el_GR/LC_MESSAGES/messages.mo, languages/el_GR/LC_MESSAGES, languages/el_GR, languages/fi_FI/LC_MESSAGES/messages.mo, languages/fi_FI/LC_MESSAGES, languages/fi_FI, languages/fr_FR/LC_MESSAGES/messages.mo, languages/fr_FR/LC_MESSAGES, languages/fr_FR, languages/it_IT/LC_MESSAGES/messages.mo, languages/it_IT/LC_MESSAGES, languages/it_IT, languages/ja_JP/LC_MESSAGES/messages.mo, languages/ja_JP/LC_MESSAGES, languages/ja_JP, languages/nl_NL/LC_MESSAGES/messages.mo, languages/nl_NL/LC_MESSAGES, languages/nl_NL, languages/pl_PL/LC_MESSAGES/messages.mo, languages/pl_PL/LC_MESSAGES, languages/pl_PL, languages/pt_BR/LC_MESSAGES/messages.mo, languages/pt_BR/LC_MESSAGES, languages/pt_BR, languages/ru_RU/LC_MESSAGES/messages.mo, languages/ru_RU/LC_MESSAGES, languages/ru_RU, languages/sl_SI/LC_MESSAGES/messages.mo, languages/sl_SI/LC_MESSAGES, languages/sl_SI, languages/sv_SE/LC_MESSAGES/messages.mo, languages/sv_SE/LC_MESSAGES, languages/sv_SE, languages/tr_TR/LC_MESSAGES/messages.mo, languages/tr_TR/LC_MESSAGES, languages/tr_TR, src/Retina/RetinaImage.php, src/Retina/RetinaPicture.php, src/Retina

# concrete5 Cache Settings
Block Cache - Off
Overrides Cache - Off
Full Page Caching - Off
Full Page Cache Lifetime - Every 6 hours (default setting).

# Server Software
Apache

# Server API
cgi-fcgi

# PHP Version
5.6.20

# PHP Extensions
bcmath, calendar, cgi-fcgi, Core, ctype, curl, date, dom, ereg, exif, filter, ftp, gd, gettext, hash, iconv, imap, ionCube Loader, json, libxml, mbstring, mcrypt, mysql, mysqlnd, openssl, pcre, PDO, pdo_mysql, pdo_sqlite, Phar, posix, Reflection, session, SimpleXML, sockets, SPL, sqlite3, standard, tokenizer, xml, xmlreader, xmlwriter, Zend OPcache, zip, zlib.

# PHP Settings
max_execution_time - 30
log_errors_max_len - 1024
max_file_uploads - 20
max_input_nesting_level - 64
max_input_time - 60
max_input_vars - 1000
memory_limit - 1024M
post_max_size - 8M
sql.safe_mode - Off
upload_max_filesize - 2M
mysql.max_links - Unlimited
mysql.max_persistent - Unlimited
pcre.backtrack_limit - 1000000
pcre.recursion_limit - 100000
session.cache_limiter - <i>no value</i>
session.gc_maxlifetime - 7200
opcache.max_accelerated_files - 2000
opcache.max_file_size - 0
opcache.max_wasted_percentage - 5

Browser User-Agent String

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

Hide Post Content

This will replace the post content with the message: "Content has been removed by an Administrator"

Hide Content

Request Refund

You may not request a refund that is not currently owned by you.