Getting "Couldn't dump database" message
Permalink Browser Info Environment
Hello,
for some reason I'm getting "Couldn't dump database" error on two different servers (all servers I have tested this far). Pasted environment of a test site here there.
Cheers, Petro
for some reason I'm getting "Couldn't dump database" error on two different servers (all servers I have tested this far). Pasted environment of a test site here there.
Cheers, Petro
| Type: | Discussion |
|---|---|
| Status: | In Progress |
No symbols on passwords in either of the sites where I tested it. I will try to find time to test if this works on fresh install of the newest Concrete5 set in english language.
hmmm, that's a bummer.
How about the server name (which you can find in application/config/database.php)?
How about the server name (which you can find in application/config/database.php)?
Could you please try version 2.0.4, available from
https://www.concrete5.org/marketplace/addons/back-up-files/......
or hopefully from your c5 site's dashboard.
https://www.concrete5.org/marketplace/addons/back-up-files/......
or hopefully from your c5 site's dashboard.
Did a fresh install of Concrete5 as a test and installed 2.0.4 did not help. Here is my database.php as well, no symbols there either.
<?php return [ 'default-connection' => 'concrete', 'connections' => [ 'concrete' => [ 'driver' => 'c5_pdo_mysql', 'server' => 'mysql5.sigmatic.fi', 'database' => 'roihucre_testi', 'username' => ' ', 'password' => ' ', 'charset' => 'utf8', ], ], ];
Could you please install and run the attached version? It won't work - but it should save some diagnostic info.
The job should now create up to three files in the base directory of your site: mysqldump-error-log.txt, mysqldump-stdout.txt and mysqldump-stderr.txt. (Two of them may be empty.) Could you please send me those files? You can delete them after doing so.
It could also be useful if you could send me application/files/backup/gond-backup.cnf - although you might want to remove the password from it first.
I assume that there's a zero length database file in application/files/backup.
I have a nasty suspicion that the problem is an inability to connect to the database server.
The job should now create up to three files in the base directory of your site: mysqldump-error-log.txt, mysqldump-stdout.txt and mysqldump-stderr.txt. (Two of them may be empty.) Could you please send me those files? You can delete them after doing so.
It could also be useful if you could send me application/files/backup/gond-backup.cnf - although you might want to remove the password from it first.
I assume that there's a zero length database file in application/files/backup.
I have a nasty suspicion that the problem is an inability to connect to the database server.
These are the only files it generated, I have actually not even gotten an empty database dump file. I have few servers where I could test this too if you want to see what other servers do?
gond-backup.cnf
gond-backup.cnf
[mysqldump] host="mysql5.sigmatic.fi" user="roihucre_inc" password="" default-character-set=utf8 complete-insert events routines
Simple Backup (Backup Database): failed to execute /usr/bin/mysqldump --defaults-extra-file=application/files/backup/gond-backup.cnf --verbose --log-error="mysqldump-log-error.txt" -r "application/files/backup/POHJAv2 database.sql" roihucre_pohjav2 > mysqldump-stdout.txt 2> mysqldump-stderr.txt (return code: 127)
Thanks. That info was illuminating. The issue you're suffering is actually different to that being experienced here:
https://www.concrete5.org/community/forums/customizing_c5/simple-bac...
In your case, my add-on isn't finding the mysqldump program that does the hard stuff. This could be because your host hasn't installed it, or because the add-on is looking for it in the wrong place. The fact that your database server is on a separate system (ie, not localhost) could be an issue; I don't have access to such a system to investigate.
Unfortunately, ftp or file managers probably won't be sufficient to track down where mysqldump might be. Do you have access to SSL (eg, putty)? If so, you could do some dredging there to see whether you can find it. Try looking in /usr and /usr/bin. If nothing, see what your $PATH is, and see if anything therein could be relevant. If this is not feasible, not to worry: I can try to put together some code that would do these things for you.
It would indeed be interesting to see if the add-on works on the other servers to which you have access. In addition to the diagnostic files it creates, it should also write an entry to your c5 site's log (accessible from dashboard) which could be helpful.
Many thanks for working with me on this! :)
https://www.concrete5.org/community/forums/customizing_c5/simple-bac...
In your case, my add-on isn't finding the mysqldump program that does the hard stuff. This could be because your host hasn't installed it, or because the add-on is looking for it in the wrong place. The fact that your database server is on a separate system (ie, not localhost) could be an issue; I don't have access to such a system to investigate.
Unfortunately, ftp or file managers probably won't be sufficient to track down where mysqldump might be. Do you have access to SSL (eg, putty)? If so, you could do some dredging there to see whether you can find it. Try looking in /usr and /usr/bin. If nothing, see what your $PATH is, and see if anything therein could be relevant. If this is not feasible, not to worry: I can try to put together some code that would do these things for you.
It would indeed be interesting to see if the add-on works on the other servers to which you have access. In addition to the diagnostic files it creates, it should also write an entry to your c5 site's log (accessible from dashboard) which could be helpful.
Many thanks for working with me on this! :)
Here is another server, same problem, database not at localhost either.
[mysqldump] host="172.17.0.1" user="ippuser1" password="" default-character-set=utf8 complete-insert events routines
Simple Backup (Backup Database): failed to execute /usr/bin/mysqldump --defaults-extra-file=application/files/backup/gond-backup.cnf --verbose --log-error="mysqldump-log-error.txt" -r "application/files/backup/POHJAv2 database.sql" roihucre_pohjav2 > mysqldump-stdout.txt 2> mysqldump-stderr.txt (return code: 127)
I was in contact with our server provider, and they said that this should work:
mysqldump -u mysql5.sigmatic.fi -p roihucre_pohjav2 > dump.sql
tho this command asks for a password of course.
also said that this command puts the dump to the folder where the command is ran from:
-u = mysql5.sigmatic.fi
I'm no expert with servers or with MySQL, but what I gathered from this is that you don't need to run the mysqldump from a location like "/usr/bin/mysqldump" cause the MySQL server is here mysql5.sigmatic.fi, hence mysqldump is there and not in some certain location.
mysqldump -u mysql5.sigmatic.fi -p roihucre_pohjav2 > dump.sql
tho this command asks for a password of course.
also said that this command puts the dump to the folder where the command is ran from:
-u = mysql5.sigmatic.fi
I'm no expert with servers or with MySQL, but what I gathered from this is that you don't need to run the mysqldump from a location like "/usr/bin/mysqldump" cause the MySQL server is here mysql5.sigmatic.fi, hence mysqldump is there and not in some certain location.
THIS IS THE LATEST MESSAGE, wrote this here because of the odd arrangement of the posts.
After some more talks with our host, I also conducted a lot of more tests and noticed some super odd things happening, I managed to create dump file but it was empty (possibly the same problem as other people have), so I found out after a lot of testing that when I ran mysqldump through test php file the result was always a empty, but when the host ran the the same command (not through php file) the dump was created every time correctly.
This all was result of the mysqldump is not usable when running .php files, odd problem to have, but probably not uncommon.
They then recommended using php mysqldumping such as this:
https://github.com/ifsnop/mysqldump-php/blob/master/src/Ifsnop/Mysql...
Not sure if you are willing to try implementing this, but if you are, I'm glad to help testing it.
After some more talks with our host, I also conducted a lot of more tests and noticed some super odd things happening, I managed to create dump file but it was empty (possibly the same problem as other people have), so I found out after a lot of testing that when I ran mysqldump through test php file the result was always a empty, but when the host ran the the same command (not through php file) the dump was created every time correctly.
This all was result of the mysqldump is not usable when running .php files, odd problem to have, but probably not uncommon.
They then recommended using php mysqldumping such as this:
https://github.com/ifsnop/mysqldump-php/blob/master/src/Ifsnop/Mysql...
Not sure if you are willing to try implementing this, but if you are, I'm glad to help testing it.
Your database password may have been exposed here: "mysqldump -u mysql5.sigmatic.fi -p...". If so, see if you can edit/delete this post (it's visible to all). You may also want to change your database password.
Another server, it is starting to look like we have some really "secure" servers here in Finland.
These are only logs and files it managed to do, quite certain there is nothing that can be done about this one other than messaging the host.
These are only logs and files it managed to do, quite certain there is nothing that can be done about this one other than messaging the host.
[mysqldump] host="mysql3.shellit.org" user="u36961" password="" default-character-set=utf8 complete-insert events routines
system() has been disabled for security reasons
mysql5.sigmatic.fi
==================
You're probably right: the directory path may not be necessary in your case. Can you please relace packages\gond_backup_files\jobs\gond_backup_database.php with the attached version, and rerun the job? (Doing it this way saves me from creating a whole new version of the add-on, and saves you from having to install it using c5.) If it doesn't work, we might still get some useful new info from the c5 log entry (to check that it's trying to execute what we want), and the various mysqldump-*.txt files.
Your service provider's command line for mysqldump looks odd to me. -u should be followed by the username, but is being followed by the host/server name (which requires -h). In addition, -p seems to be followed by your password, so mysqldump shouldn't have to prompt for it. (PS: specifying the password on the command line isn't very secure, which is why I use a config file instead.) You probably worked all this out for yourself in your subsequent tests.
If you managed to create empty database files in your tests, you may have made useful progress. You could have managed to get mysqldump to run, but it's failing. My guess would be server not found. You can capture its error messages in a couple of ways; the easiest is to append this to your command line:
2>errors.txt
then look in errors.txt.
Some hosts automatically kill processes that cause CPU/memory spikes, which could prevent mysqldump from completing. I doubt that this is the problem.
172.17.0.1
==========
This could well be the same problem as mysql5.sigmatic.fi, so please try the attached job.
I note that 172.17.0.1 is a private IP address, but I don't think that should matter.
mysql3.shellit.org
==================
As you say, this would need the host to relax this restriction, or use straight php-based solution such as the one you mentioned. Such a solution does exist for c5: seehttps://datatainment.de/this-n-that/concrete5-backup-and-restore-dat... I'm loathe to implement something like this myself for various reasons (robustness, commplexity, and it's already been done).
==================
You're probably right: the directory path may not be necessary in your case. Can you please relace packages\gond_backup_files\jobs\gond_backup_database.php with the attached version, and rerun the job? (Doing it this way saves me from creating a whole new version of the add-on, and saves you from having to install it using c5.) If it doesn't work, we might still get some useful new info from the c5 log entry (to check that it's trying to execute what we want), and the various mysqldump-*.txt files.
Your service provider's command line for mysqldump looks odd to me. -u should be followed by the username, but is being followed by the host/server name (which requires -h). In addition, -p seems to be followed by your password, so mysqldump shouldn't have to prompt for it. (PS: specifying the password on the command line isn't very secure, which is why I use a config file instead.) You probably worked all this out for yourself in your subsequent tests.
If you managed to create empty database files in your tests, you may have made useful progress. You could have managed to get mysqldump to run, but it's failing. My guess would be server not found. You can capture its error messages in a couple of ways; the easiest is to append this to your command line:
2>errors.txt
then look in errors.txt.
Some hosts automatically kill processes that cause CPU/memory spikes, which could prevent mysqldump from completing. I doubt that this is the problem.
172.17.0.1
==========
This could well be the same problem as mysql5.sigmatic.fi, so please try the attached job.
I note that 172.17.0.1 is a private IP address, but I don't think that should matter.
mysql3.shellit.org
==================
As you say, this would need the host to relax this restriction, or use straight php-based solution such as the one you mentioned. Such a solution does exist for c5: seehttps://datatainment.de/this-n-that/concrete5-backup-and-restore-dat... I'm loathe to implement something like this myself for various reasons (robustness, commplexity, and it's already been done).
This should clear what is what.
mysql5.sigmatic.fi
==================
This is actually the host I was in contact with that had "restricted" access to mysqldump. We use this host for most of our clients sites. Also no worry, no password visible above there. -p works in weird ways and when using password it should be like this (no space after -p) -pTHISISPASSWORD THISISDATABASE
172.17.0.1
==========
This is the one where I tested that new gond_backup_database.php, oddly seems like nothing changed and I get very little logging of any kind.
mysqldump-stdout.txt = empty
mysqldump-stderr.txt
Concrete5 log
gond-backup.cnf
Hopefully this was all the files and logs
mysql3.shellit.org
==================
This one was the hopeless case, did not contact the host yet, probably wont as we only have one client there.
mysql5.sigmatic.fi
==================
This is actually the host I was in contact with that had "restricted" access to mysqldump. We use this host for most of our clients sites. Also no worry, no password visible above there. -p works in weird ways and when using password it should be like this (no space after -p) -pTHISISPASSWORD THISISDATABASE
172.17.0.1
==========
This is the one where I tested that new gond_backup_database.php, oddly seems like nothing changed and I get very little logging of any kind.
mysqldump-stdout.txt = empty
mysqldump-stderr.txt
sh: mysqldump: command not found
Concrete5 log
Simple Backup (Backup Database): failed to execute mysqldump --defaults-extra-file=application/files/backup/gond-backup.cnf --verbose --log-error="mysqldump-log-error.txt" -r "application/files/backup/wordpress database.sql" ippuser1 > mysqldump-stdout.txt 2> mysqldump-stderr.txt (return code: 127)
gond-backup.cnf
[mysqldump] host="172.17.0.1" user="ippuser1" password="********" default-character-set=utf8 complete-insert events routines
Hopefully this was all the files and logs
mysql3.shellit.org
==================
This one was the hopeless case, did not contact the host yet, probably wont as we only have one client there.
Thanks very much for persevering!
I think the best chance of success may be with sigmatic.fi, since the creation of zero-length database files may indicate that mysqldump is actually being executed (unless the zero-length files are being created by the operating system when it sees that you're redirecting stdout). Plus, the host says that it should be possible (albeit maybe not from within php).
I don't suppose you have command line access to a sigmatic.fi account (eg, ssh/putty)? If so, you could experiment with command line calls to mysqldump without needing php.
I'd be interested in the content of mysqldump-stderr.txt when running the new job on sigmatic.fi. That should indicate whether mysqldump is being executed or not, and if it is, what it's problem is. The c5 log entry is useful to verify that the job is trying to execute the command that we expect.
If we can crack it with sigmatic.fi, the same fix might work with 172.17.0.1 since the error message is essentially the same. It depends whether this host has more restricted access.
I think the best chance of success may be with sigmatic.fi, since the creation of zero-length database files may indicate that mysqldump is actually being executed (unless the zero-length files are being created by the operating system when it sees that you're redirecting stdout). Plus, the host says that it should be possible (albeit maybe not from within php).
I don't suppose you have command line access to a sigmatic.fi account (eg, ssh/putty)? If so, you could experiment with command line calls to mysqldump without needing php.
I'd be interested in the content of mysqldump-stderr.txt when running the new job on sigmatic.fi. That should indicate whether mysqldump is being executed or not, and if it is, what it's problem is. The c5 log entry is useful to verify that the job is trying to execute the command that we expect.
If we can crack it with sigmatic.fi, the same fix might work with 172.17.0.1 since the error message is essentially the same. It depends whether this host has more restricted access.
Hello, tried to run it on Sigmatic server, same problem, no other errors got done except for mysqldump-stderr.txt
And this most certainly is because of the odd way how Sigmatic servers are set up that when you run mysqldump from command line call it works, but not when run from the same command from php file. This simply is because the command line is a different user and has access to mysqldump but the other one cant access it. It is silly and I'm pretty sure there is no way to go around it.
Tho oddly when I manually run mysqldump from manually made php file I get an empty dump file.
sh: 1: mysqldump: not found
And this most certainly is because of the odd way how Sigmatic servers are set up that when you run mysqldump from command line call it works, but not when run from the same command from php file. This simply is because the command line is a different user and has access to mysqldump but the other one cant access it. It is silly and I'm pretty sure there is no way to go around it.
Tho oddly when I manually run mysqldump from manually made php file I get an empty dump file.
Unfortunately, you're probably right. I spent ages setting up a home network to test this. If the mysql utilities are not mounted in the php user's file system, 'not found' will result. However, if the utilities are mounted but not executable, 'permission denied' will result.
We can't really blame your hosts for this level of security. Presumably they must have made special provisions for things like phpmyadmin, which can execute mysqldump from php code.
I'm now wondering whether all non-localhost configurations will fail like this. I'll ask around to see if there are any working examples.
Perhaps you're getting zero-length database dumps from your php experiments because the output file is being created before mysqldump runs. Try doing it without redirecting the output (ie, don't end with > dump.sql). You're probably not using the -r option, but it may do the same thing (so try without it).
My apologies that this job doesn't work in your environments, and many thanks for working through it! :)
I'll update the job to provide better diagnostic info for such problems, and update the documentation likewise. But that won't help you much. :)
We can't really blame your hosts for this level of security. Presumably they must have made special provisions for things like phpmyadmin, which can execute mysqldump from php code.
I'm now wondering whether all non-localhost configurations will fail like this. I'll ask around to see if there are any working examples.
Perhaps you're getting zero-length database dumps from your php experiments because the output file is being created before mysqldump runs. Try doing it without redirecting the output (ie, don't end with > dump.sql). You're probably not using the -r option, but it may do the same thing (so try without it).
My apologies that this job doesn't work in your environments, and many thanks for working through it! :)
I'll update the job to provide better diagnostic info for such problems, and update the documentation likewise. But that won't help you much. :)
I suspect the issue may be caused by symbols (non-alphanumeric characters) in the database password. Do you know if your password contains symbols?
On the assumption that this is the problem, I'm currently testing a version of the add-on that handles such characters. It should be available in a few hours. I'll post a message here when it's available to try.