Import question
Permalink Browser Info Environment
Hi,
I ran an import with 360 records but without thinking I put it on immediate instead of cron job. Now its stopped after 10 minutes and importing 191 records (they are smallish entries). Is it possible to re-start the import? I believe that if I choose the same file and import it again that it it will create duplicates of the 190 done so far.
Best
Phil
I ran an import with 360 records but without thinking I put it on immediate instead of cron job. Now its stopped after 10 minutes and importing 191 records (they are smallish entries). Is it possible to re-start the import? I believe that if I choose the same file and import it again that it it will create duplicates of the 190 done so far.
Best
Phil
| Type: | Pre-Sale |
|---|---|
| Status: | Resolved |
I thought that would be the case, but there is no such entry in the jobs list? I just have the below. I notice no number 10 or 11 entry
12 Fill thumbnail database table Never
13 Deactivate Users Never
3 Check Automated Groups Apr 18, 2018, 6:50:08 PM Active users updated.
4 Generate the sitemap.xml file Apr 18, 2018, 6:50:10 PM /sitemap.xml file saved (166 pages).
5 Process Email Posts Apr 18, 2018, 6:50:12 PM The Job was run successfully.
6 Remove Old Page Versions Apr 18, 2018, 6:50:15 PM 3 versions deleted from 113 pages (3)
7 Update Gatherings Apr 18, 2018, 6:50:25 PM The Job was run successfully.
8 Update Statistics Trackers Apr 18, 2018, 6:50:32 PM The Job was run successfully.
9 HonestWebsites Store Geocode Aug 3, 2018, 5:25:15 PM The Job was run successfully.
2 Index Search Engine - All Aug 22, 2018, 10:08:01 PM Index performed on: 678 pages, 2 users, 342 files, and 1 site
1 Index Search Engine - Updates Aug 22, 2018, 10:09:30 PM Index performed on: 0 pages, 2 users, 342 files, and 1 site
12 Fill thumbnail database table Never
13 Deactivate Users Never
3 Check Automated Groups Apr 18, 2018, 6:50:08 PM Active users updated.
4 Generate the sitemap.xml file Apr 18, 2018, 6:50:10 PM /sitemap.xml file saved (166 pages).
5 Process Email Posts Apr 18, 2018, 6:50:12 PM The Job was run successfully.
6 Remove Old Page Versions Apr 18, 2018, 6:50:15 PM 3 versions deleted from 113 pages (3)
7 Update Gatherings Apr 18, 2018, 6:50:25 PM The Job was run successfully.
8 Update Statistics Trackers Apr 18, 2018, 6:50:32 PM The Job was run successfully.
9 HonestWebsites Store Geocode Aug 3, 2018, 5:25:15 PM The Job was run successfully.
2 Index Search Engine - All Aug 22, 2018, 10:08:01 PM Index performed on: 678 pages, 2 users, 342 files, and 1 site
1 Index Search Engine - Updates Aug 22, 2018, 10:09:30 PM Index performed on: 0 pages, 2 users, 342 files, and 1 site
Hi PhilYoung,
Thanks for sharing your jobs list. We are not sure why the "Staff Directory Import" job is missing from your list.
Any chance it got deleted?
Restoring the job would allow you to run it and continue importing. However the recommended way to do that is via sql.
Do you have access to your database and feel comfortable running a mysql query to add the "Staff Directory Import" job back?
Let us know and we can provide you a script that you can run.
We will likely need the installed Staff Directory pkgID from the Jobs table to provide the script.
Thanks,
Informatics Inc
Thanks for sharing your jobs list. We are not sure why the "Staff Directory Import" job is missing from your list.
Any chance it got deleted?
Restoring the job would allow you to run it and continue importing. However the recommended way to do that is via sql.
Do you have access to your database and feel comfortable running a mysql query to add the "Staff Directory Import" job back?
Let us know and we can provide you a script that you can run.
We will likely need the installed Staff Directory pkgID from the Jobs table to provide the script.
Thanks,
Informatics Inc
Hi,
I don't think it got deleted - the import froze and i suspect it never got written as after 10 minutes of the site being frozen I had to cancel import.
The packages table in the db shows the pkgID as 18
Thanks for your support with this. I can run the script you provide.
Best
Phil
I don't think it got deleted - the import froze and i suspect it never got written as after 10 minutes of the site being frozen I had to cancel import.
The packages table in the db shows the pkgID as 18
Thanks for your support with this. I can run the script you provide.
Best
Phil
Hi Phil,
Please find attached the sql query you need to run to add the job.
We would suggest that you make a database backup of your site before running the query/ Import job just as a precautionary measure.
Thanks,
Informatics Inc
Please find attached the sql query you need to run to add the job.
We would suggest that you make a database backup of your site before running the query/ Import job just as a precautionary measure.
Thanks,
Informatics Inc
Hi,
I ran the sql command and i now have staff directory in the jobs list but when I run it i get the message below
Staff Directory Import Aug 28, 2018, 9:30:15 PM array_combine(): Both parameters should have an equal number of elements
Best
Phil
I ran the sql command and i now have staff directory in the jobs list but when I run it i get the message below
Staff Directory Import Aug 28, 2018, 9:30:15 PM array_combine(): Both parameters should have an equal number of elements
Best
Phil
Hi Phil,
Please send us the csv file via a Private Message. We will take a look.
Thanks,
Informatics Inc
Please send us the csv file via a Private Message. We will take a look.
Thanks,
Informatics Inc
Hi,
Being based in Europe and subject to stringent GDPR regulations. I cant send you the csv. I assume the error message indicates an issue with one of the fields in the csv which, of course, may be why the import stalled after 191 lines and the job never got written. There does not seem to be an order to the import from the csv (unless you know the sequence?) which makes creating an abbreviated file a bit problematic but I can quickly eliminate about 120 that were imported from the 191. That will mean I get some duplicates from a new import but I can weed them out manually fairly quickly.
I have studied the csv and it looks fine. Would the import stall with an entry that had a foreign text character that rendered like this "Grażyna" as a first name? Actually there a few of those in some addresses too.
Thanks as always for your great support
Best
Phil
Being based in Europe and subject to stringent GDPR regulations. I cant send you the csv. I assume the error message indicates an issue with one of the fields in the csv which, of course, may be why the import stalled after 191 lines and the job never got written. There does not seem to be an order to the import from the csv (unless you know the sequence?) which makes creating an abbreviated file a bit problematic but I can quickly eliminate about 120 that were imported from the 191. That will mean I get some duplicates from a new import but I can weed them out manually fairly quickly.
I have studied the csv and it looks fine. Would the import stall with an entry that had a foreign text character that rendered like this "Grażyna" as a first name? Actually there a few of those in some addresses too.
Thanks as always for your great support
Best
Phil
Here is an oddity on my installation.
i can import a file with 10 entries on immediate and it runs fine. If I try and do it using the same file but using a cron job it gives the same error message
array_combine(): Both parameters should have an equal number of elements
and the log says
Exception Occurred: /home/polarity/public_html/polarityeducation.net/packages/staff_directory/single_pages/dashboard/directory_extended/directory_import/view.php:265 Invalid argument supplied for foreach() (2)
So its not an issue with the csv...
i can import a file with 10 entries on immediate and it runs fine. If I try and do it using the same file but using a cron job it gives the same error message
array_combine(): Both parameters should have an equal number of elements
and the log says
Exception Occurred: /home/polarity/public_html/polarityeducation.net/packages/staff_directory/single_pages/dashboard/directory_extended/directory_import/view.php:265 Invalid argument supplied for foreach() (2)
So its not an issue with the csv...
Hi Phil,
We reviewed the foreign text character, and it does not break the import.
Since we can't review the csv file, our best targeted guess, is that there may be at least one record that has a comma(,) in the field. Since this is a comma separated file, that would break the import.
Can you verify if your csv file contains any commas?
Regarding the chron job, the code is set up to return to the file in queue when you start the import again. Meaning, it will reference to the file you initially uploaded, and re-running it would return the same error. Thus, even if you've fixed something in the csv you're attempting to import, it'll still error on the first file.
To help determine & provide options, can you let us know what are the updates/steps that you have taken so far?
1) Did you delete any imported pages from the site map?
2) Did you run another import using a new import file?
Thanks,
Informatics Inc
We reviewed the foreign text character, and it does not break the import.
Since we can't review the csv file, our best targeted guess, is that there may be at least one record that has a comma(,) in the field. Since this is a comma separated file, that would break the import.
Can you verify if your csv file contains any commas?
Regarding the chron job, the code is set up to return to the file in queue when you start the import again. Meaning, it will reference to the file you initially uploaded, and re-running it would return the same error. Thus, even if you've fixed something in the csv you're attempting to import, it'll still error on the first file.
To help determine & provide options, can you let us know what are the updates/steps that you have taken so far?
1) Did you delete any imported pages from the site map?
2) Did you run another import using a new import file?
Thanks,
Informatics Inc
Hi,
I have no idea why the original large import broke but I uploaded a new file with 250 records (just a truncated version of the original) late last night and it went well the whole import took about 25 minutes and then it took me about 20 minutes to delete the duplicated imports.
So the full import is now done.
I think your package is more fault tolerant than you indicate. Today, I cleared the sql table of all the imports and I can upload a file with multiple commas in the address field and it does not break the import either as immediate or as a cron job.
Thanks for your help. In spite of my issue and delay the import is complete and has saved me hours and hours of manual copy and paste of some 380 records.
Perhaps in a future update you could make it possible to delete more than one record at a time as I can imagine some scenarios where this would be very helpful.
Staff directory is still one of the best c5 packages!
Best wishes
Phil
I have no idea why the original large import broke but I uploaded a new file with 250 records (just a truncated version of the original) late last night and it went well the whole import took about 25 minutes and then it took me about 20 minutes to delete the duplicated imports.
So the full import is now done.
I think your package is more fault tolerant than you indicate. Today, I cleared the sql table of all the imports and I can upload a file with multiple commas in the address field and it does not break the import either as immediate or as a cron job.
Thanks for your help. In spite of my issue and delay the import is complete and has saved me hours and hours of manual copy and paste of some 380 records.
Perhaps in a future update you could make it possible to delete more than one record at a time as I can imagine some scenarios where this would be very helpful.
Staff directory is still one of the best c5 packages!
Best wishes
Phil
Thanks for the kind words Phil! Always a pleasure.
Let us know if you need anything else!
- Informatics Inc.
Let us know if you need anything else!
- Informatics Inc.
To continue from the last record that was imported, you can go to System & Settings > Automated Jobs and Run the "Staff Directory Import".
This should import the remaining items.
We would suggest that you make a database backup of your site before running the Import job just as a precautionary measure.
Thanks,
Informatics Inc