Index Search Engine (All or Updates) Fails

Permalink 1 0 Browser Info Environment
When trying to run the Index Search Engine - All (or Update), the process gets to about 150 pages of over 1200 pages and never goes any further. The CPU on the server runs up to 100% and the site becomes unresponsive.
This is setup on an AWS system and even when it automatically scales up, the site is still unresponsive.

I have tried to allow the job to run for a few days, still with it never getting further than 150 pages indexed.

There was a reported bug specifically about this issue that was supposedly fixed in v 8.4.3 which we have recently updated to from 8.3.2.

Is there something in the DB from the fix in 8.4.3 that needs to be manually cleared out?


Status: New
HDMZDevelopment replied on at Permalink Reply
There are also no errors being displayed in the server logs, nor in the C5 Reports dashboard. The job just will not finish
carl101lee replied on at Permalink Reply
carl101lee
I have the Same issue, After Moving a site to the Live (new server) which is running php 7.1 .
I had to stop the Porcess runnign in the jobs table in the SQL. But it also Maxs out the CPU and I have set the mem limit 1048M and exicution time to 900s also the sevrer IS quad Core with SSD and 6 GIG ram, and is only running this site and 3 development sites (not live yet)

Yet this also still causes the PHP to fall over, I belive this is defently an issue with PHP7 and Concrete 5 and I see that No fix has yet to be Relased.

I hope this thread gets Bumped Or I will ahev to stat my own.

Carl

concrete5 Environment Information

# concrete5 Version
Core Version - 8.4.3
Version Installed - 8.4.3
Database Version - 20180716000000

# concrete5 Packages
ExchangeCore reCAPTCHA (1.1.1), HDMZ Build (1.5.8), HDMZ Express Custom (0.9), HDMZ Site Search (0.0.1), Page Redirect (2.0), Page Selector Attribute (2.0)

# concrete5 Overrides
blocks/accordion/view.css, blocks/accordion/add.php, blocks/accordion/edit.php, blocks/accordion/view.php, blocks/accordion/controller.php, blocks/accordion/icon.png, blocks/accordion/form.php, blocks/accordion/Layer_24.png, blocks/accordion/view.js, blocks/accordion/templates/leadership/view.css, blocks/accordion/templates/leadership/view.php, blocks/accordion/templates/leadership/view.js, blocks/accordion/templates/leadership, blocks/accordion/templates/dropdown-list/view.css, blocks/accordion/templates/dropdown-list/view.php, blocks/accordion/templates/dropdown-list/view.js, blocks/accordion/templates/dropdown-list, blocks/accordion/templates, blocks/accordion/db.xml, blocks/accordion, blocks/page_list/templates/2_col_color_partial_overlay.php, blocks/page_list/templates/4_col_partial_overlay.php, blocks/page_list/templates/2_col_full_overlay.php, blocks/page_list/templates/3_col_color_partial_overlay.php, blocks/page_list/templates/2_col_partial_overlay.php, blocks/page_list/templates/4_col_full_overlay.php, blocks/page_list/templates/3_col_full_overlay.php, blocks/page_list/templates/3_col_partial_overlay.php, blocks/page_list/templates/4_col_color_partial_overlay.php, blocks/page_list/templates, blocks/page_list, blocks/image/controller.php, blocks/image/form.php, blocks/image/templates/descripton_on_overlay.php, blocks/image/templates/image_with_description/view.php, blocks/image/templates/image_with_description, blocks/image/templates/image_with_blocks.php, blocks/image/templates, blocks/image/db.xml, blocks/image, blocks/switch_language/view.css, blocks/switch_language/view.php, blocks/switch_language/view.js, blocks/switch_language, blocks/hdmz_accordion_filter/add.php, blocks/hdmz_accordion_filter/edit.php, blocks/hdmz_accordion_filter/view.php, blocks/hdmz_accordion_filter/controller.php, blocks/hdmz_accordion_filter/icon.png, blocks/hdmz_accordion_filter/db.xml, blocks/hdmz_accordion_filter/form_setup_html.php, blocks/hdmz_accordion_filter, blocks/hdmz_tab_content/add.php, blocks/hdmz_tab_content/edit.php, blocks/hdmz_tab_content/view.php, blocks/hdmz_tab_content/controller.php, blocks/hdmz_tab_content/icon.png, blocks/hdmz_tab_content/templates/tabs_view/view.php, blocks/hdmz_tab_content/templates/tabs_view, blocks/hdmz_tab_content/templates, blocks/hdmz_tab_content/db.xml, blocks/hdmz_tab_content/form_setup_html.php, blocks/hdmz_tab_content, blocks/hdmz_google_search/add.php, blocks/hdmz_google_search/edit.php, blocks/hdmz_google_search/view.php, blocks/hdmz_google_search/controller.php, blocks/hdmz_google_search/icon.png, blocks/hdmz_google_search/db.xml, blocks/hdmz_google_search/form_setup_html.php, blocks/hdmz_google_search, blocks/hdmz_fess_site_search/view.css, blocks/hdmz_fess_site_search/view.php, blocks/hdmz_fess_site_search, blocks/rss_displayer/view.css, blocks/rss_displayer/view.php, blocks/rss_displayer/view.js, blocks/rss_displayer/templates/stock_quote/view.php, blocks/rss_displayer/templates/stock_quote, blocks/rss_displayer/templates, blocks/rss_displayer, blocks/index.html, blocks/hdmz_pipeline/view.css, blocks/hdmz_pipeline/add.php, blocks/hdmz_pipeline/edit.php, blocks/hdmz_pipeline/view.php, blocks/hdmz_pipeline/scrapbook.php, blocks/hdmz_pipeline/controller.php, blocks/hdmz_pipeline/icon.png, blocks/hdmz_pipeline/db.xml, blocks/hdmz_pipeline/form_setup_html.php, blocks/hdmz_pipeline, blocks/image_slider/templates/random_image/view.css, blocks/image_slider/templates/random_image/view.php, blocks/image_slider/templates/random_image, blocks/image_slider/templates, blocks/image_slider, blocks/express_entry_list/controller.php, blocks/express_entry_list/templates/krossflo_parts/view.css, blocks/express_entry_list/templates/krossflo_parts/view.php, blocks/express_entry_list/templates/krossflo_parts, blocks/express_entry_list/templates, blocks/express_entry_list, blocks/core_area_layout/templates/accordion/view.php, blocks/core_area_layout/templates/accordion, blocks/core_area_layout/templates/horizontal-image-scroll/view.php, blocks/core_area_layout/templates/horizontal-image-scroll, blocks/core_area_layout/templates/image-layout/view.php, blocks/core_area_layout/templates/image-layout, blocks/core_area_layout/templates, blocks/core_area_layout, blocks/hdmz_vimeo/view.css, blocks/hdmz_vimeo/add.php, blocks/hdmz_vimeo/edit.php, blocks/hdmz_vimeo/view.php, blocks/hdmz_vimeo/controller.php, blocks/hdmz_vimeo/icon.png, blocks/hdmz_vimeo/view.js, blocks/hdmz_vimeo/db.xml, blocks/hdmz_vimeo/form_setup_html.php, blocks/hdmz_vimeo, blocks/autonav/templates/repligen_mega_menu/view.css, blocks/autonav/templates/repligen_mega_menu/view.php, blocks/autonav/templates/repligen_mega_menu/view.js, blocks/autonav/templates/repligen_mega_menu, blocks/autonav/templates, blocks/autonav, controllers/single_page/video_library.php, controllers/single_page/register.php, controllers/single_page/repligen_store.php, controllers/single_page/dashboard/tangenxrfq/tangenxrfq_requests.php, controllers/single_page/dashboard/tangenxrfq/import.php, controllers/single_page/dashboard/tangenxrfq, controllers/single_page/dashboard/configurator.php, controllers/single_page/dashboard/configurator/configurator_invoices.php, controllers/single_page/dashboard/configurator/import.php, controllers/single_page/dashboard/configurator, controllers/single_page/dashboard/tangenxrfq.php, controllers/single_page/dashboard, controllers/single_page/configurator.php, controllers/single_page/download_file.php, controllers/single_page/login.php, controllers/single_page/tangenxrfq.php, controllers/single_page/krosflo.php, controllers/single_page/resins.php, controllers/single_page, controllers/index.html, elements/tangenxrfq/list.php, elements/tangenxrfq, elements/dashboard/tangenxrfq/form.php, elements/dashboard/tangenxrfq/list.php, elements/dashboard/tangenxrfq, elements/dashboard/configurator/form.php, elements/dashboard/configurator/list.php, elements/dashboard/configurator, elements/dashboard, elements/configurator.php, elements/configurator/list.php, elements/configurator/order.php, elements/configurator, elements/tangenxrfq.php, elements/index.html, elements/header_required.php, attributes/select/composer.php, attributes/select, attributes/textarea/composer.php, attributes/textarea, attributes/text/composer.php, attributes/text, authentication/concrete/form.php, authentication/concrete, authentication/index.html, jobs/index.html, css/tangenxrfq/form.css, css/tangenxrfq, css/configurator/form.css, css/configurator, css/app.css, js/scripts.js, js/mobile-menu.js, js/date.js, js/configurator/import_admin.js, js/configurator, js/main.js, js/jquery.tablesorter.min.js, js/scrollreveal.min.js, js/bootstrap/bootstrap.tooltip.js, js/bootstrap, js/bootstrap.min.js, js/modernizr.min.js, mail/configurator_user_notification.php, mail/tangenxrfq_user_notification.php, mail/tangenxrfq_notification.php, mail/configurator_notification.php, mail/salesforce_error_notice.php, mail/shopify_error_notice.php, mail/index.html, mail/blacklist_registrant_user.php, mail/blacklist_registrant.php, single_pages/tangenxrfq/view.php, single_pages/tangenxrfq, single_pages/register.php, single_pages/repligen_store.php, single_pages/dashboard/tangenxrfq/tangenxrfq_requests.php, single_pages/dashboard/tangenxrfq/import.php, single_pages/dashboard/tangenxrfq, single_pages/dashboard/configurator.php, single_pages/dashboard/configurator/configurator_invoices.php, single_pages/dashboard/configurator/import.php, single_pages/dashboard/configurator, single_pages/dashboard/tangenxrfq.php, single_pages/dashboard, single_pages/configurator/view.php, single_pages/configurator, single_pages/download_file.php, single_pages/krosflo/view.php, single_pages/krosflo, single_pages/login.php, single_pages/account/edit_profile.php, single_pages/account, single_pages/video_library/view.php, single_pages/video_library, single_pages/contact/view-old.php, single_pages/contact/view.php, single_pages/contact, single_pages/index.html, single_pages/resins/view.php, single_pages/resins, themes/repligen2018/elements/header.php, themes/repligen2018/elements/header_top.php, themes/repligen2018/elements/footer_bottom.php, themes/repligen2018/elements/footer.php, themes/repligen2018/elements, themes/repligen2018/thumbnail.png, themes/repligen2018/accordions.php, themes/repligen2018/page_forbidden.php, themes/repligen2018/left_sidebar.php, themes/repligen2018/view.php, themes/repligen2018/js/scripts.js, themes/repligen2018/js/bootstrap-select.js, themes/repligen2018/js/dura-main.js, themes/repligen2018/js/carousel.js, themes/repligen2018/js, themes/repligen2018/page_not_found.php, themes/repligen2018/page_theme.php, themes/repligen2018/default.php, themes/repligen2018/images/pin.png, themes/repligen2018/images/background-slider-royal.png, themes/repligen2018/images/icons/delete_small.png, themes/repligen2018/images/icons/remove_icon.png, themes/repligen2018/images/icons/user.png, themes/repligen2018/images/icons, themes/repligen2018/images/hq-pin.png, themes/repligen2018/images/background.png, themes/repligen2018/images/background-slider-night-road.png, themes/repligen2018/images/default-banner.jpg, themes/repligen2018/images/fpo.png, themes/repligen2018/images/background-slider-blue-sky.png, themes/repligen2018/images/background-slider-default.png, themes/repligen2018/images/map.png, themes/repligen2018/images/map-mobile.png, themes/repligen2018/images, themes/repligen2018/home.php, themes/repligen2018/right_sidebar.php, themes/repligen2018/blank.php, themes/repligen2018/css/presets/defaults.less, themes/repligen2018/css/presets/hdmz.less, themes/repligen2018/css/presets/royal.less, themes/repligen2018/css/presets/night-road.less, themes/repligen2018/css/presets/blue-sky.less, themes/repligen2018/css/presets, themes/repligen2018/css/bootstrap-modified.css, themes/repligen2018/css/carousel.css, themes/repligen2018/css/color-gray.css, themes/repligen2018/css/bootstrap-select.css, themes/repligen2018/css/main.less, themes/repligen2018/css/dura-responsive.css, themes/repligen2018/css/styles.xml, themes/repligen2018/css/blocks/page-list.less, themes/repligen2018/css/blocks/page-title.less, themes/repligen2018/css/blocks/image.less, themes/repligen2018/css/blocks/accordions.less, themes/repligen2018/css/blocks, themes/repligen2018/css/dura-main.css, themes/repligen2018/css/import.less, themes/repligen2018/css/menu.less, themes/repligen2018/css/build/breadcrumb.less, themes/repligen2018/css/build/core/include/mixins.less, themes/repligen2018/css/build/core/include, themes/repligen2018/css/build/core, themes/repligen2018/css/build/errors.less, themes/repligen2018/css/build/pagination.less, themes/repligen2018/css/build/header.less, themes/repligen2018/css/build/body.less, themes/repligen2018/css/build/captcha.less, themes/repligen2018/css/build/add-ons/calendar.less, themes/repligen2018/css/build/add-ons, themes/repligen2018/css/build/search.less, themes/repligen2018/css/build/mobile/navigation.less, themes/repligen2018/css/build/mobile, themes/repligen2018/css/build/mixins.less, themes/repligen2018/css/build/footer.less, themes/repligen2018/css/build/blocks/testimonial.less, themes/repligen2018/css/build/blocks/page-list.less, themes/repligen2018/css/build/blocks/page-list-option-buttons.less, themes/repligen2018/css/build/blocks/image-slider.less, themes/repligen2018/css/build/blocks/faq.less, themes/repligen2018/css/build/blocks/page-title.less, themes/repligen2018/css/build/blocks/page-attribute-display.less, themes/repligen2018/css/build/blocks/topic-list.less, themes/repligen2018/css/build/blocks/feature.less, themes/repligen2018/css/build/blocks/next-previous.less, themes/repligen2018/css/build/blocks/feature-template-hover-description.less, themes/repligen2018/css/build/blocks/image.less, themes/repligen2018/css/build/blocks/page-title-option-byline.less, themes/repligen2018/css/build/blocks/tags.less, themes/repligen2018/css/build/blocks/date-navigation.less, themes/repligen2018/css/build/blocks/page-list-template-thumbnail-grid.less, themes/repligen2018/css/build/blocks, themes/repligen2018/css/build/reset.less, themes/repligen2018/css/build/fonts/defaults.less, themes/repligen2018/css/build/fonts/hdmz.less, themes/repligen2018/css/build/fonts/royal.less, themes/repligen2018/css/build/fonts/night-road.less, themes/repligen2018/css/build/fonts/blue-sky.less, themes/repligen2018/css/build/fonts, themes/repligen2018/css/build/miscellaneous-styles.less, themes/repligen2018/css/build/edit-mode.less, themes/repligen2018/css/build/bootstrap-3.2.0/popovers.less, themes/repligen2018/css/build/bootstrap-3.2.0/badges.less, themes/repligen2018/css/build/bootstrap-3.2.0/input-groups.less, themes/repligen2018/css/build/bootstrap-3.2.0/jumbotron.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/popovers.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/opacity.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/grid-framework.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/resize.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/pagination.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/labels.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/modals.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/tab-focus.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/responsive-visibility.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/hide-text.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/clearfix.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/forms.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/gradients.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/text-emphasis.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/panels.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/list-group.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/reset-filter.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/text-overflow.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/image.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/progress-bar.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/grid.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/alerts.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/vendor-prefixes.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/border-radius.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/center-block.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/dropdowns.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/tooltip.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/table-row.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/nav-divider.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/nav-vertical-align.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/size.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/background-variant.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins/buttons.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins, themes/repligen2018/css/build/bootstrap-3.2.0/pagination.less, themes/repligen2018/css/build/bootstrap-3.2.0/breadcrumbs.less, themes/repligen2018/css/build/bootstrap-3.2.0/labels.less, themes/repligen2018/css/build/bootstrap-3.2.0/pager.less, themes/repligen2018/css/build/bootstrap-3.2.0/normalize.less, themes/repligen2018/css/build/bootstrap-3.2.0/modals.less, themes/repligen2018/css/build/bootstrap-3.2.0/type.less, themes/repligen2018/css/build/bootstrap-3.2.0/navbar.less, themes/repligen2018/css/build/bootstrap-3.2.0/carousel.less, themes/repligen2018/css/build/bootstrap-3.2.0/wells.less, themes/repligen2018/css/build/bootstrap-3.2.0/tables.less, themes/repligen2018/css/build/bootstrap-3.2.0/utilities.less, themes/repligen2018/css/build/bootstrap-3.2.0/progress-bars.less, themes/repligen2018/css/build/bootstrap-3.2.0/forms.less, themes/repligen2018/css/build/bootstrap-3.2.0/glyphicons.less, themes/repligen2018/css/build/bootstrap-3.2.0/bootstrap.less, themes/repligen2018/css/build/bootstrap-3.2.0/panels.less, themes/repligen2018/css/build/bootstrap-3.2.0/list-group.less, themes/repligen2018/css/build/bootstrap-3.2.0/mixins.less, themes/repligen2018/css/build/bootstrap-3.2.0/close.less, themes/repligen2018/css/build/bootstrap-3.2.0/component-animations.less, themes/repligen2018/css/build/bootstrap-3.2.0/responsive-utilities.less, themes/repligen2018/css/build/bootstrap-3.2.0/theme.less, themes/repligen2018/css/build/bootstrap-3.2.0/print.less, themes/repligen2018/css/build/bootstrap-3.2.0/scaffolding.less, themes/repligen2018/css/build/bootstrap-3.2.0/grid.less, themes/repligen2018/css/build/bootstrap-3.2.0/alerts.less, themes/repligen2018/css/build/bootstrap-3.2.0/thumbnails.less, themes/repligen2018/css/build/bootstrap-3.2.0/media.less, themes/repligen2018/css/build/bootstrap-3.2.0/dropdowns.less, themes/repligen2018/css/build/bootstrap-3.2.0/variables.less, themes/repligen2018/css/build/bootstrap-3.2.0/tooltip.less, themes/repligen2018/css/build/bootstrap-3.2.0/code.less, themes/repligen2018/css/build/bootstrap-3.2.0/responsive-embed.less, themes/repligen2018/css/build/bootstrap-3.2.0/button-groups.less, themes/repligen2018/css/build/bootstrap-3.2.0/navs.less, themes/repligen2018/css/build/bootstrap-3.2.0/buttons.less, themes/repligen2018/css/build/bootstrap-3.2.0, themes/repligen2018/css/build/content.less, themes/repligen2018/css/build/buttons.less, themes/repligen2018/css/build/custom-design-classes/blog-entry-list.less, themes/repligen2018/css/build/custom-design-classes/testimonial-bio.less, themes/repligen2018/css/build/custom-design-classes/recent-blog-entry.less, themes/repligen2018/css/build/custom-design-classes/block-sidebar-padded.less, themes/repligen2018/css/build/custom-design-classes/area-content-accent.less, themes/repligen2018/css/build/custom-design-classes/block-sidebar-wrapped.less, themes/repligen2018/css/build/custom-design-classes, themes/repligen2018/css/build, themes/repligen2018/css/typography.less, themes/repligen2018/css, themes/repligen2018/full.php, themes/repligen2018, themes/index.html, tools/track_protected_video.php, tools/index.html, tools/async_news_feed.php, page_templates/index.html, views/index.html, src/Mail/Service.php, src/Mail/MailServiceProvider.php, src/Mail, src/TangenxrfqHtmlRfqProvider.php, src/SalesforceProvider.php, src/Area/GlobalArea.php, src/Area, src/ConfiguratorHtmlInvoiceProvider.php, src/UserCurrencyProvider.php, src/ShopifyMultipassProvider.php

# concrete5 Cache Settings
Block Cache - Off
Overrides Cache - Off
Full Page Caching - Off
Full Page Cache Lifetime - Every 6 hours (default setting).

# Server Software
Apache/2.4.6 (CentOS)

# Server API
apache2handler

# PHP Version
7.1.22

# PHP Extensions
apache2handler, bz2, calendar, Core, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, gmp, hash, iconv, igbinary, json, ldap, libxml, mbstring, mcrypt, mysqli, openssl, pcre, PDO, pdo_mysql, pdo_sqlite, Phar, posix, redis, Reflection, session, shmop, SimpleXML, snmp, soap, sockets, SPL, sqlite3, standard, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zip, zlib

# PHP Settings
max_execution_time - 240
log_errors_max_len - 1024
max_file_uploads - 20
max_input_nesting_level - 64
max_input_time - 220
max_input_vars - 1000
memory_limit - 1024M
post_max_size - 256M
sql.safe_mode - Off
upload_max_filesize - 512M
ldap.max_links - Unlimited
mysqli.max_links - Unlimited
mysqli.max_persistent - Unlimited
pcre.backtrack_limit - 1000000
pcre.recursion_limit - 100000
session.cache_limiter - <i>no value</i>
session.gc_maxlifetime - 7200
soap.wsdl_cache_limit - 5
opcache.max_accelerated_files - 30000
opcache.max_file_size - 0
opcache.max_wasted_percentage - 5

Browser User-Agent String

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36