[UX improvement] Express Form, inform user input is invalid BEFORE submitting, on all types

Permalink 17 16 Browser Info Environment
Currently on many Attribute Types in Express Forms, when you submit invalid data, it spits out SQL vomit errors.

This causes two issues:

1. It creates incomplete entries in the Express Entity.
2. It exposes SQL table info which could lead to security issues.
3. The UX is very poor.

Currently when you input an invalid Email address, the form doesn't even let you submit. But when you input invalid data in almost any other type, it leads to a SQL error. This can include data that is too long, or the wrong type, or other SQL errors.

What should happen is that the form should check for validity BEFORE submission, and only submit if it is for sure valid. Otherwise it should advise the user as they type why it's invalid, and never let them submit until it becomes valid.

Furthermore, the SQL error vomit can lead to information leak about the SQL structure that could be a security issue. Users should never see SQL errors presented.


Status: New
BloodyIron replied on at Permalink Reply
I've thought more about this, and I'm pretty sure the spitting out of the SQL errors could be used for injection/exploit, as it spits out the error, this could potentially lead to the invalid text triggering queries. This really should never happen! I am now particularly concerned about the security of this.
Mnkras replied on at Permalink Reply
Mnkras
Can you provide exact reproduction steps to get an error?

And since this is open source knowing DB structure isn't really bad.
BloodyIron replied on at Permalink Reply
Well, when you input an extremely long entry into a Phone Number type, I got this:


An exception occurred while executing 'UPDATE GetInfoExpressSearchIndexAttributes SET ak_get_info_phone = ? WHERE exEntryID = ?' with params ["000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", 53]: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'ak_get_info_phone' at row 1
Mnkras replied on at Permalink Reply
Mnkras
So I did some digging, the error isn't from saving the actual attribute, it is from saving the value to the search index.

The default column type is text for the attribute, the default search column type is string (255 chars) depending on your MYSQL version mysql will either silently truncate the text to 255 chars, or throw that wonderful exception above.

We have 2 options:
Change the default search column type to text and alter all existing search index columns.
OR
Truncate the data to 255 chars before saving.
mlocati replied on at Permalink Reply
mlocati
About client-side validation: it's useful only to give users a quicker feedback about wrong/malformed data.
The server side (ie the php scripts) must not trust the data they receive, since malicious users can send the server any kind of data bypassing very easily the client side checks

concrete5 Environment Information

# concrete5 Version
Core Version - 8.1.0
Version Installed - 8.1.0
Database Version - 20170123000000

# concrete5 Packages
ExchangeCore reCAPTCHA (1.1.1), Fundamental (4.0.1), Styled Maps (1.2.3)

# concrete5 Overrides
blocks/express_entry_detail/templates/get_info_records_details/view.css, blocks/express_entry_detail/templates/get_info_records_details/view.php, blocks/express_entry_detail/templates/get_info_records_details, blocks/express_entry_detail/templates, blocks/express_entry_detail, blocks/express_form/templates/client_feedback_form/view.css, blocks/express_form/templates/client_feedback_form/view.php, blocks/express_form/templates/client_feedback_form, blocks/express_form/templates/get_info_form/view.css, blocks/express_form/templates/get_info_form/view.php, blocks/express_form/templates/get_info_form, blocks/express_form/templates/apply_to_work_form/view.css, blocks/express_form/templates/apply_to_work_form/view.php, blocks/express_form/templates/apply_to_work_form, blocks/express_form/templates, blocks/express_form, languages/ja_JP/LC_MESSAGES/messages.mo, languages/ja_JP/LC_MESSAGES, languages/ja_JP, languages/el_GR/LC_MESSAGES/messages.mo, languages/el_GR/LC_MESSAGES, languages/el_GR, languages/ru_RU/LC_MESSAGES/messages.mo, languages/ru_RU/LC_MESSAGES, languages/ru_RU, languages/it_IT/LC_MESSAGES/messages.mo, languages/it_IT/LC_MESSAGES, languages/it_IT, languages/es_PY/LC_MESSAGES/messages.mo, languages/es_PY/LC_MESSAGES, languages/es_PY, languages/nl_NL/LC_MESSAGES/messages.mo, languages/nl_NL/LC_MESSAGES, languages/nl_NL, languages/sv_SE/LC_MESSAGES/messages.mo, languages/sv_SE/LC_MESSAGES, languages/sv_SE, languages/cs_CZ/LC_MESSAGES/messages.mo, languages/cs_CZ/LC_MESSAGES, languages/cs_CZ, languages/da_DK/LC_MESSAGES/messages.mo, languages/da_DK/LC_MESSAGES, languages/da_DK, languages/fi_FI/LC_MESSAGES/messages.mo, languages/fi_FI/LC_MESSAGES, languages/fi_FI, languages/pt_BR/LC_MESSAGES/messages.mo, languages/pt_BR/LC_MESSAGES, languages/pt_BR, languages/fr_FR/LC_MESSAGES/messages.mo, languages/fr_FR/LC_MESSAGES, languages/fr_FR, languages/tr_TR/LC_MESSAGES/messages.mo, languages/tr_TR/LC_MESSAGES, languages/tr_TR, languages/de_DE/LC_MESSAGES/messages.mo, languages/de_DE/LC_MESSAGES, languages/de_DE, languages/en_GB/LC_MESSAGES/messages.mo, languages/en_GB/LC_MESSAGES, languages/en_GB, mail/block_express_form_submission.php, blocks/express_entry_detail/templates/get_info_records_details/view.css, blocks/express_entry_detail/templates/get_info_records_details/view.php, blocks/express_entry_detail/templates/get_info_records_details, blocks/express_entry_detail/templates, blocks/express_entry_detail, blocks/express_form/templates/client_feedback_form/view.css, blocks/express_form/templates/client_feedback_form/view.php, blocks/express_form/templates/client_feedback_form, blocks/express_form/templates/get_info_form/view.css, blocks/express_form/templates/get_info_form/view.php, blocks/express_form/templates/get_info_form, blocks/express_form/templates/apply_to_work_form/view.css, blocks/express_form/templates/apply_to_work_form/view.php, blocks/express_form/templates/apply_to_work_form, blocks/express_form/templates, blocks/express_form, languages/ja_JP/LC_MESSAGES/messages.mo, languages/ja_JP/LC_MESSAGES, languages/ja_JP, languages/el_GR/LC_MESSAGES/messages.mo, languages/el_GR/LC_MESSAGES, languages/el_GR, languages/ru_RU/LC_MESSAGES/messages.mo, languages/ru_RU/LC_MESSAGES, languages/ru_RU, languages/it_IT/LC_MESSAGES/messages.mo, languages/it_IT/LC_MESSAGES, languages/it_IT, languages/es_PY/LC_MESSAGES/messages.mo, languages/es_PY/LC_MESSAGES, languages/es_PY, languages/nl_NL/LC_MESSAGES/messages.mo, languages/nl_NL/LC_MESSAGES, languages/nl_NL, languages/sv_SE/LC_MESSAGES/messages.mo, languages/sv_SE/LC_MESSAGES, languages/sv_SE, languages/cs_CZ/LC_MESSAGES/messages.mo, languages/cs_CZ/LC_MESSAGES, languages/cs_CZ, languages/da_DK/LC_MESSAGES/messages.mo, languages/da_DK/LC_MESSAGES, languages/da_DK, languages/fi_FI/LC_MESSAGES/messages.mo, languages/fi_FI/LC_MESSAGES, languages/fi_FI, languages/pt_BR/LC_MESSAGES/messages.mo, languages/pt_BR/LC_MESSAGES, languages/pt_BR, languages/fr_FR/LC_MESSAGES/messages.mo, languages/fr_FR/LC_MESSAGES, languages/fr_FR, languages/tr_TR/LC_MESSAGES/messages.mo, languages/tr_TR/LC_MESSAGES, languages/tr_TR, languages/de_DE/LC_MESSAGES/messages.mo, languages/de_DE/LC_MESSAGES, languages/de_DE, languages/en_GB/LC_MESSAGES/messages.mo, languages/en_GB/LC_MESSAGES, languages/en_GB, mail/block_express_form_submission.php

# concrete5 Cache Settings
Block Cache - On
Overrides Cache - On
Full Page Caching - On - In all cases.
Full Page Cache Lifetime - Every 6 hours (default setting).

# Server Software
Apache/2.4.18 (Ubuntu)

# Server API
apache2handler

# PHP Version
7.0.15-0ubuntu0.16.04.4

# PHP Extensions
apache2handler, calendar, Core, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imagick, json, libxml, mbstring, mcrypt, mysqli, mysqlnd, openssl, pcre, PDO, pdo_mysql, Phar, posix, readline, Reflection, session, shmop, SimpleXML, sockets, SPL, standard, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zip, zlib

# PHP Settings
max_execution_time - 60
log_errors_max_len - 1024
max_file_uploads - 20
max_input_nesting_level - 64
max_input_time - 60
max_input_vars - 1000
memory_limit - 128M
post_max_size - 20M
sql.safe_mode - Off
upload_max_filesize - 20M
mysqli.max_links - Unlimited
mysqli.max_persistent - Unlimited
pcre.backtrack_limit - 1000000
pcre.recursion_limit - 100000
session.cache_limiter - <i>no value</i>
session.gc_maxlifetime - 7200
opcache.max_accelerated_files - 2000
opcache.max_file_size - 0
opcache.max_wasted_percentage - 5

Browser User-Agent String

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/56.0.2924.76 Chrome/56.0.2924.76 Safari/537.36