Serverside validation missing / field values sometimes get lost after edit

Bobblue

DTRegister version 4.0.9

I edit often the records in the backend. For example when i fill in hidden fields or set the payment status for offlinepayment. I realized since 4.0.5 that sometimes some of the conitional field values get lost, even if they are mandatory. This brings me to the point that there is no serverside validation.

I do not know if it is the browser eating up the fields
connection brocken while sending
cache issue
or what ever but in the end the field value is gone also in the database (j25_dtregister_user_field_values)

this is extreamly bad when it is field billed field with a limited amount and all availible options are out of stock.
Losing the field value db means it becomes availible in the frontend for the next customer again. Than it is soled a second time.

I cant exactly say when this happens, and i cant really reproduce it. But i had already 6 customers with this issue.
Please help!

saguaros

Bobblue

Hi,

Could you tell us the fields which we can replicate the same issue and check for you?

Bobblue

saguaros :
Hi, like i wrote aboth it is not happening every time. It is realy rearly. But of cause i can give you an example booking.
look at this confirmation codes (Records)

IDM-19-T0-SC-F-4176927
IDM-19-T0-SC-U-8203702
IDM-19-T0-SC-F-4388439
IDM-19-T0-SC-D-7693584
IDM-19-T0-SC-F-9164257
IDM-19-T0-SC-U-0485596
IDM-19-T0-SC-O-5011086
IDM-19-T0-SC-F-2268559
IDM-19-T0-SC-ST-4437953

All of them have no field SC_Staffel_Tertium

SC_Staffel_Tertium is conditional on Altersgruppe (value id 0 or 3) but all of them have Altersgruppe 0.
SC_Staffel_Tertium is mandatory.... so not having it is an error!

Next example:

look at this confirmation codes (Records)

IDM-19-T0-SC-F-2047185
IDM-19-T0-SC-F-3025908
IDM-19-T0-SC-F-1086012
IDM-19-T0-SC-F-1597263
IDM-19-T0-SC-F-3368160
IDM-19-T0-SC-D-4191198
IDM-19-T0-SC-F-4619180
IDM-19-T0-SC-F-0921774
IDM-19-T0-SC-F-5424592
IDM-19-T0-SC-O-9251923
IDM-19-T0-SC-O-0237940

All of them have no field KFZ. KFZ is coditional on Anreiseart (value id 0,1,2) KFZ is mandatory.
All of them have Anreisart 0 or 1 so not having KFZ is an error.

Manit

Hi Bobblue

Thank you for your provided info. Would you mind sending us the event id where the issue happens? It will be helpful for us to check and debug the issue.

Regards.

Bobblue

Manit :
sure thats eventid 196, 197,198,199,200,201,202 and 203

saguaros

Bobblue
We tested on your site:

Register event ID: 200 from frontend with field id 140 value "Test"
Record in backend reflects the same entered data while registering from frontend
Try to save records several times in backend but the data still be correct: field id 140: KFZ

It's weird but we can't replicate the issue.

Bobblue

saguaros
Yes.... like i said :

I do not know if it is the browser eating up the fields: connection brocken while sending, cache issue, or what ever but in the end the field value is gone also in the database (j25_dtregister_user_field_values)

I cant exactly say when this happens, and i cant really reproduce it....

Fact is that i have this customers with bad data! Fact is also that it is not so often because from 300 bookings its around 11.... My impression is that it has something to do with https://www.joomlart.com/forums/d/19736-conditional-hidden-fields-bug-in-recursion/5 and happens maybe when in the edit screen of a record not all conditinal fields are load and then get lost vile save the edited record....

Nether the less.... I think your aproche here is wrong! We are talking about a race condition not about something you can reproduce easy with 5 clicks. Thats why there should be a server-side / or even better database-side validation in place.... this is missed totally. So please put this on the to do for the next iteration!

Bobblue

Bobblue : Maybee it is also when the User edited his booking, and not all conditional fields are loaded ?

saguaros

Bobblue

Hi,

We tried all ways but still can't replicate the issue, edit records but all conditional fields are still loaded. It would be great if you can share screenshots or record video so we can reproduce it.

Bobblue

saguaros

Nice try. I will do. First look told me that you changed something because there is now a gray box above the record...
diff between playground and live page tells me:

diff -r inseln-der-macht/administrator/components/com_dtregister/controllers/field.php idm-playground/administrator/components/com_dtregister/controllers/field.php
408,409c408,412
< 
<             $where[] = "LOWER(name) LIKE '%" . $database->escape(trim(strtolower($search))) . "%'";
---
> 			if($search == "hidden"){
> 				$where[] = " hidden =1 ";
> 			}else{
>             	$where[] = "LOWER(name) LIKE '%" . $database->escape(trim(strtolower($search))) . "%'";
> 			}
Only in inseln-der-macht/administrator/components/com_dtregister/controllers: field.php.bkp
diff -r inseln-der-macht/administrator/components/com_dtregister/controllers/user.php idm-playground/administrator/components/com_dtregister/controllers/user.php
592a593,595
> 		echo "<pre>";
> 		//print_r($tUser);
> 		echo "</pre>";
diff -r inseln-der-macht/administrator/components/com_dtregister/models/devent.php idm-playground/administrator/components/com_dtregister/models/devent.php
4483c4483
< 
---

let me find out what that exactly is!

Bobblue

So gray box is comming from the echo pre stuff in user.php.... doesnt change anything. but nice that you left the garbage in.

The stuff in the field.php looks quit good. but field.php i exchanged because of this bug:
https://www.joomlart.com/forums/d/19279-conditional-hidden-fields-bug-in-recursion
but bringing the diff to production:
dosn't change there the behavior in Safari (with https)....: hidden, conditional fields are not initial loaded once opening a record. and this stays so till i click on the conditional field
changes the behavior in Chrome and Firefox (with https)...: hidden, conditional fields are not initial loaded once opening a record. this changes after waiting some time.... it are reloaded incrementally

both is somehow wrong. My impression is, when it is not yet opened, reloaded and i already save the record, the not loaded field will be thrown away.

saguaros

Bobblue

Could you share the screenshot of that graybox?

Bobblue

saguaros
Hi I made a video for you 😉 Showing
a) the gray box caused by your <pre> and
b) showing the reload of hidden conditional fields which are not initial loaded once opening a record...

http://playground.inseln-der-macht.de/images/screencast.mp4

saguaros

Hi,

The gray box is removed, the conditional fields are loaded via Ajax so it takes time a bit to load. I just tested in Safari on my Mac PC and it works fine now.

We're also planning to improve on this by removing the ajax load for fields loading so it can be better.

Regards

Bobblue

Ok removing the Ajax load and put it to initial load is the quick solution on it. But when the list of fields becomes very long we can reach server limits. A flag that prevents the user / admin saving a only partial loaded record after modifications would be also a good idea from my point of view. And of cause a serverside Check if conditional mandatory fields ar set with throwing an error back to user/admin is also needed

saguaros

Bobblue

Thanks for your feedback, I will share with the team also.

Bobblue

saguaros:
any Update?

saguaros

Bobblue

I replied in this thread for conditional fields issue: https://www.joomlart.com/forums/d/19736-conditional-hidden-fields-bug-in-recursion/10