[Erp5-dev] Validating W3 Confomance of ERP5 Forms

[Jacek Medrzycki]

I have written a small test that validates forms using external
validator on w3.org page.

As it's not typical unit test and I'm in doubts of how such test should
look like, it's more a proposal to discussion than final solution,
thought it works.

Currently it checks login form, main form and some view and listbox
forms. As any form of its type is rendered by the same renderer, it
seems that it is enough to check some forms, not all - probably the
representative probe to make sure all field types are included in test.
For now it checks Person_view and Person_viewProfile,
PersonModule_viewPersonList, Organisation_view and
OrganisationModule_viewOrganisationList, but it rather an example.
Perhaps other forms should be taken into account.

test_03_StandardForms checks all forms defined in
getFormsToValidateList() and launches the validator for each form. It
gatheres all errors and fails if the error list is not empty, so all
forms are tested. Test results are shown in the exception message (if
the verbose is set to false, only form names for which validation failed
are shown)

The heart of the test is W3FormValidator class that fires the
validator.w3.org page and parses the result. Its validate_form method
gets a page source code and then parses the W3 org validator output and
returns a list of tuples: line, column and error message.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: testW3Conformance.py
Type: text/x-python
Size: 5991 bytes
Desc: not available
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20070326/b9f5ee8d/attachment.py>

[Pelletier Vincent]

Post by Jacek Medrzycki
I have written a small test that validates forms using external
validator on w3.org page.

It would be better to use a local validation mean. Yoshinori suggested, for
example, http://openjade.sourceforge.net/ .

Post by Jacek Medrzycki
As it's not typical unit test and I'm in doubts of how such test should
look like, it's more a proposal to discussion than final solution,
thought it works.

eval is evil :)

Instead of
eval('object.%s' % (some_id, ))
please use
getattr(object, some_id)

Post by Jacek Medrzycki
The heart of the test is W3FormValidator class

I think it should be the code responsible for the sleep() used to limit the
access frequency - but if we use a local validation method, it would not be
needed anyway. Maybe should there be multiple classes providing the same API
allowing to interface with multiple validation systems... Erm. Then again
it's just a unit test :) .

I also think verbosity should be given to this class' constructor in order to
factorise the "append" code.

[Jacek Medrzycki]

Post by Pelletier Vincent

Post by Jacek Medrzycki
I have written a small test that validates forms using external
validator on w3.org page.

It would be better to use a local validation mean. Yoshinori suggested, for
example, http://openjade.sourceforge.net/ .

I'll have a look. BTW, I think that w3org validator can also be
installed lokally, why not use it?

Post by Pelletier Vincent
eval is evil :)
Instead of
eval('object.%s' % (some_id, ))
please use
getattr(object, some_id)

I know that eval could be a security threat, especially when dealing
with user input, but I thought it is safe in unit test. But I'll fix the
code of course.

Post by Pelletier Vincent
I also think verbosity should be given to this class' constructor in order to
factorise the "append" code.

Can you explain more closely. I don't understand fully what you mean.

Jacek

[Pelletier Vincent]

Post by Jacek Medrzycki
I know that eval could be a security threat, especially when dealing
with user input, but I thought it is safe in unit test. But I'll fix the
code of course.

Well, more generally I think it's better to use a "static" code than dynamic
string execution - if not for performance reasons, at least for readability.
And I'm afraid that once the first "eval" ever get accepted in the repository
it would become some kind of example and would lead to a more generalised use
of it, without taking appropriate care.

Post by Jacek Medrzycki
Can you explain more closely. I don't understand fully what you mean.

I was refering to this code:
results = self.w3_validator.validate_form(form_source)
if len(results)>0:
if self.verbose:
test_errors.append((form, results))
else:
test_errors.append((form, ))

I guess it would be a bit nicer like:
self.w3_validator = W3FormValidator(self.w3_validator_url,
verbose_level=verbose_level)
[...]
test_errors.append(self.w3_validator.validate_form(form_source))