Pelletier Vincent
2008-02-20 14:08:33 UTC
Hi.
I just (r19431) added a new feature to CMFActivity: "serialization_tag".
This can be used to address activity concurency problem with storages which
don't detect conflict errors (ie: MySQL) with MVCC-enabled Zope (2.8 and
higher).
This change requires re-creating activity tables, since it adds a new column.
One problem which mighthappen without using this feature when having multiple
activity execution node is the following execution plan:
Suppose activities A and B, both reading the same object that an user can
modify:
T+0 user modifies object
T+1 node 1 executes activity A (duration: 4)
T+2 user modifies object
T+3 node 2 executed activity B (duration 1)
T+4 ... activity B commits in MySQL
T+5 ... activity A commits in MySQL
If both activities modify the same table lines, there is a possibility that
the final MySQL content will not be coherent with final object state.
This is because MySQL locks rows it updates in the order they are found, and
update each one immediately. So 2 requests modifying the same rows can get
locks in an order which is not consistent with query start order.
A concrete case of this problem is when indexing objects.
Indexations use SQLDict, which regroups indexations to lower the number of
individual SQL queries to MySQL. So 2 invocations might have significant run
duration differencies, exacerbating above described concurency problem.
So the final state of the catalog can be incoherent with final object state.
serialization_tag solves this by basicaly forcing activities sharing the same
tag value to be sequentialy executed.
Technicaly, it's in fact a tiny bit different: if multiple activities with the
same serialization tag are in -1 processing_node (validation pending) state,
they are all distributed. This is because the order they are executed in
doesn't matter *if* they were all waiting validation at the same time (all
will see the same object data, and any further object modification will cause
a new activity to come up). This is done to shorten the time needed to
validate multiple pending activities.
Also, another technical detail about tag matching: tag is matched by MySQL
using "LIKE" string operator. So it's possible to use SQL wildcards in such
tag.
Of course, this modification alone does not cure the indexation problem - I'll
fix that shortly using this new feature.
I just (r19431) added a new feature to CMFActivity: "serialization_tag".
This can be used to address activity concurency problem with storages which
don't detect conflict errors (ie: MySQL) with MVCC-enabled Zope (2.8 and
higher).
This change requires re-creating activity tables, since it adds a new column.
One problem which mighthappen without using this feature when having multiple
activity execution node is the following execution plan:
Suppose activities A and B, both reading the same object that an user can
modify:
T+0 user modifies object
T+1 node 1 executes activity A (duration: 4)
T+2 user modifies object
T+3 node 2 executed activity B (duration 1)
T+4 ... activity B commits in MySQL
T+5 ... activity A commits in MySQL
If both activities modify the same table lines, there is a possibility that
the final MySQL content will not be coherent with final object state.
This is because MySQL locks rows it updates in the order they are found, and
update each one immediately. So 2 requests modifying the same rows can get
locks in an order which is not consistent with query start order.
A concrete case of this problem is when indexing objects.
Indexations use SQLDict, which regroups indexations to lower the number of
individual SQL queries to MySQL. So 2 invocations might have significant run
duration differencies, exacerbating above described concurency problem.
So the final state of the catalog can be incoherent with final object state.
serialization_tag solves this by basicaly forcing activities sharing the same
tag value to be sequentialy executed.
Technicaly, it's in fact a tiny bit different: if multiple activities with the
same serialization tag are in -1 processing_node (validation pending) state,
they are all distributed. This is because the order they are executed in
doesn't matter *if* they were all waiting validation at the same time (all
will see the same object data, and any further object modification will cause
a new activity to come up). This is done to shorten the time needed to
validate multiple pending activities.
Also, another technical detail about tag matching: tag is matched by MySQL
using "LIKE" string operator. So it's possible to use SQL wildcards in such
tag.
Of course, this modification alone does not cure the indexation problem - I'll
fix that shortly using this new feature.
--
Vincent Pelletier
Vincent Pelletier