[Erp5-dev] Sale Supplies - problem with performance, some other questions

[Łukasz K. Nowak]

Hello,

On fresh revision (14673) I've create site with erp5_[base,trade,pdm].

There I've created 2 resources: 'car' and 'wheel'.

I've created 2 organisations: 'we' and 'them'.

I've created Sale Supply, which has destination 'them'.

In those Sale Supply I've created one Sale Supply Line for 'car' and 550
for 'wheel'.

I've patched 'SupplyLine_asPredicate' with:
script.log('%s'%(context.getRelativeUrl()))

I've created Sale Order from 'we' to 'them'. I added Sale Order Line,
set resource to 'car' and...system is invoking 551 times
'SupplyLine_asPredicate' on every line in Sale Supplies - for every line
from this Sale Supply.. Huh - its quite performance disaster, I've seen,
that there might be many invocations of such predicate search.

I was just curious, and added default_supply_line to both resources -
only that 'correct' one was searched.

Is it bug or feature?

Another thing. Please look at SupplyLine_asPredicate[1]. I thought, that
defining of Purchase Supplies is done based on its source, am I right?
Is that script able to support such situation? Shall I propose tested
conditions to support sale and purchase supplies in this script?

One another. On TechnicalNoteOnPredicates[2] one can read about:
"You may want to configure the list of local properties which works like
a criterion property and the list of base categories which works like a
membership criterion base category with 2 portal defaults values:

* portal_criterion_base_category_list, default to
(source,destination,resource,group)
* portal_criterion_property_list, default to
(start_date,stop_date)"

Where is it?


Thanks in advance for any help or tips,
Luke

[1]
http://svn.erp5.org/*checkout*/erp5/trunk/bt5/erp5_trade/SkinTemplateItem/portal_skins/erp5_trade/SupplyLine_asPredicate.xml?revision=14673&content-type=text%2Fplain
[2] http://www.erp5.org/TechnicalNoteOnPredicates

[Shresth K]

Hello Lukasz,

Post by Łukasz K. Nowak
"You may want to configure the list of local properties which works like
a criterion property and the list of base categories which works like a
* portal_criterion_base_category_list, default to
(source,destination,resource,group)
* portal_criterion_property_list, default to
(start_date,stop_date)"
Where is it?

As several other default portal wide properties it should have been in
ERP5Defaults.py and this is where it was last seen, until someone else
confirm otherwise-

http://svn.erp5.org/erp5/trunk/products/ERP5/ERP5Defaults.py?r1=4173&r2=4381

I too am trying to find my way around the predicate and mapped value world.

Regards,
Shresth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20070605/f890e13a/attachment.htm>

[Łukasz K. Nowak]

Hello,

Dnia 05-06-2007, wto o godzinie 14:33 +0200, ?ukasz K. Nowak napisa?(a):
(...)

Post by Łukasz K. Nowak
Another thing. Please look at SupplyLine_asPredicate[1]. I thought, that
defining of Purchase Supplies is done based on its source, am I right?
Is that script able to support such situation? Shall I propose tested
conditions to support sale and purchase supplies in this script?

I've attached SupplyLine_asPredicate, which, I hope, resolves correctly
issue of purchase system. I haven't written unit test for it - I'm still
waiting for comments on problems with invoking asPrediacte on all Supply
Lines. I'd like to know, if idea provided in this script is
ERP5-oriented and correct?

Thanks,
Luke
--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: [phone] fax: [phone]
``Use the Source, Luke...''

-------------- next part --------------
## Script (Python) "SupplyLine_asPredicate"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=*args,**kw
##title=
##
"""
This is modified version of SupplyLine_asPredicate, which supports sale and purchase system.
Well - at least I (Luke) hope...
"""
source = context.getSourceValue()
destination = context.getDestinationValue()
script.log('me = %s source = %s destination = %s'%(context.getRelativeUrl(),source,destination))
if source is None and destination is None: # we are on supply line, which parent is Resource - eg. default_supply_line
return context.generatePredicate(membership_criterion_base_category_list = ('resource',),
criterion_property_list = ('start_date',))

if destination is not None: # we are on supply line, which is sale supply
return context.generatePredicate(membership_criterion_base_category_list = ('destination', 'resource'),
criterion_property_list = ('start_date',))
elif source is not None: # we are on supply line, which is purchase supply
return context.generatePredicate(membership_criterion_base_category_list = ('source', 'resource'),
criterion_property_list = ('start_date',))

[Łukasz K. Nowak]

Hello,

Post by Łukasz K. Nowak
On fresh revision (14673) I've create site with erp5_[base,trade,pdm].
There I've created 2 resources: 'car' and 'wheel'.
I've created 2 organisations: 'we' and 'them'.
I've created Sale Supply, which has destination 'them'.
In those Sale Supply I've created one Sale Supply Line for 'car' and 550
for 'wheel'.
script.log('%s'%(context.getRelativeUrl()))
I've created Sale Order from 'we' to 'them'. I added Sale Order Line,
set resource to 'car' and...system is invoking 551 times
'SupplyLine_asPredicate' on every line in Sale Supplies - for every line
from this Sale Supply.. Huh - its quite performance disaster, I've seen,
that there might be many invocations of such predicate search.
I was just curious, and added default_supply_line to both resources -
only that 'correct' one was searched.
Is it bug or feature?

Well. After playing some with Purchase and Sale Supplies in one system,
I've seen, that for given context (eg. Sale Order Line) all *Supply
Lines are searched. I've found, that patching Resource with given patch
make this search narrowed to only proper context (it is still much, but
in our system it is making things almost 2 times faster).

Any comments welcome.

Thanks,
Luke
--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: [phone] fax: [phone]
``Use the Source, Luke...''

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ERP5-Document-Resource.py-only-sale-only-purchase.patch
Type: text/x-patch
Size: 1096 bytes
Desc: not available
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20070606/9b6ea4c9/attachment.bin>

[Sébastien Robin]

Post by Łukasz K. Nowak
Hello,

Post by Łukasz K. Nowak
On fresh revision (14673) I've create site with erp5_[base,trade,pdm].
There I've created 2 resources: 'car' and 'wheel'.
I've created 2 organisations: 'we' and 'them'.
I've created Sale Supply, which has destination 'them'.
In those Sale Supply I've created one Sale Supply Line for 'car' and 550
for 'wheel'.
script.log('%s'%(context.getRelativeUrl()))
I've created Sale Order from 'we' to 'them'. I added Sale Order Line,
set resource to 'car' and...system is invoking 551 times
'SupplyLine_asPredicate' on every line in Sale Supplies - for every line
from this Sale Supply.. Huh - its quite performance disaster, I've seen,

When is it called 551 times ? When you create new lines ?

If I remember correctly it might be called when it tries to get a price for
your line. And if we call each time the method you said, this is because we
are looking for a price for every line. And the price does not depend only
about the resource, it can depends on quantity, variation... wich might be
different for every line. So this is not that easy to do optimisation.

Post by Łukasz K. Nowak

Post by Łukasz K. Nowak
that there might be many invocations of such predicate search.
I was just curious, and added default_supply_line to both resources -
only that 'correct' one was searched.
Is it bug or feature?

Well. After playing some with Purchase and Sale Supplies in one system,
I've seen, that for given context (eg. Sale Order Line) all *Supply
Lines are searched. I've found, that patching Resource with given patch
make this search narrowed to only proper context (it is still much, but
in our system it is making things almost 2 times faster).

There is a discussion about predicate and performance here :
http://erp5.org/Discussion/PredicatesPerformances

Your patch is only Sale and Purchase specific, this is not generic, so we
should not apply. But I understand that it makes the research faster.

So the discussion might be updated, and time must be found in order to improve
well that part.

Seb.

Post by Łukasz K. Nowak
Any comments welcome.
Thanks,
Luke

[Łukasz Nowak]

Hello,

Dnia 08-06-2007, pi? o godzinie 16:27 +0200, S?bastien Robin napisa?(a):
(...)

Post by Sébastien Robin
Your patch is only Sale and Purchase specific, this is not generic, so we
should not apply. But I understand that it makes the research faster.

Ok, understood. What about another way (attached)? It will only touch
list of portal types for correct context, so for any other usage it will
be stable.

Another thing - in Resource.py there is function _pricingSortMethod[1].
If I understand correctly it is used to find "best" supplies by its
destination category, which is fetched from context. So:

* Purchase Supplies are compared by destination? I thought (and
implemented it like this...) to use source (well, not exactly by
source_section in our case, but that's another thing...)
* if I'd like to modify system, to got Sale Supplies by
destination_section, and not destination - I've updated
SupplyLine_asPredicate to achive it - shall I modify this to
getDestinationSection?

Sorry for such - might be - obvious questions, but I wasn't able to find
more documents about trade, and source code analysis is not so good to
learn such behaviour.

Thanks for any tips,
Luke

[1]http://svn.erp5.org/erp5/trunk/products/ERP5/Document/Resource.py?view=markup#_pricingSortMethod
--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: [phone] fax: [phone]
``Use the Source, Luke...''

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Resource.py-optimized-another-way.patch
Type: text/x-patch
Size: 1408 bytes
Desc: not available
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20070612/89e0bf57/attachment.bin>