[Erp5-dev] hot reindexing

[Yoshinori Okuji]

Hello,

To support hot reindexing, many things in ZSQLCatalog were modified.
FYI, hot reindexing is a feature that you can reindex all objects in
the whole site on the fly without disturbing users. This will be
documented in near future in details.

This mail specifies how to migrate to this new ZSQLCatalog. Now
ZSQLCatalog can contain multiple catalogs at a time, and you can choose
which catalog should be used by default. Catalogs are instances of
SQLCatalog, so I call catalogs in ZSQLCatalog "SQL Catalogs".

In the previous implementation, ZSQLCatalog stored methods in itself.
The current implementation moved these methods to SQL Catalogs and
ZSQLCatalog contains SQL Catalogs instead.

For example, you can make a SQL Catalog named "erp5_mysql" in
ZSQLCatalog, and you can cut and paste all methods to this "erp5_mysql"
SQL Catalog. Then, you can choose the default SQL Catalog in the tab
Properties in ZSQLCatalog.

When you view objects and modify parameters through ZSQLCatalog (or
ERP5Catalog), ZSQLCatalog works as a proxy to the default SQL Catalog
actually. So, say, if you modify filters in ZSQLCatalog, the result
would be reflected to the default SQL Catalog. And, this is the same as
modifying filters in the default SQL Catalog directly, indeed.


For other related changes:

You need to recreate tables for portal_activities, if you use
CMFActivity. I'd recommend making a different database specific to
portal_activities at this time. This is useful for hot reindexing,
because activities are independent of catalogged objects (this
description is not enough, so I will write more info afterwards). If
you make a new database for portal_activities, please make a connection
named "cmf_activity_sql_connection". This is compatible with the skins
inside CMFActivity.

The way to set up initial properties for SQL Catalogs has been changed.
Rather than writing the pairs of key/value in Python code, an XML file
named "properties.xml" is used. This file is generated automatically
from the ZMI (see Advanced). Because of this change, I think it is
broken to set up CPS, since I didn't make properties.xml for CPS.
(Sorry, I don't have CPS in my box.)

The table "catalog" is divided into two, because many columns were not
used in ERP5. So columns used in ERP5 are left in "catalog" and the
others were moved to a new table "compatibility". Probably ERP5 does
not need "compatibility" at all, but it might be useful in the future,
if ERP5 uses non-ERP5 objects inside an ERP5 Site (CPS Documents, for
example).


How to use hot reindexing:

You can use the ZMI to start hot reindexing. You need to make a Python
Script named "Catalog_getReindexablePathList" which returns a list of
the paths of folders to be reindexed (e.g. ('portal_templates',
'portal_simulation', 'accounting', )). This script can be in
portal_catalog or portal_skins. You need to have at least two seperate
databases (you can make any number of databases in MySQL). That's it.
It should still have bugs, but it will work somehow.

YO