The CMFActivity Tool allows to start methods and scripts in background. If you need to run the same method on many objects, this is not good to start a single transaction and modify all of them. If you do so, then a very long transaction may be required and there is a very high chance that you will get locks if the zope server is under use.
With the CMFActivity Tool, you can start an activity for each object, then each activity is called one by one.
Table of Contents
How to start an activity¶
You have to call the activate method on the object on which you want to call a method or script. Let's say you want to start an activity in order to set the title, you can do like this:
my_object.activate().setTitle('foobar')
Like this, the title will not be set immediately, but it will be set some time after by the activity tool.
All arguments and parameters of the method or the script must be pickleable, so you can use tuples, strings, dictionaries, integers... but it is not possible to use objects from the ZODB (in that case you can just pass object path and retrieve them later using restrictedTraverse).
Note:
This Thread has some more detailed explanations, which should be integrated in this page.
Manage the list of activities¶
In the zope management interface of your site, you can select the object with the id portal_activities. Then you can select the activity tab and you will get the full list of activities.
Usually, activities are executed automatically. If there is many activities, it may require some time. On each activity, it is possible to select invoke in order to execute the activity right now, or you may select cancel if you do not want to run the activity.
You will see several columns:
- Object: show the path of the object on which the activity was started.
- Method Id: the method called.
- Activity Kw: keyword argument passed to activate
- Arguments: arguments used when the method was activated.
- Named Parameters: additional parameters used when the method was activated.
- Processing Node: the number of the node where the method will be executed.
- Priority: a priority (smaller is more important).
- Processing: if 1, this activity is currently executed.
>The Processing Node¶
The Processing Node can have many different values:
- -3: The object does not exist any more.
- -2: The method or script failed.
- -1: New message.
- 1 or more: Usual number, the message will be executed soon.
If the Processing Node is more than 0, this is the number of the node where the message will be executed. If you don't run a cluster, then the number ill be always 1, because there is only one node. If you are running a cluster with many Zope servers, then the processing_node can be as big as the total number of nodes where you are running activities.
How to define an ordered execution of activities¶
It is often required to start an activity when another one (or many ones) will be finished. You can use several kinds of parameters for that.
The first one is after_method_id, it allows to start an activity after another only by giving the name of the previous method.
Let's say you want to define the description after the title, you can do like this:
my_object.activate(after_method_id='setTitle').setDescription('aa')
List of possible control mechanisms:
- at_date
request execution date for this activate call
- after_method_id
never validate message if after_method_id is in the list of methods which are going to be executed
- after_message_uid
never validate message if after_message_uid is in the list of messages which are going to be executed
- after_path
never validate message if after_path is in the list of path which are going to be executed
- after_path_and_method_id
never validate message if a message for method_id on path is in the queue.
- tag
add a tag to a message
- after_tag
never validate message if there is a message tagged with this tag
- activate_kw
a way to pass activate parameters to subsequent activity calls, see below
How to control the behavior of the activity queue¶
CMFActivity model is pluggable, it's possible to define an activity queue class, and later pass the id of this class as activity= keyword to activate. CMFActivity comes with the following activity queues:
- SQLDict (the default)
SQLDict uses SQL as a backend, so this ensures that activities will be executed even if the server is restarted, and allows to distributes activities in a ZEO cluster. It guarantees that if a message for a method M on an object O will be executed only once at a time (think that (M, O) are the keys of a dictionary and the message is the value). If there is a tuple (M, O) in the queue, calling O.activate().M() will not add a new message, when this message will be processed, calling O.activate().M() will add the message.
- SQLQueue
Like SQLDict, SQLQueue uses a SQL backend, but all messages are executed, as a result, if a message M on an object O is n times in the queue, then M will be called n times.
- RAMDict
It haves the same semantics as SQLDict for duplicate messages, but uses a RAM storage, so it is not peristent across server restarts and is not shared on the cluster, but can be faster.
- RAMQueue
Mixes RAMDict and SQLQueue
Usage of activate_kw¶
/!\ Note: activity_kw is a way to pass parameters to the activate method which is invoked internally from another method. That is the only way to pass such parameters at the moment, but the design is flawed. Originally, I (yo) invented this parameter only for reindexObject, and it was good enough in this case, because a single method call happened here at one level. Other developers, however, extended the meaning, and started to propagate activity_kw at multiple levels to multiple methods, and this was wrong, since the scope is too large.
Using such code snippet:
object.activate(tag='my_tag').edit(activate_kw={'tag':'my_tag'})
object.activate(after_tag='my_tag').myMethod()
it is possible to have such scenario:
- edit and myMethod are put into activity queue
- edit is invoked with tag
my_tag, and generates side-effects (like reindexObject) which has same tags
- more methods, which supports
activity_kw are putting tag into activity
- myMethod is waiting in activity queue as long as all messages with tag my_tag validates and then is invoked
Below code snippet:
object.activate(tag='my_tag').edit()
object.activate(after_tag='my_tag').myMethod()
will do:
- edit and myMethod are put into activity queue
- edit is invoked, and generates side-effects
- myMethod is invoked just after edit validates, without waiting for other activities, which might be related to edit.
/!\ More discussion are on mailing list.
How to store the result of an activity¶
It is sometimes interesting to get the result of an activity. There is a special type of object that we can create in the CMFActivity Tool called : Active Process. This object can be used in order to store the result of some activities.
You can create an active process like this:
active_process = context.portal_activities.newActiveProcess()
Then if you want to store the result of an activity on this object, there is the parameter active_process that you can give when you call the activate method.
my_object.activate( active_process = active_process.getPath()).getTitle()
my_object2.activate(active_process = active_process.getPath()).getTitle()
Then, with the parameter active_process, the result will be automatically stored on the active process. Then you can get the list of result like this:
active_process.getResultList()
Here the list of result might be: ['title1','title2']
It might be interesting to start an activity with the result of a previous activity. You can do this by giving the path of the active process to the last activity.
active_process = context.portal_activities.newActiveProcess()
my_object.activate(active_process = active_process.getPath()).getTitle()
my_object2.activate(active_process = active_process.getPath()).getTitle()
context.activate(after_method_id = 'getTitle'
).Base_processResultList(activate_process = active_process.getPath()
)
Like this, when the script Base_processResultList will be called, then we will be sure that all activities getTitle will be finished and the result will be available on the active process. So the script will be able to use it.
What to do if no messages are executed any more¶
One problem may happen with TimerService, it uses gethostbyname(gethostname()) to guess the address of the current node, so if you change your network configuration, you better check the IP address corresponding to your hostname in /etc/hosts
It might happens that some messages are set to processing but the macking looks like to do nothing (for example the load stay very low). In this case you might only restart Zope and then everything should working again.
How to monitor activities in a Linux shell¶
There is script watch_activities available in CMFActivity product bin directory: https://lab.nexedi.com/nexedi/erp5/blob/master/product/CMFActivity/bin/watch_activities.
The result look like this and will be updated every 5 seconds:
+-------------+------------------------+------------+-----------------+
| count(path) | method_id | processing | processing_node |
+-------------+------------------------+------------+-----------------+
| 1 | expand | 0 | -2 |
| 16439 | immediateReindexObject | -1 | -1 |
| 2 | immediateReindexObject | 0 | -2 |
| 192 | immediateReindexObject | 0 | 1 |
| 101 | immediateReindexObject | 1 | 1 |
| 45 | vincent_repairUids | -1 | -1 |
+-------------+------------------------+------------+-----------------+
Related Articles¶