DCWorkflow's hidden gems

A while ago, Laurence, Alex and I were sat over a beer thinking about all the parts of Plone we'd like to rip out. Alex asked, "what part of Plone is actually really good". And the first thing that came to (Laurence's) mind was DCWorkflow, better known as the portal_workflow tool.

On my most recent project, I've started appreciating it even more. Our workflow system is actually really cool, and can do a bunch of things that you may not have used, but which may come in handy one day. In general, if you are trying to solve a problem that involves one or more of:

• the state (or status) of a content object
• per-object security
• user-triggered events/check points
• approvals, sign-off or handover

then you should look to solve those using the workflow system.

How well do you know DCWorkflow?

As a refresher, DCWorkflow is a states-and-transitions system. That means that your workflow starts in a particular state (the initial state) and then moves to other states via transitions (also called actions in CMF).

When the workflow enters a particular state (including the initial state), the workflow is given a chance to update permissions on the object. A workflow manages a number of permissions - typically the "core" CMF permissions like View, Modify portal content and so on - and will set those on the object at each state change. Note that this is an event-driven thing, not a real-time security check: only by changing the state is the security information updated. That's why you need to click "Update security settings" at the bottom of the portal_workflow screen in the ZMI when you change your workflows' security settings and want to update existing objects.

A state can also assign local roles to groups. This is akin to assigning roles to groups on Plone's Sharing tab, but the mapping of roles to groups happens on each state change, much like the mapping of roles to permissions. Thus, you can say that in the pending_secondary'state, members of the Secondary reviewers group has the Reviewer local role. This is powerful stuff when combined with the more usual role-to-permission mapping.

State changes result in a number of variables being recorded, such as the actor (the user that invoked the transition), the action (the id of the transition), the date and time and so on. The list of variables is dynamic, so each workflow can define any number of variables linked to TALES expressions that are invoked to calculate the current value at the point of transition. And of course, the workflow keeps track of the current state. The state is exposed as a special type of workflow variable called the state variable. Most workflows in Plone uses the name review_state as the state variable.

Workflow variables are recorded for each state change in the workflow history. This allows you to see when a transition occurred, who effected it, and what state the object was in before or after. In fact, the "current state" of the workflow is internally considered to be the most recent entry in the workflow history.

Workflow variables are also the basis for worklists. They are basically canned queries run against the current state of workflow variables. Plone's review portlet shows all current worklists from all installed workflows. This can be a bit slow, but it does meant that you can use a single portlet to display an amalgamated list of all items on all worklists that apply to the current user. Most Plone workflows have a single worklist that matches on the review_state variable, e.g. showing all items in the pending state.

If states are the static entities in the workflow system, transitions (actions) are dynamic. Each state defines zero or more possible exit transitions, and each transition defines exactly one target state. It is also possible to mark a transition as "stay in current state". This can be useful if you want to do something in reaction to a transition and record that the transition happened in the workflow history, but not change the state (or security) of the object.

Transitions are controlled by one or more guards. These can be permissions (the preferred approach), roles (mostly useful for the Owner role - in other cases it is normally better to use permissions) or TALES expressions. A transition is available if all its guard conditions pass. A transition with no guard conditions is available to everyone (including anonymous!).

Transitions are user-triggered by default, but may be automatic. An automatic transition triggers immediately following another transition provided its guard conditions pass. It will not necessarily trigger as soon as the guard condition becomes true (that would involve continually re-evaluating guards for all active workflows on all objects!), but you can explicitly ask the workflow sytem to check them and trigger them - see below.

When a transition is triggered, the IBeforeTransitionEvent and IAfterTransitionEvent events are triggered. These are low-level events from Products.DCWorkflow that can tell you a lot about the previous and current states. There is a higher level IActionSucceededEvent in Products.CMFCore that is more commonly used to react after a workflow action has completed.

In addition to the events, you can configure workflow scripts. These are either created through-the-web or (more commonly) as External Methods, and may be set to execute before a transition is complete (i.e. before the object enters the target state) or just after it has been completed (the object is in the new state). Note that if you are using event handlers, you'll need to check the event object to find out which transition was invoked, since the events are fired on all transitions. The per-transition scripts are only called for specific transitions.

Workflows are mapped to types via the portal_workflow tool. There is a default workflow, indicated by the string (Default). Some types have no workflow, which means that they hold no state information and typically inherit permissions from their parent. It is also possible for types to have multiple workflows. You can list multiple workflows by separating their names by commas. This is called a workflow chain.

Note that in Plone, the workflow chain of an object is looked up by multi-adapting the object and the workflow to the IWorkflowChain interface. The adapter factory should return a tuple of string workflow names (IWorkflowChain is a specialisation of IReadSequence, i.e. a tuple). The default obviously looks at the mappings in the portal_workflow tool, but it is possible to override the mapping, e.g. in resopnse to some marker interface.

Multiple workflows applied in a single chain always co-exist in time. Typically, you need each workflow in the chain to have different state variables. The standard portal_workflow API (in particular, doActionFor(), which is used to change the state of an object) also asumes the transition ids are unique. If you have two workflows in the chain and both currently have a submit action available, only the first workflow will be transitioned if you do portal_workflow.doActionFor(context, 'submit'). Plone will show all available transitions from all workflows in the current object's chain in the State drop-down, so you do not need to create any custom UI for this. However, note that Plone always assumes the state variable is called review_state (which is also the variable indexed in portal_catalog). Therefore, the states of any secondary workflow probably won't show up unless you build some custom UI.

In terms of security, remember that the role-to-permission (and group-to-local-role) mappings are event-driven and are effected after each transition. If you have two concurrent workflows that manage the same permissions, the settings from the last transition invoked will apply. If they manage different permissions (or there is a partial overlap) then only the permissions managed by the most-recently-invoked workflow will change, leaving the settings for other permissions untouched. See also the note about subtractive workflows below.

Multiple workflows can be very useful in case you have concurrent processes. For example, an object may be published, but require translation. You can track the review state in the main workflow and the translation state in another. If you index the state variable for the second workflow in the catalog (the state variable is always available on the indexable object wrapper so you only need to add an index with the appropriate name to portal_catalog) you can search for all objects pending translation, for example using a Collection. See also the note about reactive workflows below.

Some interesting things I've done lately

All that, you get for free. I've also been working on some additional products around workflow.

Workflows from spreadsheet

When you have complex workflows managing lots of permissions, it can be pretty difficult to get a good overview of what states and transitions you have, how they relate to one another, and what the permission settings are. This is especially true if you're working on the filesystem and making repeatable workflows to be installed via GenericSetup. Like most GenericSetup handlers, the workflow tool handler uses an XML syntax, and managing ten states with ten permissions and a mixture of roles in XML is not fun.

Enter collective.wtf. The name has something to do with how I was feeling at the time. It lets you write your workflows as spreadsheets (which you need to save to CSV format). It tries to remove all the boilerplate to get Plone-standard workflows and avoids as much repetition as possible. I use an OpenOffice template that provides nicely formatted workflows and then save to CSV for the final import.

collective.wtf also provides a number of debugging aids around workflows. For example, you can easily get a CSV view of an currently installed workflows to sanity-check permissions, and there is a view that runs some heuristics on your installed workflows to check against Plone conventions and best practice.

Subtractive workflows

I needed to model a situation where a content object could go through a number of states managing the 'View' permission. In some states, the object would ordinarily be published to anonymous, in other states to authenticated members. However, it also had to be possible to mark an object as "confidential". A confidential item should never, ever be shown to the Anonymous, Authenticated or Member roles, but would otherwise go through the usual workflow.

To avoid having to create a parallel workflow for confidential items, I created collective.subtractiveworkflow. With this workflow, the permissions you chose for a given state are taken away from the selected roles, rather than granted to them (group/local role mappings are unaffected). I use it as a secondary workflow. If the user selects 'mark confidential' from the state drop-down, the subtractive workflow moves to the 'confidential' state, where View is taken away from Anonymous, Authenticated and Member. In the default 'non-confidential' state, no roles are ticked and so no permissions are taken away.

Remember from above that when a workflow manages permissions, only the last state applies. For something like a subtractive workflow, that is a problem, because its very role is to moderate the permissions managed by the previous state. Therefore, collective.subtractiveworkflow installs an event handler that will correctly calculate the "merged" set of permissions across all states for any workflow event that takes place on a multi-workflow chain where at least one element in the chain is subtractive.

Reactive workflows

Remember also that automatic transitions are only checked and potentially invoked immediately following a state change. In fact, they are only invoked after a state change in the same workflow, on the same object.

Laurence Rowe wrote Products.ReactiveWorkflow, which I've stolen and rolled into collective.wtf. It contains two event handlers which are not registered by default, but can be registered in ZCML, or called directly e.g. from a workflow post-transition script, to:

• invoke any available automatic transitions for all workflows on the current object
• invoke any available automatic transitions on the parent object

With these two event handlers installed, you can make complex co-reactive workflows. You could have an automatic transition in one workflow with a guard that is "freed" by a specific action on the second workflow, or you could have a state change in a a child object effect a state change in its parent.

Here is an example: The system I'm working on has an Idea folderish type. When it enters the ready-for-review state, the Review type becomes addable (the workflow manages the add permission). The Review type can be added by any user with the Reviewer role. There are also users with the OfficialReviewer role. A review is added as draft, and can then be transitioned to the complete state by its creator. There is an automatic transition from complete to official protected by a permission only granted to an OfficialReviewer. Thus, when an official reviewer completes his or her review, the review object automatically enters the official state. In the parent idea, there is an automatic transition from ready-for-review to review-completed that is guarded on whether or not there is an official review in the object. With the reactive workflow event handlers, that triggers as soon as the child review object is transitioned.

To have a bit more control in the GUI, you can also use content rules to make these types of checks and trigger automatic or regular transitions in the current or parent object. To help with that, a new product collective.contentrules.parentchild provides a set of conditions and actions that you case.

Smart guards

In the example above, there was a condition "the object contains a Review object in the official review state". This looks like object/@@wf-utils/contains/Review/official. The @@wf-utils view contains a few handlers like this, using traversal logic to receive arguments. I haven't managed to release that as a separate product yet, but if there's demand, I'll do so (or post the code).

Detailed workflow steps portlet

Plone's workflow UI is pretty un-obtrusive, but sometimes you want to push the workflow steps in people's faces. For this project, we need to have a rich text description associated with each workflow state, including HTML and some dynamic elements (mostly links). The same goes for each state, which should be presented as an action link and associated rich text description.

We achieve that using collective.portlet.workflowsteps. This pulls text from the description of the current state and transitions. It allows HTML and simple variable interpolation to enter the current object's URL, the portal root URL and so on. We assign an instance of this portlet as a content-type portlet, making it available on the types where it makes sense without taking up space everywhere.

The True Review portlet

We also have another custom portlet: collective.portlet.truereview. Unlike the standard review portlet, it does not use worklists. Instead, it searches for objects (usually limited by type and review state) where the current user has the Review portal content permission. This means that the review list contains only items that the user can actually review, as opposed to all items in the pending state. Since we use local roles extensively to grant review rights to specific users and groups, this makes the review list a lot more useful. As a bonus, this product also lets you configure collections using "user can review" as a criteria, making it easier to set up custom review lists.

Baton permissions

Finally, not so much a product as a pattern. I call it "baton permission". I've used this in the past where review of an item passes between multiple roles, with one role at a time responsible for the review. In such workflows, all review checkpoint transitions are guarded by the Review portal content permission, and this permission is also managed by the workflow itself. In each state, the workflow controls who is allowed to review the object and transition it to the next state.

Documentation and releases

Hopefully, this post has been useful (if long!). I'd welcome some help from anyone who wants to help turn the generic sections about DCWorkflow into a more comprehensive tutorial. That'd probably require some screenshots, a bit more detail on the UI and GenericSetup formats, and a description of the standard roles, but a lot of the details are outlined above.

Also, the products I've described are released to PyPI. At some point, I will also upload them to plone.org. I'm just saying that because Alex Clark will almost certainly remind me. :-)