You are here: Home Articles Of babies and bathwater (or: Why I love the Zope Component Architecture)
Navigation
OpenID Log in

 

Of babies and bathwater (or: Why I love the Zope Component Architecture)

by Martin Aspeli last modified Dec 01, 2009 10:30 AM

In defence of one of the great Python frameworks of the past decade.

I went on holiday for four days. When I came back and opened Thunderbird, I saw that there were 97 new emails in the zope-dev newsgroup (mailing list). Controversial debate, here we come.

This particular debate was instigated in reaction to an earlier thread by Chris McDonough, who was trying to bring some BFG API improvements back up to zope.component. I always value Chris' opinions on design and development - few people are as committed to software maintenance, programmer friendliness or high-quality documentation as he and his colleagues at Agendaless are. And even though Chris' proposal didn't quite fit in the end, he inadvertently re-kindled a debate about how we can improve the API of the Zope Component Architecture (ZCA), i.e. zope.interface and zope.component.

Recently, there's been a sense of backlash against the ZCA. It's too hard. It's too Zope-specific. It breaks expectations and has a sub-optimal API. This has sometimes spilled over into a generic mistrust of frameworks. Frameworks are too hard. They're too application-specific. Many of them are too esoteric and break expectations in one way or another. We all just want to write Python code with simple functions and simple data structures and maybe a decorator or two, dammit. None of the complexity that requires us to read documentation and learn new concepts.

Don't get me wrong. Many of these criticism are justified in part. ZCA over-use is a common affliction in some parts (and I shall certainly take my share of the blame). Over-generalisation is the enemy of discoverability. The ZCA, like all inversion-of-control frameworks, encourage separation of concerns to the point where you can't always find the implementation for something without understanding what registrations are currently in effect. Ian Bicking used the term "non-locality" of code, which is definitely something that ZCA espouses - even encourages - and this comes at a cost.

And yet. The ZCA rocks. It's an incredibly advanced and powerful way to build software. When I work in other environments (or other programming languages), I often miss it, and I sometimes end up re-inventing parts of it for my own needs. Zope, Plone, Grok, BFG and the other frameworks that use the ZCA constitute some of the largest and most complex codebases in the Python world. They are also some of the most extensible, customisable products in existence - probably in all of software. The ZCA sits at the heart of that, and has (ignoring a huge amount of legacy code that clouds the picture) facilitated a type of consistency in extensibility that is difficult to emulate.

There is no doubt also that the ZCA adds complexity. It demands that you swallow its core concepts up front (interfaces, adapters, utilities, events) and get to know them pretty well. It has a few nooks where people can do crazy things, usually doing as much harm as good.

But my take on it all is that the "framework backlash" is wrong. It seems to happen when people who've been working on high-complexity software starts building small and simple things, using elegant frameworks like BFG or Pylons. A whole application is just a couple of functions, a few dicts and lists, and maybe a page template or two. Why can't all of our code be like this? But of course it can't. When you start building software that is tens of thousands of lines of code, you need architecture, you need formalisms, you need common patterns and repeatable techniques. You need to cater for teams that change, skills that vary and codebases that get so big no one person can manage it all at once. You need to consider how your code will evolve, because the code you wrote first will be in dire need of refactoring by the time you're half way through your project.

That doesn't mean the ZCA is right for all situations. Rather, I'd argue that learning to use the ZCA effectively is a bit like learning a new programming language on top of Python. You need to "get" those concepts, just like you need to "get" functions, classes, for loops and if statements. You need to gain some experience and learn some common patterns and techniques. In short, you need to invest some time and become familiar with your tools. That investment will pay off for a big project, or a complex one. It probably won't if you're building a small, stand-alone application. And for the huge grey-zone in-between, you're going to have to make informed decisions.

For the ZCA itself, there is clearly an opportunity right now to evolve, and a degree of energy poised to make this happen. My personal wish-list would be the following:

  • Improve the documentation. Chris McDonough pointed out that when you try to explain something, you'll find holes in your concepts. This tutorial is my latest attempt at explaining the ZCA concepts - it may be helpful, either if you'd like to learn more about the ZCA, or if you'd like to try to write canonical ZCA documentation (in which case, feel free to borrow liberally).
  • Document clearly when the ZCA adds value, and when it may be overkill.
  • Improve the API, but don't sacrifice backwards compatibility and don't break existing documentation. We've had a long education process to date, getting people familiar with the ZCA concepts. We need to avoid the kind of damaging mega-confusion that surrounded the whole Zope 2 / Zope 3 / Five debacle. Giving a new name to an existing concept or a new signature to an existing method is the kind of thing you do only to spite your users. Live up to past decisions, even if they weren't optimal, and find a pragmatic way forward rather than wish every effort can be turned into "green fields" development.
  • Build better tools for introspection and debugging. The ZCA maintains a lot of runtime state about components, which could be used more effectively to help people see which registrations are in effect.

And finally, some advice for projects, such as Plone, that use the ZCA for extensibility and inversion-of-control : Don't let the ZCA be the first API that people see, and don't assume everyone has yet reached ZCA mastery. Think of the ZCA as a building block, not the end goal.

BFG does this very well. A particular feature may be implemented using the ZCA, in order to be allow different policies to be swapped in, say. But for the casual user of the framework, there is a helper method which performs the necessary ZCA lookups and presents a domain-specific, documented API for a particular purpose. And it documents its design decisions. That's maturity for you.

We're trying to do something similar with Dexterity and the ecosystem of tools around it. The same will hopefully be true for other technologies as we move towards Plone 5 and what we hope will be a much improved integrator learning curve. Stay with us. The ZCA certainly will.

Document Actions

The Initial Investment

Posted by http://hgeldenhuys.myopenid.com/ at Dec 01, 2009 11:28 AM
The ZCA is a tough nut to crack, especially if your stuck in an old paradigm and your tendency is to look for quick wins or instant gratification -- and Python is the kind of environment, when undisciplined, instant gratification is almost made deal. It's always tempting to look for easy ways to "hack" at frameworks that at first don't make much sense.

I took an entire weekend to work through the Zope Component Architecture basics with five.grok tutorial, resisting the urge to immediately start coding my own software with the little knowledge that I progressively gained by reading the documentation.

It paid off. The initial investment to try and understand how the ZCA works and how five.grok and other tools make working with these frameworks way much easier, paid off.

It took me 2 days of reading and working through tutorials and a whole lot less time writing the actual code that was in my head. The payoff is, I better understand how ZCA works, my code is shorter, more readable and by implication much more maintainable.

I'm sure that I won't HAVE to consult the documentation each and every time I do a new project, so the initial investment in educating myself will pay off even more from here on.

I wrote a small URL and bookmarking service using your tutorial, xdv for skinning and I'm quite impressed with how much painless effort I managed to pull it off. It's still way early in alpha phase, but you can go and have a look if you'd like: http://my.own.io/

Like I said, its very alpha but already makes tiny urls, takes a preview-snapshot of the URL, indexes the tags and title all by just adapting a normal Link type. I only work on this in my spare time.

The tutorial is very thorough and systematic and I appreciate the effort that went into five.grok. So in a way my site was inspired by the framework.

Thanks for the insightful input.

The Initial Investment

Posted by http://hgeldenhuys.myopenid.com/ at Dec 01, 2009 11:30 AM
PS: Here's a snapshot incase you don't want to bother: http://skitch.com/hgeldenhuys/nji5n/yahoo-own.io

The Initial Investment

Posted by http://pigeonflight.myopenid.com/ at Dec 01, 2009 10:50 PM
I like the site, I'd love to learn more about your approach. One question, Isn't Plone a bit on the heavy side for such a service? If you're going to go with the ZCA, Why not Grok or BFG?

The Initial Investment

Posted by Martin Aspeli at Dec 01, 2009 08:31 PM
Hi,

That's very cool! Thanks for sharing your experiences! The site looks really nice too.

Martin

ZCA or PCA ?

Posted by http://openid-provider.appspot.com/luigi.scarso at Dec 02, 2009 05:25 AM
Quoting from
http://www.muthukadan.net/docs/zca.html#getting-started
""Zope Component Architecture (ZCA) is a Python framework for supporting component based design and programming. It is very well suited to developing large Python software systems. The ZCA is not specific to the Zope web application server: it can be used for developing any Python application. Maybe it should be called as Python Component Architecture.""
Sometime I think something like this (CA=ComponentArch.)
1) light CA -- using AbstractBaseClasses ?
2) medium CA -- "something" from ZCA
3) heavy CA -- full ZCA

They are just names, of course, but sometime names are important to avoid confusion.

ZCA instandard lib? Yes, for me.
Is too much ? Maybe.
Maybe ZCA as "recommended/supported framework?" Maybe.
I don't want to touch this argument.


Anyway, many thanks for this post.
Plone Book
Professional Plone 4 Development

I am the author of a book called Professional Plone Development. You can read more about it here.

About this site

This Plone site is kindly hosted by: 

Six Feet Up