Let me start off by saying that I'm not a Plone developer. I've got a
lot of experience with Zope 2 development though, and I keep an eye on
the Plone community. I hope the Plone developers will not mind a
relative outsider's observations and suggestions. I understand that many of the people developing with Plone, or
learning Plone, are frustrated. Now this is nothing unusual - I think
fundamentally it's pretty difficult to extend and customize such a
feature-rich system. We should ever strive to move forward however, and the situation is
clearly worse than it could be: Plone presents many different ways of
doing things to the developer. This is largely a result of its
evolution over time. Things a Plone developer has to deal with are
Zope 2, the CMF, various Zope 3 technologies, the Five integration
layer, and many technologies the Plone developers created. I'm responsible for creating the Five project some years ago (many
others did a lot too), bringing Zope 3 technology to the Zope 2
platform. This ultimately led to the introduction of Zope 3
technologies into Plone. This has been, I take it, a mixed blessing. I think Zope 3 technologies have undoubtedly given Plone developers
new abilities to create powerful, pluggable features for Plone. That's
great. It's also allowed Plone to adopt well-engineered Zope 3
libraries. It has also aligned the interests of the Plone community
better with those of the Zope 3 community, and the Python community as
a whole. This led to us all working together better; a not to be
underestimated benefit in open source development. As a result, Plone
developers are now working on code that is useful outside of Plone
(for instance with Grok), and similarly Zope 3 developers are working
on code useful in Plone. On the other hand, the addition of Zope 3 technology adds yet another
way to do it for those people who want to extend the customize
Plone. It's yet another set of concepts to learn about. What's worse,
these concepts interact in "interesting" ways with older bits that are
already there. This results in frustration and head-scratching. In addition, when given a powerful tool for creating pluggable
frameworks, it's easy for a developer to go overboard, and create
something powerful, very pluggable, but not very agile. That's not
directly the fault of the tool itself. Building a good framework and
using tools well is a learning process. But in the mean time this is
results in yet more complexity for people to deal with. Why can't there be a simple, uniform way of extending and customizing
Plone? In fact, the Zope 3 developers created at least the basis of such a
uniform system. It's been informed by history: it built on lessons
learned with Zope 2 and CMF. This uniform system for customization and
extending software is called the Zope 3 component architecture. In
addition, there's an explicit, and valuable, notion of configuration
that supports extending and customization of software by changing the
configuration. Plone is a victim of its success. It's been around for a while. Plone
is therefore a piece of software with a history. There are many useful
features in Plone that use the older ways of doing things. The
community has finite resources to rewrite these. It's generally not a
good idea to start rewriting everything at once, either. And if you
manage to rewrite something, you'll have people complain about
breaking backwards compatibility... Still, Plone is being rewritten, step by step, to make use of Zope 3's
component and configuration technology. Misteps are made. People are
frustrated. But I do believe Plone is fundamentally going in the right
direction with this. The Plone developers have been working for
years on creating a uniform system for customizing and extending
Plone, building on Zope 3 foundations. Ironically, but entirely
understandably, this effort is also a source of frustration. Why can't this component and configuration system "Just Be Python"? Zope 3 is far from perfect. Traditionally, configuration is done with
Zope 3's ZCML; the bit that's making life harder for many people
working with Plone today. I can understand this frustration very well
myself, as I've felt it when developing with Zope 3. Now I'm not someone who is fundamentally against the notion of
domain-specific configuration languages. As I indicated above, I think
there is a lot of value in having a notion of explicit configuration
in your system. ZCML and Zope 3's configuration system are valuable.
So I understood why ZCML was there. I even defended it - you can
dislike it because it's an XML dialect, but that's really superficial. Then I worked with it. I learned over time that ZCML is one of the
aspects of Zope 3 that cuts down on my agility as a developer. Not
because it's XML, but because it's separate. I have to do too many
mental context switches - Python code to ZCML and back. If I forget to
write a bit of ZCML to hook things up, which is easy to do, things
won't work. I don't see the complete picture when reading the code,
and I don't see the complete picture when reading ZCML. I concluded it
didn't fit my brain. I think it fits the brains of some people, but I
also think there are quite a lot of people like me. ZCML wasn't the only thing that made Zope 3 harder for me to work with
than I thought it should be. Other bits are its very explicit and
invasive security system (but that doesn't bother a Zope 2 developer
already, as it's not in Five), and the tendency (partially culturally
and partially encouraged by Zope 3's security system) to write a lot
of explicit interfaces before they are really needed. I see a lot of
value in the explicit interfaces that zope.interface supports, but
I typically want to add them to my code gradually as I evolve it. I
don't want to feel the pressure to write them up front. My response to these issues back in the summer of 2006, was to start
thinking about Grok. In the fall of 2006, now two years ago, some of
us got together and fleshed out the design of Grok into more detail,
then started building it. Along with improving the technical aspects
of the Zope 3 experience, we also wanted to take a fresh look at the
way the Zope 3 community works and how it presents itself. We got
quite far in two years. The Grok technologies we developed are now finding their way back into
Zope 2. The process started with the extraction of the
grokcore.component library from Grok. grokcore.component works
out of the box in Zope 2. Seeing this, Zope 2 developers then started
working on the five.grok project, bringing back more Grok technologies
to Zope 2. There has been a nice synergy between the efforts to bring Grok
technology to Zope 2, and the efforts to integrate Grok technology in
non-Grok ("legacy" :) Zope 3 applications. The libraries we've been
extracting for reuse in Zope 2 are reusable in Zope 3 a well, and vice
versa, with only a little bit of extra effort. Both motivations (use
in Zope 2, use in Zope 3) nicely drive development in the same
direction. It is, I think, a nice example of how our larger community
can work together, something the Five project has helped make
possible. I hope Grok technologies makes their way into Plone. I think it can
help the developer's experience a lot. Why should Plone developers believe a guy who is partially responsible
for bringing the whole ZCML mess to you in the first place? Above I
make the case that the Five project brought a lot of value to us, not
just frustration. And the Five project allows us to share our
frustrations in a larger community: although I'm not a Plone
developer, I've shared some of your frustrations, and helped start
work on a solution: Grok. Grok is here today. The technology is there for the taking. It's essentially "Just Python": you do not write ZCML, and you don't even have to write much more than the Python code you'd need to write anyway - it just gets configured for you. Frustrated with the Zope 3 component and configuration system? Grok technology is
designed to make those things easier! People have a legitimate fear it will result in yet another layer they
will have to learn about, and have to deal with during
development. This will be true in part, as ZCML isn't going to go away
right away everywhere. This means, I fear, people will need to know
about it at least a bit. But at least by adopting Grok technology
you'd be moving forward, in backwards compatible fashion. You'd keep
the good things that Zope 3 already offers while getting rid of some
of the bad aspects of it. Grok, as a sub-community, cares intensely about the developer
experience. We care about DRY: don't repeat yourself. We care about
bringing a smooth learning curve towards the advanced technologies of
Zope 3, while not breaking compatibility with it. We try to make our
"new layer" as thin as possible, choosing to replace an existing layer
instead of piling yet another new one on top of it. We're about making
the easy things easy; thanks to Zope 3, the hard things are already
possible and we do not take away from this power. I hope the Plone project will align its interests with those of the
Grok project. I admit am self-interested; I think adoption of Grok
technology in Plone would be of benefit to the Grok project at least
as much as it would be to the Plone project. But that's what good open
source development is about: aligning interests. Thanks for listening!
![[Comments]](/blog-content//themes/Default/comments.png "6 comments")
(6) Sat Oct 18 2008 18:36 Why I think Grok can make Plone easier:
- Comments:
Posted by Alexander Limi at Sat Oct 18 2008 21:07
Hi Martijn,Thanks for writing this article, I think most people in the Plone community are interested in Grok and Grok-style development patterns — and what they can do for the developer experience.The good news is that we are actively focusing on reducing the stack with Plone 4. Moving Grok closer to Plone still needs a concerted push, though. Having some sort of weekend sprint (since a lot of the developers live not too far from where you are) might be a good first step.I know that Martin (Aspeli) is very interested in bringing Grok to Plone, as are most of our other developers. There's also interest from the Grok side — Luciano Ramalho mentioned that he is really interested in seeing it happen.As often happens, I don't think there's a fundamental disconnect or disagreement about whether Grok is right for Plone — it's mostly getting the time to do it, and the oversight to do it properly. I believe everyone would be much more comfortable if you were involved too, so it follows the Grok philosophy. :)Great to see this being talked about!
Posted by Martijn Faassen at Sat Oct 18 2008 22:49
Hey Alexander,Actually there has been quite a push to move Grok to Zope 2 in recent months, so things are in good shape there, and I've seen Martin work on it too.I wasn't suggesting there was a disconnect - I was writing this down as I saw some people who hadn't heard of Grok or might not understand what it was about if they did.I have deliberately stayed away from Grok in Zope 2 myself actually - I feel I've done my duty to Zope 2 for a while. :) But from a bit more distance I am interested in the process, and Plone, nonetheless.
Posted by Martin Aspeli at Sun Oct 19 2008 01:10
Hi Martijn,Thanks for writing this! I've said the same things before, not at least because Grok seems to be where the majority of innovation in Zope is happening currently.However, there are three charges that the advocates of Grok-in-Plone would need to counter: - Not enough people are using Grok, i.e. it's yet another esoteric technology choice - It's too much magic (or too hard to debug when things don't do what you'd expect) - It's yet another way of doing things that can't completely displace the existing waysI'd be interested to hear your responses to these. :-)Martin
Posted by Paul McGavin at Sun Oct 19 2008 04:15
Hi, Martijn.I am new to Grok, but a veteran of Zope2 and Plone 2.x development. I have also built some interesting WSGI web apps using Pylons and Django. I have a interest in bringing Grok closer to the Plone world and would be happy to help out in any way that makes sense. As I continue my immersion into the Zope 3/Grok code over the next several weeks, I will float a few ideas of how I might be able to help. Thanks for your work in making Zope3 more approachable to the greater Python and Plone communities.
Posted by Jean-Paul Ladage at Sun Oct 19 2008 14:00
Hi Martijn,Thank you for this article. I just would like to express my interest in
getting Grok closer to Plone.At Zest Software we have been looking at the five.grok for a time tracking
tool we included in our eXtreme Management tool, which is build on top of
Plone. But at that time five.grok was 2 weeks old and security didn't seem to
work ;)Although it was easy to create the basic features for XM using content types
in Plone, we now all agree that for a project management tool we get way to
much with Plone. Actually we have multiple Apps that are built on top of Plone
like PloneHRM, Poi, etc. We are currently moving towards building a set of
smaller tools that are lean and better focused on the tasks at hand. Grok
really provides the agility to do that.Since completely rewriting any plone based app from scratch would be quite an
effort. Gradually moving peaces to grok to me seems like a sane strategy. So
me and a few other Zesters would like to assist at a sprint. I know you have
done more than enough for the Zope2 community, but we could use a little
guidance.Kind regards,Jean-Paul LadagePosted by Martijn Faassen at Sun Oct 19 2008 18:15
Concerning a sprint: if people are organizing a sprint on this topic and it's somewhere I can get to, I'll of course happily attend!
