portal factory causes zodb writes ( compounded by kss )

[hazmat]

Content created in a portal factory generates spurious writes to portal and reference catalogs. Moreover with KSS inline validation enabled (default), these spurious writes happen everytime a user tabs through fields in a form. Also as these spurious writes are to centralized application hotspots, this behavior greatly increases the probability of WriteConflictErrors.

Packing removes these, but clearly Plone should never be writing to the database for portal factory content.

I came upon this issue, while digging through an application where the data.fs has grown to large for packing in reasonable time, and found approximately 25% of the zodb transactions are due to portal_factory writes.

I think the appropriate solution is to make archetypes portal factory aware, and to shortcircuit indexing methods (indexObject) and Referenceable.manage_afterAdd if the content is portal factory contained... ie. aq_parent( object ).class.name == 'TempFolder'

[hazmat]

actually it was more like 40% of the total zodb transactions, each of those being about a 1mb. insanity.. every tab through an add form adding a 1mb to your zodb..

[hazmat]

the attached file, seems to mitigate the issue for me.

[hazmat]

patch to for portal factory sanity ( redux sans debug w/ fix )

[jonstahl]

Hanno, Eric-

This seems like something to consider for 4.0 release, too.

[matth]

Another approach to this same problem was put forward 2 years ago in http://svn.plone.org/svn/collective/experimental.contentcreation|experimental.contentcreation. Your patch is much smaller though ;) But I think experimental.contentcreation does some other things too.

[hannosch]

What Kapil describes here, happens with experimental.contentcreation installed as well. And it also happens in Plone 4.0 where an extended version of "e.contentcreation" is integrated into Plone itself.

[hazmat]

One other data point regarding this ticket, even with the patch, if your using CacheSetup (CacheFoo) it will separately generate writes to the portal cache tool ( via its monkey patch of manage_permissions), adding roughly 443 bytes per portal factory request ( typically multiplied by a number of kss inline validation requests ).

As noted by hanno, this patch is still useful with experimental.contentcreation. The main effect of experimental.contentcreation is to reduce the amount of *redundant* reindexing in a given request, which in turn means for a portal factory request turning 4 index ops into 1. The effect of this patch is to attempt to generate zero writes to the zodb, during portal factory requests.

[maurits]

I merged the patch in archetypes changeset 11950.

hazmat: can you explain this comment in your patch? "It does potentially cause breakage for reference browser usage." I don't immediately spot problems.

Tests still pass. Adding a print statement next to the return statements from the patch gives over 70 print lines when adding a page, so seems like a useful fix indeed. Note that in plonenext 3.3 there is one test failure when you use Products.Archetypes trunk; fix that by 'bin/develop Products.validation'.

Shall I merge this to Plone 4 as well?

[gotcha]

Kapil, do you see any need to fix sthing in archetypes.kss as well ? Or is your patch sufficient ?

IOW, do you imply that archetypes.kss is also triggering useless code on content instances outside portal_factory ?

[maurits]

Merged to Archetypes trunk in archetypes changeset 12056.