Ticket #8629 (closed Feature Request: fixed)

Opened 6 years ago

Last modified 4 years ago

chown and chmod commands might apply to ${buildout:directory}/var/????storage (that is: both var/blobstorage and var/filestorage)

Reported by: grahamperrin Owned by: smcmahon
Priority: blocker Milestone: 4.0
Component: Installers Version:
Keywords: 4.0 blobs Cc:

Description

Considerations

  1.  plone.app.blob in ZEO cluster configuration
  1. the greater security applied to subdirectories of var by the more recent/ experimental versions of the unified installer
  1.  http://dev.plone.org/plone/browser/Installers/UnifiedInstaller/branches/3.1-ex/install.sh#L106 lines 106—108
# user ids for effective user in root installs; ignored in non-root.
EFFECTIVE_USER=plone
ZEO_USER=zeo

Observations

If I'm not mistaken: the defaults associated with point (1) above, at or around first start time, are incompatible with (2) and/or (3).

Suggestions

i) create directory hierarchy var/blobstorage at an appropriate time

— that is, the Short summary of ticket:8495 (but maybe ignore its  first comment)

ii) chown and chmod commands should apply to both var/blobstorage and var/filestorage

The [chown] section of one of my buildout configuration files

[chown]
# This recipe is used to set permissions -- and ownership for root mode installs
recipe = plone.recipe.command
command =
    chmod 600 .installed.cfg
    touch ${buildout:directory}/var/zeoserver/zeoserver.log
    find ${buildout:directory} -type d -name LC_MESSAGES -exec chown -R ${client1:effective-user} \{\} \;
    chown -R ${zeoserver:effective-user} ${buildout:directory}/var/????storage
    chmod 700 ${buildout:directory}/var/????storage
    chown -R ${zeoserver:effective-user} ${buildout:directory}/var/zeoserver
    chmod 700 ${buildout:directory}/var/zeoserver
    chown -R ${client1:effective-user} ${buildout:directory}/var/client?
    chmod 700 ${buildout:directory}/var/client?

— highlight: the four question marks in ????storage

There may be a more graceful approach. I'm experimenting.

Assumptions

a) blobstorage (probably plone.app.blob) remains the preferred option within  Plone improvement proposal 154

b) ultimate acceptance of that proposal

c) no objections (from those who opt to not install plone.app.blob) to the presence of an empty blobstorage within a Plone directory structure.

Components

We might debate that plone.app.blob should be improved in due course — in line with considerations (2) and (3) above. I realise that plone.app.blob is not within Plone core.

In the meantime: if assumption (c) above is agreeable, then one or two non-disruptive improvements to Installer (Unified) could improve the situation for (at least) testers.

Selection from a transcript

[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/startcluster.sh
zeoserver: . daemon process started, pid=17420
This is the first start of this instance.
Creating Data.fs and a Plone site.
We only need to do this once, but it takes some time.
Creating Plone site at /Plone in ZODB...
================================================
atReal : ARFilePreview
================================================
2008-10-24 07:43:53 WARNING ZEO.zrpc (17434) CW: error connecting to ('127.0.0.1', 8100): ECONNREFUSED
2008-10-24 07:43:58 WARNING ZEO.zrpc (17434) CW: error connecting to ('127.0.0.1', 8100): ECONNREFUSED
^CTraceback (most recent call last):
Traceback (most recent call last):
  File "/Applications/Plone/zeocluster/bin/plonectl", line 14, in ?
    plone.recipe.unifiedinstaller.ctl.main(server='zeoserver', clients=['client1', 'client2'], location='/Applications/Plone/zeocluster', binDirectory='/Applications/Plone/zeocluster/bin', fileStorage='/Applications/Plone/zeocluster/var/filestorage/Data.fs')
  File "/Applications/Plone/buildout-cache/eggs/plone.recipe.unifiedinstaller-0.6b4-py2.4.egg/plone/recipe/unifiedinstaller/ctl.py", line 96, in main
    controller.init_storage()
  File "/Applications/Plone/buildout-cache/eggs/plone.recipe.unifiedinstaller-0.6b4-py2.4.egg/plone/recipe/unifiedinstaller/ctl.py", line 61, in init_storage
    "run %s %s" % (os.path.join(self.modulePath, 'mkPloneSite.py'),
  File "/Applications/Plone/buildout-cache/eggs/plone.recipe.unifiedinstaller-0.6b4-py2.4.egg/plone/recipe/unifiedinstaller/ctl.py", line 48, in runCommand
    po.communicate()
  File "/Applications/Plone/Python-2.4/lib/python2.4/subprocess.py", line 1083, in communicate
  File "<string>", line 1, in ?
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/__init__.py", line 51, in app
    startup()
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/__init__.py", line 47, in startup
    _startup()
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/App/startup.py", line 59, in startup
    DB = dbtab.getDatabase('/', is_root=1)
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/Startup/datatypes.py", line 280, in getDatabase
    db = factory.open(name, self.databases)
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/Startup/datatypes.py", line 178, in open
    DB = self.createDB(database_name, databases)
  File "/Applications/Plone/zeocluster/parts/zope2/lib/python/Zope2/Startup/datatypes.py", line 175, in createDB
    return ZODBDatabase.open(self, databases)
  File "/Applications/Plone/buildout-cache/eggs/ZODB3-3.8.1-py2.4-macosx-10.3-i386.egg/ZODB/config.py", line 97, in open
    storage = section.storage.open()
  File "/Applications/Plone/buildout-cache/eggs/ZODB3-3.8.1-py2.4-macosx-10.3-i386.egg/ZODB/config.py", line 168, in open
    realm=self.config.realm)
  File "/Applications/Plone/buildout-cache/eggs/ZODB3-3.8.1-py2.4-macosx-10.3-i386.egg/ZEO/ClientStorage.py", line 347, in __init__
    self._wait(wait_timeout)
    self.wait()
  File "/Applications/Plone/Python-2.4/lib/python2.4/subprocess.py", line 1007, in wait
  File "/Applications/Plone/buildout-cache/eggs/ZODB3-3.8.1-py2.4-macosx-10.3-i386.egg/ZEO/ClientStorage.py", line 362, in _wait
    pid, sts = os.waitpid(self.pid, 0)
    self._rpc_mgr.connect(sync=1)
KeyboardInterrupt
  File "/Applications/Plone/buildout-cache/eggs/ZODB3-3.8.1-py2.4-macosx-10.3-i386.egg/ZEO/zrpc/client.py", line 153, in connect
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22%     self.cond.wait(30)
  File "/Applications/Plone/Python-2.4/lib/python2.4/threading.py", line 222, in wait
    _sleep(delay)
KeyboardInterrupt

[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/zeoserver fg
/Applications/Plone/zeocluster/parts/zeoserver/bin/runzeo
^C[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/startcluster.sh
Password:
zeoserver: . daemon process started, pid=17484
client1: . . daemon process started, pid=17494
client2: . . . daemon process started, pid=17504
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/shutdowncluster.sh 
zeoserver: daemon manager not running
client1: . daemon process stopped
client2: . daemon process stopped
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% ls -l var
total 8
-rw-r--r--  1 root   staff   79  6 Aug 02:38 README.txt
drwx------  4 root   staff  136 24 Oct 07:44 blobstorage
drwx------  7 plone  staff  238 24 Oct 07:51 client1
drwx------  5 plone  staff  170 24 Oct 07:52 client2
drwx------  6 zeo    staff  204 24 Oct 07:50 filestorage
drwx------  3 zeo    staff  102 24 Oct 07:51 zeoserver
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/buildout -c grahamperrin-wip.cfg
Uninstalling products-svn.
Running uninstall recipe.
-------- WARNING --------
Directory /Applications/Plone/zeocluster/parts/products-svn have been removed.
Changes might be lost.
-------- WARNING --------
Updating zope2.
Updating fake eggs
Updating fop.
Updating fop-fixup.
Updating productdistros.
Updating zeoserver.
Updating client1.
Updating client2.
Updating varnish-build.
Updating varnish-instance.
Updating zopepy.
Updating zopeskel.
Updating chown.
chown: Running 
chmod 600 .installed.cfg
touch /Applications/Plone/zeocluster/var/zeoserver/zeoserver.log
find /Applications/Plone/zeocluster -type d -name LC_MESSAGES -exec chown -R plone \{\} \;
chown -R zeo /Applications/Plone/zeocluster/var/????storage
chmod 700 /Applications/Plone/zeocluster/var/????storage
chown -R zeo /Applications/Plone/zeocluster/var/zeoserver
chmod 700 /Applications/Plone/zeocluster/var/zeoserver
chown -R plone /Applications/Plone/zeocluster/var/client?
chmod 700 /Applications/Plone/zeocluster/var/client?
Installing products-svn.
Updating unifiedinstaller.
Updating precompile.
  precompiling python scripts in /Applications/Plone/zeocluster/products
  precompiling python scripts in /Applications/Plone/zeocluster/parts/productdistros
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% ls -l var
total 8
-rw-r--r--  1 root   staff   79  6 Aug 02:38 README.txt
drwx------  4 zeo    staff  136 24 Oct 07:44 blobstorage
drwx------  7 plone  staff  238 24 Oct 07:51 client1
drwx------  5 plone  staff  170 24 Oct 07:52 client2
drwx------  6 zeo    staff  204 24 Oct 07:50 filestorage
drwx------  3 zeo    staff  102 24 Oct 07:51 zeoserver
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/startcluster.sh
zeoserver: . daemon process started, pid=17575
client1: . . daemon process started, pid=17581
client2: . . . daemon process started, pid=17587
[macbookpro03-centrim:/Applications/Plone/zeocluster] gjp22% sudo bin/shutdowncluster.sh
Password:
zeoserver: . daemon process stopped
client1: . daemon process stopped
client2: . daemon process stopped

— abstract: the ultimate buildout allows chown and chmod to become effective on previously missing blobstorage (AFAIR that directory was created when, whilst running zeoserver in foreground, I ran client1 in foreground in a separate Terminal window)

— highlight: ownership of blobstorage is changed from root to zeo

I might attach a buidout configuration file to this ticket.

Change History

comment:1 in reply to: ↑ description Changed 6 years ago by grahamperrin

Replying to grahamperrin:

We might debate that plone.app.blob should be improved in due course — in line with considerations (2) and (3) above.

 http://plone.org/products/plone.app.blob/issues/5 is harmonise plone.app.blob with greater security applied to directories and to Python processes in (for example) ZEO cluster environment.

comment:2 Changed 5 years ago by smcmahon

  • Status changed from new to assigned
  • Keywords 3.4 added

This will probably be deferred until 3.4, when we'll probably have blob filestorage specification support added to plone.recipe.zeoserver and zope2instance. That will give us a standard location for whatever owner/permission fixups the installers need to make.

comment:4 Changed 5 years ago by hannosch

  • Component changed from Installer (Unified) to Installers

comment:5 Changed 5 years ago by hannosch

  • Milestone changed from 3.x to Ongoing

comment:6 Changed 5 years ago by smcmahon

  • Priority changed from minor to blocker
  • Keywords 4.0 added; 3.4 removed
  • Milestone changed from Ongoing to 4.0

With blobs finally going mainstream in 4.0, this is a must for the installers.

comment:7 Changed 5 years ago by witsch

  • Keywords blobs added

comment:8 follow-up: ↓ 9 Changed 4 years ago by kleist

Just for the record: With Plone 4 coredev buildout on Windows 7 (user has administrator role), starting the server with ".\bin\instance.exe fg":

2009-12-01 11:59:29 WARNING ZODB.blob (1756) Blob dir E:\buildout\plone4coredev\var\blobstorage\ has insecure mode setting

comment:9 in reply to: ↑ 8 Changed 4 years ago by smcmahon

Replying to kleist:

Just for the record: With Plone 4 coredev buildout on Windows 7 (user has administrator role), starting the server with ".\bin\instance.exe fg":

2009-12-01 11:59:29 WARNING ZODB.blob (1756) Blob dir E:\buildout\plone4coredev\var\blobstorage\ has insecure mode setting

Development buildouts are not meant to be secure in terms of file system ownership and permissions. This is an issue for installers (and general deployment security), but a non-issue for development buildouts.

comment:10 follow-up: ↓ 11 Changed 4 years ago by grahamperrin

Experimenting: < http://pastebin.ca/1744042>

should be good for Mac OS X 10.6 (Snow Leopard) and for other versions of the OS that support Access Control Lists (ACLs).

comment:11 in reply to: ↑ 10 Changed 4 years ago by grahamperrin

Experimenting a little further, corrections: < http://pastebin.ca/t.php/chown-zeo>.

comment:12 Changed 4 years ago by smcmahon

  • Status changed from assigned to closed
  • Resolution set to fixed

Fixed in changeset #9951.

Note: See TracTickets for help on using tickets.