Ticket #2530 (closed Bug: fixed)

Opened 10 years ago

Last modified 8 years ago

Grossly inefficient member search

Reported by: tesdal Owned by:
Priority: major Milestone: 2.5
Component: Users/Groups Version:
Keywords: Cc:

Description (last modified by hannosch) (diff)

The member search (used for localrole form among other things) is grossly inefficient. It lists all users from the user DB and calls wrapUser which adds member directory if member directory creation is turned on. As the localrole form can be used quite a lot, we should rather find a better way of doing this search.

CMFMember would probably help, as member objects are put in the catalog.

Also, LDAPUserFolder is commonly used for intranets, and shouldn't require too much fiddling to work without doing wrapUser on all the users in the DB.

Change History

comment:1 Changed 9 years ago by mustard

The prefs_users_overview is basically unusable - we have a site with over 10000 members and clicking the user management links slows the whole thing down badly - you have to wait several minutes just to get the initial search form, because it actually does a full searhc for members just to show you the first 20.

The standard users and the memberdata needs to be indexed. I know CMFMember addresses this to an extent, but it also does much more. Don't know if CMFMember is slated for including in future zopes, but this issue needs addressing somehow.

comment:2 Changed 9 years ago by rafrombrc

there is already a PLIP submitted for integrating CMFMember into plone ( http://members.plone.org/products/plone/roadmap/42). it has not been officially approved yet, but the sentiment among developers seems to be that it is a good idea, pending some further improvements to the CMFMember code. i will certainly be working to get CMFMember into shape and will be lobbying hard to get it integrated into plone 2.2.

the default plone member search IS grossly inefficient. the most sensible way to handle this would be to catalog the MemberData objects. however, since CMFMember already does this, and provides a number of other advantages besides, it would be foolish (IMO) to invest time into implementing a catalog-backed search for the default MemberData implementation. development time would be better spent on improving CMFMember so that it can replace the default implementation altogether.

in the meantime, mustard, i suggest you experiment w/ using CMFMember for your site. it provides a tool to migrate your existing members. of course it is a good idea to do this on a test installation and to do heavy testing before considering deployment on a live site.

comment:3 Changed 9 years ago by limi

What we *can* (and *should*) do for 2.1 is to remove the default search from the templates, so it doesn't take 2-3 minutes when you navigate to the page and before you can do anything. When you do a search, it will still take a while, but currently you do *two* searches before you get anything of use.

It should just show the overview with an empty table and wait for a search before it supplies anything.

Ideally, if it's possible to enumerate the number of users in the site with a non-expensive API call, there should be a threshold here - show the search unpopulated if there are more than 100 users, do the default search if there are less. Not sure if this is possible, though - I assume it isn't.

comment:4 Changed 9 years ago by Anonymous User

There certainly exist cheap counting calls.

I need that for PlonePAS, so I'll just do it for Plone. Probably Tues 10 May.

comment:5 Changed 9 years ago by shh

Anything happened on the "large amount of users" detection front?

comment:6 Changed 9 years ago by optilude

  • Status changed from new to closed
  • Resolution set to fixed

I've fixed the template so that it no longer does a search by default. Instead, there is a "Find all" button to find all members.

I'm closing this issue, since there's no followup on the efficient determination of how many members there are so we can have the old behaviour on small sites. This is now more of a feature request (and a fairly marginal one), so let's not waste any time on this 2.1 wise.

Martin

comment:7 Changed 9 years ago by geoff

Let's reassign this to 2.2 rather than closing it -- this is something that should be fixed eventually.

Also, you can get a reasonably efficient count of the # of members via portal_memberdata.getMemberDataContents()member_count?

comment:8 Changed 9 years ago by geoff

  • Status changed from closed to new
  • Resolution fixed deleted

comment:9 Changed 9 years ago by optilude

There was a similar member search appearing on prefs_group_members.pt, now converted to a form controller template and made to use the same no-search/FindAll pattern. Note that there are still search-on-load happening in both the group-listings (groups overview, user group memberships), though the number of groups should be smaller. Additionally, on the group membership template, all members of the group are shown automatically, no search interface. Fixing this is fairly easy, but the templates are messy and poorly generalised, so it's a bit painful. :)

comment:10 Changed 8 years ago by hannosch

  • Component changed from Infrastructure to Users/Groups
  • Description modified (diff)

comment:11 Changed 8 years ago by limi

In Plone 2.1, this is mostly fixed, apart from the Sharing screen, it seems. plone.org suffers quite a bit from this, and I'd like to see it made more efficient (drop the wrapUser call) for 2.1.x if we can.

comment:12 Changed 8 years ago by hannosch

  • Milestone changed from 2.1.x to 2.1.3

comment:13 Changed 8 years ago by wichert

(In [9041]) Rewire so we do not list all groups by default but allow searching for them. Also move the user search results down into under the user settings header and add a header for the advanced settings to make things a bit more logical. refs #2530

comment:14 Changed 8 years ago by wichert

I fixed this for the group membership management templates in PlonePAS as well; once PlonePAS has been merged we can close this issue.

comment:15 Changed 8 years ago by hannosch

  • Status changed from new to closed
  • Resolution set to fixed
  • Milestone changed from 2.1.x to 2.5

PlonePAS is merged, we have only search forms now.

Note: See TracTickets for help on using tickets.