防止 django 管理员在列表表单上运行 SELECT COUNT(*)

问题描述 投票：0回答：9

每次我使用 Admin 列出模型的条目时，Admin 都会计算表中的行数。更糟糕的是，即使您正在过滤查询，它似乎也会这样做。

例如，如果我只想显示 id 为 123、456、789 的模型，我可以这样做：

/admin/myapp/mymodel/?id__in=123,456,789

但是运行的查询（除其他外）是：

SELECT COUNT(*) FROM `myapp_mymodel` WHERE `myapp_mymodel`.`id` IN (123, 456, 789) # okay
SELECT COUNT(*) FROM `myapp_mymodel` # why???

这正在杀死mysql+innodb。似乎问题在这张票中得到了部分承认，但我的问题似乎更具体，因为它计算了所有行，即使它不应该这样做。 有没有办法禁用全局行计数？

注意：我使用的是 django 1.2.7。

django django-models django-admin
9个回答
32
投票
show_full_result_count = False

来禁用此功能。

https://docs.djangoproject.com/en/stable/ref/contrib/admin/#django.contrib.admin.ModelAdmin.show_full_result_count

    




    

    
    
    

    

        

            

                

                    
25
投票

                    

                

            

        

        
count
 属性，并且可以通过使用自定义查询集（如
这篇文章
中所示）覆盖它来完成，该查询集专门使用近似等效的计数：

from django.db import connections, models
from django.db.models.query import QuerySet

class ApproxCountQuerySet(QuerySet):
    """Counting all rows is very expensive on large Innodb tables. This
    is a replacement for QuerySet that returns an approximation if count()
    is called with no additional constraints. In all other cases it should
    behave exactly as QuerySet.

    Only works with MySQL. Behaves normally for all other engines.
    """

    def count(self):
        # Code from django/db/models/query.py

        if self._result_cache is not None and not self._iter:
            return len(self._result_cache)

        is_mysql = 'mysql' in connections[self.db].client.executable_name.lower()

        query = self.query
        if (is_mysql and not query.where and
                query.high_mark is None and
                query.low_mark == 0 and
                not query.select and
                not query.group_by and
                not query.having and
                not query.distinct):
            # If query has no constraints, we would be simply doing
            # "SELECT COUNT(*) FROM foo". Monkey patch so the we
            # get an approximation instead.
            cursor = connections[self.db].cursor()
            cursor.execute("SHOW TABLE STATUS LIKE %s",
                    (self.model._meta.db_table,))
            return cursor.fetchall()[0][4]
        else:
            return self.query.get_count(using=self.db)




然后在管理中：



class MyAdmin(admin.ModelAdmin):

    def queryset(self, request):
        qs = super(MyAdmin, self).queryset(request)
        return qs._clone(klass=ApproxCountQuerySet)




近似函数可能会弄乱第 100000 页上的内容，但对于我的情况来说已经足够了。

    





    

    
    
    

    

        

            

                

                    
12
投票

                    
                

            

        

        



这是 pg 版本。



class ApproxCountPgQuerySet(models.query.QuerySet):
  """approximate unconstrained count(*) with reltuples from pg_class"""

  def count(self):
      if self._result_cache is not None and not self._iter:
          return len(self._result_cache)

      if hasattr(connections[self.db].client.connection, 'pg_version'):
          query = self.query
          if (not query.where and query.high_mark is None and query.low_mark == 0 and
              not query.select and not query.group_by and not query.having and not query.distinct):
              # If query has no constraints, we would be simply doing
              # "SELECT COUNT(*) FROM foo". Monkey patch so the we get an approximation instead.
              parts = [p.strip('"') for p in self.model._meta.db_table.split('.')]
              cursor = connections[self.db].cursor()
              if len(parts) == 1:
                  cursor.execute("select reltuples::bigint FROM pg_class WHERE relname = %s", parts)
              else:
                  cursor.execute("select reltuples::bigint FROM pg_class c JOIN pg_namespace n on (c.relnamespace = n.oid) WHERE n.nspname = %s AND c.relname = %s", parts)
          return cursor.fetchall()[0][0]
      return self.query.get_count(using=self.db)


    



    

    
    
    

    

        

            

                

                    
6
投票

                    
                

            

        

        



class MyAdmin(admin.ModelAdmin):

    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        return qs._clone(klass=ApproxCountQuerySet)


    



    

    
    
    

    

        

            

                

                    
4
投票

                    
                

            

        

        



查看 1.3.1 安装的代码，我发现管理代码正在使用 
get_paginator()
 返回的分页器。默认的分页器类似乎位于 
django/core/paginator.py
 中。该类有一个名为 
_count
 的私有值，它在 
Paginator._get_count()
 中设置（我的副本中的第 120 行）。这又用于设置 Paginator 类的一个名为 
count
 的 property。我认为
_get_count()
就是你的目标。现在舞台已经搭建好了。


您有几个选择：





    直接修改源码。我确实
  1. 不
    推荐这样做，但由于您似乎被困在 1.2.7 中，您可能会发现这是最方便的。 记住记录此更改！未来的维护者（可能包括你自己）会感谢您的提醒。

    2. Monkeypatch 班级。这比直接修改更好，因为 a) 如果您不喜欢更改，只需注释掉 Monkeypatch，b) 它更有可能与 Django 的未来版本一起使用。我有一个可以追溯到 4 年前的猴子补丁，因为他们仍然没有修复模板变量 
  2. _resolve_lookup()
     代码中的错误，该错误无法在评估的顶层识别可调用对象，只能在较低级别识别可调用对象。尽管补丁（包装类的方法）是针对 0.97-pre 编写的，但它仍然可以在 1.3.1 上运行。
    

    


    3. 我没有花时间确切地弄清楚你必须为你的问题做出哪些改变，但它可能是沿着将 
_approx_count
 成员添加到适当的类 
class META
 然后测试以查看该属性是否存在。如果是并且是 
None
，那么您执行 
sql.count()
 并设置它。如果您位于（或接近）列表的最后一页，您可能还需要重置它。如果您需要更多帮助，请联系我；我的电子邮件在我的个人资料中。
    





    

    
    
    

    

        

            

                

                    
4
投票

                    
                

            

        

        
更改管理类使用的默认分页器
。这是在短时间内缓存结果的一个：https://gist.github.com/e4c5/6852723
    




    

    
    
    

    

        

            

                

                    
4
投票

                    
                

            

        

        



enter image description here

使用的技巧是从数据库中获取
per_page + 1
元素，以查看是否有更多元素，然后提供假计数。


假设我们想要第三页，该页面有 25 个元素 => 我们想要 
object_list[50:75]
。当调用 
Paginator.count
 时，查询集将被评估为 
object_list[50:76]
（请注意，我们采用 
75+1
 元素），然后如果我们从 db 获得 25+1 个元素，则返回计数为 76，或者返回 50 + 元素数量如果我们没有收到 26 个元素，则已收到。 




TL；博士：
我为 
ModelAdmin
:
 创建了一个 mixin


from django.core.paginator import Paginator
from django.utils.functional import cached_property


class FastCountPaginator(Paginator):
    """A faster paginator implementation than the Paginator. Paginator is slow
    mainly because QuerySet.count() is expensive on large queries.

    The idea is to use the requested page to generate a 'fake' count. In
    order to see if the page is the final one  it queries n+1 elements
    from db then reports the count as page_number * per_page + received_elements.
    """

    use_fast_pagination = True

    def __init__(self, page_number, *args, **kwargs):
        self.page_number = page_number
        super(FastCountPaginator, self).__init__(*args, **kwargs)

    @cached_property
    def count(self):
        # Populate the object list when count is called. As this is a cached property,
        # it will be called only once per instance
        return self.populate_object_list()

    def page(self, page_number):
        """Return a Page object for the given 1-based page number."""
        page_number = self.validate_number(page_number)
        return self._get_page(self.object_list, page_number, self)

    def populate_object_list(self):
        # converts queryset object_list to a list and return the number of elements until there
        # the trick is to get per_page elements + 1 in order to see if the next page exists.
        bottom = self.page_number * self.per_page
        # get one more object than needed to see if we should show next page
        top = bottom + self.per_page + 1
        object_list = list(self.object_list[bottom:top])
        # not the last page
        if len(object_list) == self.per_page + 1:
            object_list = object_list[:-1]
        else:
            top = bottom + len(object_list)
        self.object_list = object_list
        return top


class ModelAdminFastPaginationMixin:
    show_full_result_count = False  # prevents root_queryset.count() call

    def changelist_view(self, request, extra_context=None):
        # strip count_all query parameter from the request before it is processed
        # this allows all links to be generated like this parameter was not present and without raising errors
        request.GET = request.GET.copy()
        request.GET.paginator_count_all = request.GET.pop('count_all', False)

        return super().changelist_view(request, extra_context)

    def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
        # use the normal paginator if we want to count all the ads
        if hasattr(request.GET, 'paginator_count_all') and request.GET.paginator_count_all:
            return Paginator(queryset, per_page, orphans, allow_empty_first_page)
        page = self._validate_page_number(request.GET.get('p', '0'))
        return FastCountPaginator(page, queryset, per_page, orphans, allow_empty_first_page)

    def _validate_page_number(self, number):
        # taken from Paginator.validate_number and adjusted
        try:
            if isinstance(number, float) and not number.is_integer():
                raise ValueError
            number = int(number)
        except (TypeError, ValueError):
            return 0
        if number < 1:
            number = 0
        return number




pagination.html
模板：


{% if cl and cl.paginator and cl.paginator.use_fast_pagination %}
    {# Fast paginator with only next button and show the total number of results#}
    {% load admin_list %}
    {% load i18n %}
    {% load admin_templatetags %}
    <p class="paginator">
        {% if pagination_required %}
            {% for i in page_range %}
                {% if forloop.last %}
                    {% fast_paginator_number cl i 'Next' %}
                {% else %}
                    {% fast_paginator_number cl i %}
                {% endif %}
            {% endfor %}
        {% endif %}
        {% show_count_all_link cl "showall" %}
    </p>
{% else %}
    {#  use the default pagination template if we are not using the FastPaginator  #}
    {% include "admin/pagination.html" %}
{% endif %}





以及使用的模板标签：



from django import template
from django.contrib.admin.views.main import PAGE_VAR
from django.utils.html import format_html
from django.utils.safestring import mark_safe

register = template.Library()

DOT = '.'


@register.simple_tag
def fast_paginator_number(cl, i, text_display=None):
    """Generate an individual page index link in a paginated list.

    Allows to change the link text by setting text_display
    """
    if i == DOT:
        return '… '
    elif i == cl.page_num:
        return format_html('<span class="this-page">{}</span> ', i + 1)
    else:
        return format_html(
            '<a href="{}"{}>{}</a> ',
            cl.get_query_string({PAGE_VAR: i}),
            mark_safe(' class="end"' if i == cl.paginator.num_pages - 1 else ''),
            text_display if text_display else i + 1,
        )


@register.simple_tag
def show_count_all_link(cl, css_class='', text_display='Show the total number of results'):
    """Generate a button that toggles between FastPaginator and the normal
    Paginator."""
    return format_html(
        '<a href="{}"{}>{}</a> ',
        cl.get_query_string({PAGE_VAR: cl.page_num, 'count_all': True}),
        mark_safe(f' class="{css_class}"' if css_class else ''),
        text_display,
    )





您可以这样使用它：



class MyVeryLargeModelAdmin(ModelAdminFastPaginationMixin, admin.ModelAdmin):
# ...






或者更简单的版本，不显示 
Next
 按钮和 显示结果总数 :

from django.core.paginator import Paginator
from django.utils.functional import cached_property


class FastCountPaginator(Paginator):
    """A faster paginator implementation than the Paginator. Paginator is slow
    mainly because QuerySet.count() is expensive on large queries.

    The idea is to use the requested page to generate a 'fake' count. In
    order to see if the page is the final one  it queries n+1 elements
    from db then reports the count as page_number * per_page + received_elements.
    """

    use_fast_pagination = True

    def __init__(self, page_number, *args, **kwargs):
        self.page_number = page_number
        super(FastCountPaginator, self).__init__(*args, **kwargs)

    @cached_property
    def count(self):
        # Populate the object list when count is called. As this is a cached property,
        # it will be called only once per instance
        return self.populate_object_list()

    def page(self, page_number):
        """Return a Page object for the given 1-based page number."""
        page_number = self.validate_number(page_number)
        return self._get_page(self.object_list, page_number, self)

    def populate_object_list(self):
        # converts queryset object_list to a list and return the number of elements until there
        # the trick is to get per_page elements + 1 in order to see if the next page exists.
        bottom = self.page_number * self.per_page
        # get one more object than needed to see if we should show next page
        top = bottom + self.per_page + 1
        object_list = list(self.object_list[bottom:top])
        # not the last page
        if len(object_list) == self.per_page + 1:
            object_list = object_list[:-1]
        else:
            top = bottom + len(object_list)
        self.object_list = object_list
        return top


class ModelAdminFastPaginationMixin:
    show_full_result_count = False  # prevents root_queryset.count() call

    def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
        page = self._validate_page_number(request.GET.get('p', '0'))
        return FastCountPaginator(page, queryset, per_page, orphans, allow_empty_first_page)

    def _validate_page_number(self, number):
        # taken from Paginator.validate_number and adjusted
        try:
            if isinstance(number, float) and not number.is_integer():
                raise ValueError
            number = int(number)
        except (TypeError, ValueError):
            return 0
        if number < 1:
            number = 0
        return number



    



    

    
    
    

    

        

            

                

                    
0
投票

                    
                

            

        

        


class HyperTablePaginator(Paginator):
    @cached_property
    def count(self):
        query = self.object_list.query
        if query.where:
            return super().count

        with connection.cursor() as cursor:
            cursor.execute("SELECT * FROM approximate_row_count(%s);", [query.model._meta.db_table])
            return int(cursor.fetchone()[0])


    



    

    
    
    

    

        

            

                

                    
0
投票

                    
                

            

        

        


pip 安装 django-lazy-admin-pagination 

    

	

    

    


  

  

    
最新问题


  





  

    © www.soinside.com 2019 - 2024. All rights reserved.