使用 Loguru 日志库记录导入文件中发出的请求

Question

所以我在 python 文件（main.py）中编写了程序，该文件使用 api 包装文件（bluerev.py）中的类。我想使用 main.py 中的 loguru 记录器来收集程序中的所有异常 + api 包装器中发出的所有请求。因此，bluerev.py api 包装器中设置的日志记录如下所示：

import logging

#Logging setup
logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

class BluerevApiRequestHandler:

def __init__(self):
    self.date_str_format = "%Y-%m-%dT%H:%M:%S.%f"

@staticmethod
def setup_logger_for_requests():
    """
    Sets up the requests library to log api requests
    """
    logger.setLevel(logging.DEBUG)
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.DEBUG)
    requests_log.propagate = True

main.py 日志记录代码如下所示：

from blurev import *
from loguru import logger
#more imports and code

@logger.catch
def main():
    # associated file and credential locations
    gmail_creds_file = "gmail_creds.json"
    revu_creds_file = r"revu_credentials.json"
    token_file = r"revu_refresh_token.json"
    files_location = os.path.join(os.getcwd(), "rev_dist_files")
    review_files_location = os.path.join(os.getcwd(), "files_to_distribute")

    # Set up Logging
    logging_file = r"plan_review_distributor.log"
    #logging_format_str = "%(levelname)s - %(asctime)-15s - %(filename)s - line %(lineno)d --> %(message)s"
    logger.add(os.path.join(files_location, logging_file),  level="WARNING")
    logger.add(os.path.join(files_location, logging_file), filter="blurev", level="DEBUG")

    #call the requests logging function here in main()
   request_handler = BluerevApiRequestHandler()
   request_handler.setup_logger_for_requests()

   #more code

所以我想知道应该如何更改它，以便 main.py 中的函数和代码在 blueev.py 文件中调用的请求被记录到logging_file中。现在他们还没有。

Answer 1

问题在于

loguru

使用与经典

logging

库完全不同的日志记录机制。经典的

logging

库构建了记录器的层次结构，日志记录向上传播到根（请参阅 Python 文档中的 高级日志记录教程 以供参考）。但是

loguru

根本不使用这种层次结构，它的运作方式与它完全脱节。
因此，如果您希望使用经典

logging

库发出的日志最终由

loguru

处理，您必须拦截它们。

这是我的解决方案的最小可重复示例：

# main.py

import logging as classic_logging
import os

from blurev import BluerevApiRequestHandler
from loguru import logger as loguru_logger


@loguru_logger.catch
def main():
    logging_file = r"plan_review_distributor.log"
    loguru_logger.add(os.path.join(".", logging_file), filter=lambda rec: rec["name"] != "blurev", level="WARNING")
    loguru_logger.add(os.path.join(".", logging_file), filter="blurev", level="DEBUG")

    root_logger = classic_logging.getLogger("")
    root_logger.handlers = [InterceptHandler()]  # replace ANY pre-existing handler
    root_logger.setLevel("DEBUG")

    request_handler = BluerevApiRequestHandler()
    request_handler.do_something_using_requests()


########################################################################################################################
# recipe from https://loguru.readthedocs.io/en/stable/overview.html#entirely-compatible-with-standard-logging
class InterceptHandler(classic_logging.Handler):
    def emit(self, record):
        # Get corresponding Loguru level if it exists
        try:
            level = loguru_logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        # Find caller from where originated the logged message
        frame, depth = classic_logging.currentframe(), 2
        while frame.f_code.co_filename == classic_logging.__file__:
            frame = frame.f_back
            depth += 1

        loguru_logger.opt(depth=depth, exception=record.exc_info).log(level, record.getMessage())
########################################################################################################################


if __name__ == "__main__":
    main()

# bluerev.py

import logging as classic_logging

import requests

bluerev_logger = classic_logging.getLogger(__name__)


class BluerevApiRequestHandler:

    def do_something_using_requests(self):
        print(requests.get("http://example.com").text.splitlines()[3])  # expecting: "    <title>Example Domain</title>"
        classic_logging.getLogger("requests.packages.urllib3").warning("warning from requests")  # simulating
        bluerev_logger.debug("debug from bluerev")
        bluerev_logger.critical("critical from bluerev")

这个想法是向经典的根记录器添加一个处理程序，它将日志记录传输到 loguru 记录器。我们不需要

setup_logger_for_requests

，我们只是让日志记录从

"requests.packages.urllib3"

自然地沿着层次结构传播。
我向第一个

loguru

sink 添加了一个过滤器，这样

"blurev"

日志记录就不会被写入文件两次（因为被两个接收器捕获）。

这是我得到的输出：

# stdout

2022-01-07 11:58:02.480 | DEBUG    | urllib3.connectionpool:_new_conn:227 - Starting new HTTP connection (1): example.com:80
2022-01-07 11:58:02.653 | DEBUG    | urllib3.connectionpool:_make_request:452 - http://example.com:80 "GET / HTTP/1.1" 200 648
2022-01-07 11:58:02.654 | WARNING  | blurev:do_something_using_requests:14 - warning from requests
2022-01-07 11:58:02.654 | DEBUG    | blurev:do_something_using_requests:15 - debug from bluerev
2022-01-07 11:58:02.654 | CRITICAL | blurev:do_something_using_requests:16 - critical from bluerev
    <title>Example Domain</title>

# plan_review_distributor.log

2022-01-07 11:58:02.654 | WARNING  | blurev:do_something_using_requests:14 - warning from requests
2022-01-07 11:58:02.654 | DEBUG    | blurev:do_something_using_requests:15 - debug from bluerev
2022-01-07 11:58:02.654 | CRITICAL | blurev:do_something_using_requests:16 - critical from bluerev

请参阅 Loguru 的文档部分“完全兼容标准日志记录”。
另外，你有时写“bluerev”，有时写“bluerev”，最好小心！

Answer 2

只需使用 loguru 设置创建另一个 python 模块：loguru_logger.py:

from loguru import logger
logger.add('Logs/{time:YYYY-MM-DD}_log.json', 
           format = '{time} {level} {message}', 
           level = 'DEBUG', rotation = "1 day", 
           compression = 'zip', 
           serialize = True, 
           retention="90 days")

然后你就可以在其他模块中使用这个模块了：

from loguru_logger import logger
logger.debug('my message')