6

我正在尝试使用TimedRotatingFileHandler将每日日志保存在单独的日志文件中。旋转按预期完美地工作,但我不喜欢它的工作方式是文件的命名。

如果我将日志文件设置为 my_log_file.log,这将是“今天的”日志文件,当它在午夜更改一天时,它将被重命名为my_log_file.log.2014-07-08最后没有 .log 扩展名,并且my_log_file.log将为新的一天。

我想得到的是旧文件被重命名为my_log_file.2014-07-08.logor even my_log_file-2014-07-08.log,主要是 .log 在最后,而不是在中间。另外,我希望“今天的”日志文件已经用今天的日期命名,就像旧的一样。

有什么办法吗?

我发现我可以通过以下方式个性化后缀:

handler.suffix = "%Y-%m-%d"

但是我没有办法删除内部 .log 部分并强制当前日志文件添加后缀。

4

4 回答 4

5

I have created a class ParallelTimedRotatingFileHandler mainly aimed at allowing multiple processes writing in parallel to a log file. The problems with parallel processes solved by this class, are:

  • The rollover moment when all processes are trying to copy or rename the same file at the same time, gives errors.
  • The solution for this problem was exactly the naming convention you suggest. So, for a file name Service that you supply in the handler, logging does not go to e.g. Service.log but today to Service.2014-08-18.log and tomorrow Service.2014-08-19.log.
  • Another solution is to open the files in a (append) mode instead of w to allow parallel writes.
  • Deleting the backup files also needs to be done with caution as multiple parallel processes are deleting the same files at the same time.
  • This implementation does not take into account leap seconds (which is not a problem for Unix). In other OS, it might still be 30/6/2008 23:59:60 at the rollover moment, so the date has not changed, so, we take the same file name as yesterday.
  • I know that the standard Python recommendation is that the logging module is not foreseen for parallel processes, and I should use SocketHandler, but at least in my environment, this works.

The code is just a slight variation of the code in the standard Python handlers.py module. Of course copyright to the copyright holders.

Here is the code:

import logging
import logging.handlers
import os
import time
import re

class ParallelTimedRotatingFileHandler(logging.handlers.TimedRotatingFileHandler):
    def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, postfix = ".log"):

        self.origFileName = filename
        self.when = when.upper()
        self.interval = interval
        self.backupCount = backupCount
        self.utc = utc
        self.postfix = postfix

        if self.when == 'S':
            self.interval = 1 # one second
            self.suffix = "%Y-%m-%d_%H-%M-%S"
            self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$"
        elif self.when == 'M':
            self.interval = 60 # one minute
            self.suffix = "%Y-%m-%d_%H-%M"
            self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}$"
        elif self.when == 'H':
            self.interval = 60 * 60 # one hour
            self.suffix = "%Y-%m-%d_%H"
            self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}$"
        elif self.when == 'D' or self.when == 'MIDNIGHT':
            self.interval = 60 * 60 * 24 # one day
            self.suffix = "%Y-%m-%d"
            self.extMatch = r"^\d{4}-\d{2}-\d{2}$"
        elif self.when.startswith('W'):
            self.interval = 60 * 60 * 24 * 7 # one week
            if len(self.when) != 2:
                raise ValueError("You must specify a day for weekly rollover from 0 to 6 (0 is Monday): %s" % self.when)
            if self.when[1] < '0' or self.when[1] > '6':
                 raise ValueError("Invalid day specified for weekly rollover: %s" % self.when)
            self.dayOfWeek = int(self.when[1])
            self.suffix = "%Y-%m-%d"
            self.extMatch = r"^\d{4}-\d{2}-\d{2}$"
        else:
            raise ValueError("Invalid rollover interval specified: %s" % self.when)

        currenttime = int(time.time())
        logging.handlers.BaseRotatingHandler.__init__(self, self.calculateFileName(currenttime), 'a', encoding, delay)

        self.extMatch = re.compile(self.extMatch)
        self.interval = self.interval * interval # multiply by units requested

        self.rolloverAt = self.computeRollover(currenttime)

    def calculateFileName(self, currenttime):
        if self.utc:
             timeTuple = time.gmtime(currenttime)
        else:
             timeTuple = time.localtime(currenttime)

        return self.origFileName + "." + time.strftime(self.suffix, timeTuple) + self.postfix

    def getFilesToDelete(self, newFileName):
        dirName, fName = os.path.split(self.origFileName)
        dName, newFileName = os.path.split(newFileName)

        fileNames = os.listdir(dirName)
        result = []
        prefix = fName + "."
        postfix = self.postfix
        prelen = len(prefix)
        postlen = len(postfix)
        for fileName in fileNames:
            if fileName[:prelen] == prefix and fileName[-postlen:] == postfix and len(fileName)-postlen > prelen and fileName != newFileName:
                 suffix = fileName[prelen:len(fileName)-postlen]
                 if self.extMatch.match(suffix):
                     result.append(os.path.join(dirName, fileName))
        result.sort()
        if len(result) < self.backupCount:
            result = []
        else:
            result = result[:len(result) - self.backupCount]
        return result

     def doRollover(self):
         if self.stream:
            self.stream.close()
            self.stream = None

         currentTime = self.rolloverAt
         newFileName = self.calculateFileName(currentTime)
         newBaseFileName = os.path.abspath(newFileName)
         self.baseFilename = newBaseFileName
         self.mode = 'a'
         self.stream = self._open()

         if self.backupCount > 0:
             for s in self.getFilesToDelete(newFileName):
                 try:
                     os.remove(s)
                 except:
                     pass

         newRolloverAt = self.computeRollover(currentTime)
         while newRolloverAt <= currentTime:
             newRolloverAt = newRolloverAt + self.interval

         #If DST changes and midnight or weekly rollover, adjust for this.
         if (self.when == 'MIDNIGHT' or self.when.startswith('W')) and not self.utc:
             dstNow = time.localtime(currentTime)[-1]
             dstAtRollover = time.localtime(newRolloverAt)[-1]
             if dstNow != dstAtRollover:
                 if not dstNow:  # DST kicks in before next rollover, so we need to deduct an hour
                     newRolloverAt = newRolloverAt - 3600
                 else:           # DST bows out before next rollover, so we need to add an hour
                     newRolloverAt = newRolloverAt + 3600
         self.rolloverAt = newRolloverAt
于 2014-08-19T15:21:25.773 回答
4

据我所知,没有办法直接实现这一点。

您可以尝试的一种解决方案是覆盖默认行为。

  • 创建自己的TimedRotatingFileHandler class并覆盖doRollover() function.
  • 检查 python 安装中的源代码<PythonInstallDir>/Lib/logging/handlers.py

像这样的东西:

class MyTimedRotatingFileHandler(TimedRotatingFileHandler):
    def __init__(self, **kwargs):
        TimedRotatingFileHandler.__init__(self, **kwargs)

    def doRollover(self):
        # Do your stuff, rename the file as you want 
于 2014-07-09T09:43:56.260 回答
2

这是一个简单的解决方案:将自定义命名器函数添加到处理程序。日志实用程序将调用您的命名器函数来创建滚动文件的名称,正如 Jester 最初在 6 年前(!)使用 filename.log.YYYYMMDD 所指出的那样,因此我们需要将 .log 部分“移动”到末尾:

def namer(name):
    return name.replace(".log", "") + ".log"

然后在你设置你的处理程序之后,只需将你的函数分配给它的 namer 属性:

handler.namer = namer

这是我的完整日志初始化脚本,我是 python 新手,欢迎批评/建议:

import os
import logging
from logging.handlers import TimedRotatingFileHandler
from config import constants

def namer(name):
    return name.replace(".log", "") + ".log"

def init(baseFilename):
    logPath = constants.LOGGING_DIR
    envSuffix = '-prod' if constants.ENV == 'prod' else '-dev'
    logFilename = os.path.join(logPath, baseFilename + envSuffix + '.log')
    print(f"Logging to {logFilename}")

    handler = TimedRotatingFileHandler(logFilename,
    when = "midnight", 
    backupCount = 30,
    encoding = 'utf8')
    handler.setLevel(logging.DEBUG)
    handler.suffix = "%Y%m%d"
    handler.namer = namer # <-- Here's where I assign the custom namer.

    formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s [%(module)s:%(lineno)d]')
    handler.setFormatter(formatter)

    logging.basicConfig(
        handlers = [handler],
        format = '%(asctime)s %(levelname)s %(message)s [%(module)s:%(lineno)d]',
        level = logging.DEBUG,
        datefmt = '%Y-%m-%d %H:%M:%S')


if __name__ == '__main__':
    init('testing')
    logging.error("ohai")
    logging.debug("ohai debug")
    logging.getLogger().handlers[0].doRollover()
    logging.error("ohai next day")
    logging.debug("ohai debug next day")
于 2021-03-02T18:28:32.700 回答
0

我在 Python 3.7 中使用了解决方案https://stackoverflow.com/a/25387192/6619512 ,这是一个很好的解决方案。

但是对于“午夜”参数和以“W”开头的参数时它不起作用,因为在TimedRotatingFileHandler 类中引入并使用了atTime参数。

要使用此解决方案,请使用以下 __init__ 行:

def __init__(self, filename, when='h', interval=1, backupCount=0,
             encoding=None, delay=False, utc=False, atTime=None, postfix = ".log"):

还将以下内容添加到 __init__ 声明的内容中:

self.postfix = postfix
于 2019-11-22T09:15:00.473 回答