1

我正在通过 gunicorn + uvicorn 在 VM 上提供模型。

它由 supervisord 自动启动，正在运行api.sh。

api.sh包含：

source /home/asd/.virtual_envs/myproject/bin/activate

/home/asd/.virtual_envs/myproject/bin/gunicorn --max-requests-jitter 30 -w 6 -b 0.0.0.0:4080 api:app -k uvicorn.workers.UvicornWorker

无需过多介绍api.py，它包含以下主要部分：

from starlette.applications import Starlette
from models import SomeModelClass


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    return UJSONResponse(result)

发生的情况是，在 api 启动几天后，我开始收到这些错误：

[INFO] Starting gunicorn 20.0.3
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
...

在 supervisord 中重新启动 api 什么也没做，我得到与上面相同的消息。我发现可行的唯一方法是：

  1. 在 supervisord 中停止 api
  2. 查看 4080 端口（一个python3.8进程）上运行的是哪个 pid：sudo netstat -tulpn | grep LISTEN
  3. 杀死它运行kill -9 [PID]
  4. 重复步骤 2-3 1-2 次，直到没有任何东西占用 4080 端口
  5. 在 supervisord 中启动 api

你有什么想法如何解决这个问题吗？

4

1 回答 1

1

实际使用的代码Pool很multiprocessing可能是导致此问题的原因。

例子：

from starlette.applications import Starlette
from models import SomeModelClass
from multiprocessing import Pool
from utils import myfun


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    # Start of the offending code
    pool = Pool(4)
    result = pool.map(myfun, result, chunksize=1)
    # End of the offending code
    return UJSONResponse(result)

解决方案是替换multiprocessing为concurrency：

from starlette.applications import Starlette
from models import SomeModelClass
import concurrent.futures
from utils import myfun


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    # Start of the fix
    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        result = executor.map(myfun, result)
    result = list(result)
    # End of the fix
    return UJSONResponse(result)
于 2021-01-19T07:38:36.330 回答