我有一个可以处理异步请求询问库和三重奏的刮板项目。我想根据输入选择多少并发任务,但我的代码很长而且很原始
我将 trio 的产卵和托儿所对象用于并发任务(文档:https ://trio.readthedocs.io/en/latest/reference-core.html )
这是我草率的代码:
import trio
import asks
Number_of_workers = input("how many workers do you want?: ") #How many tasks I want between 1 and 5
async def child1(s):
r = await s.get("https://example.com", params={"example":"example"})
print("do something with", r.text)
async def child2():
r = await s.get("https://example.com", params={"example":"example"})
print("do something with", r.text)
async def child3():
r = await s.get("https://example.com", params={"example":"example"})
print("do something with", r.text)
async def child4():
r = await s.get("https://example.com", params={"example":"example"})
print("do something with", r.text)
async def child5():
r = await s.get("https://example.com", params={"example":"example"})
print("do something with", r.text)
async def parent():
s = Session(connections=5)
async with trio.open_nursery() as nursery:
if int(Number_of_workers) == 1:
nursery.start_soon(child1, s)
elif int(Number_of_workers) == 2:
nursery.start_soon(child1, s)
nursery.start_soon(child2, s)
elif int(Number_of_workers) == 3:
nursery.start_soon(child1, s)
nursery.start_soon(child2, s)
nursery.start_soon(child3, s)
elif int(Number_of_workers) == 4:
nursery.start_soon(child1, s)
nursery.start_soon(child2, s)
nursery.start_soon(child3, s)
nursery.start_soon(child4, s)
elif int(Number_of_workers) == 5:
nursery.start_soon(child1, s)
nursery.start_soon(child2, s)
nursery.start_soon(child3, s)
nursery.start_soon(child4, s)
nursery.start_soon(child5, s)
trio.run(parent)
我想你可以理解我的意思,这个示例代码在理论上是有效的,但是对于可能会减少到更少代码行的东西来说它很长。
这种方案在处理 10 或 20 名工人时会变得特别长,并且总是限制在预定义的数量内。
就其本身而言,每个孩子都是相同的,相同的代码,它只是从带有 importlib 的外部模块 .py 文件中获取不同的数据(例如参数和 url)。
有没有办法将其缩减为更优化的代码?