Python怎么實(shí)現(xiàn)熱加載配置文件
本文小編為大家詳細(xì)介紹“Python怎么實(shí)現(xiàn)熱加載配置文件”，內(nèi)容詳細(xì)，步驟清晰，細(xì)節(jié)處理妥當(dāng)，希望這篇“Python怎么實(shí)現(xiàn)熱加載配置文件”文章能幫助大家解決疑惑，下面跟著小編的思路慢慢深入，一起來學(xué)習(xí)新知識吧。
我們提供的服務(wù)有：成都做網(wǎng)站、成都網(wǎng)站設(shè)計(jì)、微信公眾號開發(fā)、網(wǎng)站優(yōu)化、網(wǎng)站認(rèn)證、武漢ssl等。為近1000家企事業(yè)單位解決了網(wǎng)站和推廣的問題。提供周到的售前咨詢和貼心的售后服務(wù)，是有科學(xué)管理、有技術(shù)的武漢網(wǎng)站制作公司
背景
由于最近工作需求，需要在已有項(xiàng)目添加一個新功能，實(shí)現(xiàn)配置熱加載的功能。所謂的配置熱加載，也就是說當(dāng)服務(wù)收到配置更新消息之后，我們不用重啟服務(wù)就可以使用最新的配置去執(zhí)行任務(wù)。
如何實(shí)現(xiàn)
下面我分別采用多進(jìn)程、多線程、協(xié)程的方式去實(shí)現(xiàn)配置熱加載。
使用多進(jìn)程實(shí)現(xiàn)配置熱加載
如果我們代碼實(shí)現(xiàn)上使用多進(jìn)程， 主進(jìn)程1來更新配置并發(fā)送指令，任務(wù)的調(diào)用是進(jìn)程2，如何實(shí)現(xiàn)配置熱加載呢？
使用signal信號量來實(shí)現(xiàn)熱加載
當(dāng)主進(jìn)程收到配置更新的消息之后(配置讀取是如何收到配置更新的消息的? 這里我們暫不討論)， 主進(jìn)程就向進(jìn)子程1發(fā)送kill信號，子進(jìn)程1收到kill的信號就退出，之后由信號處理函數(shù)來啟動一個新的進(jìn)程，使用最新的配置文件來繼續(xù)執(zhí)行任務(wù)。
main 函數(shù)
def main():
# 啟動一個進(jìn)程執(zhí)行任務(wù)
p1 = Process(target=run, args=("p1",))
p1.start()
monitor(p1, run) # 注冊信號
processes["case100"] = p1 #將進(jìn)程pid保存
num = 0
while True: # 模擬獲取配置更新
print(
f"{multiprocessing.active_children()=}, count={len(multiprocessing.active_children())}\n")
print(f"{processes=}\n")
sleep(2)
if num == 4:
kill_process(processes["case100"]) # kill 當(dāng)前進(jìn)程
if num == 8:
kill_process(processes["case100"]) # kill 當(dāng)前進(jìn)程
if num == 12:
kill_process(processes["case100"]) # kill 當(dāng)前進(jìn)程
num += 1signal_handler 函數(shù)
def signal_handler(process: Process, func, signum, frame):
# print(f"{signum=}")
global counts
if signum == 17: # 17 is SIGCHILD
# 這個循環(huán)是為了忽略SIGTERM發(fā)出的信號，避免搶占了主進(jìn)程發(fā)出的SIGCHILD
for signame in [SIGTERM, SIGCHLD, SIGQUIT]:
signal.signal(signame, SIG_DFL)
print("Launch a new process")
p = multiprocessing.Process(target=func, args=(f"p{counts}",))
p.start()
monitor(p, run)
processes["case100"] = p
counts += 1
if signum == 2:
if process.is_alive():
print(f"Kill {process} process")
process.terminate()
signal.signal(SIGCHLD, SIG_IGN)
sys.exit("kill parent process")完整代碼如下
#! /usr/local/bin/python3.8
from multiprocessing import Process
from typing import Dict
import signal
from signal import SIGCHLD, SIGTERM, SIGINT, SIGQUIT, SIG_DFL, SIG_IGN
import multiprocessing
from multiprocessing import Process
from typing import Callable
from data import processes
import sys
from functools import partial
import time
processes: Dict[str, Process] = {}
counts = 2
def run(process: Process):
while True:
print(f"{process} running...")
time.sleep(1)
def kill_process(process: Process):
print(f"kill {process}")
process.terminate()
def monitor(process: Process, func: Callable):
for signame in [SIGTERM, SIGCHLD, SIGINT, SIGQUIT]:
# SIGTERM is kill signal.
# No SIGCHILD is not trigger singnal_handler,
# No SIGINT is not handler ctrl+c,
# No SIGQUIT is RuntimeError: reentrant call inside <_io.BufferedWriter name='<stdout>'>
signal.signal(signame, partial(signal_handler, process, func))
def signal_handler(process: Process, func, signum, frame):
print(f"{signum=}")
global counts
if signum == 17: # 17 is SIGTERM
for signame in [SIGTERM, SIGCHLD, SIGQUIT]:
signal.signal(signame, SIG_DFL)
print("Launch a new process")
p = multiprocessing.Process(target=func, args=(f"p{counts}",))
p.start()
monitor(p, run)
processes["case100"] = p
counts += 1
if signum == 2:
if process.is_alive():
print(f"Kill {process} process")
process.terminate()
signal.signal(SIGCHLD, SIG_IGN)
sys.exit("kill parent process")
def main():
p1 = Process(target=run, args=("p1",))
p1.start()
monitor(p1, run)
processes["case100"] = p1
num = 0
while True:
print(
f"{multiprocessing.active_children()=}, count={len(multiprocessing.active_children())}\n")
print(f"{processes=}\n")
time.sleep(2)
if num == 4:
kill_process(processes["case100"])
if num == 8:
kill_process(processes["case100"])
if num == 12:
kill_process(processes["case100"])
num += 1
if __name__ == '__main__':
main()執(zhí)行結(jié)果如下
multiprocessing.active_children()=[<Process name='Process-1' pid=2533 parent=2532 started>], count=1
processes={'case100': <Process name='Process-1' pid=2533 parent=2532 started>}
p1 running...
p1 running...
kill <Process name='Process-1' pid=2533 parent=2532 started>
multiprocessing.active_children()=[<Process name='Process-1' pid=2533 parent=2532 started>], count=1
processes={'case100': <Process name='Process-1' pid=2533 parent=2532 started>}
signum=17
Launch a new process
p2 running...
p2 running...
multiprocessing.active_children()=[<Process name='Process-2' pid=2577 parent=2532 started>], count=1
processes={'case100': <Process name='Process-2' pid=2577 parent=2532 started>}
p2 running...
p2 running...
multiprocessing.active_children()=[<Process name='Process-2' pid=2577 parent=2532 started>], count=1
processes={'case100': <Process name='Process-2' pid=2577 parent=2532 started>}
p2 running...
p2 running...
multiprocessing.active_children()=[<Process name='Process-2' pid=2577 parent=2532 started>], count=1
processes={'case100': <Process name='Process-2' pid=2577 parent=2532 started>}
p2 running...
p2 running...
kill <Process name='Process-2' pid=2577 parent=2532 started>
signum=17
Launch a new process
multiprocessing.active_children()=[<Process name='Process-2' pid=2577 parent=2532 stopped exitcode=-SIGTERM>], count=1
processes={'case100': <Process name='Process-3' pid=2675 parent=2532 started>}
p3 running...
p3 running...
multiprocessing.active_children()=[<Process name='Process-3' pid=2675 parent=2532 started>], count=1總結(jié)
好處：使用信號量可以處理多進(jìn)程之間通信的問題。
壞處：代碼不好寫，寫出來代碼不好理解。信號量使用必須要很熟悉，不然很容易自己給自己寫了一個bug.(所有初學(xué)者慎用，老司機(jī)除外。)
還有一點(diǎn)不是特別理解的就是process.terminate() 發(fā)送出信號是SIGTERM number是15，但是第一次signal_handler收到信號卻是number=17，如果我要去處理15的信號，就會導(dǎo)致前一個進(jìn)程不能kill掉的問題。歡迎有對信號量比較熟悉的大佬，前來指點(diǎn)迷津，不甚感謝。
采用multiprocessing.Event 來實(shí)現(xiàn)配置熱加載
實(shí)現(xiàn)邏輯是主進(jìn)程1 更新配置并發(fā)送指令。進(jìn)程2啟動調(diào)度任務(wù)。
這時候當(dāng)主進(jìn)程1更新好配置之后，發(fā)送指令給進(jìn)程2，這時候的指令就是用Event一個異步事件通知。
直接上代碼
scheduler 函數(shù)
def scheduler():
while True:
print('wait message...')
case_configurations = scheduler_notify_queue.get()
print(f"Got case configurations {case_configurations=}...")
task_schedule_event.set() # 設(shè)置set之后, is_set 為True
print(f"Schedule will start ...")
while task_schedule_event.is_set(): # is_set 為True的話，那么任務(wù)就會一直執(zhí)行
run(case_configurations)
print("Clearing all scheduling job ...")event_scheduler 函數(shù)
def event_scheduler(case_config): scheduler_notify_queue.put(case_config) print(f"Put cases config to the Queue ...") task_schedule_event.clear() # clear之后，is_set 為False print(f"Clear scheduler jobs ...") print(f"Schedule job ...")
完整代碼如下
import multiprocessing
import time
scheduler_notify_queue = multiprocessing.Queue()
task_schedule_event = multiprocessing.Event()
def run(case_configurations: str):
print(f'{case_configurations} running...')
time.sleep(3)
def scheduler():
while True:
print('wait message...')
case_configurations = scheduler_notify_queue.get()
print(f"Got case configurations {case_configurations=}...")
task_schedule_event.set()
print(f"Schedule will start ...")
while task_schedule_event.is_set():
run(case_configurations)
print("Clearing all scheduling job ...")
def event_scheduler(case_config: str):
scheduler_notify_queue.put(case_config)
print(f"Put cases config to the Queue ...")
task_schedule_event.clear()
print(f"Clear scheduler jobs ...")
print(f"Schedule job ...")
def main():
scheduler_notify_queue.put('1')
p = multiprocessing.Process(target=scheduler)
p.start()
count = 1
print(f'{count=}')
while True:
if count == 5:
event_scheduler('100')
if count == 10:
event_scheduler('200')
count += 1
time.sleep(1)
if __name__ == '__main__':
main()執(zhí)行結(jié)果如下
wait message... Got case configurations case_configurations='1'... Schedule will start ... 1 running... 1 running... Put cases config to the Queue ... Clear scheduler jobs ... Schedule job ... Clearing all scheduling job ... wait message... Got case configurations case_configurations='100'... Schedule will start ... 100 running... Put cases config to the Queue ... Clear scheduler jobs ... Schedule job ... Clearing all scheduling job ... wait message... Got case configurations case_configurations='200'... Schedule will start ... 200 running... 200 running...
總結(jié)
使用Event事件通知，代碼不易出錯，代碼編寫少，易讀。相比之前信號量的方法，推薦大家多使用這種方式。
使用多線程或協(xié)程的方式，其實(shí)和上述實(shí)現(xiàn)方式一致。唯一區(qū)別就是調(diào)用了不同庫中，queue和 event.
# threading scheduler_notify_queue = queue.Queue() task_schedule_event = threading.Event() # async scheduler_notify_queue = asyncio.Queue() task_schedule_event = asyncio.Event()
讀到這里，這篇“Python怎么實(shí)現(xiàn)熱加載配置文件”文章已經(jīng)介紹完畢，想要掌握這篇文章的知識點(diǎn)還需要大家自己動手實(shí)踐使用過才能領(lǐng)會，如果想了解更多相關(guān)內(nèi)容的文章，歡迎關(guān)注創(chuàng)新互聯(lián)行業(yè)資訊頻道。
名稱欄目：Python怎么實(shí)現(xiàn)熱加載配置文件
標(biāo)題網(wǎng)址：http://jinyejixie.com/article14/gpidde.html
成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián)，為您提供外貿(mào)網(wǎng)站建設(shè)、微信小程序、面包屑導(dǎo)航、做網(wǎng)站、商城網(wǎng)站、網(wǎng)站導(dǎo)航
聲明：本網(wǎng)站發(fā)布的內(nèi)容（圖片、視頻和文字）以用戶投稿、用戶轉(zhuǎn)載內(nèi)容為主，如果涉及侵權(quán)請盡快告知，我們將會在第一時間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場，如需處理請聯(lián)系客服。電話：028-86922220；郵箱：631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載，或轉(zhuǎn)載時需注明來源： 創(chuàng)新互聯(lián)
- 響應(yīng)式網(wǎng)站設(shè)計(jì)必須考慮到哪些方面 2021-09-17
- H5響應(yīng)式網(wǎng)站怎么做？有什么優(yōu)勢特點(diǎn)？ 2016-09-11
- 什么情況下適用響應(yīng)式網(wǎng)站？ 2016-09-19
- 【成都網(wǎng)站建設(shè)】自適應(yīng)網(wǎng)站和響應(yīng)式網(wǎng)站之間的區(qū)別？ 2022-12-14
- 成都網(wǎng)站制作公司淺析為什么響應(yīng)式網(wǎng)站建設(shè)很流行？ 2016-10-26
- 企業(yè)響應(yīng)式網(wǎng)站建設(shè)為何大勢所趨勢不可擋 2022-05-21
- 響應(yīng)式網(wǎng)站有什么優(yōu)勢？ 2023-03-21
- 創(chuàng)新互聯(lián)教你響應(yīng)式網(wǎng)站建設(shè)如何制作？ 2020-11-02
- 響應(yīng)式網(wǎng)站這么多人選擇建原由是什么 2022-07-14
- 延慶企業(yè)建設(shè)響應(yīng)式網(wǎng)站的必要性 2023-03-20
- 為什么要選擇響應(yīng)式網(wǎng)站？ 2023-01-04
- 建設(shè)響應(yīng)式網(wǎng)站好不好？有哪些優(yōu)勢 2023-01-25