境全流程腳本集)
海光DCU大模型推理環(huán)境全流程腳本集純代碼可直接復(fù)用整理一套從環(huán)境校驗(yàn)、依賴安裝、服務(wù)部署到壓測驗(yàn)證的全鏈路可運(yùn)行腳本全程無額外依賴復(fù)制到服務(wù)器按順序執(zhí)行即可完成整套推理環(huán)境搭建適合快速部署生產(chǎn)級推理服務(wù)。1. DTK環(huán)境一鍵校驗(yàn)?zāi)_本執(zhí)行前先確認(rèn)DTK環(huán)境是否正常、硬件是否可識別避免后續(xù)安裝全做無用功。保存為check_dtk_env.sh#!/bin/bashecho DCU 環(huán)境基線校驗(yàn) # 1. 檢查rocm-smi工具echo-n1. rocm-smi 工具: ifcommand-vrocm-smi/dev/null;thenecho正常elseecho未找到請先加載DTK環(huán)境變量exit1fi# 2. 檢查HIP編譯器echo-n2. hipcc 編譯器: ifcommand-vhipcc/dev/null;thenhipcc--version2/dev/null|grepHIP versionelseecho未找到exit1fi# 3. 設(shè)備識別檢查echo-n3. DCU 設(shè)備識別: card_count$(rocm-smi--showhw2/dev/null|grep-cDevice ID)if[$card_count-gt0];thenecho檢測到$card_count張卡elseecho未識別到設(shè)備請檢查驅(qū)動與權(quán)限exit1fi# 4. 輸出核心硬件信息echo-e\n 硬件明細(xì) rocm-smi--showhw2/dev/null|grep-EDevice ID|VRAM Total|Compute Unitecho-e\n? 環(huán)境校驗(yàn)通過使用方式bash check_dtk_env.sh任意一項(xiàng)失敗都會直接提示對應(yīng)問題。2. PyTorch與推理依賴一鍵安裝自動匹配DTK版本安裝對應(yīng)ROCm版PyTorch同時(shí)安裝vLLM、transformers等推理必備依賴。保存為install_deps.sh#!/bin/bashecho 安裝 DCU 推理依賴 # 自動識別DTK對應(yīng)ROCm版本DTK_VER$(hipcc--version2/dev/null|grep-oPHIP version: \K[0-9]|head-1)if[$DTK_VER-ge26];thenROCM_VER6.0TORCH_VER2.4.0elif[$DTK_VER-ge25];thenROCM_VER5.7TORCH_VER2.3.1elseechoDTK版本過低建議升級至25.04以上exit1fiecho匹配 ROCm 版本:$ROCM_VERPyTorch 版本:$TORCH_VER# 安裝PyTorchpipinstalltorch$TORCH_VERtorchvision0.19.0\--index-url https://download.pytorch.org/whl/rocm$ROCM_VER# 安裝推理常用依賴pipinstalltransformers accelerate sentencepiece\modelscopevllm0.18.1 requests# 驗(yàn)證安裝結(jié)果echo-e\n 安裝結(jié)果驗(yàn)證 python3EOF import torch import transformers import vllm print(fPyTorch: {torch.__version__}) print(fTransformers: {transformers.__version__}) print(fvLLM: {vllm.__version__}) print(fDCU可用: {torch.cuda.is_available()}) print(fDCU數(shù)量: {torch.cuda.device_count()}) EOFecho? 依賴安裝完成使用方式bash install_deps.sh自動適配DTK版本無需手動對應(yīng)。3. vLLM推理服務(wù)一鍵啟停腳本帶進(jìn)程守護(hù)、日志管理的服務(wù)化腳本支持啟動、停止、狀態(tài)查詢無需手動管理進(jìn)程。保存為vllm_service.sh#!/bin/bash# 配置項(xiàng) - 根據(jù)實(shí)際環(huán)境修改MODEL_PATH./models/Qwen2-7B-InstructMODEL_NAMEQwen2-7B-InstructPORT8000TP_SIZE1DTK_ENV/opt/dtk-26.04/env.shLOG_FILE./vllm_service.logPID_FILE./vllm.pid# 加載DTK環(huán)境source$DTK_ENVexportTRITON_HIP_LLD_PATH/opt/dtk-26.04/llvm/bin/ld.lldstart_service(){if[-f$PID_FILE]kill-0$(cat$PID_FILE)2/dev/null;thenecho服務(wù)已在運(yùn)行PID:$(cat$PID_FILE)exit1fiecho啟動 vLLM 推理服務(wù)...nohupnumactl--cpunodebind0--membind0\vllm serve$MODEL_PATH\--served-model-name$MODEL_NAME\--tensor-parallel-size$TP_SIZE\--dtypebfloat16\--max-model-len8192\--gpu-memory-utilization0.9\--enforce-eager\--max-num-seqs128\--host0.0.0.0\--port$PORT\$LOG_FILE21echo$!$PID_FILEecho服務(wù)已啟動PID:$!echo日志文件:$LOG_FILEecho端口:$PORT}stop_service(){if[!-f$PID_FILE];thenecho服務(wù)未運(yùn)行exit1fipid$(cat$PID_FILE)ifkill-0$pid2/dev/null;thenecho停止服務(wù)PID:$pidkill$pidsleep3kill-9$pid2/dev/nullfirm-f$PID_FILEecho服務(wù)已停止}status_service(){if[-f$PID_FILE]kill-0$(cat$PID_FILE)2/dev/null;thenecho? 服務(wù)運(yùn)行中PID:$(cat$PID_FILE)echo端口:$PORTelseecho? 服務(wù)未運(yùn)行fi}case$1instart)start_service;;stop)stop_service;;restart)stop_service;sleep2;start_service;;status)status_service;;*)echo用法:$0start|stop|restart|status;exit1;;esac使用方式bashvllm_service.sh start# 啟動服務(wù)bashvllm_service.sh status# 查看狀態(tài)bashvllm_service.sh stop# 停止服務(wù)4. 推理接口壓測腳本服務(wù)啟動后批量并發(fā)壓測輸出吞吐、平均延遲等核心指標(biāo)量化驗(yàn)證服務(wù)性能。保存為benchmark.pyimporttimeimportrequestsimportthreadingfromstatisticsimportmean# 配置API_URLhttp://localhost:8000/v1/chat/completionsMODEL_NAMEQwen2-7B-InstructCONCURRENT16# 并發(fā)數(shù)TOTAL_REQUESTS64MAX_TOKENS256PROMPT請?jiān)敿?xì)介紹海光DCU的主要應(yīng)用場景和技術(shù)特點(diǎn)# 全局統(tǒng)計(jì)success0total_tokens0latencies[]lockthreading.Lock()defworker():globalsuccess,total_tokens payload{model:MODEL_NAME,messages:[{role:user,content:PROMPT}],max_tokens:MAX_TOKENS,temperature:0.1,stream:False}starttime.perf_counter()try:resprequests.post(API_URL,jsonpayload,timeout60)costtime.perf_counter()-startifresp.status_code200:dataresp.json()tokensdata[usage][completion_tokens]withlock:success1total_tokenstokens latencies.append(cost)exceptExceptionase:print(f請求失敗:{e})if__name____main__:print(f開始壓測: 并發(fā){CONCURRENT}, 總請求{TOTAL_REQUESTS})start_timetime.perf_counter()threads[]for_inrange(TOTAL_REQUESTS):tthreading.Thread(targetworker)threads.append(t)t.start()# 控制并發(fā)數(shù)whilethreading.active_count()CONCURRENT1:time.sleep(0.01)fortinthreads:t.join()total_timetime.perf_counter()-start_timeprint(\n 壓測結(jié)果 )print(f成功請求:{success}/{TOTAL_REQUESTS})print(f總生成Token:{total_tokens})print(f總耗時(shí):{total_time:.2f}s)print(f平均吞吐:{total_tokens/total_time:.2f}token/s)print(f單請求平均延遲:{mean(latencies):.2f}s)print(fQPS:{success/total_time:.2f}req/s)使用方式python3 benchmark.py自動輸出完整性能指標(biāo)。5. 顯存與進(jìn)程一鍵清理腳本服務(wù)異常退出、顯存殘留時(shí)執(zhí)行快速釋放所有DCU資源無需重啟節(jié)點(diǎn)。保存為clean_dcu.sh#!/bin/bashecho DCU 資源清理工具 # 列出所有占用DCU的進(jìn)程echo當(dāng)前占用DCU設(shè)備的進(jìn)程:echoPID 用戶 進(jìn)程名echo-------------------------pidsfordevin/dev/dri/renderD*;dodev_pids$(fuser$dev2/dev/null)forpidin$dev_pids;doif!echo$pids|grep-q$pid;thenpids$pids$piduser$(ps-p$pid-ouser2/dev/null)comm$(ps-p$pid-ocomm2/dev/null)printf%-6s %-8s %s\n$pid$user$commfidonedoneif[-z$pids];thenecho無占用進(jìn)程顯存空閑exit0firead-p是否終止以上所有進(jìn)程并釋放顯存? (y/n): confirmif[$confirmy];thenforpidin$pids;dokill-9$pid2/dev/nullecho已終止進(jìn)程$piddone# 清理PyTorch緩存python3-cimport torch; torch.cuda.empty_cache()2/dev/nullecho? 清理完成elseecho已取消操作fi使用方式bash clean_dcu.sh先列進(jìn)程再確認(rèn)避免誤殺正常任務(wù)。以上5個(gè)腳本覆蓋了DCU大模型推理環(huán)境從部署到運(yùn)維的全流程高頻場景全部基于原生命令與官方工具實(shí)現(xiàn)無額外依賴存到服務(wù)器固定目錄即可長期復(fù)用。 海光 DCU 大模型推理環(huán)境全流程腳本集
海光DCU大模型推理環(huán)境全流程腳本集純代碼可直接復(fù)用整理一套從環(huán)境校驗(yàn)、依賴安裝、服務(wù)部署到壓測驗(yàn)證的全鏈路可運(yùn)行腳本全程無額外依賴復(fù)制到服務(wù)器按順序執(zhí)行即可完成整套推理環(huán)境搭建適合快速部署生產(chǎn)級推理服務(wù)。1. DTK環(huán)境一鍵校驗(yàn)?zāi)_本執(zhí)行前先確認(rèn)DTK環(huán)境是否正常、硬件是否可識別避免后續(xù)安裝全做無用功。保存為check_dtk_env.sh#!/bin/bashecho DCU 環(huán)境基線校驗(yàn) # 1. 檢查rocm-smi工具echo-n1. rocm-smi 工具: ifcommand-vrocm-smi/dev/null;thenecho正常elseecho未找到請先加載DTK環(huán)境變量exit1fi# 2. 檢查HIP編譯器echo-n2. hipcc 編譯器: ifcommand-vhipcc/dev/null;thenhipcc--version2/dev/null|grepHIP versionelseecho未找到exit1fi# 3. 設(shè)備識別檢查echo-n3. DCU 設(shè)備識別: card_count$(rocm-smi--showhw2/dev/null|grep-cDevice ID)if[$card_count-gt0];thenecho檢測到$card_count張卡elseecho未識別到設(shè)備請檢查驅(qū)動與權(quán)限exit1fi# 4. 輸出核心硬件信息echo-e\n 硬件明細(xì) rocm-smi--showhw2/dev/null|grep-EDevice ID|VRAM Total|Compute Unitecho-e\n? 環(huán)境校驗(yàn)通過使用方式bash check_dtk_env.sh任意一項(xiàng)失敗都會直接提示對應(yīng)問題。2. PyTorch與推理依賴一鍵安裝自動匹配DTK版本安裝對應(yīng)ROCm版PyTorch同時(shí)安裝vLLM、transformers等推理必備依賴。保存為install_deps.sh#!/bin/bashecho 安裝 DCU 推理依賴 # 自動識別DTK對應(yīng)ROCm版本DTK_VER$(hipcc--version2/dev/null|grep-oPHIP version: \K[0-9]|head-1)if[$DTK_VER-ge26];thenROCM_VER6.0TORCH_VER2.4.0elif[$DTK_VER-ge25];thenROCM_VER5.7TORCH_VER2.3.1elseechoDTK版本過低建議升級至25.04以上exit1fiecho匹配 ROCm 版本:$ROCM_VERPyTorch 版本:$TORCH_VER# 安裝PyTorchpipinstalltorch$TORCH_VERtorchvision0.19.0\--index-url https://download.pytorch.org/whl/rocm$ROCM_VER# 安裝推理常用依賴pipinstalltransformers accelerate sentencepiece\modelscopevllm0.18.1 requests# 驗(yàn)證安裝結(jié)果echo-e\n 安裝結(jié)果驗(yàn)證 python3EOF import torch import transformers import vllm print(fPyTorch: {torch.__version__}) print(fTransformers: {transformers.__version__}) print(fvLLM: {vllm.__version__}) print(fDCU可用: {torch.cuda.is_available()}) print(fDCU數(shù)量: {torch.cuda.device_count()}) EOFecho? 依賴安裝完成使用方式bash install_deps.sh自動適配DTK版本無需手動對應(yīng)。3. vLLM推理服務(wù)一鍵啟停腳本帶進(jìn)程守護(hù)、日志管理的服務(wù)化腳本支持啟動、停止、狀態(tài)查詢無需手動管理進(jìn)程。保存為vllm_service.sh#!/bin/bash# 配置項(xiàng) - 根據(jù)實(shí)際環(huán)境修改MODEL_PATH./models/Qwen2-7B-InstructMODEL_NAMEQwen2-7B-InstructPORT8000TP_SIZE1DTK_ENV/opt/dtk-26.04/env.shLOG_FILE./vllm_service.logPID_FILE./vllm.pid# 加載DTK環(huán)境source$DTK_ENVexportTRITON_HIP_LLD_PATH/opt/dtk-26.04/llvm/bin/ld.lldstart_service(){if[-f$PID_FILE]kill-0$(cat$PID_FILE)2/dev/null;thenecho服務(wù)已在運(yùn)行PID:$(cat$PID_FILE)exit1fiecho啟動 vLLM 推理服務(wù)...nohupnumactl--cpunodebind0--membind0\vllm serve$MODEL_PATH\--served-model-name$MODEL_NAME\--tensor-parallel-size$TP_SIZE\--dtypebfloat16\--max-model-len8192\--gpu-memory-utilization0.9\--enforce-eager\--max-num-seqs128\--host0.0.0.0\--port$PORT\$LOG_FILE21echo$!$PID_FILEecho服務(wù)已啟動PID:$!echo日志文件:$LOG_FILEecho端口:$PORT}stop_service(){if[!-f$PID_FILE];thenecho服務(wù)未運(yùn)行exit1fipid$(cat$PID_FILE)ifkill-0$pid2/dev/null;thenecho停止服務(wù)PID:$pidkill$pidsleep3kill-9$pid2/dev/nullfirm-f$PID_FILEecho服務(wù)已停止}status_service(){if[-f$PID_FILE]kill-0$(cat$PID_FILE)2/dev/null;thenecho? 服務(wù)運(yùn)行中PID:$(cat$PID_FILE)echo端口:$PORTelseecho? 服務(wù)未運(yùn)行fi}case$1instart)start_service;;stop)stop_service;;restart)stop_service;sleep2;start_service;;status)status_service;;*)echo用法:$0start|stop|restart|status;exit1;;esac使用方式bashvllm_service.sh start# 啟動服務(wù)bashvllm_service.sh status# 查看狀態(tài)bashvllm_service.sh stop# 停止服務(wù)4. 推理接口壓測腳本服務(wù)啟動后批量并發(fā)壓測輸出吞吐、平均延遲等核心指標(biāo)量化驗(yàn)證服務(wù)性能。保存為benchmark.pyimporttimeimportrequestsimportthreadingfromstatisticsimportmean# 配置API_URLhttp://localhost:8000/v1/chat/completionsMODEL_NAMEQwen2-7B-InstructCONCURRENT16# 并發(fā)數(shù)TOTAL_REQUESTS64MAX_TOKENS256PROMPT請?jiān)敿?xì)介紹海光DCU的主要應(yīng)用場景和技術(shù)特點(diǎn)# 全局統(tǒng)計(jì)success0total_tokens0latencies[]lockthreading.Lock()defworker():globalsuccess,total_tokens payload{model:MODEL_NAME,messages:[{role:user,content:PROMPT}],max_tokens:MAX_TOKENS,temperature:0.1,stream:False}starttime.perf_counter()try:resprequests.post(API_URL,jsonpayload,timeout60)costtime.perf_counter()-startifresp.status_code200:dataresp.json()tokensdata[usage][completion_tokens]withlock:success1total_tokenstokens latencies.append(cost)exceptExceptionase:print(f請求失敗:{e})if__name____main__:print(f開始壓測: 并發(fā){CONCURRENT}, 總請求{TOTAL_REQUESTS})start_timetime.perf_counter()threads[]for_inrange(TOTAL_REQUESTS):tthreading.Thread(targetworker)threads.append(t)t.start()# 控制并發(fā)數(shù)whilethreading.active_count()CONCURRENT1:time.sleep(0.01)fortinthreads:t.join()total_timetime.perf_counter()-start_timeprint(\n 壓測結(jié)果 )print(f成功請求:{success}/{TOTAL_REQUESTS})print(f總生成Token:{total_tokens})print(f總耗時(shí):{total_time:.2f}s)print(f平均吞吐:{total_tokens/total_time:.2f}token/s)print(f單請求平均延遲:{mean(latencies):.2f}s)print(fQPS:{success/total_time:.2f}req/s)使用方式python3 benchmark.py自動輸出完整性能指標(biāo)。5. 顯存與進(jìn)程一鍵清理腳本服務(wù)異常退出、顯存殘留時(shí)執(zhí)行快速釋放所有DCU資源無需重啟節(jié)點(diǎn)。保存為clean_dcu.sh#!/bin/bashecho DCU 資源清理工具 # 列出所有占用DCU的進(jìn)程echo當(dāng)前占用DCU設(shè)備的進(jìn)程:echoPID 用戶 進(jìn)程名echo-------------------------pidsfordevin/dev/dri/renderD*;dodev_pids$(fuser$dev2/dev/null)forpidin$dev_pids;doif!echo$pids|grep-q$pid;thenpids$pids$piduser$(ps-p$pid-ouser2/dev/null)comm$(ps-p$pid-ocomm2/dev/null)printf%-6s %-8s %s\n$pid$user$commfidonedoneif[-z$pids];thenecho無占用進(jìn)程顯存空閑exit0firead-p是否終止以上所有進(jìn)程并釋放顯存? (y/n): confirmif[$confirmy];thenforpidin$pids;dokill-9$pid2/dev/nullecho已終止進(jìn)程$piddone# 清理PyTorch緩存python3-cimport torch; torch.cuda.empty_cache()2/dev/nullecho? 清理完成elseecho已取消操作fi使用方式bash clean_dcu.sh先列進(jìn)程再確認(rèn)避免誤殺正常任務(wù)。以上5個(gè)腳本覆蓋了DCU大模型推理環(huán)境從部署到運(yùn)維的全流程高頻場景全部基于原生命令與官方工具實(shí)現(xiàn)無額外依賴存到服務(wù)器固定目錄即可長期復(fù)用。