Paraformer分角色语音识别-中文-通用 FunASR demo测试与训练

0 资料

https://github.com/modelscope/FunASR/blob/main/README_zh.md

https://github.com/modelscope/FunASR/blob/main/model_zoo/readme_zh.md

训练实时语音识别Paraformer模型

PyTorch / 2.3.0 / 3.12(ubuntu22.04) / 12.1

1 Paraformer分角色语音识别-中文-通用

https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc-spk_asr_nat-zh-cn

安装ffmpeg

source /etc/network_turbo
conda install x264 ffmpeg -c conda-forge -y
# 或者
conda install -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/ x264 ffmpeg -y

source /etc/network_turbo
pip install torchaudio
pip install -U funasrpython -c "import torchaudio; print(torchaudio.__version__)"
python -c "import funasr; print(funasr.__version__)"

1 模型下载

模型下载：https://modelscope.cn/models/iic/speech_paraformer-large-vad-punc-spk_asr_nat-zh-cn/files

使用SDK下载下载：

开始前安装

source /etc/network_turbo
pip install modelscope

脚本下载

# source /etc/network_turbo
from modelscope import snapshot_download# 指定模型的下载路径
cache_dir = '/root/autodl-tmp'
# 调用 snapshot_download 函数下载模型model_dir = snapshot_download('iic/speech_paraformer-large-vad-punc-spk_asr_nat-zh-cn', cache_dir=cache_dir)print(f"模型已下载到: {model_dir}")

2 音频识别测试

音频下载

wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav

from funasr import AutoModel
# paraformer-zh is a multi-functional asr model
# use vad, punc, spk or not as you need
# model = AutoModel(model="paraformer-zh", model_revision="v2.0.4",
model = AutoModel(model="/root/autodl-tmp/iic/speech_paraformer-large-vad-punc-spk_asr_nat-zh-cn", model_revision="v2.0.4",vad_model="fsmn-vad", vad_model_revision="v2.0.4",punc_model="ct-punc-c", punc_model_revision="v2.0.4",# spk_model="cam++", spk_model_revision="v2.0.2",)
# res = model.generate(input=f"{model.model_path}/example/asr_example.wav", 
res = model.generate(input=f"vad_example.wav", batch_size_s=300, hotword='魔搭')
print(res)

结果如下：

[{‘key’: ‘vad_example’, ‘text’:
‘试错的过程很简单啊，今特别是今天冒名插血卡的同学，你们可以听到后面的有专门的活动课，它会大大降低你的思错成本。其实你也可以不要来听课，为什么你自己写嘛？我先今天写五个点，我就实试实验一下，反正这五个点不行，我再写五个点，再是再不行，那再写五个点嘛。你总会所谓的活动大神和所谓的高手都是只有一个，把所有的错。所有的坑全部趟一遍，留下正确的你就是所谓的大神明白吗？所以说关于活动通过这一块，我只送给你们四个字啊，换位思考。如果说你要想降低你的试错成本，今天来这里你们就是对的。因为有创企创需要搞这个机会。所以说关于活动过于不过这个问题或者活动很难通过这个话题。呃，如果真的要坐下来聊的话，要聊一天。但是我觉得我刚才说的四个字足够好，谢谢。好，非常感谢那个三毛老师的回答啊，三毛老师说我们在整个店铺的这个活动当中，我们要学会换位思考。其实。’,
‘timestamp’: [[380, 620], [640, 740], [740, 940], [940, 1020], [1020,
1260], [1500, 1740], [1740, 1840], [1840, 2135], [2830, 3010], [3010,
3210], [3210, 3290], [3290, 3370], [3370, 3470], [3470, 3590], [3590,
3830], [3950, 4130], [4130, 4270], [4270, 4350], [4350, 4470], [4470,
4590], [4590, 4690], [4690, 4770], [4770, 5010], [5250, 5410], [5410,
5530], [5530, 5650], [5650, 5975], [6670, 6830], [6830, 6970], [6970,
7110], [7110, 7230], [7230, 7470], [7490, 7730], [8070, 8310], [8310,
8430], [8430, 8670], [8690, 8910], [8910, 9030], [9030, 9270], [9550,
9750], [9750, 9910], [9910, 10110], [10110, 10350], [10670, 10910],
[10950, 11130], [11130, 11250], [11250, 11370], [11370, 11490],
[11490, 11630], [11630, 11730], [11730, 11970], [12310, 12490],
[12490, 12610], [12610, 12710], [12710, 12790], [12790, 12910],
[12910, 13110], [13110, 13270], [13270, 13350], [13350, 13490],
[13490, 13630], [13630, 13870], [14030, 14250], [14250, 14350],
[14350, 14589], [14630, 14850], [14850, 14950], [14950, 15070],
[15070, 15250], [15250, 15490], [15950, 16150], [16150, 16390],
[16390, 16490], [16490, 16610], [16610, 16750], [16750, 16850],
[16850, 16970], [16970, 17210], [17270, 17390], [17390, 17570],
[17570, 17810], [17990, 18230], [18310, 18410], [18410, 18550],
[18550, 18650], [18650, 18870], [18870, 19010], [19010, 19090],
[19090, 19190], [19190, 19310], [19310, 19390], [19390, 19490],
[19490, 19590], [19590, 19830], [19970, 20130], [20130, 20250],
[20250, 20430], [20430, 20550], [20550, 20670], [20670, 20910],
[21090, 21270], [21270, 21510], [21510, 21650], [21650, 21750],
[21750, 21990], [22470, 22609], [22609, 22710], [22710, 22890],
[22890, 22970], [22970, 23070], [23070, 23190], [23190, 23485],
[24060, 24280], [24280, 24380], [24380, 24620], [25120, 25360],
[25500, 25660], [25660, 25985], [27030, 27230], [27230, 27410],
[27410, 27510], [27510, 27750], [27810, 27990], [27990, 28150],
[28150, 28270], [28270, 28350], [28350, 28430], [28430, 28755],
[30180, 30320], [30320, 30560], [30600, 30720], [30720, 30840],
[30840, 30940], [30940, 31235], [32020, 32260], [32280, 32440],
[32440, 32620], [32620, 32700], [32700, 32940], [33200, 33340],
[33340, 33440], [33440, 33560], [33560, 33800], [33960, 34160],
[34160, 34360], [34360, 34600], [34800, 35000], [35000, 35240],
[35540, 35700], [35700, 35900], [35900, 36000], [36000, 36180],
[36180, 36420], [36440, 36600], [36600, 36700], [36700, 36840],
[36840, 36940], [36940, 37020], [37020, 37100], [37100, 37200],
[37200, 37455], [38480, 38600], [38600, 38720], [38720, 38960],
[39180, 39320], [39320, 39440], [39440, 39560], [39560, 39740],
[39740, 39840], [39840, 40040], [40040, 40220], [40220, 40340],
[40340, 40440], [40440, 40540], [40540, 40620], [40620, 40840],
[40840, 40980], [40980, 41120], [41120, 41220], [41220, 41340],
[41340, 41460], [41460, 41620], [41620, 41800], [41800, 41920],
[41920, 42160], [42340, 42580], [42720, 42940], [42940, 43100],
[43100, 43240], [43240, 43480], [43800, 44020], [44020, 44200],
[44200, 44440], [44740, 44960], [44960, 45140], [45140, 45380],
[45440, 45680], [45680, 45860], [45860, 45980], [45980, 46080],
[46080, 46220], [46220, 46360], [46360, 46460], [46460, 46725],
[47580, 47760], [47760, 48000], [48040, 48220], [48220, 48340],
[48340, 48580], [48660, 48800], [48800, 48920], [48920, 49000],
[49000, 49080], [49080, 49260], [49260, 49495], [50180, 50300],
[50300, 50540], [50600, 50780], [50780, 50900], [50900, 51140],
[51180, 51320], [51320, 51440], [51440, 51520], [51520, 51640],
[51640, 51740], [51740, 51820], [51820, 51960], [51960, 52200],
[52400, 52560], [52560, 52640], [52640, 52740], [52740, 52860],
[52860, 52980], [52980, 53100], [53100, 53280], [53280, 53380],
[53380, 53500], [53500, 53620], [53620, 53860], [54080, 54220],
[54220, 54320], [54320, 54400], [54400, 54640], [54640, 54760],
[54760, 54880], [54880, 54980], [54980, 55100], [55100, 55220],
[55220, 55380], [55380, 55520], [55520, 55700], [55700, 55820],
[55820, 55900], [55900, 56060], [56060, 56295], [57010, 57210],
[57210, 57310], [57310, 57430], [57430, 57530], [57530, 57650],
[57650, 57890], [57950, 58130], [58130, 58230], [58230, 58330],
[58330, 58430], [58430, 58530], [58530, 58630], [58630, 58750],
[58750, 58950], [58950, 59050], [59050, 59345], [60070, 60230],
[60230, 60330], [60330, 60470], [60470, 60570], [60570, 60810],
[60970, 61170], [61170, 61290], [61290, 61410], [61410, 61490],
[61490, 61610], [61610, 61770], [61770, 61890], [61890, 62130],
[62230, 62430], [62430, 62670], [63030, 63270], [63290, 63430],
[63430, 63670], [63890, 64130], [64170, 64270], [64270, 64470],
[64470, 64570], [64570, 64750], [64750, 64870], [64870, 65050],
[65050, 65170], [65170, 65290], [65290, 65410], [65410, 65530],
[65530, 65610], [65610, 65710], [65710, 65830], [65830, 66010],
[66010, 66110], [66110, 66230], [66230, 66310], [66310, 66410],
[66410, 66550], [66550, 66650], [66650, 66890], [67110, 67350],
[67410, 67550], [67550, 67690], [67690, 67790], [67790, 67910],
[67910, 68030], [68030, 68110], [68110, 68210], [68210, 68310],
[68310, 68490], [68490, 68610], [68610, 68750], [68750, 68850],
[68850, 68930], [68930, 69030], [69030, 69150], [69150, 69290],
[69290, 69450], [69450, 69610], [69610, 69750], [69750, 69990],
[69990, 70110], [70110, 70315]]}]

3 FunASR安装 （训练用）

source /etc/network_turbo
git clone https://github.com/alibaba/FunASR.git && cd FunASR

进入到：FunASR/examples/industrial_data_pretraining/paraformer

4 训练