NCNN部署

一、安装ncnn以及检查ncnn是否安装成功

• 参考文档:https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux

sudo apt install build-essential git cmake libprotobuf-dev protobuf-compiler libomp-dev libopencv-dev
安装:
cd ncnn
mkdir -p build
cd build
cmake ..
make -j

验证方法:

cd ../examples
../build/examples/squeezenet ../images/256-ncnn.png
[0 AMD RADV FIJI (LLVM 10.0.1)]  queueC=1[4]  queueG=0[1]  queueT=0[1]
[0 AMD RADV FIJI (LLVM 10.0.1)]  bugsbn1=0  buglbia=0  bugcopc=0  bugihfa=0
[0 AMD RADV FIJI (LLVM 10.0.1)]  fp16p=1  fp16s=1  fp16a=0  int8s=1  int8a=1
532 = 0.163452
920 = 0.093140
716 = 0.061584

pytorch转ncnn

(坑：python3.10+torch2.5可行，python3.8+ torch1.9不行，转换pnnx yolov8n.torchscript 会失败)
-说明文档：https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx

• pnnx的使用文档:https://github.com/pnnx/pnnx
• yolov8转ncnn示例代码地址：https://github.com/Tencent/ncnn/blob/master/examples/yolov8.cpp
• yolov8转ncnn示例代码地址： /home/standard/yolo/ncnn/examples/yolo11.cpp
• 转换方法:
• *说明: pnnx20241223 ultralytics8.3.25 否则可能出现以下修改方式不一样的情况

// 1. 安装依赖
//      pip3 install -U ultralytics pnnx ncnn
// 2. 将pt导出为torchscript
//      yolo export model=yolov8n.pt format=torchscript
// 3. 将torchscript 转换为ncnn
//      pnnx yolov8n.torchscript
// 4. 因为动态尺寸,修改yolov8n_pnnx.py如下
//      A. modify reshape to support dynamic image sizes
//      B. permute tensor before concat and adjust concat axis
//      C. drop post-process part
//      before:
//          v_165 = v_142.view(1, 144, 6400)
//          v_166 = v_153.view(1, 144, 1600)
//          v_167 = v_164.view(1, 144, 400)
//          v_168 = torch.cat((v_165, v_166, v_167), dim=2)
//          ...
//      after:
//          v_165 = v_142.view(1, 144, -1).transpose(1, 2)
//          v_166 = v_153.view(1, 144, -1).transpose(1, 2)
//          v_167 = v_164.view(1, 144, -1).transpose(1, 2)
//          v_168 = torch.cat((v_165, v_166, v_167), dim=1)
//          return v_168
// 5. 重新运行以下脚本
//      python3 -c 'import yolov8n_pnnx; yolov8n_pnnx.export_torchscript()'
// 6. 运行以下脚本
//      pnnx yolov8n_pnnx.py.pt inputshape=[1,3,640,640] inputshape2=[1,3,320,320]
// 7. 重命名
//      mv yolov8n_pnnx.py.ncnn.param yolov8n.ncnn.param
//      mv yolov8n_pnnx.py.ncnn.bin yolov8n.ncnn.bin

二、量化int8推理

-说明文档:https://github.com/Tencent/ncnn/wiki/quantized-int8-inference

1.优化模型

• 说明文档:https://github.com/Tencent/ncnn/wiki/quantized-int8-inference
• 源码位置 /home/standard/yolo/ncnn/tools/ncnnoptimize.cpp

/home/standard/yolo/ncnn/build/tools/ncnnoptimize yolov8n.ncnn.param yolov8n.ncnn.bin yolov8n.ncnn_opt.param yolov8n.ncnn_opt.bin 65536

2.创建校准表文件(不是必须操作)

• 源码位置:/home/standard/yolo/ncnn/tools/quantize/ncnn2table.cpp
• 说明文档:https://github.com/Tencent/ncnn/wiki/quantized-int8-inference

find $3 -type f > imagelist.txt  # 图片名字全部保存到imagelist
/home/standard/yolo/ncnn/build/tools/quantize/ncnn2table yolov8n.ncnn_opt.param yolov8n.ncnn_opt.bin imagelist.txt yolov8n.ncnn/yolov8n.ncnn_opt.table mean=[255,255,255] norm=[0,0,0] shape=[640,640,3] pixel=BGR thread=5 method=kl

3.量化模型

• 源码位置:/home/standard/yolo/ncnn/tools/quantize/ncnn2int8.cpp

./home/standard/yolo/ncnn/build/tools/quantize/ncnn2int8 yolov8n.ncnn_opt.param yolov8n.ncnn_opt.bin yolov8n.ncnn_opt_int8.param yolov8n.ncnn_opt_int8.bin yolov8n.ncnn_opt.table

三、推理例程

• python版本:/home/standard/yolo/ncnn/python/examples/yolov8.py
• c++版本：/home/standard/yolo/ncnn/examples/yolov8.cpp
  编译后的地址:/home/standard/yolo/ncnn/build/examples/yolov8 (可用于gdb调试)

四、调试CMAKE的写法：

断点调试：/home/standard/yolo/ncnn/examples/CMakeLists.txt
```python
{
        "name": "CMake Debug",
        "type": "cmake",
        "request": "launch",
        "scriptPath": "/home/standard/yolo/catkin_workspace/src/standard_perception/CMakeLists.txt",
        "cmakeDebugType": "script",
    },
    {
        "name": "CMake Configure",
        "type": "cmake",
        "request": "launch",
        "cmakeDebugType": "configure"
    }

```