【開發資源】TinyML MCU 等級開源推論引擎

作者：Jack Hsu

本文轉載自歐尼克斯實境互動工作室(OmniXRI)，2025.6.5 整理製作

開源推論引擎在人工智慧應用中扮演著關鍵角色，提供了多樣化且高效能的解決方案，以應對從嵌入式系統到大型資料中心的廣泛需求。這些引擎不僅支援各種硬體平台，還積極參與開源社群，促進了模型部署的標準化和跨平台互通。

舉例來說，Arm的CMSIS-6包括了NN和DSP模組，專為微控制器和DSP優化，而Google的LiteRT和TensorFlow Lite系列則強調在邊緣裝置上的低功耗高效運算。同時，PyTorch的Executorch和QNNPack以及Alibaba的MNN和TinyNeuralNetwork，以及Tencent的TNN和NCNN，都提供了多樣化的運算解決方案，涵蓋了從高效能計算到低功耗嵌入式系統的各種需求。

這開源專案透過其獨特的技術和演算法貢獻，豐富了開發者在機器學習和深度學習模型轉換及優化的工具選擇。這些引擎和工具的共同目標是推動AI技術在端點裝置上的普及，並促進其在各種應用領域中的實用化和擴展。以下整理一些連結資源供開發者朋友參考：

開源推論引擎 #

• Arm CMSIS-6 (NN, DSP)
  https://github.com/ARM-software/CMSIS_6
• Google LiteRT （TensorFlow Lite）
  https://github.com/google-ai-edge/litert
• Google TensorFlow Lite for Microcontroller (TFLite-Micro)
  https://github.com/tensorflow/tflite-micro
• PyTorch ExecuTorch
  https://github.com/pytorch/executorch
• PyTorch QNNPack
  https://github.com/pytorch/qnnpack
• Alibaba MNN
  https://github.com/alibaba/MNN
• Alibaba TinyNeuralNetwork
  https://github.com/alibaba/TinyNeuralNetwork
  
• Tencent TNN
  https://github.com/Tencent/TNN
• Tencent NCNN
  https://github.com/Tencent/ncnn
• uTensor
  https://github.com/uTensor/uTensor
• Fraunhofer-IMS AIfES
  https://github.com/Fraunhofer-IMS/AIfES_for_Arduino
• Sipeed TinyMaix
  https://github.com/sipeed/TinyMaix
• XiaoMi MACE
  https://github.com/XiaoMi/mace
• PaddlePaddle Paddle-Lite
  https://github.com/PaddlePaddle/Paddle-Lite
• Eloquent EloquentMicroML
  https://github.com/eloquentarduino/EloquentMicroML
• Kraiskil ONNX2C
  https://github.com/kraiskil/onnx2c
• Huawei MindSpore
  https://github.com/mindspore-ai/mindspore
• SenseTime OpenPPL (PPLNN)
  https://github.com/OpenPPL/ppl.nn
• Open AI Lab Tengine
  https://github.com/OAID/Tengine

AI 相關編譯器 #

• MicroTVM
  https://tvm.hyper.ai/docs/topic/microtvm/
• Skymizer ONNC
  https://github.com/ONNC/onnc
• Arm Ethos-U Vela Compiler
  https://pypi.org/project/ethos-u-vela/

MCU 專用工具 #

• Edge Impulse
  https://docs.edgeimpulse.com/docs
• STM32
  • CubeAI (X-CUBE-AI)
    https://stm32ai.st.com/stm32-cube-ai/
  • NanoEdge AI Studio
    https://stm32ai.st.com/nanoedge-ai/
  • ST Edge AI Suit
    https://www.st.com/content/st_com/en/st-edge-ai-suite.html
• NXP eIQ
  https://www.nxp.com/design/design-center/software/eiq-ai-development-environment:EIQ
• Renesas eAI
  https://www.renesas.com/en/key-technologies/artificial-intelligence/e-ai/e-ai-development-environment-microcontrollers