ais-bench

AISBench Benchmark is a model evaluation tool built based on OpenCompass. It supports evaluation scenarios for both accuracy and performance testing of AI models on Ascend NPU.

| Accuracy Evaluation | Model accuracy on text/multimodal datasets | | Performance Evaluation | Latency, throughput, stress testing | | Steady-State Performance | Obtain true optimal system performance | | Real Traffic Simulation | Simulate real business traffic patterns | | Multi-turn Dialogue | Evaluate multi-turn conversation models |

| Function Call (BFCL) | Function calling capability evaluation |

AISBench Benchmark - Ascend NPU 的 AI 模型评估工具。支持准确性评估（文本、多模态数据集上的服务/本地模型）、性能评估（延迟、吞吐量、压力测试、稳态、真实流量模拟）、vLLM/Triton 推理服务、15+ 基准测试（MMLU、GSM8K、MMMU、docvqa、ocrbench_v2 等）、多轮对话、函数调用 (BFCL) 和自定义数据集。 来源：ascend-ai-coding/awesome-ascend-skills。