03-大模型部署之基于Kubernetes+vLLM+Ray多机多卡集群部署与容器调度方案

1. 架构概述

1.1 为什么需要Ray集成

Ray是一个分布式计算框架,当与vLLM结合时,可以提供以下优势:

  • 智能资源调度:Ray能够智能分配和调度GPU资源
  • 弹性伸缩:根据工作负载动态调整集群规模
  • 容错机制:自动处理节点故障和任务重试
  • 多任务并行:支持同时运行多个推理任务
  • 资源隔离:为不同用户提供资源隔离

1.2 整体架构设计

┌─────────────────────────────────────────────────────────────────────────────┐
│                           Kubernetes集群                                    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                        Ray集群                                      │   │
│  │                                                                     │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐ │   │
│  │  │   Ray Head  │  │Ray Worker 1 │  │Ray Worker 2 │  │Ray Worker N │ │   │
│  │  │             │  │             │  │             │  │             │ │   │
│  │  │ vLLM Driver │  │ vLLM Worker │  │ vLLM Worker │  │ vLLM Worker │ │   │
│  │  │             │  │ (GPU 0,1)   │  │ (GPU 2,3)   │  │ (GPU N,N+1) │ │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                        服务层                                       │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │   │
│  │  │API Gateway  │  │  负载均衡器  │  │  监控系统   │  │  日志收集   │   │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘

2. 环境准备与依赖

2.1 硬件要求

最小配置
  • Kubernetes Master节点:4 CPU,8GB内存
  • Ray Head节点:8 CPU,32GB内存,1张GPU
  • Ray Worker节点:16 CPU,64GB内存,2-4张GPU
  • 网络:节点间万兆网络
推荐配置
  • 存储:高速SSD存储用于模型缓存
  • 内存:每个节点至少128GB内存
  • GPU:NVIDIA A100或V100,每节点4-8张
  • 网络:InfiniBand或高速以太网

2.2 软件依赖

# Kubernetes组件
kubectl version >= 1.24
helm version >= 3.8

# NVIDIA组件
nvidia-driver >= 470
cuda-toolkit >= 11.7
nvidia-docker2

# Ray组件
ray[default] >= 2.0
ray-dashboard

# vLLM组件
vllm >= 0.2.5
torch >= 2.0.0

3. Ray集群部署

3.1 Ray Head节点配置

# ray-head-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ray-head
  namespace: ray-vllm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ray-head
      component: head
  template:
    metadata:
      labels:
        app: ray-head
        component: head
    spec:
      containers:
      - name: ray-head
        image: rayproject/ray:2.8.0-py38-gpu
        ports:
        - containerPort: 6379
          name: gcs-server
        - containerPort: 8265
          name: dashboard
        - containerPort: 10001
          name: client
        command: ["/bin/bash", "-c", "--"]
        args:
        - |
          ray start --head --port=6379 --dashboard-host=0.0.0.0 --dashboard-port=8265 \
            --num-cpus=8 --num-gpus=1 --block
        env:
        - name: RAY_BACKEND_LOG_LEVEL
          value: "debug"
        resources:
          requests:
            cpu: 4
            memory: "16Gi"
            nvidia.com/gpu: 1
          limits:
            cpu: 8
            memory: "32Gi"
            nvidia.com/gpu: 1
        volumeMounts:
        - name: ray-logs
          mountPath: /tmp/ray
        livenessProbe:
          exec:
            command:
            - ray
            - status
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - ray
            - status
          initialDelaySeconds: 10
          periodSeconds: 5
      volumes:
      - name: ray-logs
        emptyDir: {}

3.2 Ray Worker节点配置

# ray-worker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ray-worker
  namespace: ray-vllm
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ray-worker
      component: worker
  template:
    metadata:
      labels:
        app: ray-worker
        component: worker
    spec:
      containers:
      - name: ray-worker
        image: rayproject/ray:2.8.0-py38-gpu
        command: ["/bin/bash", "-c", "--"]
        args:
        - |
          ray start --address=ray-head-svc:6379 \
            --num-cpus=16 --num-gpus=2 --block
        env:
        - name: RAY_BACKEND_LOG_LEVEL
          value: "info"
        resources:
          requests:
            cpu: 8
            memory: "32Gi"
            nvidia.com/gpu: 2
          limits:
            cpu: 16
            memory: "64Gi"
            nvidia.com/gpu: 2
        volumeMounts:
        - name: ray-logs
          mountPath: /tmp/ray
        - name: model-cache
          mountPath: /models
      volumes:
      - name: ray-logs
        emptyDir: {}
      - name: model-cache
        persistentVolumeClaim:
          claimName: model-cache-pvc

3.3 Ray服务配置

# ray-head-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: ray-head-svc
  namespace: ray-vllm
spec:
  selector:
    app: ray-head
    component: head
  ports:
  - name: gcs-server
    port: 6379
    targetPort: 6379
  - name: dashboard
    port: 8265
    targetPort: 8265
  - name: client
    port: 10001
    targetPort: 10001
  type: ClusterIP

3.4 Ray集群自动扩缩容

# ray-cluster-autoscaler.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-cluster-config
  namespace: ray-vllm
data:
  cluster.yaml: |
    cluster_name: vllm-cluster
    max_workers: 20
    upscaling_speed: 2.0
    idle_timeout_minutes: 10
    
    provider:
      type: kubernetes
      namespace: ray-vllm
      use_internal_ips: true
      
    head_node:
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: ray-head
      spec:
        replicas: 1
        template:
          spec:
            containers:
            - name: ray-head
              image: rayproject/ray:2.8.0-py38-gpu
              resources:
                requests:
                  cpu: 4
                  memory: "16Gi"
                  nvidia.com/gpu: 1
                limits:
                  cpu: 8
                  memory: "32Gi"
                  nvidia.com/gpu: 1
    
    worker_nodes:
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: ray-worker
      spec:
        replicas: 3
        template:
          spec:
            containers:
            - name: ray-worker
              image: rayproject/ray:2.8.0-py38-gpu
              resources:
                requests:
                  cpu: 8
                  memory: "32Gi"
                  nvidia.com/gpu: 2
                limits:
                  cpu: 16
                  memory: "64Gi"
                  nvidia.com/gpu: 2

4. vLLM与Ray集成

4.1 vLLM Ray部署器

# vllm_ray_deployer.py
import ray
from ray import serve
from vllm import LLM, SamplingParams

@serve.deployment(
    name="vllm-deployment",
    num_replicas=3,
    ray_actor_options={"num_gpus": 1}
)
class VLLMDeployment:
    def __init__(self, model_name: str):
        self.llm = LLM(
            model=model_name,
            tensor_parallel_size=1,
            gpu_memory_utilization=0.9
        )
        self.sampling_params = SamplingParams(
            temperature=0.8,
            top_p=0.95,
            max_tokens=1000
        )
    
    async def generate(self, prompt: str):
        outputs = self.llm.generate([prompt], self.sampling_params)
        return {"generated_text": outputs[0].outputs[0].text}
    
    async def generate_batch(self, prompts: list):
        outputs = self.llm.generate(prompts, self.sampling_params)
        return [
            {"prompt": output.prompt, "generated_text": output.outputs[0].text}
            for output in outputs
        ]

# 启动服务
def start_vllm_service(model_name: str = "meta-llama/Llama-2-7b-chat-hf"):
    ray.init(address="auto")  # 连接到Ray集群
    
    serve.start(http_options={"host": "0.0.0.0", "port": 8000})
    
    # 部署vLLM服务
    VLLMDeployment.options(name=model_name.split("/")[-1]).deploy(model_name)
    
    print(f"vLLM服务已启动,模型: {model_name}")

if __name__ == "__main__":
    start_vllm_service()

4.2 Ray Serve配置文件

# ray-serve-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-serve-config
  namespace: ray-vllm
data:
  config.yaml: |
    serve:
      deployments:
        - name: vllm-llama2-7b
          import_path: vllm_ray_deployer:VLLMDeployment
          init_args:
            model_name: "meta-llama/Llama-2-7b-chat-hf"
          autoscaling_config:
            min_replicas: 1
            max_replicas: 10
            target_num_ongoing_requests_per_replica: 2
          ray_actor_options:
            num_gpus: 1
            num_cpus: 4
            
        - name: vllm-llama2-13b
          import_path: vllm_ray_deployer:VLLMDeployment
          init_args:
            model_name: "meta-llama/Llama-2-13b-chat-hf"
          autoscaling_config:
            min_replicas: 1
            max_replicas: 5
            target_num_ongoing_requests_per_replica: 1
          ray_actor_options:
            num_gpus: 2
            num_cpus: 8

4.3 多模型管理

# multi_model_manager.py
import ray
from ray import serve
from typing import Dict, List
import asyncio

class MultiModelManager:
    def __init__(self):
        self.models = {}
        self.model_configs = {
            "llama2-7b": {
                "model_name": "meta-llama/Llama-2-7b-chat-hf",
                "tensor_parallel_size": 1,
                "max_tokens": 1000
            },
            "llama2-13b": {
                "model_name": "meta-llama/Llama-2-13b-chat-hf",
                "tensor_parallel_size": 2,
                "max_tokens": 1500
            },
            "llama2-70b": {
                "model_name": "meta-llama/Llama-2-70b-chat-hf",
                "tensor_parallel_size": 4,
                "max_tokens": 2000
            }
        }
    
    async def deploy_model(self, model_id: str):
        if model_id not in self.model_configs:
            raise ValueError(f"未知模型ID: {model_id}")
        
        config = self.model_configs[model_id]
        
        @serve.deployment(
            name=f"vllm-{model_id}",
            num_replicas=1,
            ray_actor_options={"num_gpus": config["tensor_parallel_size"]}
        )
        class ModelDeployment:
            def __init__(self):
                from vllm import LLM, SamplingParams
                self.llm = LLM(
                    model=config["model_name"],
                    tensor_parallel_size=config["tensor_parallel_size"]
                )
                self.sampling_params = SamplingParams(
                    max_tokens=config["max_tokens"]
                )
            
            async def generate(self, prompt: str):
                outputs = self.llm.generate([prompt], self.sampling_params)
                return outputs[0].outputs[0].text
        
        # 部署模型
        ModelDeployment.deploy()
        self.models[model_id] = ModelDeployment.get_handle()
        
        return f"模型 {model_id} 部署成功"
    
    async def generate(self, model_id: str, prompt: str):
        if model_id not in self.models:
            await self.deploy_model(model_id)
        
        handle = self.models[model_id]
        return await handle.generate.remote(prompt)
    
    async def generate_batch(self, model_id: str, prompts: List[str]):
        if model_id not in self.models:
            await self.deploy_model(model_id)
        
        handle = self.models[model_id]
        tasks = [handle.generate.remote(prompt) for prompt in prompts]
        return await asyncio.gather(*tasks)

# 启动多模型管理器
def start_multi_model_service():
    ray.init(address="auto")
    serve.start(http_options={"host": "0.0.0.0", "port": 8000})
    
    manager = MultiModelManager()
    
    @serve.deployment(name="model-manager")
    class ManagerDeployment:
        def __init__(self):
            self.manager = manager
        
        async def __call__(self, request):
            import json
            data = await request.json()
            
            if data["action"] == "generate":
                result = await self.manager.generate(
                    data["model_id"], 
                    data["prompt"]
                )
                return {"result": result}
            elif data["action"] == "generate_batch":
                results = await self.manager.generate_batch(
                    data["model_id"],
                    data["prompts"]
                )
                return {"results": results}
            elif data["action"] == "deploy":
                result = await self.manager.deploy_model(data["model_id"])
                return {"message": result}
    
    ManagerDeployment.deploy()
    print("多模型管理服务已启动")

if __name__ == "__main__":
    start_multi_model_service()

5. 资源调度与负载均衡

5.1 自定义调度器

# custom_scheduler.py
import ray
from ray.util.scheduling_strategies import PlacementGroupSchedulingStrategy
from ray.util.placement_group import placement_group

class VLLMScheduler:
    def __init__(self):
        self.placement_groups = {}
        self.model_requirements = {
            "llama2-7b": {"num_gpus": 1, "num_cpus": 4},
            "llama2-13b": {"num_gpus": 2, "num_cpus": 8},
            "llama2-70b": {"num_gpus": 4, "num_cpus": 16}
        }
    
    def create_placement_group(self, model_id: str, num_replicas: int = 1):
        """为模型创建放置组"""
        if model_id not in self.model_requirements:
            raise ValueError(f"未知模型: {model_id}")
        
        reqs = self.model_requirements[model_id]
        
        # 创建捆绑包
        bundles = [
            {"GPU": reqs["num_gpus"], "CPU": reqs["num_cpus"]}
            for _ in range(num_replicas)
        ]
        
        # 创建放置组
        pg = placement_group(bundles, strategy="STRICT_SPREAD")
        ray.get(pg.ready())
        
        self.placement_groups[model_id] = pg
        return pg
    
    def get_scheduling_strategy(self, model_id: str):
        """获取调度策略"""
        if model_id not in self.placement_groups:
            self.create_placement_group(model_id)
        
        pg = self.placement_groups[model_id]
        return PlacementGroupSchedulingStrategy(
            placement_group=pg,
            placement_group_capture_child_tasks=True
        )
    
    def release_placement_group(self, model_id: str):
        """释放放置组"""
        if model_id in self.placement_groups:
            ray.util.remove_placement_group(self.placement_groups[model_id])
            del self.placement_groups[model_id]

# 使用自定义调度器
scheduler = VLLMScheduler()

@ray.remote
def deploy_vllm_with_scheduler(model_id: str):
    """使用自定义调度器部署vLLM"""
    from vllm import LLM
    
    strategy = scheduler.get_scheduling_strategy(model_id)
    
    with strategy:
        llm = LLM(model=f"meta-llama/{model_id}-chat-hf")
        return llm

5.2 负载均衡策略

# load_balancer.py
import ray
from typing import List, Dict
import time
import random

class VLLMLoadBalancer:
    def __init__(self):
        self.model_instances = {}  # model_id -> list of instance handles
        self.instance_loads = {}   # instance_handle -> current_load
        self.last_update = {}      # instance_handle -> last_update_time
    
    def register_instance(self, model_id: str, instance_handle):
        """注册vLLM实例"""
        if model_id not in self.model_instances:
            self.model_instances[model_id] = []
        
        self.model_instances[model_id].append(instance_handle)
        self.instance_loads[instance_handle] = 0
        self.last_update[instance_handle] = time.time()
    
    def select_instance(self, model_id: str, strategy: str = "least_loaded"):
        """选择实例处理请求"""
        if model_id not in self.model_instances:
            raise ValueError(f"模型 {model_id} 没有可用实例")
        
        instances = self.model_instances[model_id]
        
        if strategy == "least_loaded":
            # 选择负载最少的实例
            return min(instances, key=lambda x: self.instance_loads[x])
        elif strategy == "round_robin":
            # 轮询选择
            return instances[random.randint(0, len(instances)-1)]
        elif strategy == "random":
            # 随机选择
            return random.choice(instances)
        else:
            raise ValueError(f"未知策略: {strategy}")
    
    async def generate(self, model_id: str, prompt: str, strategy: str = "least_loaded"):
        """使用负载均衡生成文本"""
        instance = self.select_instance(model_id, strategy)
        
        # 增加负载计数
        self.instance_loads[instance] += 1
        
        try:
            # 调用实例生成文本
            result = await instance.generate.remote(prompt)
            return result
        finally:
            # 减少负载计数
            self.instance_loads[instance] -= 1
            self.last_update[instance] = time.time()
    
    def get_load_stats(self):
        """获取负载统计"""
        stats = {}
        for model_id, instances in self.model_instances.items():
            stats[model_id] = {
                "total_instances": len(instances),
                "total_load": sum(self.instance_loads[i] for i in instances),
                "avg_load": sum(self.instance_loads[i] for i in instances) / len(instances)
            }
        return stats

# 全局负载均衡器
load_balancer = VLLMLoadBalancer()

6. 监控与运维

6.1 Ray Dashboard集成

# ray-dashboard-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: ray-dashboard
  namespace: ray-vllm
spec:
  selector:
    app: ray-head
    component: head
  ports:
  - port: 8265
    targetPort: 8265
  type: LoadBalancer

6.2 自定义监控指标

# monitoring.py
import ray
import time
import psutil
from prometheus_client import Counter, Histogram, Gauge, start_http_server

# Prometheus指标
REQUEST_COUNT = Counter('vllm_requests_total', 'Total vLLM requests', ['model_id'])
REQUEST_LATENCY = Histogram('vllm_request_duration_seconds', 'vLLM request latency')
GPU_MEMORY_USAGE = Gauge('vllm_gpu_memory_usage_bytes', 'GPU memory usage', ['gpu_id'])
ACTIVE_INSTANCES = Gauge('vllm_active_instances', 'Number of active vLLM instances', ['model_id'])

class VLLMMonitor:
    def __init__(self):
        self.start_time = time.time()
        start_http_server(8001)  # Prometheus metrics端口
    
    @ray.remote
    def collect_metrics(self):
        """收集指标"""
        import torch
        
        # GPU内存使用情况
        if torch.cuda.is_available():
            for i in range(torch.cuda.device_count()):
                memory_used = torch.cuda.memory_allocated(i)
                GPU_MEMORY_USAGE.labels(gpu_id=str(i)).set(memory_used)
        
        # 系统资源使用情况
        cpu_percent = psutil.cpu_percent()
        memory = psutil.virtual_memory()
        
        return {
            "timestamp": time.time(),
            "cpu_percent": cpu_percent,
            "memory_percent": memory.percent,
            "memory_used_gb": memory.used / (1024**3)
        }
    
    def record_request(self, model_id: str, duration: float):
        """记录请求指标"""
        REQUEST_COUNT.labels(model_id=model_id).inc()
        REQUEST_LATENCY.observe(duration)
    
    def update_active_instances(self, model_id: str, count: int):
        """更新活跃实例数"""
        ACTIVE_INSTANCES.labels(model_id=model_id).set(count)

# 全局监控实例
monitor = VLLMMonitor()

6.3 告警配置

# prometheus-rules.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: vllm-alerts
  namespace: ray-vllm
spec:
  groups:
  - name: vllm.rules
    rules:
    - alert: VLLMHighLatency
      expr: histogram_quantile(0.95, rate(vllm_request_duration_seconds_bucket[5m])) > 10
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: "vLLM高延迟告警"
        description: "vLLM请求延迟超过10秒"
    
    - alert: VLLMGPUMemoryHigh
      expr: vllm_gpu_memory_usage_bytes / (1024^3) > 30
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: "vLLM GPU内存使用率过高"
        description: "GPU内存使用超过30GB"
    
    - alert: VLLMNoActiveInstances
      expr: vllm_active_instances == 0
      for: 1m
      labels:
        severity: critical
      annotations:
        summary: "vLLM没有活跃实例"
        description: "所有vLLM实例都已停止"

7. 故障处理与恢复

7.1 自动故障恢复

# fault_tolerance.py
import ray
import time
from typing import Dict, List

class VLLMFailoverManager:
    def __init__(self):
        self.health_check_interval = 30  # 30秒检查一次
        self.max_retries = 3
        self.retry_delay = 5
        self.unhealthy_instances = set()
    
    @ray.remote
    def health_check(self, instance_handle):
        """检查实例健康状态"""
        try:
            # 发送健康检查请求
            result = ray.get(instance_handle.health_check.remote(), timeout=10)
            return result.get("status") == "healthy"
        except Exception as e:
            print(f"健康检查失败: {e}")
            return False
    
    async def monitor_instances(self):
        """监控所有实例"""
        while True:
            # 获取所有实例
            all_instances = ray.state.actors()
            
            for actor_id, actor_info in all_instances.items():
                if "vllm" in actor_info["name"]:
                    instance_handle = ray.get_actor(actor_id.split("-")[-1])
                    
                    # 执行健康检查
                    is_healthy = await self.health_check.remote(instance_handle)
                    
                    if not is_healthy:
                        if actor_id not in self.unhealthy_instances:
                            print(f"实例 {actor_id} 不健康")
                            self.unhealthy_instances.add(actor_id)
                            await self.restart_instance(actor_id)
                    else:
                        if actor_id in self.unhealthy_instances:
                            print(f"实例 {actor_id} 恢复健康")
                            self.unhealthy_instances.remove(actor_id)
            
            await asyncio.sleep(self.health_check_interval)
    
    async def restart_instance(self, actor_id: str):
        """重启不健康的实例"""
        try:
            # 获取实例配置
            actor_info = ray.state.actors()[actor_id]
            model_name = actor_info.get("model_name", "default")
            
            # 杀死旧实例
            ray.kill(actor_id)
            
            # 等待一段时间
            await asyncio.sleep(self.retry_delay)
            
            # 重启实例
            new_actor = VLLMDeployment.options(name=actor_id).remote(model_name)
            print(f"实例 {actor_id} 已重启")
            
        except Exception as e:
            print(f"重启实例 {actor_id} 失败: {e}")

# 启动故障恢复管理器
failover_manager = VLLMFailoverManager()

7.2 数据备份与恢复

# backup-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: vllm-backup
  namespace: ray-vllm
spec:
  schedule: "0 2 * * *"  # 每天凌晨2点执行
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: your-registry/vllm-backup:latest
            command:
            - /bin/bash
            - -c
            - |
              # 备份模型缓存
              tar -czf /backup/model-cache-$(date +%Y%m%d).tar.gz /models
              
              # 备份配置文件
              kubectl get configmap -n ray-vllm -o yaml > /backup/config-$(date +%Y%m%d).yaml
              
              # 清理旧备份(保留7天)
              find /backup -name "*.tar.gz" -mtime +7 -delete
              find /backup -name "*.yaml" -mtime +7 -delete
            volumeMounts:
            - name: model-cache
              mountPath: /models
            - name: backup-storage
              mountPath: /backup
          volumes:
          - name: model-cache
            persistentVolumeClaim:
              claimName: model-cache-pvc
          - name: backup-storage
            persistentVolumeClaim:
              claimName: backup-pvc
          restartPolicy: OnFailure

8. 最佳实践

8.1 性能优化

  1. 资源分配:根据模型大小合理分配GPU资源
  2. 批处理:启用连续批处理提高吞吐量
  3. 缓存策略:使用PVC缓存模型文件
  4. 网络优化:使用高速网络减少通信延迟

8.2 安全考虑

  1. 网络隔离:使用NetworkPolicy限制Pod间通信
  2. 密钥管理:使用Kubernetes Secret管理API密钥
  3. 镜像安全:定期扫描镜像漏洞
  4. 访问控制:实施RBAC权限控制

8.3 运维建议

  1. 版本管理:使用Git管理配置文件
  2. 渐进式部署:使用金丝雀发布策略
  3. 监控告警:建立完善的监控告警体系
  4. 文档维护:保持运维文档的及时更新

通过以上方案,可以构建一个高可用、可扩展的基于Kubernetes+vLLM+Ray的多机多卡集群部署系统,满足生产环境中的大语言模型推理需求。

Logo

聚焦RDK与具身智能,汇集机器人开发教程、开源案例和技术交流

更多推荐