搜索

按 Enter 键快速全文搜索，也可直接发起 AI 搜索
 首页 文库资料 文章资讯  上传文档  发布文章  登录账户

分类

全部综合其他(20)人工智能(20)云计算&大数据(1)机器学习(1)

语言

全部英语(19)中文（简体）(2)

格式

全部PDF文档 PDF(21)
 
本次搜索耗时 0.058 秒，为您找到相关结果约 21 个.

  • pdf文档 vLLM v0.6.1 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 215 页 | 1.29 MB | 3 月前
    3

  • pdf文档 vLLM v0.4.2 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 99 页 | 982.83 KB | 3 月前
    3

  • pdf文档 vLLM v0.6.0 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 201 页 | 1.26 MB | 3 月前
    3

  • pdf文档 vLLM v0.5.0.post1 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 144 页 | 1.09 MB | 3 月前
    3

  • pdf文档 vLLM v0.5.3.post1 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 143 页 | 1.07 MB | 3 月前
    3

  • pdf文档 vLLM v0.5.3 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 143 页 | 1.07 MB | 3 月前
    3

  • pdf文档 vLLM v0.5.1 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 162 页 | 1.14 MB | 3 月前
    3

  • pdf文档 vLLM v0.5.4 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 152 页 | 1.10 MB | 3 月前
    3

  • pdf文档 vLLM v0.4.0.post1 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 68 页 | 810.15 KB | 3 月前
    3

  • pdf文档 vLLM v0.5.2 Documentation

    State-of-the-art serving throughput - Efficient management of attention key and value memory with PagedAttention - Continuous batching of incoming requests - Fast model execution with CUDA/HIP graph - Quantization: support For more information, check out the following: - vLLM announcing blog post (intro to PagedAttention) - vLLM paper (SOSP 2023) - How continuous batching enables 23x throughput in LLM inference input_ids and positions are now flattened tensors. 2. Replace the attention operation with either PagedAttention, PagedAttentionWithRoPE, or PagedAttentionWithALiBi depending on the model's architecture
    0 码力 | 166 页 | 1.15 MB | 3 月前
    3
共 21 条
  • 1
  • 2
  • 3
前往
页
相关搜索词
vLLMKV cachePagedAttentionLoRA多模态模型量化投资LLM分布式推理KV缓存管理量化分批处理模型支持多模态推理引擎性能监控Vision Language Modelsmulti_modal_datapreemptionchunked prefillperformance tuningOffline Batched InferencePreemptionChunked PrefillMultiModalDataDictpaged attention多模态数据连续批量处理预emptioncontinuous batchingLLM inferencequantizationproduction metricsusage statisticsmulti-modal models