Browsing Tag

System optimization

4 posts
retrieval metrics and embeddings

Evaluating Retrieval Quality: Recall@K, Ndcg, and Embedding Choices

Understanding retrieval metrics like Recall@K and NDCG, along with embedding choices, unlocks better system performance—discover how to optimize your results.
kv cache offloading techniques

KV Cache Offloading: Techniques, Trade‑offs, and Hardware Support

Learn how offloading KV cache tasks with specialized hardware can enhance performance but involves critical trade-offs worth exploring.