CacheBench: Building a Benchmarking Suite for Cache Performance Evaluation

Fri, 28 Feb 2025 00:00:00 +0000

Overview

In this project, we aim to develop a comprehensive benchmarking suite, CacheBench, for evaluating the performance of cache systems in modern computing environments. Caches play a crucial role in enhancing system performance by reducing latency and improving data access speeds. However, evaluating cache performance is a complex task that requires a diverse set of workloads and metrics to capture the cache’s behavior accurately. The current focus is on the eviction algorithms and if time permits, we will extend to other components of cache design.

This project will have three main components:

Implementing and benchmarking existing cache eviction algorithms in libCacheSim using large-scale simulation. This part will mainly focus on reproducing existing works.
Developing a set of microbenchmarks and a platform for researchers to evaluate new designs with little effort in the future. This part will focus on building the open-source infrastructure for future research.
Developing a leaderboard for the community to submit new algorithms and workloads. This part will focus on building the community and fostering adoption and collaboration.

Topics: storage systems, benchmarking, performance evaluation
Skills: C programming, web programming (e.g., node.js, React), database management
Difficulty: Moderate
Size: Large (350 hours).
Mentors: Juncheng Yang, Yazhuo Zhang (yazhuo@inf.ethz.ch)

IO logger: IO tracing in the modern computing era

Fri, 28 Feb 2025 00:00:00 +0000

Overview

Storage systems are critical components of modern computing infrastructures, and understanding their performance characteristics is essential for optimizing system efficiency. There were many works from twenty to thirty years ago, but the landscape has changed significantly with the advent of

cloud computing, virtualization, and storage disaggregation on the server side
ubiquitous fast wireless networking for end users that make remote storage feasible
AI and ML workloads that generate and move massive data both in the cloud and on the edge.

In this project, we aim to develop an IO logger, a tool for tracing, logging and analyzing IO operations in various computing environments. The IO logger will capture detailed information about read and write operations, latency, throughput, and other metrics to help researchers and practitioners understand the behavior of storage systems under different workloads and configurations. By providing a comprehensive view of IO performance, the IO logger will enable users to identify bottlenecks, optimize resource utilization, and improve system efficiency.

This project will have two phases:

IO logger for *NIX systems: Develop a tool leveraging eBPF and other tools for tracing IO operations on Linux and other Unix-like systems. The tool will capture detailed information about disk reads and writes, network transfers, and other IO activities, providing insights into system performance. The tool will be open-sourced, and we will work with industry partners and testbeds to integrate it into existing monitoring and analysis tools. Moreover, we will collect and open source the IO traces to benefit the community.
IO logger for personal computing environment: Develop a tool for end-users to trace IO operations on their personal devices, such as laptops, desktops, and mobile phones. We will design and implement tools for three different platforms, Window, MacOS and Andriod. We will use the tools to collect IO traces from volunteers and real-world applications. providing insights into storage usage, network activity, and application performance. The tool will be user-friendly, lightweight, and privacy-preserving, ensuring that users can monitor their IO activities without compromising their data security.

Notable difference and challenges compared to the existing works are:

more IO requests with rich features: open-source traces from previous works were collected all after page cache, which are often write-heavy, lose most IO requests, and do not provide enough features, e.g., process name. To address this, we will build a tool that can also records requests served by page cache, which requires the tool to be efficient and cannot impose significant overhead to the ruuning systems.
focus on new applications and workloads: the existing works were mostly outdated from the 1990s, during which the Internet has not been widely used, and applications are mostly processing local data and does not communicate with outside world. While there have been a few works looked into mobile storage a decade ago. The landscape has changed significantly since then, especially with the advent of AI and ML workloads that generate and move massive data both in the cloud and on the edge. This project will look into the difference and challenges brought by these new applications and workloads.

Topics: tracing tool, operating system, eBPF, performance evaluation
Skills: C programming, system programming, eBPF, Linux kernel, mobile application development
Difficulty: Hard
Size: Large (350 hours).
Mentors: Juncheng Yang

Juncheng Yang | UCSC OSPO

CacheBench: Building a Benchmarking Suite for Cache Performance Evaluation

Overview

IO logger: IO tracing in the modern computing era

Overview

Related works