Jiayuan Zhu | UCSC OSPO

Public Artifact and Data Visualization: A Journey to Empower

Tue, 24 Oct 2023 00:00:00 +0000

Hola Amigos! As we draw the curtains on our project titled Public Artifact and Data Visualization we’re thrilled to present the incredible advancements we’ve achieved since our mid-term update. Our mission has been to foster a deeper understanding of data and empower users to make informed decisions. Let’s delve into the remarkable evolution of our project.

Unveiling New Functionalities

Modular Architecture: Your Way, Your Choice

At the core of our project is a modular architecture designed to cater to your unique preferences. We firmly believe that choice empowers users. Thus, we’ve given you the option to select between a Graphical User Interface (GUI) and a Command-Line Interface (CLI). It’s about providing a platform that adapts to your specific requirements and style of interaction.

Real-time Backend Environment Monitoring: Data as it Happens

Real-time monitoring of backend environment data is at the heart of our project. It’s not just about collecting data; it’s about providing continuous insights into system performance. This feature empowers you to make real-time, data-driven decisions—an essential capability in today’s fast-paced computing landscape.

Visualizing Environment Variables: Clarity Amidst Complexity

We’ve placed a strong emphasis on user-friendly data visualization. Our enhancements enable you to navigate through detected variables effortlessly and compare iterations within different buckets. The result is a visual representation of complex data, making it easier to comprehend and analyze.

Predefined Monitoring Commands: Your Head Start

We understand that monitoring can be a daunting task. To simplify the process, we’ve introduced predefined monitoring commands such as mpstat and iostat. These templates serve as a launchpad for monitoring common system metrics, helping you get started quickly and efficiently.

Comprehensive Customization: Tailoring the Experience

Recognizing that every user has unique needs, our platform now offers extensive documentation. This documentation serves as a guide, enabling users to fine-tune their monitoring commands. It’s about tailoring the platform to match your specific requirements and preferences. The power to customize is firmly in your hands.

Import and Export Functionality: Seamless Collaboration

In an era where collaboration and data management are essential, we’ve introduced the capability to import and export environment data. This feature simplifies data management and supports collaborative efforts, making it easy to share monitoring data and conduct analysis across various environments.

Exploring Our Repositories

As mentioned earlier, we have completed the core functionalities of our platform, and we would love to have you try it out and provide us with valuable feedback. Here are the links to our repositories where you can explore and experiment with our platform:

GUI Repository and CLI Repository
- The journey begins with a choice. Our repositories cater to a diverse range of user preferences. Inside the README.md file of the GUI repository, you’ll find meticulous installation instructions to guide you through setting up the Graphical User Interface (GUI). It’s your portal to a user-friendly experience
Sample Repository
- For those eager to embark on their monitoring journey, our Sample Repository is a valuable resource. It provides scripts that not only enable you to run our program but also serve as templates. These templates are designed to simplify the monitoring of your own programs, tailored to your unique requirements.

Project Demo

To provide you with a glimpse of what our project can do, here are some demo images showcasing the capabilities and features of “Public Artifact and Data Visualization.”

Thank You for Joining Us

We appreciate your support and participation in this journey of data visualization and empowerment. Our commitment to enhancing the world of data comprehension remains unwavering. As we mark the end of this chapter, we eagerly anticipate the exciting future that awaits in the realm of data visualization. The path doesn’t end here; it’s just the beginning of a new chapter in our collective exploration of data’s potential.`

Mid-term blog post for Public Artifact Data and Visualization

Mon, 31 Jul 2023 00:00:00 +0000

Over the past few weeks, our platform development has been progressing steadily, and we are excited to share the milestones we have achieved so far. As planned in our introductory blog, we have successfully laid the groundwork for the platform with the guidance and support of our mentor.

Milestones and Accomplishments

Here are some of the key functionalities we have implemented so far:

Modular Architecture: We successfully designed the platform with a modular architecture, separating the Graphical User Interface (GUI) and Command-Line Interface (CLI) functionalities. This modularity allows users to interact with the platform in their preferred way.
Experiment and Bucket Creation: Users can now create experiments, buckets (for storing different implementations of experiments), and iterations using either the GUI or CLI.
Real-time Backend Environment Monitoring: Through the command line interface, users have the capability to control the monitoring of backend environment data, allowing for real-time tracking and analysis of important metrics.
Visualizing Environment Variables: Users can now visualize detected environment variables on the platform. Moreover, they can compare iterations within different buckets and gain more insights by observing the timeseries data, such as CPU usage, in a graphical format.

Challenges

In the early stages of designing our platform, we encountered significant challenges at the system design level. One of the most daunting obstacles we faced was devising an effective method to monitor backend environment variables. To tackle this obstacle, we engaged in extensive discussions and sought guidance from our mentor. After careful consideration, we decided to adopt a multi-process approach to monitor the backend environment variables effectively. Specifically, we devised a meticulous strategy of creating a separate process in the background for each specific metric we needed to monitor. By allocating a dedicated process to each metric, we ensured a streamlined and efficient monitoring process.

Currently, we are facing a challenge related to monitoring metrics. Since different users have varying monitoring requirements, it is impractical for us to manually write monitoring solutions for each user. To address this issue, we are actively working on implementing a pluggable design that allows users to configure their own monitoring preferences.

Our approach involves providing users with the flexibility to define their custom configuration files or write monitoring programs following our documented guidelines. This way, users can specify the specific metrics they wish to monitor and tailor the monitoring process to their individual needs.

Try it Out!

GUI Repository and CLI Repository
- In the README.md file of GUI repo, you will find detailed installation instructions to set up the Graphical User Interface (GUI). Follow the steps provided to get started with our platform.
Sample Repository
- In this repository, we have included scripts that allow you to run our program. Additionally, you can use these scripts as templates to monitor your own programs according to your specific requirements.

We welcome you to take the platform for a test drive and feel free to raise any issues you encounter during the installation process. Your feedback is invaluable to us, as it helps us identify and address any potential installation challenges and improve the user experience.

Public Artifact Data and Visualization

Sat, 17 Jun 2023 00:00:00 +0000

Hello! As part of the Public Artifact Data and Visualization our proposals (proposal from Jiayuan Zhu and proposal from Krishna Madhwani) under the mentorship of Anjo Vahldiek-Oberwagner aims to design a system that allows researchers to conveniently record and compare the environmental information, such as CPU utilization, of different iterations and versions of code during an experiment.

In academic experiments, there is often a need to compare results and performance between different iterations and versions. This comparative analysis helps researchers evaluate the impact of different experimental parameters and algorithms on the results and enables them to optimize experimental design and algorithm selection. However, to conduct effective comparative analysis, it is essential to record and compare environmental information, alongside the experimental data. This information provides valuable insights into the factors that may influence the observed outcomes.

Through this summer, we aim to develop a system that offers a streamlined interface, enabling users to effortlessly monitor their running programs using simple command-line commands. Moreover, our system will feature a user-friendly dashboard where researchers can access historical runtime information and visualize comparisons between different iterations. The dashboard will present comprehensive graphs and charts, facilitating the analysis of trends and patterns in the environmental data.