data visualization | UCSC OSPO

Environmental NeTworked Sensor (ENTS)

Fri, 30 Jan 2026 00:00:00 +0000

ENTS I: Usability improvements for visualization dashboard

Topics: Data Visualization, Backend, Frontend, UI/UX, Analytics
Skills:
- Required: React, Javascript, Python, SQL, Git
- Nice to have: Flask, Docker, CI/CD, AWS, Authentication
Difficulty: Medium
Size: Large (350 hours)
Mentors: Colleen Josephson, Alec Levy, John Madden

The Environmental NeTworked Sensor (ENTS) platform, formally Open Sensing Platform (OSP), implements data visualization website for monitoring microbial fuel cell sensors (see GitHub). The mission is to scale up the current platform to support other researchers or citizen scientists in integrating their novel sensing hardware or microbial fuel cell sensors for monitoring and data analysis. Examples of the types of sensors currently deployed are sensors measuring soil moisture, temperature, current, and voltage in outdoor settings. The focus of the software half of the project involves building upon our existing visualization web platform, and adding additional features to support the mission. A live version of the website is available here.

Below is a list of project ideas that would be beneficial to the ENTS project. You are not limited to the following projects, and encourage new ideas that enhance the platform:

Drag and drop charts functionality
Creation of unique charts by users (with unique equations)
Customizable options of charts (color, line width, datapoint/line style, axis labels)
Exportable charts (with customizable options)
Saving layouts via url

ENTS II: Migration to TockOS

Topics: Embedded system, operating system
Skills:
- Required: Rust, C/C++, Git, Github
- Nice to have: STM32 HAL, python
Difficulty: Hard
Size: Large (350 hours)
Mentors: Colleen Josephson, John Madden

The current version of the hardware firmware is implemented in baremetal through the use of STM hardware abstraction layer (HAL) drivers. We are interested in porting the firmware implementation to an operating system (OS) to allow for additional functionality to support environmental data logging. TockOS is an embedded operating system designed for running multiple concurrent, mutually distrustful applications on low-memory and low-power microcontrollers that will be used. TockOS allows for OTA updates, dynamic app loading, hardware multiplexing, and more. We envision multiple users utilizing shared ENTS hardware that provides communication and measurement capabilities. Thus, the initial cost of deploying wireless sensor networks is reduced.

The TockOS kernel is written in Rust to enhance security. Userspace apps can be written in either C, C++, or Rust. Development will be done through a remote development server to access the hardware. See the following repos for the current status of the project:

Userspace library: libtock-c
Kernel: tock
Baremetal: ENTS-node-firmware

Scope of work:

Writing kernel peripheral drivers.
- Done entirely in Rust.
- Low-level understanding of microcontroller
- Basic kernel functionality knowledge.
Porting baremetal components to userland apps.
- Involves porting STM HAL calls to TockOS syscalls.
- Primarily done in C.
- Understanding of syscalls.

Midway Through GSoC: ENTS

Thu, 24 Jul 2025 00:00:00 +0000

Midway Through GSoC

Hi everyone! I’m Devansh Kukreja, and I’m excited to share a midterm update on my Google Summer of Code 2025 project with the University of California, Santa Cruz Open Source Program Office (UC OSPO) under the Open Source Research Experience (OSRE). I’m contributing to ENTS, a platform that supports real-time monitoring and visualization of environmental sensor networks.

Project Overview

The Environmental NeTworked Sensor (ENTS) platform is an open-source web portal designed to collect, visualize, and analyze data from distributed sensor networks. It’s used by researchers and citizen scientists to monitor field-deployed sensors measuring soil moisture, temperature, voltage, and current—supporting critical research on sustainability and environmental change.

My project focuses on improving the platform’s stability, usability, and extensibility through:

Fixing bugs in the data visualization components.
Enhancing real-time chart synchronization and data point selection.
Improving overall system error handling and reliability.
Building a Logger Registration System that enables users to register and configure their logging devices.
Exploring integration with The Things Network (TTN) to support LoRaWAN-based wireless sensor connectivity.

Progress So Far

During the first half of the GSoC period, I focused on laying the groundwork for a more robust and user-friendly system. Highlights include:

Enhanced date range logic: Improved the way the dashboard selects time periods by automatically choosing a recent two-week window with valid sensor data. This ensures charts always display meaningful insights and avoids showing blank states.
Improved chart rendering: Refined how charts behave when there’s no data or when unusual values (like negatives) are present. This includes smoother axis alignment and fallback messaging when data is unavailable.
Refactored cell management UI: Cleaned up and improved the modals used to manage cells and sensors, fixing several UI/UX issues and bugs to make interactions more intuitive and consistent.
Enabled smart URL syncing: The dashboard state now stays in sync with the URL, making it easier to share specific views or navigate back to previous states without losing context.

What’s Next

In the second half of the program, I’ll be focusing on:

Building out and polishing the Logger Registration UI based on the backend schema and wireframes.
Finalizing the onboarding flow for field loggers, linking registration data to ingestion and dashboard views.
Continuing work on LoRaWAN support with TTN, aiming to enable basic OTA provisioning for future deployments.
Exploring an admin dashboard that helps visualize device health, sync status, and alert on any anomalies.

Final Thoughts

Working on ENTS has been incredibly rewarding—it’s more than just code. It’s about making tools that help scientists and conservationists understand our changing environment, and I’m honored to be a part of that.

Big thanks to my mentors Colleen Josephson, John Madden, and Alec Levy for their support and thoughtful feedback throughout. I’ve learned a ton already, and I can’t wait to keep building.

Scaling Sensor Networks for Environmental Research

Sun, 15 Jun 2025 00:00:00 +0000

Hi! I’m Devansh Kukreja, a researcher, indie developer, and Computer Science undergrad. I’m interested in distributed systems, orchestration services, and real-time data platforms. I enjoy working on systems that help different components connect and run smoothly at scale.

This summer, I’m contributing to the ENTS (Environmental NeTworked Sensor) platform with the University of California, Santa Cruz Open Source Program Office as part of Google Summer of Code 2025.

ENTS is an open-source web portal designed to collect, visualize, and analyze data from large-scale environmental sensor networks. It helps researchers and citizen scientists monitor sensors like soil moisture, temperature, current, and voltage supporting real-time environmental research in outdoor settings.

My work this summer focuses on improving the platform’s reliability and usability. I’ll be fixing visualization bugs, enhancing chart synchronization, making data point selection more intuitive, and improving error handling. Alongside that, I’m building a Logger Registration System that lets users easily add and configure their data loggers, with potential support for over-the-air provisioning via The Things Network (TTN) for LoRaWAN-based devices.

You can check out my full proposal here. I’m grateful to be mentored by Colleen Josephson, John Madden, and Alec Levy, who are guiding the project with incredible insight and support.

By the end of the summer, ENTS will be a more stable, user-friendly, and extensible platform—better equipped to support environmental research at scale. I’m super excited to learn, build, and contribute to something meaningful!

Environmental NeTworked Sensor (ENTS)

Fri, 31 Jan 2025 00:00:00 +0000

ENTS I: Web portal for large-scale sensor networks

Topics: Data Visualization, Backend, Frontend, UI/UX, Analytics
Skills:
- Required: React, Javascript, Python, SQL, Git
- Nice to have: Flask, Docker, CI/CD, AWS, Authentication
Difficulty: Medium
Size: Large (350 hours)
Mentors: Colleen Josephson, John Madden, Alec Levy

Below is a list of project ideas that would be beneficial to the ENTS project. You are not limited to the following projects, and encourage new ideas that enhance the platform:

Improve streaming functionality
Generic interface for sensor measurements
Logger registration
Over the air (OTA) configuration updates
Implement unit tests and API documentation

ENTS II: Hardware to for large-scale field sensor networks

Topics: Embedded system, wireless communication, low-power remote sensing
Skills:
- Required: C/C++, Git, Github, PlatformIO
- Nice to have: STM32 HAL, ESP32 Arduino, protobuf, python, knowledge of standard communication protocols (I2C, SPI, and UART)
Difficulty: Hard
Size: Large (350 hours)
Mentors: Colleen Josephson, John Madden, Jack Lin

The Environmental NeTworked Sensor (ENTS) node aims to be a general purpose hardware platform for outdoor sensing (e.g. agriculture, ecological monitoring, etc.). The typical use case involves a sensor deployment in an agricultural field, remotely uploading measurements without interfering with farming operations. The current hardware revision (Soil Power Sensor was originally designed for monitoring power output of microbial fuel cells using high fidelity voltage and current measurement channels, as well as auxiliary sensors such as the SDI-12 TEROS-21 soil moisture sensor. The primary activities of this project will involve low-level firmware design and implementation, but may also incorporate hardware design revisions if necessary. We are looking to expand functionality to other external sensors, as well as optimize for power consumption, via significant firmware design activities.

Long-range, low-power wireless communication is achieved through a LoRa capable STM32 microcontroller with in-lab experiments using an ESP32 microcontroller to enable the simpler WiFi interface. Both wireless interfaces communicate upload measurements to our data visualization dashboard, ENTS I. The combined goal across both of these projects is to create a system that enables researchers to test and evaluate novel sensing solutions. We are looking to make the device usable to a wide range of researchers which may not have a background in electronics, so are interested in design activities that enhance user friendliness.

In total there will be 2-4 people working on the hardware with progress being tracked on GitHub. Broader project planning is tracked through a Jira board. We intend to have weekly meetings to provide updates on current issue progress along with assigning tasks. Please reach out to John Madden if there are any questions or specific ideas for the project.

Below is a list of project ideas that would be beneficial to the ENTS project. You are not limited to the following projects, and encourage new ideas that enhance the platform:

Backup logging via SD card
I2C multiplexing for multiple of the same sensors
Batch sensor measurement uploading

Static and Interactive Visualization Capture

Fri, 30 Aug 2024 00:00:00 +0000

Introduction

Hello! My name is Arya Sarkar a machine learning engineer and researcher based out of Kolkata, a city in Eastern India dubbed the City of Joy. During summer of 2024, I worked closely with Professor David Koop on the project titled Reproducibility in Data Visualization. We explored multiple existing solutions and tested different stratergies and made great progress in the capture of visualiations using a relatively less used method of embedding visualization meta-information into the final resultant visualizations jpg as a json object.

Progress and Challenges

Static Visualization Capture

We successfully developed a method to capture static visualizations as .png files along with embedded metadata in a JSON format. This approach enables seamless reproducibility of the visualization by storing all necessary metadata within the image file itself. Our method supports both Matplotlib and Bokeh libraries and demonstrated near-perfect reproducibility, with only a minimal 1-2% pixel difference in cases where jitter (randomness) was involved.

Interactive Visualization Capture

For interactive visualizations, our focus shifted to capturing state changes in Plotly visualizations on the web. We developed a script that tracks user interactions (e.g., zoom, box, lasso, slider) using event listeners and automatically captures the visualization state as both image and metadata files. This script also maintains a history of interactions to ensure reproducibility of all interaction states.

The challenge of capturing web-based visualizations from platforms like ObservableHq remains, as iframe restrictions prevent direct access to SVG elements. Further exploration is needed to create a more robust capture method for these environments.

Future Work

We aim to package our interactive capture script into a Google Chrome extension.

Temporarily store interaction session files in the browser’s local storage.

Enable users to download captured files as a zip archive, using base64 encoding for images.

Conclusion

The last summer, we made significant strides in enhancing data visualization reproducibility. Our innovative approach to embedding metadata directly into visualization files offers a streamlined method for recreating static visualizations. The progress in capturing interactive visualization states opens new possibilities for tackling a long-standing challenge in the field of reproducibility.

Reproducibility in Data Visualization

Thu, 18 Jul 2024 00:00:00 +0000

Introduction

Hello! My name is Arya Sarkar a machine learning engineer and researcher based out of Kolkata, a city in Eastern India dubbed the City of Joy. For the last month and a half I have been working closely with Professor David Koop on the project titled Reproducibility in Data Visualization. I’m thrilled to be able to make my own little mark on this amazing project and aid in exploring solutions to capture visualizations in hopes of making reproducibility easier in this domain.

Progress and Challenges

The last month and a half have mostly been spent trying to explore best possible solutions to facilitate the reproducibility of STATIC visualizations from local sources and/or the web. We have taken inspiration from existing work in the domain and successfully captured meta-information required to ensure reproducibility in the regenerated visualizations from the said metadata. The metadata extracted is saved into the generated .png figure of the visualization therefore allowing reproducibility as long as you have (a) The original dataset (b) The generated .png of the visualization. Every other information is stored inside the .png file as a json object and can be used to regenerate the original image with a very high accuracy.

The problem however remains with visualizations where randomness such as jitter is involved. Capturing the randomness has not been 100% successful as of now, and we are looking into options to ensure the capture of certain plots that contains randomness.

The following images can be used to highlight some results from our reproducibility experiments: Original Histogram using Matplotlib on the iris dataset:

Reproduced Histogram using metainformation from the original:

The next steps

We have already started looking into solutions and ways to capture visualizations from the web i.e. from platforms such as ObservableHq and use these experiments to transition into capturing interactive visualizations from the web.

Capturing user interactions and all states in an interactive visualization can prove to be very useful as it is a very known pain-point in the reproducibility community and has been a challenge that needs to be solved. My next steps involve working on finding a solution to capture these interactive visualizations especially those living on the web and ensuring their reproducibility.

Reproducibility in Data Visualization

Fri, 14 Jun 2024 00:00:00 +0000

Hello! My name is Arya Sarkar and I will be contributing to the research project titled Reproducibility in Data Visualization, with a focus on investigating and coming up with novel solutions to capture both static and dynamic visualizations from different sources. My project is titled Investigate Solutions for Capturing Visualizations and I am mentored by Prof. David Koop.

Open-source has always piqued my interest, but often I found it hard to get started in as a junior in university. I spent a lot of time working with data visualizations but had never dived into the problem of reproducibility before diving into this project. When I saw a plethora of unique and interesting projects during the contribution phase of OSRE-2024, I was confused at the beginning. However, the more I dived into this project and understood the significance of research in this domain to ensure reproducibility, the more did I find myself getting drawn towards it. I am glad to be presented this amazing opportunity to work in the Open-source space as a researcher in reproducibility.

This project aims to investigate, augment, and/or develop solutions to capture visualizations that appear in formats including websites and Jupyter notebooks. We have a special interest on capturing the state of interactive visualizations and preserving the user interactions required to reach a certain visualization in an interactive environment to ensure reproducibility.My proposal can be viewed here!

Reproducibility in Data Visualization

Tue, 06 Feb 2024 15:00:00 -0500

At the heart of evaluating reproducibility is a judgment about whether two results are indeed the same. This can be complicated in the context of data visualization due to rapidly evolving technology and differences in how users perceive the results. First, due to the rapid evolution of libraries including web technologies, visualizations created in the past may look different when rendered in the future. Second, as the goal of data visualization is communicating data to people, different people may perceive visualizations in a different way. Thus, when a reproduced visualization does not exactly match the original, judging whether they are “similar enough” is complicated by these factors. For example, changes in a colormap may be deemed minor by a computer but could lead people to different understandings of the data. The goals of this research are to capture visualizations in a way that allows their reproducibility to be evaluated and to develop methods to categorize the differences when a reproduced visualization differs from the original.

Investigate Solutions for Capturing Visualizations

Topics: Reproducibility, Data Visualization
Skills: Python and/or JavaScript, Data Visualization Tools
Difficulty: Moderate
Size: Medium or Large (175 or 350 hours)
Mentors: David Koop

The goal of this project is to investigate, augment, and/or develop solutions to capture visualizations that appear in formats including websites and Jupyter notebooks. In past work, we implemented methods to capture thumbnails as users interacted with visualizations. Other solutions can be used to capture interactive visualizations. We wish to understand the feasibility of recording such visualizations and their utility in evaluating reproducibility in the future.

Specific tasks:

Evaluate tools for capturing static visualizations on the web
Investigate tools for capturing dynamic visualizations on the web
Investigate how data including code or metadata can be captured with visualizations
Augment or develop tools to aid in capturing reproducible visualizations

Categorize Differences in Reproduced Visualizations

Topics: Reproducibility, Data Visualization
Skills: Python and/or JavaScript, Data Visualization Tools
Difficulty: Moderate/Hard
Size: Medium or Large (175 or 350 hours)
Mentors: David Koop

The goal of this project is to organize types of differences in reproduced visualizations and create tools to detect them. Publications and computational notebooks record renderings of visualizations. When they also include the code to reproduce the visualization, we can regenerate them in order to compare them. Often, the reproduced visualization does not match the original (see examples in this manuscript). This project seeks to categorize the types of differences that can occur in order and start understanding how they impact judgments of reproducibility.

Specific tasks:

Evaluate and/or develop tools to compare two visualizations
Evaluate the utility of artificial intelligence solutions
Organize and categorize the detected differences
Develop tools to determine the types or categories of differences present in two visualizations

Open Sensing Platform (OSP)

Mon, 05 Feb 2024 00:00:00 +0000

Open Sensing Platform I: Software to enable large scale outdoor sensor networks

Topics: Data Visualization, Backend, Web Development, UI/UX, Analytics
Skills:
- Required: React, Javascript, Python, SQL, Git
- Nice to have: Flask, Docker, CI/CD, AWS, Authentication
Difficulty: Medium
Size: Large (350 hours)
Mentors: Colleen Josephson, John Madden, Aaron Wu

Open Sensing Platform (OSP) is a new initiative expanding from our prior project DirtViz, a data visualization web platform for monitoring microbial fuel cell sensors (see GitHub). The mission is to scale up the current platform to support other researchers or citizen scientists in integrating their novel sensing hardware or microbial fuel cell sensors for monitoring and data analysis. Examples of the types of sensors currently deployed are sensors measuring soil moisture, temperature, current, and voltage in outdoor settings. The focus of the software half of the project involves building upon our existing visualization web platform, and adding additional features to support the mission. A live version of the website is available here.

Deliverables:
- Create a system for remote collaborators/citizen scientists to set up their sensors and upload securely, eg. designing user flow to create sensors
- Craft an intuitive navigation system so that data from deployment sites around the world can be easily viewed, eg. designing experience/system to locate deployment sites.
- Refine our web-based visualization tools to add additional features for users to analyze collected data, eg. lazy loading out-of-range data or caching queried data.
- Document the tool thoroughly for future maintenance

Open Sensing Platform II: Hardware to enable large scale outdoor sensor networks

Topics: Embedded system, wireless communication, low-power remote sensing
Skills:
- Required: C/C++, Git, Github, Platformio
- Nice to have: PCB design and debugging experience, STM32 HAL, ESP32 Arduino, protobuf, python, knowledge of standard communication protocols (I2C, SPI, and UART)
Difficulty: Hard
Size: Large (350 hours)
Mentors: Colleen Josephson, John Madden, Stephen Taylor

The Open Sensing Platform hardware aims to be a general purpose hardware platform for outdoor sensing (e.g. agriculture, ecological monitoring, etc.). The typical use case involves a sensor deployment in an agricultural field, remotely uploading measurements without interfering with farming operations. The current hardware revision (Soil Power Sensor) was originally designed for monitoring power output of microbial fuel cells using high fidelity voltage and current measurement channels, as well as auxiliary sensors such as the SDI-12 TEROS-12 soil moisture sensor. The primary activities of this project will involve low-level firmware design and implementation, but may also incorporate hardware design revisions if necessary. We are looking to expand functionality to other external sensors, as well as optimize for power consumption, via significant firmware design activities.

Long-range, low-power wireless communication is achieved through a LoRa capable STM32 microcontroller with in-lab experiments using an ESP32 microcontroller to enable the simpler WiFi interface. Both wireless interfaces communicate upload measurements to our data visualization dashboard, Open Sensing Platform I. The combined goal across both of these projects is to create a system that enables researchers to test and evaluate novel sensing solutions. We are looking to make the device usable to a wide range of researchers which may not have a background in electronics, so are interested in design activities that enhance user friendliness.

Deliverables: Contribution via commits to the GitHub repository with documentation on completed work. A changelog of contributions to the firmware.

DirtViz 2.0 (2023)

Mon, 07 Feb 2022 00:00:00 +0000

DirtViz is a project to visualize data collected from sensors deployed in sensor networks. We have deployed a number of sensors measuring qualities like soil moisture, temperature, current and voltage in outdoor settings. This project involves extending our existing visualization stack, DirtViz 1.0 (see github), and expanding it to version 2.0. The project goal is to create a fully-fledged dataviz tool tailored to the types of data collected from embedded systems sensor networks.

Visualize Sensor Data

Topics: Data Visualization, Analytics
Skills: javascript, python, bash, webservers, git, embedded systems
Difficulty: Easy/Moderate
Size: Large, 350 hours
Mentors: Colleen Josephson, Sonia Naderi, Stephen Taylor, John Madden

Specific tasks:

Refine our web-based visualization tools to easily allow users to zoom in on date ranges, change axes, etc.
Create a system for remote collaborators/citizen scientists to upload their own data in a secure manner
Craft an intuitive navigation system so that data from deployment sites around the world can be easily viewed
Document the tool thoroughly for future maintenance
If interested, we are also open to you investigating correlations between different data streams and doing self-directed data analysis