Hebbian Robotics introduces HFlow to standardize chaotic robotics data workflows.
Robotics developers currently depend on scattered scripts to manage recordings, which remain functional with a handful of samples but turn into an opaque process when scaling beyond dozens. The sheer volume obscures why certain segments are excluded, erasing reproducibility and making debugging nearly impossible.
Hebbian Robotics, backed by Y Combinator’s S26 cohort, has developed HFlow to tackle this inefficiency. This SDK transforms raw, synchronized recordings—including video, joint states, actions, and sensor metadata—into vetted segments and queryable catalogs.
How HFlow streamlines data processing
HFlow functions as a middleware layer that integrates existing tools rather than creating replacements. The foundation relies on MCAP, the open Foxglove container format, to store sensor data, robot states, and video in a single unified file. During research, developers write processing steps as Python functions for quick iteration. Once ready, HFlow packages these steps into Airflow 3 DAGs, enabling production-grade task tracking, retry mechanisms, and logging.
Instead of manually inspecting bloated recording files, HFlow logs version details, metadata, and measurements in an append-only Parquet table. Users then query this structured dataset using DuckDB SQL to generate versioned manifests instantly.
Addressing the "Garbage In, Garbage Out" dilemma
Maintaining data quality remains a persistent challenge in embodied AI. Hebbian reframes quality checks as modular evidence rather than fixed rules. Their approach includes two verification types: deterministic checks for drift, frozen frames, or missing data streams, and model-based validation using VLMs or MediaPipe to flag environmental anomalies or hand poses.
HFlow isolates problematic data while preserving the evidence, decoupling the data from its rejection criteria. This separation allows teams to salvage useful samples for alternative training pipelines.
Frustrations often surface during quality assessments—identifying whether cameras malfunctioned, streams lost synchronization, or critical data streams vanished. As datasets expand, ad-hoc scripts fail to track execution, audit outcomes, or replicate datasets. With HFlow’s built-in checks, developers can start with predefined validation steps before customizing additional transformations.
All Replies （3）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Nightmare! I once wasted a full week on a single script error causing sensor lag. To avoid that, start with HFlow's built-in checks to catch issues like camera freezes or stream drift before they snowball.
High-res video and LiDAR latency is killing me. How are you guys managing the time-stamps? One concrete step from the basis is to use HFlow's built-in checks to help manage timestamps by determining whether streams drifted out of sync.
Syncing sensor data across ROS bags is a total nightmare. Which tool actually handles this without crashing? I've heard good things about Hebbian Robotics' HFlow SDK. It's open source and designed for scalable multimodal data pipelines in robotics. You can start with HFlow's built-in checks to ensure data quality and sync, and then write new transforms as needed.