An open-source visual canvas where LLM, agent, RAG, vision, and fine-tuning steps become typed block graphs — and the graph compiles straight to runnable Python.
01 — Overview
AI Blocks is an open-source visual programming environment for building AI and ML pipelines. Instead of writing boilerplate to stitch together an LLM call, a retrieval step, and a classifier, you drag blocks onto a canvas, wire their ports together, and the tool assembles a runnable Python program from the graph.
It was built at the vibeFORWARD Hackathon, where it won 1st place. The premise: most people who want to prototype an agent or a RAG flow think about it as a diagram — boxes and arrows — long before they think about it as code. AI Blocks makes that diagram the actual source of truth.
02 — The problem
Assembling a multi-step AI pipeline is mostly plumbing. Every new experiment means re-wiring the same transformer, embedding, retrieval, and control-flow steps by hand, and keeping their inputs and outputs compatible as the graph grows.
- Boilerplate tax — the interesting decisions (which model, which retriever) are buried under glue code.
- Diagram-to-code gap — teams whiteboard a pipeline, then spend hours translating it into a working script.
- No type safety across steps — mismatched inputs and outputs fail at runtime, not at design time.
03 — What I built
I worked on the editor experience, the typed block schema, and the Python runtime layer — and proposed the feature that changed the product's ceiling: graph-to-code assembly.
- Drag-and-drop block components and editor UI in React / Vite for composing pipelines visually — LLM, agent, RAG, vision, and fine-tuning steps as connectable nodes.
- Contributed to the typed block schema system and Python runtime layer, defining input/output ports across transformers, embeddings, classical ML, and control flow.
- Built the block connection system handling port-to-port wiring between nodes, enforcing which outputs may feed which inputs.
- Proposed and helped design the graph-to-code assembly — shifting the tool from a static pipeline uploader into a live Python code generator driven by the canvas.
04 — Architecture
The canvas isn't a picture of the pipeline — it is the pipeline. Every edit updates a typed graph, which the assembler lowers into Python.
01 · Author
Visual Canvas
React / Vitedrag-drop blocks~500 typed defs
↓edits
02 · Model
Typed Block Graph
nodes + typed I/O portsport-to-port connections
↓typed nodes
03 · Validate
Compatibility Check
port-type matchrequired inputscycle detection
↓valid graph
04 · Assemble
Graph → Code Generator
recurse if/for/whileimport de-dupliteral escaping
↓codegen
05 · Output
Runnable Python
valid, properly-nested .py
05 — Challenges
- Typed ports across very different categories. A transformer's output, an embedding vector, and a classifier's label are not interchangeable. The schema had to encode compatibility so the canvas could reject invalid wiring at design time rather than runtime.
- Deterministic graph-to-code. Turning an arbitrary node graph into correct, readable Python means resolving execution order, handling branches, and generating code a human would actually trust to run.
06 — Lessons & what's next
- A typed schema is what makes a visual tool feel safe — the moment the canvas can say "no, that connection is invalid," it stops being a toy.
- The biggest leap in value came from reframing the deliverable: not "upload a pipeline" but "generate the code." Same UI, completely different product.
- Next: in-canvas execution with live previews, a shareable block library, and round-trip editing that imports existing Python back onto the canvas.
ReactVitePythonTyped GraphsCode Generation
07 — Forked & extended
After the hackathon I forked AI Blocks to my own GitHub and shipped production-minded improvements as reviewed pull requests — each deepening a part of the system I originally built, all covered by a passing test suite and a clean typecheck.
- Test suite for the type system, graph & codegen (19 tests) — port-type compatibility, graph validation (required inputs, type mismatches, cycle detection), and the live graph→Python generator (import de-duplication, output-to-input wiring, parameter interpolation). PR #1 ↗
- Safe Python string literals — the generator emitted params verbatim, so a quote or newline could corrupt or inject into the output; I added an opt-in
{{params.x|literal}} escaping filter, leaving all ~580 existing templates untouched. PR #2 ↗
- Real control-flow code generation — made the generator recurse into if / for / while branches and indent them to the right column, so nested loops and conditionals produce valid, properly-nested Python — completing the code-assembly path I helped design. PR #3 ↗