AI Text-to-Draw
Synthesizes normalized contour coordinates, ARGB fills, and initial behavior blocks directly from natural language.
Originally pioneered in science education research to let students learn by modeling through freehand drawings—now rebuilt from the ground up in Flutter/Dart with four specialized domain studios, 2D rigid-body physics, and Socratic multimodal AI scaffolding.
Investigated in IEEE TLT (2013) · JSET (2015, 2018) · Springer (2019) · Interactive 2026 Studio at /simsketch
Behaviors
Conventional modeling environments require learners to master programming code or differential equations before testing a hypothesis—confining classroom practice to passive “learning with models” rather than active “learning by modeling.” Conceived at the University of Twente and extensively researched at Utrecht University's Freudenthal Institute, SimSketch demonstrated across multiple classroom and museum studies that annotated freehand sketches enable young learners to construct, execute, and revise genuine computational models.
In 2026, we revitalized this foundational research software in Flutter/Dart—preserving its constructivist sketch-to-agent pedagogy while introducing domain-specific physics, ecology, kinematics, mathematics, and Socratic AI scaffolding.
Bollen & van Joolingen formalized sketch-based modeling: decomposing learner drawings into autonomous ModelUnit glyphs and AbstractBehavior primitives—featuring qualitative “low–medium–high” sliders and Section 4.2's interlocking gear-train debugging case study.
Evaluated by van Joolingen, Heijnes, Leenaars & Schouten across museum astronomy (N = 248) and Cepaea nemoralis biology cohorts (N = 204): proving gains on nature of models (d = 0.74) while uncovering the p = 0.380 model-testing gap, black-box predation misconceptions, and Web-over-USB exploration advantages.
Translating a decade of design-based research into a modern cross-platform architecture: four domain-tailored laboratories (Sandbox, progressive Ecology, involute GearSketch, and MathAnim 2D) augmented by multimodal vision and Socratic conversational inquiry.
Pioneered drawing-based inquiry in primary classrooms, science museums, and biology field studies.
Engineered specifically to resolve the empirical bottlenecks identified across the 2013–2019 studies.
SimSketch is one of the few educational modeling tools tested repeatedly across science museums and secondary classrooms. Each empirical finding directly guided a concrete capability in our 2026 re-architecture.
Statistically significant gain in modeling competence across 7 ninth-grade biology classes (N = 204).
110 complete pre/post datasets: mean score rose from 6.92 to 8.32 (max 24) after four 45-minute lessons.
Medium gain on epistemological competence—while 'testing models' (p = 0.380) exposed a critical hypothesis-revision gap.
Students gained on 'nature of models' (d = 0.74) and 'multiple models' (d = 0.28), but treated simulation runs as a technical check—motivating 2026's 60s Time-Travel & Socratic AI.
Complete learner datasets in the Dutch Science Museum planetary modeling study.
204 learners modeled Sun–Earth–Moon orbits; rated 3.21–3.36/4 on valuing and 2.71–3.05/4 on perceived competence.
Design-based research cycles revealing the 'black-box' predation fallacy and Web vs. USB exploration effects.
Hidden rules triggered leaf-pigment misconceptions (solved by 2026 FOV Radar rings), while Cycle 4's USB deployment reduced playful exploration ('uitgespeeld')—validating 2026's zero-install Web/PWA.
Empirical Insight: Section 4.2's interlocking gear-train debugging case study and bounded qualitative sliders showed learners need both mechanical formalisms and safe parameter ranges that cannot destabilize the simulation.
Directly evolves Section 4.2 into GearSketch (involute teeth, coaxial axles, roller chains, jam detection) and translates 2013's sliders into bounded Inspector controls (Speed: 3.2 px/f, FOV: 120px, Restitution: 65%).
Empirical Insight: Science-museum planetary modeling (N = 248) demonstrated that learners grasp periodic orbits and spatial trajectories best when geometric motion links to continuous mathematical curves.
Couples orbital Circle and Gravitational Pull multi-agent primitives with MathAnim 2D—a 60 FPS Cartesian, Parametric, and Polar function visualizer with step-by-step Euler ODE animation.
Empirical Insight: Hidden predation rules caused 'black-box' misconceptions (e.g., snails absorbing green pigment from leaves), excess parameters overwhelmed novices, and Iteration 4's USB deployment reduced playful exploration ('uitgespeeld') vs. web delivery.
Live FOV radar rings make detection radii visible on canvas, Ecology Sub-Modes (Basic / Standard / Advanced) adapt behavior dialogs by age & onboarding stage, and the zero-install Web/PWA build (/simsketch) preserves spontaneous inquiry.
Empirical Insight: While 'nature of models' rose strongly (d = 0.74), 'testing models' did not improve significantly (p = 0.380) because students viewed running the simulation as a technical operation—and ad-hoc teacher help masked diagnostic assessment.
Directly targets the p = 0.380 model-testing gap: 60s circular snapshot rewind enables systematic before/after hypothesis comparison, Socratic AI prompts model revision, and JSON/chat logs capture unconfounded process-tracing data.
Empirical metrics and effect sizes reported above originate strictly from the cited 2013–2019 publications on SimSketch. The 2026 multi-domain studios, glass-box FOV radar, 60s time-travel, and Socratic AI represent the modern re-architecture addressing those empirical findings.
Beyond modernizing the runtime, the 2026 studio directly addresses methodological challenges documented in the 2013–2019 literature: black-box predation fallacies, the p = 0.380 model-testing gap, qualitative-to-quantitative staging, and unconfounded diagnostic logging.
In the 2018 and 2019 classroom studies, students modeled how song thrushes (Turdus philomelos) selectively prey on banded grove snails (Cepaea nemoralis). When predation probability operated as a hidden “black box” in 2018, students invented non-evolutionary explanations—such as snails “absorbing green pigment from leaves.” Toggle below to see how 2026's Glass-Box FOV Radar Rings make differential detection visually explicit.
To serve different age groups and first-time onboarding without overwhelming novices (Heijnes et al., 2018), Ecology behaviors (e.g., Photosynthesis, Split, Competition) adapt their parameter dialogs across three complexity tiers:
Evolves 2013's qualitative sliders into crash-proof Bounded Inspector Controls (Speed: 3.2 px/frame, FOV: 120px, Restitution: 65%) so learners experiment safely without destabilizing the engine.
Heijnes et al. (2018) evaluated student dialogue using Hogan et al.'s (1999) six-dimensional coding scheme for peer scientific reasoning. In 2026, our Socratic Modeling Assistant maps its dialogic prompts directly to these six dimensions:
Rubric Definition: Constructing multi-step mechanistic accounts of why emergent system behavior arises from individual agent rules. (Counteracts the 2018 'black-box' fallacy where learners claimed snails absorbed green pigment from leaves.)
"Watch the cyan FOV radar ring around the predator: does an individual snail change color, or does differential survival shift the population ratio?"
→ Target Model Revision: Learner articulates natural selection via differential predation rather than Lamarckian individual transformation.
van Joolingen et al. (2019) noted that ad-hoc teacher interventions in the classroom make it difficult to measure a student's unassisted modeling competence diagnostically. In 2026, portable JSON Scene Serialization, 60s Snapshot Diffs, and Socratic Turn Logs provide standardized process-tracing data for researchers:
// Standardized Process-Tracing Snapshot Diff
{
"session": "cepaea_cohort_09B",
"rewindEvent": { "fromSec": 42.0, "toSec": 12.0, "key": "[" },
"hypothesisRevision": {
"unit": "SongThrush_01",
"behavior": "Follow",
"fovRadiusPx": { "before": 120, "after": 55 },
"targetMorph": "Yellow_Unbanded"
},
"testingModelsCompetence": "verified_revision"
}In Heijnes et al. (2018, Iteration 4), distributing SimSketch on local USB sticks instead of the web reduced students' spontaneous, playful exploration (“uitgespeeld”). Delivering the 2026 studio as an instant, zero-install Flutter Web/PWA app at /simsketch directly preserves frictionless browser inquiry.
Expanding beyond a single generic drawing pad, the 2026 re-architecture pairs open-ended multi-agent sketching with dedicated ecological, mechanical, and mathematical laboratories.
Interactive Studio Preview · Switch tabs or use arrow keys
Hover or focus any chip to inspect its rule & parameters.
2D Rigid-Body Physics
Circular snapshot buffer captures state every 1.5s (`[`) so learners systematically test and revise hypotheses instead of treating runs as a passive check.
Exports vector strokes, behavior graphs, and Socratic logs as standardized `.json` telemetry—separating student competence from teacher scaffolding.
Informed by 2018 Cycle 4 (where USB sticks reduced playful exploration vs. web): instant browser access at `/simsketch` plus 6 native targets.
Earlier classroom studies highlighted that responsive Socratic scaffolding is crucial when learners interpret unexpected simulation results. In 2026, SimSketch introduces vision-language import, generative sketching, and a Socratic dialogic tutor to support inquiry without giving away answers.
Synthesizes normalized contour coordinates, ARGB fills, and initial behavior blocks directly from natural language.
Extracts hand-drawn paper sketches via multimodal vision, isolating distinct contours into live, centered agents.
Calibrated to Hogan et al.'s (1999) 6-dimension reasoning rubric to close the 2019 p = 0.380 model-testing gap through guided hypothesis revision.
Provider-agnostic by design. Keys stay on the device.
Every stroke normalizer, kinematic solver, and differential compiler executes natively inside the Flutter/Dart client. Multimodal LLM calls route directly from device memory through an abstract ILlmService contract.
drawing_engineisolatedperfect_freehand ink contours, bounding-box auto-centering & undo stacks
simulation_engineisolatedModelUnit, 19 behaviors, 3x sub-stepped 2D physics & raycasted shadows
gear_engineisolatedInvolute tooth synthesis, BFS phase alignment, coaxial axles & chains
math_engineisolated60 FPS zero-dependency expression AST compiler & Euler ODE stepper
pedagogy_llmisolatedAiObjectFactory JSON normalization, Vision import & Socratic prompts
Swappable multimodal repositories emitting validated ModelUnit stroke & behavior schemas:
shared_preferencesLocal-first persistence for user API keys, workspace preferences, 60-second circular snapshots, and portable JSON scene files—never touching a third-party server.
“The sketch is the interface. The simulation is the payload. The AI is the co-author.”
Uniting four simulation studios and three foundation-model providers inside one cross-platform environment required strict module isolation, typed domain models, and client-only credential storage suitable for classroom and personal devices.
lib/features/*Isolated drawing_engine, simulation_engine, gear_engine, math_engine, and pedagogy_llm packages keep four distinct studios maintainable under one workspace.
ILlmServiceGoogle Gemini, OpenAI ChatGPT, and Anthropic Claude sit behind a single typed Dart interface, decoupling multimodal inference from studio canvases.
ModelUnit · GearUnitModelUnit, GearUnit, GearChain, and MathStudioState enforce compile-time safety across spatial transforms, kinematics, and JSON serialization.
Cross-Mode QAStructured cross-mode regression checklists verify that Sandbox behaviors, time-travel snapshots, and undo stacks remain intact as GearSketch and MathAnim evolve.
Docs & CitationsShips with complete setup guides, peer-reviewed academic citations, keyboard & gesture shortcut tables, and per-engine architectural specifications.
shared_preferencesUser API keys are stored exclusively on-device via shared_preferences and connect directly to provider endpoints—never routed through an intermediary backend.