Vision–language–action (VLA) loops, then classical kinematics with WebGL (Three.js) — stepped like AlgoVision, for both ML policy intuition and geometry.
Cameras + instruction → vision encoder, LM, action tokens, decode to joint or EE deltas, and sync with the real robot.
The RL training loop: observation, policy inference, environment step, reward signal, replay buffer, and episode lifecycle.
How robot demonstrations become training data: teleoperation recording, annotation, RLDS/HDF5 conversion, and validation.
Two revolute joints, analytic inverse kinematics, 3D visualization. Drag the target or step through the derivation.
Add a wrist joint for tool orientation. Redundancy, null-space motion, and joint limits with interactive Three.js.
Numerical IK via the Jacobian matrix. Live matrix display, condition number, and damped least-squares toggle.
Domain randomization, system identification, and progressive transfer. Bridging the reality gap from sim to hardware.
One policy, many tasks: language conditioning, task embeddings, curriculum learning, and multi-task generalization.
Parallel-jaw grasping with force closure, antipodal grasps, grasp quality metrics, and friction analysis.
Configuration space, RRT path planning, spline smoothing, velocity profiles, and collision-free motion.
CLIP embeddings, open-vocabulary detection, visual grounding, spatial reasoning, and depth perception for robots.
Latent dynamics, reward prediction, MPC, Dreamer, TD-MPC, and planning in imagination instead of simulation.
Full 3D arm control: quaternions, 6×6 Jacobian, numerical IK, wrist singularities, and 7-DOF redundancy.
Impedance control, stiffness tuning, hybrid position-force, peg insertion, and compliance layers for VLA deployment.
The complete pipeline: camera → VLA → action chunk → safety filter → IK → robot hardware. The capstone.
OOD detection, workspace limits, force monitoring, E-stop, recovery strategies, and building trustworthy systems.
16 labs — 5 tiers — ~6 hours total — cognitive (ML) + physical (kinematics) tracks.