Artificial Intelligence Lab: Data, Models, and Decisions

A scatter plot of samples vs. probability density

This course is an introduction to the basic concepts of artificial intelligence (AI) as is practiced in the 2020s. At the end of this course you will be able to build AI systems that you can train to solve a variety of problems.

Graph Neural Networks

Graph Neural Networks (GNNs) are information processing architectures for signals supported on graphs. They have been developed and are presented in this course as generalizations of the convolutional neural networks (CNNs) that are used to process signals in time and space.

Introduction to Deep Learning & Applications

This course covers the fundamentals of deep learning and the basics of deep neural networks, including different network architectures and optimization algorithms for training these networks, as well as applications to computer vision, robotics, and sequence modeling.

Jailbreaking Large Language Models

This tutorial provides an overview of recent advances in jailbreaking attacks, defenses, and evaluation methods, with particular emphasis on applications in robotics and agentic systems.

Linear Control System Theory

A Boston Dynamics robot leaping through the air

This undergraduate-level course focuses on single-input single-output linear time-invariant control systems emphasizing frequency-domain methods. Topics include modeling of feedback control systems, transient and steady-state behavior, Laplace transforms, stability, root locus, frequency response, Bode plots, Nyquist plots, Nichols plots, PID control, and loop shaping.

Planning & Learning in Robotics

This course covers optimal control fundamentals and their application to motion planning and decision making in robotics. Topics include Markov decision processes (MDPs), dynamic programming, search-based and sampling-based motion planning, value and policy iteration, linear quadratic regulation (LQR), and model-free reinforcement learning.

Robot Proving Grounds

An introduction to robotics algorithms for mapping, localization, planning, and control, as well as a realistic testbed to try before deploying the algorithms on a real robot. The PyBullet physics simulator is used to demonstrate these algorithms on ground wheeled robots in simulation.

Safety in Generative AI

The rapid advancement of generative AI has brought remarkable innovations, from creating realistic images to generating human-like text. However, this power comes with significant responsibility. AI safety is crucial to ensure these systems are used ethically, fairly, and without unintended harm. Without proper safety measures, generative AI can spread misinformation, reinforce biases, or be exploited for malicious purposes. By prioritizing AI safety, researchers and developers can build trust, create guidelines for responsible use, and mitigate risks before they affect society on a large scale.

More coming soon...

Datasets

Autonomous Vehicle Laboratory Dataset (UC San Diego)
Over 13 hours of video from dynamic urban scenarios from autonomous agents and expert drivers, and over 900,000 labeled frames, useful for robust sensor fusion, trajectory prediction, dynamic planning and automatic calibration.

SemVecNet
Framework and dataset for generating vectorized HD maps from vehicle sensors.

Multi-Agent Benchmarks
Distributed coordination and resource allocation benchmarks for evaluating multi-agent autonomy.

TreeScope LiDAR Dataset
Labeled 3D LiDAR dataset for tree mapping and ecology.

Machine Learning & Hyperdimensional Computing

Uncertainty Estimation with Recursive Feature Machines
Official PyTorch implementation of Gedon et al. (2024): Uncertainty Estimation with Recursive Feature Machines (https://proceedings.mlr.press/v244/gedon24a.html).

HDnn-RRAM-Opt
Optimization software for implementing hyperdimensional neural networks on Resistive Random-Access Memory (RRAM) hardware.

EigenPro: A GPU-enabled fast and scalable solver for training kernel machines
EigenPro applies a projected stochastic gradient method with dual preconditioning to enable major speed-ups. Currently based on a PyTorch backend.

Grokking modular arithmetic with recursive feature machines (RFM)
Code for Mallinar et al. (2025): Emergence in non-neural models: grokking modular arithmetic via average gradient outer product (https://arxiv.org/abs/2407.20199).

Language Model Evaluation Harness
Unified framework to test generative language models on a large number of different evaluation tasks.

The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
HyperGen is a Rust library used to sketch genomic files and realize fast Average Nucleotide Identity (ANI) approximation.

LifeHD: Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing
Framework for continual learning on edge devices, using the efficiency of hyperdimensional computing to learn new tasks without forgetting old ones.

TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
Toolkit for compressing and optimizing large language models for deployment on edge devices with limited memory and power.

HyperGen: Compact and Efficient Genome Sketching using Hyperdimensional Vectors
A tool that applies hyperdimensional computing to genomics, allowing for extremely fast and compact comparison of DNA sequences.

Robotics & Autonomous Systems

EigenPro3: Fast training of large kernel models
High-performance library for fast training of large-scale kernel models. Implements "Projected dual-preconditioned Stochastic Gradient Descent", allowing scaling of kernel methods to billion-scale datasets on modern GPU hardware.

CORSAIR: Convolutional Object Retrieval and Symmetry-Aided Registration
Global localization and loop closure method that leverages object-level information and geometric symmetries to recognize previously visited locations.

ROAM: Riemannian Optimization for Active Mapping with Robot Teams
Framework for multi-robot exploration that uses optimization on Riemannian manifolds to coordinate robot movements for efficient mapping of large-scale environments.

SSMI: Semantic Shannon Mutual Information
C++/ROS library for active mapping that calculates Shannon Mutual Information over semantic classes, helping robots plan paths that maximize their understanding of object labels in an environment.

SAPIEN Manipulation Skill Framework (ManiSkill)
Unified framework for robot simulation and training with a strong focus on manipulation skills.

Distributionally Robust Lyapunov-stable Policy Learning
For training neural network policies that are mathematically guaranteed to be stable under a range of uncertain environmental conditions.

DR-Lyapunov-Function: Distributionally Robust Lyapunov Function Search Under Uncertainty
A framework for finding Lyapunov functions that are robust against distribution shifts and environmental uncertainties, ensuring safety for autonomous agents in unpredictable settings.

Air Router: Replaces the waypoint mission following in PX4-based flight controllers for a high-altitude robot
Used to maximize communication in a multi-robot setting, where the aerial robot can fly over ground robots and act as a data mule.

AllocNet: Learning Time Allocation for Trajectory Optimization
Neural-network-enhanced trajectory planner that predicts optimal control parameters to reduce the computational cost of real-time drone flight planning.

Learning Perception-Action-Communication Feedback Policies
Simulation environment, algorithms, and GNN-based architectures for the coverage control problem.

Edge Autonomy Reference Stack
Integrated build system for deploying mapping, control, and communication stacks on edge hardware with aerial robots.

Design and Evaluation of Motion Planners for Quadrotors
Comprehensive suite of tools for evaluating various motion planning algorithms in simulation and real-world quadrotor flight scenarios.

Convex Cover of Collision-free Space for Trajectory Generation
Geometric tool that approximates free space as a series of convex polytopes, simplifying the search for collision-free trajectories in dense environments.

MOCHA: Multi-robot Opportunistic Communication for Heterogeneous Collaboration
Communication management among robot teams in environments with intermittent connectivity, allowing them to share critical data whenever a link becomes available.

MRSL Quadrotor Simulator
Open-source software modules and drivers used by the Penn Multi-Robot Systems Lab (MRSL) for autonomous quadrotor operations, including state estimation and control.

SlideSLAM: Sparse, Lightweight, Decentralized Metric-Semantic SLAM for Multi-Robot Navigation
Lightweight, optimization-based SLAM engine for resource-constrained robots.

Semantic Lidar Odometry and Mapping in Forests (SLOAM)
Specialized system for navigating unstructured environments. Uses neural networks to segment trees as semantic landmarks, enabling robust mapping where geometric SLAM typically fails.

Quad-LCD: Layered Control Decomposition Enables Actuator-Feasible Quadrotor Trajectory Planning
A planning framework that decomposes the control problem into layers to ensure that generated quadrotor trajectories are always within the physical limits of the motors.

LieFVIN: Lie Group Forced Variational Integrator Networks for Learning and Control of Robot Systems
Physics-informed deep learning architecture for learning robotic dynamics directly on Lie groups. It preserves physical properties like energy conservation for more stable long-term control.

LieGroupHamDL: Port-Hamiltonian Neural ODE Networks on Lie Groups For Robot Dynamics Learning and Control
An extension of Hamiltonian learning that incorporates Port-Hamiltonian systems on Lie groups, allowing for the modeling of dissipative systems and external control inputs.

SE3HamDL: Hamiltonian-based Neural ODE Networks on the SE(3) Manifold For Dynamics Learning and Control
This repository implements Neural ODEs that respect the Hamiltonian structure of mechanical systems on the SE(3) manifold, which is critical for accurate modeling of 3D robotic motion.

Hierarchical LLM Framework for Multiagent Robotics
Hierarchical LLM framework for real-time multi-robot task allocation and target tracking.

VLSI Design & Hardware Optimization

BlobPlace
PPA-relevant clustering-driven placement framework for large-scale VLSI designs.

DREAMPlaceFPGA
State-of-the-art FPGA placement engine leveraging optimization-enhanced ML.

TritonPart (OpenROAD Integrated Version)
The first open-source, constraints-driven partitioner for RTL-to-GDS tools, recently updated with timing-driven capabilities and improved runtime.

DE-HNN
An effective neural model for Circuit Netlist representation.

Hypergraph Partitioning for VLSI: Benchmarks, Code and Leaderboard
A central hub for VLSI partitioning research, containing code for SpecPart (supervised spectral improvement, https://github.com/TILOS-AI-Institute/SpecPart) and K-SpecPart (multi-way partitioning, https://github.com/TILOS-AI-Institute/K-SpecPart).

MacroPlacement
MacroPlacement is an open, transparent effort to provide a public, baseline implementation of Google Brain's Circuit Training (Morpheus, https://github.com/google-research/circuit_training) deep RL-based placement method.

Introducing K-12 Students to Data Science and AI

In collaboration with middle and high school teachers and undergraduate interns, TILOS develops activities to introduce K-12 students to data science, programming, artificial intelligence, and optimization. Training is available to help teachers learn to run these activities and incorporate into curricula. Contact tilos@ucsd.edu to bring an activity to your classroom or event.

AI Robotic Flower
Data-driven Robotic Art
Data Science Automatons
Data Science Discovery Bootcamp

Micro:bit and Climate
Predictive Analytics and Income
Wearable AI
Field trips to UC San Diego campus

See the Lab 3.0 website for additional resources.

Data-driven Robotic Art

AI Robotic Flower

Wearable AI

Data Science Automatons

Field Trips to UC San Diego

Data Science Bootcamp

Micro:bit and Climate

Predictive Analytics and Income

Learning Beyond the Data: Adam Klivans on Distribution Shift and the Future of AI

Trustworthy machine learning requires models that still work when real-world data changes, and Adam Klivans, Ph.D., Director of the Institute for Foundations of Machine Learning (IFML), emphasizes learning under “distribution shift” as a major barrier to relying on models to predict disease across different patient populations. IFML focuses on foundational algorithms and mathematical techniques that push generative AI forward, including better methods for training and inference in deep learning and advances in diffusion approaches. Klivans highlights robustness and safety as core priorities, asking how to trust a model trained in one setting when it is applied in another. IFML connects these foundations to use-inspired domains such as imaging, protein engineering and biologics, and AI for mathematical discovery.

Big Data, Better Answers: Optimization at Scale with Courtney Paquette

Large-scale optimization and machine learning shape modern data science, and Courtney Paquette, Ph.D., McGill University, studies how to design and analyze algorithms for large-scale optimization problems motivated by applications and data science. Paquette draws on probability, complexity theory, and convex and non-smooth optimization, and examines scaling limits of stochastic algorithms. Speaking with Saura Naderi, UC San Diego, Paquette describes an unconventional path from finance to pure mathematics and explains how persistence and comfort with uncertainty support long-term research. She highlights the challenge of building missing foundations while advancing through graduate training, and she connects that experience to the realities of doing original work. Paquette also reflects on rapid progress in machine learning and frames AI systems as tools that can be used thoughtfully.

Artificial Intelligence and Security: A Conversation with Yaron Singer

Yaron Singer, Vice President of AI and Security at Cisco, co-founded a company specializing in artificial intelligence solutions, which was acquired by Cisco in 2024. They developed a firewall for artificial intelligence, a tool designed to protect AI from making critical mistakes. No matter how sophisticated AI is, errors can still happen, and these errors can have far-reaching consequences. The product is designed to detect and fix such mistakes. This technology was developed long before ChatGPT and its competitors burst onto the scene, making it the hottest industry in tech investment. Join Singer as he sits down with TILOS Foundations team member Mikhail (Misha) Belkin to discuss his work and the continued effort to make artificial intelligence secure.

Machine Learning, Fairness and AI: A Conversation with Adam Kalai

OpenAI researcher Adam Kalai sits down with TILOS Foundations team member Mikhail (Misha) Belkin to discuss his work in machine learning, algorithmic fairness, and artificial intelligence. Kalai has contributed research in areas like fairness in AI models, word embeddings, and human-AI collaboration. He has worked at Microsoft Research and has published influential papers on bias in machine learning models. His work has helped shape discussions on ethical AI and the development of more equitable AI systems.

Building Fast and Reliable Machine Learning Systems with Yian Ma

TILOS Foundations team member Yian Ma talks about his research using scalable inference methods for credible machine learning. This involves designing Bayesian inference methods to quantify uncertainty in the predictions of complex models; understanding computational and statistical guarantees of inference algorithms; and leveraging these scalable algorithms to learn from time series data and perform sequential decision making tasks.