Projects

Evaluation of transfer learning for roadside driving behavior in autonomous robots

Master’s thesis · Universidad de los Andes (MISIS) · Defended June 6, 2025

Evaluated transfer learning between deep reinforcement-learning agents on a roadside-driving (lane-change) task: transferring the knowledge of a previously trained TD3 self-driving policy to a new agent that has to start its task from scratch. The self-driving policies themselves were trained on a TurtleBot3 in Gazebo, with CARLA also explored as an alternative simulator. Promotor: Nicolás Cardozo.

TurtleBot3 in the Gazebo roundabout testbed, with the LiDAR scan rendered as a blue cone sweeping the environment

Fig. 3.7 — TurtleBot3 in the Gazebo roundabout testbed; the blue cone represents the LiDAR scan sweeping the simulated environment.

Highlights

  • Three formal transfer-learning metrics evaluated: Transfer Ratio, Mastery, and Pedagogy.

  • Transfer ratio of 0.9401 when reusing a pre-trained expert (vs 1.1455 for an agent learning both paradigms simultaneously) — substantial knowledge preservation.

  • ~53% fewer training steps (70 vs 149) to reach competence compared to learning from scratch.

  • A gradient-transfer variant cut training time by 44.3% (1.929 h vs 3.461 h baseline).

  • TD3 (Twin Delayed Deep Deterministic Policy Gradient) for continuous control of the robot’s linear and angular velocities — used as the underlying RL algorithm for both source and target agents.

  • Integrated ROS2 topics for sensor input (LiDAR, odometry) and actuation.

  • Read the thesis (PDF) · Source on GitHub

Bar chart comparing training time of four agents — left expert (~3.5 h), left-from-right with no transfer (~9.2 h), agent learning both paradigms simultaneously (~3.4 h), and the transfer-learning agent learning from demonstrations (~1.9 h)

Fig. 4.8 — Training time of the four evaluated agents. The transfer-learning agent (purple) reaches competence in ~1.9 h versus ~3.5 h for the left-side expert trained from scratch.

The CARLA simulator was also explored as an alternative urban-driving environment using a Cybertruck model in Town 5.

Cybertruck model placed in CARLA's Town 5 urban environment, used as the CARLA-side testbed for the autonomous-driving experiments

Fig. 3.2 — Cybertruck model placed in CARLA’s Town 5, used as the alternative urban-driving testbed.