RoboCupRescue: an interview with Adam Jacoff

The RoboCupRescue arena at RoboCup2024, Eindhoven.

RoboCup is an international scientific initiative with the goal of advancing the state of the science of intelligent robots, AI and automation. The annual RoboCup event will take place from 15-21 July in Salvador, Brazil. The RoboCupRescue League is an important element of the competition and focuses on the challenges involved in search and rescue applications. We caught up with Adam Jacoff, co-founder of the RoboCupRescue league, former RoboCup Trustee, and chair of the organising committee, to find out more.

Could you start by giving us an overview of the Rescue League?

The RoboCupRescue League is now in its 25th year hosting competitions and workshops all around the world. We’re focused on developing autonomous robots that can enable emergency responders to perform extremely hazardous tasks from safer stand-off distances. That includes search and rescue scenarios in compromised or collapsed structures, for example, but there are many tasks that firefighters and other public safety organizations do every day that would be safer with robots doing the dangerous part. We are really the only League at RoboCup that’s focused on emergency responders, so our arenas appear a little more chaotic than the other leagues, but in a controlled sort of way. We have twenty standard test methods that we use as challenge tasks that emergency responders and our league of teams around the world have helped develop and validate over many years. These competitions do three things at once.

First, they guide the research with tangible terrains, obstacles, and tasks that are representative of emergency response operations. The researchers likely do not know any emergency responders, but we do, and we have distilled their requirements into twenty or so standard test methods. So the teams know that if they can solve the challenges presented to them in the test lanes, maybe not all twenty, but ten of them, or five of them, that combination of capabilities are applicable to some public safety mission. And the more autonomy the researchers can implement into their robots, the more likely the robots will be effective and easy to use in the field. The main objective of our league is to quicken the development of effective and reliable autonomous behaviors for these complex environments.

RoboCup is an excellent incubator for developing and evaluating cutting edge research and at the same time for developing, validating, and disseminating new standard test methods. Our standard test methods are reproducible and approach the reality of very complex and hazardous environments in an incremental way. We have increasingly difficult settings for each test. So we start easy in the Preliminaries with all the terrain lanes flat so teams can optimize their approaches for each particular challenge. By Semi-Finals the terrain lanes are inclined into crossover 15° slopes with a complex terrain obstacle in the middle. The teams start performing sequences of multiple lanes, so they need to intelligently switch between different behaviors to succeed. Then for the Finals we inject more complexity with slippery features for the robots to lose traction, pinch points to force more steering, and higher step-over obstacles to make the sequences even harder. This is how we guide and challenge both autonomous and remotely operated robots beyond their comfort zone.

Second, the intensity of the experience and education for engineers and computer scientists leads to recruiting opportunities and careers. Everybody understands that teams who do well at RoboCupRescue have done something special. They can go on to more intense research or can help implement that technology in the commercial market. Our League has inspired and validated new robotic approaches to very complex environments and helped make the jump from research to commercial implementation.

The third piece of the puzzle is our connection to various communities of emergency responders. Our sequence of test lanes is exactly what emergency responders use to support their procurement and training. They use trial data captured with the robot manufacturers to help decide what robot to buy. They also use the test lanes to train remote operators and measure their proficiency. So RoboCupRescue helps emergency responders understand what the next wave of emerging technologies can do. For example, the nuclear waste cleanup industry wants to credential their remote robot operators to ensure robots don’t get stranded within extremely harsh radiation-filled environments. The robots need to keep working for 1,000 hours or more to be cost effective. They also want to ensure remote operators are proficient enough to safely perform the intended tasks without crashing the robot on the stairs and blocking the way for all other robots. Or worse yet, force humans in protective suits to go in and clear the way. Different organizations come to RoboCupRescue competitions and see various aspects of the league in action that are useful to them. In this case, as an obstacle course to get a robot driver’s license.

At Eindhoven last year (for RoboCup2024) we had the local bomb squads provide what we call a “reverse demonstration.” All these researchers give demonstrations of their robots at their schools to sponsors, family, friends, and maybe even to emergency responders. But they never get to see what the emergency responders do for a living – how dangerous their jobs are. So the RoboCupRescue league typically pauses for a while during the competition and comes together to watch a bomb technician get into their padded suit and go through the sequence of test lanes doing all the tasks that we’re asking the robots to perform. The researchers see how much of a burden it is just to wear the padded suits, which are very heavy and require forced air into the helmet. It’s not only physically difficult, they’re walking to and interacting with known explosives to render them safe – which is unconscionable given the capabilities of robots these days. They need to be working remotely from a safe stand-off distance. These “reverse demonstrations” are likely the first time the researchers ever get to meet people in bomb suits, and understand that it is these emergency responders we are working to help. Whether it’s firefighters or police, a natural or manmade disaster, we’re trying to help emergency responders stay out of harm’s way. They need capable, reliable, and easy to use robots to deal with extreme hazards more safely and effectively.

The RoboCupRescue arena at ICRA2025 (one of the qualifying events for RoboCup2025). Credit: Adam Jacoff.

So you mentioned trying to promote autonomy over teleoperation. How do you go about trying to encourage people to use more autonomy in the league?

Competitions need to have simple and clear rules. They need to challenge robots, lead them toward practical solutions, and highlight teams that are successful in a variety of challenges. So RoboCupRescue has developed scoring tasks for mobility, dexterity, and mapping within complex terrains. To score mobility points, for example, the robot needs to drive from one end of a terrain lane to the other end, or ascend and descend obstacles like stairs. They earn 1 point for each successful traverse up to 10 total repetitions to add some statistical significance. The operator is always out of sight of the robot, so working only through their interface as if the robot is inside a building. The terrain lanes and obstacles start easy and get harder, but all teams are being evaluated in the same difficulty setting as the competition progresses, so the points stay the same.

If the robot can successfully traverse the lane or perform the dexterity tasks without the operator touching the interface, we call that autonomy. At the beginning of each traverse the operator can manually set a goal point at the other end of the lane. Then they hit “go” and we all watch and learn together. If the robot gets to the other end without the operator touching the controls, the team gets 4 points instead of 1 point for that autonomous traverse. That 4:1 one ratio appears to be enough incentive to make sure teams develop autonomous behaviors and try them at each start point. However, they can always repair back to remote operation anytime the robot stalls to score at the lesser rate, so the rules encourage trying autonomy first. Also, we give Best-In-Class Autonomy awards to recognize teams that earned the most points autonomously throughout the competition.

Why do we need autonomy in emergency response operations? First, because the onboard autonomy typically makes the robot easier to use for a remote operator, which can also mean it is more likely to succeed down range or maybe be more reliable. Also, inside buildings there are often radio communications drop-out zones. Think about your cell phone reception inside concrete buildings or in basements under buildings, these places still need to be searched even when radio communication from a remote stand-off location outside the structure doesn’t support real-time video or control. So each RoboCupRescue lane, and the entire Labyrinth or Maze used for mapping, is considered a “radio drop-out zone” with tasks that score 4 to 1 if you can perform them autonomously. That applies to mobility, dexterity, and mapping tasks.

An example of one of the lanes (challenges) that teams tackle during competition. Credit: Adam Jacoff.

Will there be any new innovations to look out for this year?

What you’re going to see new this year are four-legged robots, called quadrupeds, that now have wheels as feet. There was a commercial robot that came to RoboCup2024 in Eindhoven. I invited them to demonstrate in the test lanes while the teams were at lunch, and this new robot design literally dominated our Semi-Finals sequence. Now that was a demonstration, conducted with the operator in direct line-of-sight of the robot, so much easier than remotely controlling the robot in the same terrains. But it didn’t have a remote interface. That’s exactly what our league of teams do best by adding interfaces, sensors, mapping, manipulators, and the autonomy to make it all effective and easier to use remotely.

These legged robots with wheels are a literal step function improvement in mobility, the likes of which I’ve only seen twice in thirty plus years. They have a level of mobility relative to their size that is astonishing. So we’re turning that new robot into our league’s first “standard platform.”

One of the four-legged robots with wheels as feet. Image from video taken at ICRA 2025, of the robot tackling one of the test lanes. Credit: Adam Jacoff.

Yes, I wanted to ask about the League’s plans for introducing a standard platform. Could you talk more about that?

This robot will provide a common development platform across teams with inherently good mobility within our terrains and obstacles. The league is negotiating with the manufacturer so that teams can purchase it at a relatively low price, which will bring down the cost of entry for new teams into the league. That level of mobility within our complex terrains and obstacles makes the autonomy a lot easier to develop – teams can focus on higher-level planning because the lower-level autonomous gaits and behaviors are already relatively effective. Similarly equipped teams can collaborate more closely, compare performance more directly, and share software solutions.

As a league, we can also start introducing teaming tasks for multiple robots. What kind of teaming might we explore? Maybe a certain object of interest is too heavy to drag for one robot, but could three of them work together to get it done like a team of dogs pulling a sled? Or maybe to carry payloads and supplies down-range, or even a non-ambulatory victim in a so-called litter back up-range to the base of operations. There will be lots of possibilities.

This new class of robots will operate in all the same test lanes along with all the other wheeled, tracked, and other robots developed by teams. We score every robot similarly of course, but then can separately compare the quantitative trial results for different classes of robots. It might turn out that it’s unfair to compare a new legged robot with wheels with a slower but stronger tracked robot that’s been winning RoboCupRescue for years. It’s not like tracked robots don’t have their use cases in emergency response operations. As soon as you get to tasks that involve exerting forces on the environment, like turning valves, the robots benefit from being heavier and stronger, which typically means tracks with independent front and rear flippers to negotiate the complex terrains and climb stairs. Everyone doesn’t need to adopt the new robot to compete, teams should work on implementations that push the state of the science in whatever direction they can. Then our Best-In-Class Awards can recognize the most effective robots in a variety of like-kind implementations, wheeled, tracked, legged, and maybe now legged with wheels.

When humanoids come into the league, and they will at some point, they’ll be welcomed and will be compared directly against other humanoids at least initially. We can also aggregate performance across a class of robots to identify advantages and apparent weaknesses. Emergency responders can then look at all the trial results to see which systems might work in their environments.

Do you know how many teams will be using this new robot?

I think we have almost twenty teams involved this year, but I’m not sure how many will be using the new legged robot with wheels. I expect maybe three or four initially, which would make a reasonable bracket of teams that can collaborate and compare more closely than others. There is a team from Austria that was in the German Open that is planning to come to Brazil. I’m sure when more people see the robot in action they’ll want one. Especially any new team coming into the league starting from scratch. Meanwhile, the RoboCupRescue league is poised to evaluate that robot design fully, add essential operational capabilities, and push the state of the science a bit further.

Will there be any different challenges this year?

Yes, this year we’re adding a new version of our original Stepfield terrain. They were our first rather complex terrain that inspired drastic changes in robot designs twenty years ago. The Stepfield terrain ensures the robots are always in contact with multiple different terrain elevations, and there is no respite throughout, even for legged robots with wheels. For the past couple of years, we got away from the Stepfield terrain because it was the hardest terrain to replicate at a temporary venue. But the year off for COVID was a growth time for us in terms of making the test methods as inexpensive and stowable as possible, so that small research organizations can replicate the tests to practice, refine approaches, and evaluate themselves during their development process. We first found a way to fabricate new Stepfields terrains from purchased, stackable crates. Now we have a fabricated version that accomplishes the same objectives less expensively. So RoboCupRescue teams will validate that the test apparatus as fabricated can stand up to the constant rigor of a very wide variety of robots across hundreds of trials in less than a week. That’s why RoboCupRescue is an excellent incubator for standard test methods too.

Is there anything else you’d like to highlight about the League?

One of the key features of RoboCupRescue is that the test lanes fabricated for the competition typically become a robot test facility for a local organization, like a nearby emergency responder facility or university. It can be set up perpetually, or it can move periodically to support regional competitions within the country. We usually try to connect the local organizers with their nearest emergency responders who already have robots or are seeking to purchase robots. The Olympic Village is the best analogy because it stays behind after the competition is over to be useful to the local region. So as we host our 25th year of RoboCupRescue in Brazil, we add to our world map of standard test facilities and new friends that we can continue to collaborate with professionally, and hopefully help their regional emergency responders use robots to stay out of harm’s way.

About Adam

Adam Jacoff is a robotics research engineer at the National Institute of Standards and Technology (NIST) which is part of the U.S. Department of Commerce. Over the past thirty years he has developed a variety of innovative robots and directed evaluations of more than a hundred others in a range of sizes, including the first technology readiness level assessment of autonomous mobility for the U.S. Army’s Experimental Unmanned Vehicle (XUV) (2002-2003).   His current efforts are focused toward developing 50 standard test methods to objectively evaluate ground robots, aerial drones, and aquatic vehicles for public safety operations. He has conducted 50 international robot competitions using the test methods as challenge tasks to guide innovation and measure progress (2000-present), and more than a hundred robot exercises to refine and validate the test methods with emergency responders and robot manufacturers (2005-present). These include dozens of comprehensive robot evaluations using the test methods to quantify key capabilities guiding more than $200M of purchasing decisions for civilian and military organizations (2010-present). He is now validating use of the test methods as repeatable practice tasks to focus remote operator/pilot training and measure proficiency for credentialing (2015-present).  He received a B.S. degree in Mechanical Engineering from the University of Maryland and a M.S. degree in Computer Science from Johns Hopkins University.