Up and Down the Abstraction Ladder — Code-Based Skills for Language Agents

Motivation

Developers write code to automate repetitive tasks such as renaming files, processing data, and running experiments. A correct program can carry out these tasks quickly and consistently. We can inspect its implementation, test its behavior, and reuse it across tasks.

LLM agents can do the same. They can write programs or use existing tools to automate parts of a task. The code handles the repeated steps, reducing the number of model calls and the opportunities for mistakes in those steps. The model decides when to use the tool and what to do with its result.

Code can also automate behavior in sequential decision making problems, such as playing games or controlling robots. These problems are also studied in reinforcement learning, where agents learn through interaction. Code as Policies shows how LLMs can generate programs that control robots by choosing actions from observations. We call these programs code skills. They are tools that let an agent delegate behavior over multiple steps.

That delegation creates an abstraction. A single request stands for a sequence of decisions made inside the skill. Those decisions can be useful across many situations, yet unsuitable for an unexpected one. A library may also leave out a behavior the agent needs entirely.

Other work explores how to build and reuse these capabilities. Voyager develops a library of code skills in Minecraft. Skills can also be learned with neural networks. SayCan uses a language model and estimates of success to choose among pretrained robot skills. In MaestroMotif, generated code combines learned behaviors.

Our study focuses on what happens once skills are available. How much do they help an agent act and learn, and when is it worth moving back down to individual actions? We keep the supplied skill library fixed to examine these questions separately from skill creation. We study them in NetHack, where familiar procedures must coexist with situations they cannot fully anticipate.

Why NetHack?

NetHack has long been a challenge for AI agents. The player must survive an expedition through a dungeon, exploring unfamiliar rooms, encountering monsters, and collecting equipment. Winning requires retrieving the Amulet of Yendor from Moloch’s Sanctum and bringing it back through the dungeon. The player must then survive the Elemental Planes, a final series of dangerous levels, and offer the amulet at their god’s altar on the Astral Plane. Death ends the run.

The figure below shows the variety of levels a player can encounter in NetHack’s dungeon.

NetHack’s dungeon is randomly generated, and a successful run can involve tens of thousands of turns and several days of play. The dungeon is especially dangerous in the early game, when the character is weak and has limited resources. NetHack is also incredibly complex, with hundreds of different monsters and items, a vast set of rules, and unexpected interactions between creatures, objects, and the environment.

To illustrate the complexity of NetHack, let’s take a closer look at Medusa’s Island. The player needs to reach the stairs on an island guarded by Medusa, whose gaze can turn them to stone. Crossing the surrounding water might mean levitating over it, freezing a path with a wand of cold, or filling gaps with boulders. With a pickaxe, the player can instead descend through the floor and return by the stairs from below. But digging beside water risks drowning, and the return route leads straight into Medusa’s room.

The player also needs a plan for dealing with Medusa herself. A shield of reflection can turn her to stone with her own gaze, but this will not work if the player is invisible to her. Wearing a blindfold offers another approach, protecting the player from her gaze while they fight without sight. Without a way to handle the gaze, the player may need to retreat and return with better equipment. The level is a puzzle whose solution depends on the items, abilities, and routes available in that run.

A skill could be written to beat Medusa, but it would need to choose among these plans and handle their many edge cases as the character’s role, inventory, and level layout change. Programming and testing all these combinations would take substantial work. And this is one encounter in a dungeon with more than 50 levels.

This motivates a division of work. Skills can automate familiar, repetitive procedures and return control to the LLM when they reach the limits of what they can handle. The agent can then reconsider the plan, combine skills that still apply, or use individual actions before delegating again. NetHack lets us study how much can be gained from routine automation while keeping this flexibility to deal with situations the skill code does not cover.

What we study

We study how the actions available to a language agent affect its performance, inference cost, and learning. To make this comparison possible, we built CodeHack, an open-source library of reusable Python skills and a shared runtime for NetHack and MiniHack.

CodeHack provides 78 skills for recurring tasks such as exploration, navigation, combat, and equipment management. A skill reads the game state and issues commands, adapting to new observations as it runs. When it completes, fails, or is interrupted, control returns to the model. We use this library to compare three ways of acting.

Primitives

Choose a game command

The model handles each step, from moving through a corridor to responding to a game prompt.

Move · Search · Kick

Skills

Delegate a procedure

The model selects a skill. Its code handles the intervening actions until control returns.

Explore · Fight melee · Open doors

Mixed

Choose either level

The model can use a skill, act through individual commands, and return to skills when useful.

Skills + primitives

Shared across all three · observations, map memory, inventory tracking, and execution runtime

Within each task, we keep the game settings, observation format, and prompt template fixed. The available actions and their descriptions change. The skill code stays fixed, including during training.

We examine this choice in three settings.

Before training: We compare 14 models on NetHack and test GPT-5 on seven MiniHack tasks that isolate navigation, exploration, combat, and item use.
With missing skills: We remove skill families and test whether access to primitives helps Gemma-4-31B cope with the gaps.
During learning: We train Llama-3.1-8B-Instruct and Qwen-3.5-4B with reinforcement learning, and test whether learning from demonstrations first provides a better starting point.

Together, these experiments measure the benefits of a supplied library, the cost of retaining finer control, and how the action interface shapes learning. We release the training and evaluation code alongside CodeHack.

What we learned

Zero-shot control

Skills improve success rates across MiniHack tasks and NetHack performance across model families.

Skills improve success rates on every evaluated MiniHack task, with an average gain of 55 percentage points over primitive control. These benefits extend to NetHack, where skills nearly triple average progression across 14 models. With skills, GPT-5 reaches dungeon level 4 in 84% of episodes, compared with 3% under primitive control. These are substantial early-game gains, but even GPT-5 with skills remains far from solving NetHack.

GPT-5 task success in MiniHack and milestone reach in NetHack. Skills improve all seven MiniHack tasks and make deeper NetHack milestones substantially more reachable. — Zero-shot results across the two evaluation domains. (Left) MiniHack success rates for GPT-5. (Right) NetHack milestone reach for GPT-5. Both plots present averages over 32 episodes per task. View full-size figure

For a qualitative view of how agents use these interfaces, we examine selected GPT-5 trajectories. These include a full NetHack run with mixed control and successful solutions to seven MiniHack tasks with skill-only control.

NetHack trajectory — mixed control

A full NetHack trajectory. GPT-5 uses skills and individual commands to reach dungeon depth 10. The video shows eight controller decisions per second. The playback rate is not the agent’s live speed.

MiniHack trajectories — skill-only control

Trajectories across seven MiniHack tasks. MiniHack uses the same game world to create smaller tasks. These selected successful trajectories show navigation, combat, and tool use. The decision counts refer to the agent choosing a skill, which may execute several game actions.

Inference cost

Skills improve the performance-cost frontier, enabling agents to make more progress at lower inference cost.

Averaged across models, skills reduce inference cost per episode by 86% relative to primitives. Skills lower inference cost by reducing the number of LM calls per episode by an average factor of 5.1. Adding primitives alongside skills preserves most of these performance benefits, although at a higher inference cost than skill-only control.

Average maximum dungeon depth against estimated inference cost per episode. For most models, skill-based control reaches deeper at lower cost than primitive-only control. — Pareto frontier. Each point represents a model evaluated with a particular action interface, showing its average dungeon level reached against its average inference cost per episode. For most models, switching from primitives to skills increases dungeon depth while reducing inference cost. View full-size figure

Learning with RL

The benefits of skills extend to learning. RL amplifies the performance gap between skill-based control and primitive-only control.

Under a fixed RL budget in NetHack, skill-only and mixed controllers achieve gains in dungeon depth 7.2 and 8.6 times those of primitive-only controllers, respectively, averaged across two models. These larger gains widen the absolute performance gap already present in zero-shot evaluation.

Qwen-3.5-4B learning curves for progression, score, and maximum dungeon depth. Skill-only and mixed controllers increase their lead over primitive-only control during PPO training. — Full training curves for Qwen-3.5-4B (gradient steps). We plot progression (left), score (middle), and dungeon level (right) vs. gradient steps. All curves are averages across three seeds (except for mixed, which uses 2 seeds), and error bands indicate one standard error. View full-size figure

Incomplete skill coverage

Retaining access to primitive actions helps compensate for incomplete skill coverage.

Mixed control is generally less affected by missing skills than skill-only control. Access to primitives therefore mitigates the loss of supplied skills, but does not guarantee successful recovery.

Gemma-4-31B progression, score, and dungeon depth with a full library and with individual skill families removed. Mixed control generally suffers less from missing skills than skill-only control. — Sensitivity to incomplete skill coverage for zero-shot Gemma-4-31B on NetHack. The full baseline uses all 78 skills. Each other condition removes the named skill family. Mixed control retains primitive actions. Bars show means over 64 episodes per condition. The comparison measures sensitivity to missing skill families and the extent to which primitive alternatives preserve performance. View full-size figure

For a qualitative analysis of primitive fallback, we examine two selected NetHack trajectories. Both show how an agent uses primitives to resolve a situation that the available skills cannot handle, then returns to skills.

Vault trajectory — primitive fallback

In the first trajectory, the agent encounters a vault guard, but the library lacks a skill for dropping gold to satisfy the guard’s demand. After several unproductive calls, the agent uses primitives to drop the gold and move along the exit corridor, then resumes navigation with explore.

Ranged-combat trajectory — primitive fallback

In the second trajectory, the agent faces a ranged combat situation where the fight_ranged skill returns without acting, the agent throws a javelin using primitives, then calls pickup_weapon to recover its weapons.

Mixed control requires the agent to choose when to use skills and when to act directly. These trajectories show useful recoveries. Their frequency and how RL changes those choices require further analysis.

What comes next

Our agents are still far from solving NetHack. The skills we provide cannot cover every situation, and the game demands knowledge, resource management, and planning beyond what those skills encode. Access to individual actions helps agents work around gaps in the library. Deciding when to use that flexibility remains a challenge.

A promising direction is to let agents develop the skills themselves. They could identify recurring failures, write code for missing behaviors, and test and refine it through interaction. The library could then grow to cover situations that its original designers had not anticipated.

Agents could also build tools for retrieving information from memory. Code could track previously observed items, creatures, and places, and bring back the details needed for a decision without repeatedly presenting the entire history to the model. Combining skill creation, access to relevant memories, and the ability to act directly offers a possible path toward solving NetHack.

Up and Down the
Abstraction Ladder Code-Based Skills for Language Agents

Motivation

Why NetHack?