Environments

JoltGym provides three Gymnasium-compatible environments, all registered under the JoltGym/ namespace.

HalfCheetah-v0

HalfCheetahEnv(render_mode=None, forward_reward_weight=1.0, ctrl_cost_weight=0.1, reset_noise_scale=0.1)

Bases: Env

2D planar cheetah locomotion environment powered by Jolt Physics.

A 4-legged cheetah with 6 actuated hinge joints (back thigh/shin/foot, front thigh/shin/foot) and a slide+hinge root. The goal is to run forward (positive X direction) as fast as possible.

Observation

Box(-inf, inf, (17,)) — qpos[1:] (skip root X) concatenated with qvel.

Index	Dim	Content
0	1	root Z position (height)
1	1	root Y rotation (torso angle)
2–7	6	joint angles: bthigh, bshin, bfoot, fthigh, fshin, ffoot
8–10	3	root velocities: vx, vz, angular vy
11–16	6	joint velocities

Action

Box(-1, 1, (6,)) — normalized joint torques scaled by gear ratios (120, 90, 60, 120, 60, 30).

Reward

forward_reward_weight * x_velocity - ctrl_cost_weight * sum(action²)

Attributes:

Name	Type	Description
observation_space		Gymnasium Box space of shape (17,).
action_space		Gymnasium Box space of shape (6,).

Initialize the HalfCheetah environment.

Parameters:

Name	Type	Description	Default
render_mode		Rendering mode — "human" for window, "rgb_array" for pixel output, or None to disable.	None
forward_reward_weight		Multiplier on the forward velocity reward term.	1.0
ctrl_cost_weight		Multiplier on the control cost penalty term.	0.1
reset_noise_scale		Standard deviation of Gaussian noise added to joint positions and velocities on reset.	0.1

step(action)

Run one timestep (frame_skip=5 physics steps at dt=0.01s).

Parameters:

Name	Type	Description	Default
action		Normalized joint torques, shape (6,), range [-1, 1].	required

Returns:

Name	Type	Description
obs		Observation array of shape (17,).
reward		Scalar reward (forward_vel - ctrl_cost).
terminated		Always False (HalfCheetah has no terminal state).
truncated		Always False.
info		Dict with keys x_position, x_velocity, reward_run, reward_ctrl.

reset(*, seed=None, options=None)

Reset the environment to the initial state with optional noise.

Parameters:

Name	Type	Description	Default
seed		Random seed for reproducible resets.	None
options		Unused, present for Gymnasium compatibility.	None

Returns:

Name	Type	Description
obs		Initial observation array of shape (17,).
info		Empty dict.

close()

Shut down the underlying C++ physics engine.

---

Humanoid-v0

HumanoidEnv(render_mode=None, forward_reward_weight=1.25, ctrl_cost_weight=0.1, healthy_reward=5.0, healthy_z_min=1.0, healthy_z_max=2.0, reset_noise_scale=0.005)

Bases: Env

3D bipedal humanoid locomotion environment powered by Jolt Physics.

A humanoid with 17 actuated joints (abdomen, hips, knees, shoulders, elbows) and a free 6DOF root body. The goal is to walk forward while staying upright.

Observation

Box(-inf, inf, (45,)) — qpos[2:] (skip root X, Y) concatenated with qvel.

Index	Dim	Content
0	1	root Z position (height)
1–4	4	root quaternion (w, x, y, z)
5–21	17	joint angles
22–24	3	root linear velocity (x, y, z)
25–27	3	root angular velocity (x, y, z)
28–44	17	joint velocities

Action

Box(-0.4, 0.4, (17,)) — normalized joint torques for 17 actuated joints: abdomen (3), right hip (3) + knee, left hip (3) + knee, right shoulder (2) + elbow, left shoulder (2) + elbow.

Reward

forward_reward_weight * x_velocity + healthy_reward * is_healthy - ctrl_cost_weight * sum(action²)

Termination

Episode ends when root Z position is outside [healthy_z_min, healthy_z_max].

Attributes:

Name	Type	Description
observation_space		Gymnasium Box space of shape (45,).
action_space		Gymnasium Box space of shape (17,).

Initialize the Humanoid environment.

Parameters:

Name	Type	Description	Default
render_mode		Rendering mode — "human", "rgb_array", or None.	None
forward_reward_weight		Multiplier on the forward velocity reward.	1.25
ctrl_cost_weight		Multiplier on the control cost penalty.	0.1
healthy_reward		Bonus reward for staying upright each step.	5.0
healthy_z_min		Minimum root Z height to be considered healthy.	1.0
healthy_z_max		Maximum root Z height to be considered healthy.	2.0
reset_noise_scale		Standard deviation of noise added on reset.	0.005

step(action)

Run one timestep (frame_skip=5 physics steps at dt=0.003s).

Parameters:

Name	Type	Description	Default
action		Normalized joint torques, shape (17,), range [-0.4, 0.4].	required

Returns:

Name	Type	Description
obs		Observation array of shape (45,).
reward		Scalar reward.
terminated		True when root Z leaves [healthy_z_min, healthy_z_max].
truncated		Always False.
info		Dict with keys x_position, z_position, x_velocity, reward_forward, reward_ctrl.

reset(*, seed=None, options=None)

Reset the environment to the initial state with optional noise.

Parameters:

Name	Type	Description	Default
seed		Random seed for reproducible resets.	None
options		Unused, present for Gymnasium compatibility.	None

Returns:

Name	Type	Description
obs		Initial observation array of shape (45,).
info		Empty dict.

close()

Shut down the underlying C++ physics engine.

---

CheetahRace-v0

CheetahRaceEnv(num_agents=2, render_mode=None, agent_spacing=3.0, forward_reward_weight=1.0, ctrl_cost_weight=0.1, reset_noise_scale=0.1)

Bases: Env

Multi-agent cheetah race in a shared physics world.

N cheetahs are placed side-by-side (Y-offset) and race forward (X-axis). All agents share the same PhysicsWorld, so they can physically collide and interact.

This wraps all agents into a single Gymnasium env suitable for independent-learner multi-agent training with parameter sharing. Observations and actions are flat concatenations of per-agent vectors.

Observation

Box(-inf, inf, (N*17,)) — concatenation of each agent's qpos[1:] + qvel.

Action

Box(-1, 1, (N*6,)) — concatenation of each agent's normalized joint torques.

Reward

Sum of all agents' individual rewards (forward_velocity - ctrl_cost per agent).

Attributes:

Name	Type	Description
num_agents		Number of cheetahs in the race.
observation_space		Gymnasium Box space of shape (num_agents * 17,).
action_space		Gymnasium Box space of shape (num_agents * 6,).

Initialize the CheetahRace environment.

Parameters:

Name	Type	Description	Default
num_agents		Number of cheetahs in the race.	2
render_mode		Rendering mode — "human", "rgb_array", or None.	None
agent_spacing		Y-axis distance between adjacent agents.	3.0
forward_reward_weight		Multiplier on forward velocity reward.	1.0
ctrl_cost_weight		Multiplier on control cost penalty.	0.1
reset_noise_scale		Standard deviation of noise added on reset.	0.1

step(action)

Run one timestep for all agents simultaneously.

Parameters:

Name	Type	Description	Default
action		Flat array of shape (num_agents * 6,).	required

Returns:

Name	Type	Description
obs		Flat observation of shape (num_agents * 17,).
reward		Scalar total reward (sum of all agents).
terminated		Always False.
truncated		Always False.
info		Dict with per_agent_reward array, and per-agent agent_{i}_x / agent_{i}_xvel keys.

reset(*, seed=None, options=None)

Reset all agents to their initial positions.

Parameters:

Name	Type	Description	Default
seed		Random seed for reproducible resets.	None
options		Unused, present for Gymnasium compatibility.	None

Returns:

Name	Type	Description
obs		Flat initial observation of shape (num_agents * 17,).
info		Empty dict.

close()

Clean up resources.

get_per_agent_obs(flat_obs)

Split a flat observation into per-agent arrays.

Parameters:

Name	Type	Description	Default
flat_obs		Flat observation of shape (num_agents * 17,).	required

Returns:

Type	Description
	Per-agent observations of shape (num_agents, 17).

get_per_agent_actions(flat_action)

Split a flat action into per-agent arrays.

Parameters:

Name	Type	Description	Default
flat_action		Flat action of shape (num_agents * 6,).	required

Returns:

Type	Description
	Per-agent actions of shape (num_agents, 6).