Repository navigation
jupyter_env: verification runs in the agent's own kernel, so a rollout can report any reward #1210
Description
Summary
E2BSandbox.run_shell executes shell commands by running a subprocess.run snippet through run_code, which is the same persistent kernel the agent writes to with add_and_execute_code_cell. A single cell that rebinds subprocess.run therefore decides the outcome of every shell command the environment runs afterwards, including the verify commands and the read of the reward file.
The agent does not need to write reward.txt, and it does not need the verify commands to pass. It replaces the function that runs them.
This is separate from #1199. That issue is about the contents of a file the agent can write; this is about the environment asking the agent's own kernel whether the agent succeeded.
Where
On main @ 39fdf5cd:
envs/jupyter_env/server/e2b_sandbox.py:91,run_shellbuilds asubprocess.runcell and sends it torun_code.envs/jupyter_env/server/jupyter_environment.py:385,_run_verify_commandsscorespassed / totalfrom those results.envs/jupyter_env/server/jupyter_environment.py:444,_read_reward_overridereadsreward.txtthrough the samerun_shell.
success is execution.error is None, so a cell that raises nothing counts as a passing command.
Reproduction
No E2B account needed. This drives the real E2BSandbox.run_shell and the real JupyterEnvironment against a fake kernel whose only interesting property is the one E2B's has: a namespace that persists between cells.
import contextlib, io, sys, types
from jupyter_env.models import JupyterState
from jupyter_env.server.e2b_sandbox import E2BSandbox
from jupyter_env.server.jupyter_environment import JupyterEnvironment
from openenv.core.env_server.mcp_types import CallToolAction
class FakeKernel:
"""A persistent Python kernel, which is what E2B's run_code gives you."""
def __init__(self):
self.ns: dict = {}
def run_code(self, code: str):
out, err, error = io.StringIO(), io.StringIO(), None
try:
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
exec(code, self.ns)
except BaseException as exc: # SystemExit included, as in a real kernel
error = types.SimpleNamespace(name=type(exc).__name__, value=str(exc), traceback="")
return types.SimpleNamespace(
logs=types.SimpleNamespace(
stdout=[out.getvalue()] if out.getvalue() else [],
stderr=[err.getvalue()] if err.getvalue() else [],
),
error=error,
results=[],
execution_count=1,
)
AGENT_CELL = """\
import subprocess
class _Fake:
returncode = 0
stdout = ""
stderr = ""
subprocess.run = lambda *a, **k: _Fake()
"""
sandbox = object.__new__(E2BSandbox) # the real wrapper, without an E2B account
sandbox._sbx = FakeKernel()
sandbox.sandbox_id = "fake"
env = JupyterEnvironment()
env._sandbox = sandbox
env._state = JupyterState(
episode_id="ep-1",
sandbox_id="fake",
verify_commands=["exit 1", "exit 1"], # both fail
)
env.step(CallToolAction(tool_name="add_and_execute_code_cell", arguments={"code": AGENT_CELL}))
env.step(CallToolAction(tool_name="final_answer", arguments={"answer": "done"}))
results = env.state.verify_results
print(f"{sum(1 for r in results if r.success)}/{len(results)} passed, reward={env.state.last_reward}")Output:
2/2 passed, reward=1.0
Both verify commands exit non-zero. No reward file is ever written.
#1200 does not close this
I applied #1200's run_shell change (raise SystemExit(_result.returncode) when the return code is non-zero) in the same reproduction and got the same result:
2/2 passed, reward=1.0
The check runs inside the kernel, so it reads the return code off the object the agent's function returned, which is 0. The same applies to the new pre-verify rm -f ... && test ! -e ... guard: it is another run_shell, so the agent decides whether it reports success. Setting stdout = "1.0" on the fake result instead of "" also puts the override path back in the agent's hands, within the [0, 1] range #1200 enforces.
None of that is a criticism of #1200. It fixes what it set out to fix, and the author already flagged this case as out of scope for it.
Not the same shape as terminus_env
terminus_env/server/e2b_sandbox.py builds shell commands the same way, but the agent only ever reaches that sandbox through terminal(command=...), which runs as a subprocess rather than as kernel Python. Nothing the agent sends mutates the kernel namespace, so the escape does not apply there. jupyter_env is exposed because add_and_execute_code_cell is kernel-level Python by design, which is the point of the environment.
Suggested direction
Run verification somewhere the agent's code cannot reach, and keep the notebook kernel for the notebook.
The pinned SDK already offers this: e2b_code_interpreter.Sandbox exposes commands (verified against e2b-code-interpreter>=1.0.0, the env's own pin), so self._sbx.commands.run(command) executes as a process with a real exit status that no cell can intercept. create_code_context is an alternative if a separate kernel is preferred, and a second sandbox for verification would also work at higher cost.
The reward-file read wants the same treatment, since reading it through the kernel is equally controllable.
Worth stating as an env-level rule while this is being fixed: nothing the agent can execute should be able to influence how verification runs, or how its result is read.
Scope
Happy to implement this once #1200 lands, since it touches the same two files and I would rather not cut across it.
Activity
surajsharan commentedon Sep 21, 2026
Thanks for writing this up so thoroughly, and for checking it against #1200's changes. The end-to-end reproduction is really useful. For context, I ran into the same thing while testing that fix and noted it on #1199 (#1199 (comment)), and #1200 lists it as a follow-up.
Would you be OK with me taking the implementation, since it sits right next to #1200's changes in the same files? Your reproduction would be the regression test, and I'd add you as co-author on the commit.
One thing I found while reading the SDK that's worth knowing whoever picks it up:
commands.runstarts/bin/bash -l -c, so it sources the user's login files, which the agent can also write. The fix needs to cover or at least document that.If you'd rather do it yourself, that's completely fine too, and I'm happy to review it.
k21993 commentedon Sep 21, 2026
Yes, please take the implementation. Using the reproduction as the regression test and adding me as co-author works for me.
Good catch on the login shell. I agree that commands.run alone is insufficient if the agent can modify the selected user’s startup files. Please extend the regression coverage to poison the relevant login profile and verify that the real command exit status still reaches the verifier.
A dedicated verifier user with an agent-unwritable home may be sufficient through commands.run(user=...). If that boundary cannot be enforced, a separate sandbox or another execution path may be necessary. Happy to review the PR against that invariant and the final #1200 behavior.
surajsharan commentedon Sep 21, 2026
Thanks! Before writing it I checked which user the agent actually runs as, since that decides whether a verifier-user boundary can hold, and in this template it can't:
jupyter_envcreates its sandbox withSandbox.create(api_key=...), so it gets the SDK default,code-interpreter-v1.sudo systemctl start jupyter(https://github.com/e2b-dev/code-interpreter/blob/f56a1edf750e/template/template.py#L136-L141), the unit sets noUser=(https://github.com/e2b-dev/code-interpreter/blob/f56a1edf750e/template/systemd/jupyter.service#L7-L10), and the server runs withallow_root = True(https://github.com/e2b-dev/code-interpreter/blob/f56a1edf750e/template/jupyter_server_config.py#L60). Code contexts take no user option, so the agent's cells run as root, and the defaultuserhas passwordless sudo on top.commands.runalways starts/bin/bash -l -c, so whichever user it runs as, that user's login files are sourced, and a root kernel can write all of them.So in
jupyter_envnothing inside the sandbox is out of the agent's reach, and a dedicated verifier user can't be enforced. A login-profile test would pass only because the fake says so, so I'd rather document that limit than include a test that implies it's covered. What I'd propose:commands.run, as root in the notebook's working directory so verify commands keep the permissions they have today. That closes the path in this issue, and it also stops ordinary notebook state (os.chdir,os.environ, a mockedsubprocess) from changing what verification sees, which can bite honest agents too. Tests: your reproduction, the notebook-state cases, and a non-zero exit throughCommandExitException. The PR will say plainly that against a root agent this is hardening, not a boundary.Does that split work for you?
k21993 commentedon Sep 22, 2026
That split works for me, with one framing condition: the first PR should describe
commands.runas removing notebook-kernel coupling and hardening verification, rather than establishing an isolation boundary. The root agent can still influence the same sandbox.The reproduction, notebook-state cases, and real non-zero exit coverage sound right. Please also link a follow-up before merge for fresh-sandbox verification, with the invariant that agent-controlled state cannot alter verifier startup, execution, or result collection. Happy to review both the implementation and that boundary.
Run jupyter_env verification outside the notebook kernel
Run jupyter_env verification outside the notebook kernel
Run jupyter_env verification outside the notebook kernel (openenv#1233)