Reef's OpenClaw-RL Recipe Puts Scored Weight Updates Behind a Release Gate

Scoring an OpenClaw task is the easy part. The harder question is what happens to the resulting weight update before it's allowed to touch a working serving state. Reef's repository ships a concrete OpenClaw-RL weight-evolution recipe that puts that update behind a release gate instead of trusting the score alone.

The Recipe, Concretely

The setup is bounded and specific — treat these numbers as a reproduction target, not a general benchmark:

• 72 GSM8K problems treated as 72 tasks — each problem is its own task unit in the loop.
• Qwen3-4B handles the policy and process-reward-model roles.
• Qwen3-32B is the student.
• Seven GPUs for the run.
• The project reports meeting its three-consecutive-pass criterion at session 14.
• The checked-in learning curve covers the first 36 sessions.

The gating logic is the interesting piece. A scored update doesn't get promoted to replace the working version until it clears a test — in this case, three consecutive passes. That's the difference between "the reward model liked it" and "it's safe to serve."

What These Numbers Are (and Aren't)

They're a reproduction target. They do not benchmark every OpenClaw workload, and they don't establish a generic OpenClaw harness adapter. If you read the session-14 / 36-session curve as a universal claim, you're reading it wrong. It's one recipe on one task family (GSM8K) with one model stack.

Suggested First Move

If you want to sanity-check the approach before adapting it:

1. Start with an OpenClaw task that has an objective verifier — no vibes-based scoring.
2. Reproduce the recipe as-is.
3. Confirm a deliberately bad weight candidate cannot move serving state. That's the release-gate property you actually care about.
4. Only after that passes is it worth asking whether the optimization transfers to your real task.

Step 3 is the one people skip. If a garbage update can slip past the gate, the rest of the pipeline is decoration.

Who This Is For

Developers building RL-style weight-evolution loops on top of OpenClaw who want a working reference for gating promotions rather than an abstract "it scored well, ship it" pipeline.