Sympozium’s memory-enabled agent runs never started on clusters that cannot pull from Docker Hub

The issue, in one line

The controller created the wait-for-memory init container with the literal image busybox at tag 1.36 and no way to override it, so on a mirrored or air-gapped registry the init container sat in ImagePullBackOff and every memory-enabled AgentRun pod never ran. The issue.

The cause

The AgentRun pod mutators build three utility containers from a hardcoded busybox 1.36 reference, the wait-for-memory init, the wait-for-shared-memory init and the postRun done container. Sympozium’s own images already had SYMPOZIUM_IMAGE_REGISTRY and SYMPOZIUM_IMAGE_TAG, and busybox had no equivalent, so a cluster that only pulls from an internal mirror got ErrImagePull on the init and the agent-runner, ipc-bridge and skill sidecars never started. The long-lived memory Deployment stayed Ready, so nothing surfaced except the stuck AgentRun.

Where it lives

• internal/controller/agentrun_pod_mutators.go, the wait-for-memory and wait-for-shared-memory init specs stopped hardcoding busybox at tag 1.36 and use the helper
• internal/controller/busyboximage.go, new 21-line busyboxImage() helper, SYMPOZIUM_BUSYBOX_IMAGE when set, busybox at tag 1.36 otherwise
• internal/controller/agentrun_controller.go, the postRun done container goes through the same helper
• charts/sympozium/values.yaml, new image.utility.busybox value, default busybox at tag 1.36, beside SYMPOZIUM_IMAGE_REGISTRY
• charts/sympozium/templates/controller-deployment.yaml, SYMPOZIUM_BUSYBOX_IMAGE exported from image.utility.busybox
• charts/sympozium/templates/pre-delete-finalizers.yaml, the pre-delete hook now reads image.utility.kubectl, so both utility images can point at a private mirror
• internal/controller/agentrun_memory_test.go, new test that sets SYMPOZIUM_BUSYBOX_IMAGE to a mirror and asserts the init container uses it

The fix, planned in the free diagnosis

• busyboxImage() returns SYMPOZIUM_BUSYBOX_IMAGE when it is set and busybox at tag 1.36 otherwise, and all three injected containers use it.
• the chart exposes image.utility.busybox and the controller deployment exports it as SYMPOZIUM_BUSYBOX_IMAGE, so the override travels the same road as SYMPOZIUM_IMAGE_REGISTRY.
• the pre-delete hook reads image.utility.kubectl, so a private mirror can serve both utility images.
• a controller test pins the override, and the busybox defaults in the API server and CLI editor are left as starting values a user edits.

How it’s proven, in the sprint

• agentrun_memory_test.go fails on main and passes on the branch, per the PR description, and the rest of the controller package passes with it.
• helm template renders the defaults unchanged and both overrides with a mirrored image, and helm lint is clean.
• GitHub records 30 check runs on the merged commit, all success apart from the skipped release and stubbed Cypress specs, including System tests (envtest) and build-and-push (controller).
• the maintainer re-checked the three injected containers, the chart value and the pre-delete hook line by line and approved with CI green (quote below).

What wasn’t tested, said up front

No real air-gapped or mirrored cluster was used, the proof is a unit test plus rendered Helm output, not a kubectl apply. The busybox literals left in the API server and CLI hook editor were not exercised. The web console was untouched and its Cypress specs are skipped in CI.

What happened next, after the sprint

It was merged on 2026-10-13 about 3 days after it opened, with 60 lines added and 4 removed. The reviewer wrote “All three injected containers (wait-for-memory, wait-for-shared-memory, postRun done) go through busyboxImage(), the chart sets SYMPOZIUM_BUSYBOX_IMAGE from image.utility.busybox, and the pre-delete hook reads image.utility.kubectl, with defaults identical to today.” (source). Open the pull request.