heicode-win/heicode/docs/deploy/new-api-rebuild-deploy-runbook.md

NewAPI / Heicode Manager Rebuild and Deploy Runbook

Date: 2026-05-03
Scope: rebuild and deploy the heicode Manager service on an Azure VM while keeping NewAPI as an independent model gateway.
Security rule: never put real passwords, tokens, SSH keys, Redis keys, PostgreSQL DSNs, NewAPI keys, or cloud credentials in Git, Markdown, shell history, tickets, or logs. Use environment variables, .env on the VM, Secret Store references, or REDACTED placeholders only.

1. Current repo deployment surfaces

Surface	Purpose	Notes
heicode/Dockerfile	Production image build	Builds default and classic web assets, then compiles the Go binary.
heicode/docker-compose.azure-vm.yml	Azure VM Manager service	Runs only heicode; PostgreSQL and Redis are expected to be managed Azure services.
heicode/docker-compose.override.yml	Local-source image override	Builds heicode-manager:local from the checked-out repo. Keep it in the compose file list when deploying this repo state.
heicode/bin/azure_vm_deploy.sh	SSH deployment helper	Uses env vars only; can fast-forward a remote branch, performs remote compose up, health gate, and rollback pointer capture.
heicode/bin/acceptance_agnet_local.sh	Local Agnet control-plane smoke/acceptance probe	Requires an admin session cookie supplied via env; does not store credentials.
heicode/.env.example	Env-var reference	Placeholder-only reference; production .env must stay on the VM and out of Git.

2. Required VM inputs

Prepare these values outside the repo, for example in a secret manager, CI secret store, or an uncommitted VM-local .env:

VM_HOST=<azure-vm-host-or-ip>
VM_USER=<ssh-user>
REMOTE_DIR=/opt/heicode/heicode
GIT_REF=main
HEICODE_DATA_ROOT=/var/lib/heicode/heicode
SQL_DSN=postgresql://<user>:REDACTED@<azure-postgres-host>:5432/heicode?sslmode=require
REDIS_CONN_STRING=rediss://:REDACTED@<azure-redis-host>:6380
SESSION_SECRET=REDACTED
HEICODE_AUTH_BASE_URL=https://apimtaiji.azure-api.net/api/mcp
HEICODE_ROOT_EMAILS=<comma-separated-admin-emails-if-needed>
HEICODE_ADMIN_EMAILS=<comma-separated-admin-emails-if-needed>

Do not paste real values into this document, commits, chat, or terminal transcripts that will be saved.

3. Pre-deploy checklist

Run locally from the repo root unless stated otherwise:

1. Confirm mainline docs boundary:
   • docs/heicode.md
   • docs/plan.md
   • docs/README.md
2. Confirm worktree state:
   • git status --short
3. Validate compose syntax without exposing secrets:
   • cd heicode
   • cat > /tmp/heicode.deploy.env <<'EOF_ENV'
   • HEICODE_DATA_ROOT=/tmp/heicode-data
   • SQL_DSN=REDACTED
   • REDIS_CONN_STRING=REDACTED
   • SESSION_SECRET=REDACTED
   • EOF_ENV
   • ENV_FILE=/tmp/heicode.deploy.env IMAGE_TAG=heicode-manager:local docker compose -f docker-compose.azure-vm.yml -f docker-compose.override.yml --env-file /tmp/heicode.deploy.env config --quiet
   • rm -f /tmp/heicode.deploy.env
4. Build verification options:
   • Preferred full image check: cd heicode && docker compose -f docker-compose.azure-vm.yml -f docker-compose.override.yml build heicode
   • Backend slice check when full image build is too slow: cd heicode && go test -count=1 ./controller ./model ./router
5. Confirm no secrets are staged:
   • git diff --cached -- . ':!*.jpg' ':!*.png'
   • Search for known real secret fragments only locally; never copy matches into reports.

4. VM preparation checklist

Run on the Azure VM:

export HEICODE_DATA_ROOT=/var/lib/heicode/heicode
sudo mkdir -p "$HEICODE_DATA_ROOT"/data "$HEICODE_DATA_ROOT"/logs
sudo chown -R "$USER":"$USER" "$HEICODE_DATA_ROOT"
cd /opt/heicode/heicode

Create or update VM-local .env in REMOTE_DIR; keep it untracked:

cat > .env <<'EOF_ENV'
HEICODE_DATA_ROOT=/var/lib/heicode/heicode
SQL_DSN=postgresql://<user>:REDACTED@<azure-postgres-host>:5432/heicode?sslmode=require
REDIS_CONN_STRING=rediss://:REDACTED@<azure-redis-host>:6380
SESSION_SECRET=REDACTED
HEICODE_AUTH_BASE_URL=https://apimtaiji.azure-api.net/api/mcp
HEICODE_ROOT_EMAILS=
HEICODE_ADMIN_EMAILS=
EOF_ENV
chmod 600 .env

Replace REDACTED placeholders through a secure VM-local editor or secret injection mechanism only.

5. Rebuild and deploy procedure

From the local control machine, after the VM repo has the intended commit:

cd heicode
VM_HOST=<azure-vm-host-or-ip> \
VM_USER=<ssh-user> \
REMOTE_DIR=/opt/heicode/heicode \
GIT_REF=main \
HEALTH_URL=http://127.0.0.1:3000/api/status \
./bin/azure_vm_deploy.sh

The script will:

1. SSH to the VM.
2. Verify the remote repo, compose CLI, compose file, and VM-local env file exist.
3. If GIT_REF is set, fast-forward the remote checkout from origin/<GIT_REF>.
4. Record the currently running container image in .last_success_image when available.
5. Run ENV_FILE=.env docker compose -f docker-compose.azure-vm.yml -f docker-compose.override.yml --env-file .env up -d --build --force-recreate heicode.
6. Poll /api/status through the VM-local health URL.
7. Update .last_success_image only after the health gate passes.

5.1 Agnet operator handoff

When Agnet is the executor, Manager should create a high-risk newapi-rebuild-deploy deployment using docs/integration/agnet-platform-request-contract.md and pass only references:

Field	Required reference
resource_grants[].secret_ref	Secret Store reference for SSH access and runtime env access; never plaintext.
resource_grants[].metadata.host_ref	env://NEWAPI_VM_HOST or equivalent platform secret/env ref.
resource_grants[].constraints.rollback_command_ref	runbook://newapi/rollback or this runbook section.
orchestration_plan.constraints.healthcheck_url_ref	env://NEWAPI_HEALTHCHECK_URL, expected to resolve to the VM-local /api/status probe.
orchestration_plan.metadata.commit	Intended Git commit or branch to deploy, such as origin/main after push.

Minimum evidence Agnet must return before Manager marks the operation deployed:

1. Remote commit after fetch/pull.
2. docker compose ... ps status for heicode.
3. Health check response proving "success":true.
4. Redacted log tail or log digest.
5. Rollback pointer (.last_success_image) update or explicit note that no previous image existed.

If any item is missing, Manager must keep the deployment in running, failed, or pending_operator_review; it must not report production deployment as completed.

6. Post-deploy verification checklist

Run on the VM or through secure operator access:

cd /opt/heicode/heicode
ENV_FILE=.env docker compose -f docker-compose.azure-vm.yml -f docker-compose.override.yml --env-file .env ps
ENV_FILE=.env docker compose -f docker-compose.azure-vm.yml -f docker-compose.override.yml --env-file .env logs --tail=100 heicode
curl -fsS http://127.0.0.1:3000/api/status

Expected health response includes "success":true.

Optional Manager / Agnet smoke probe after obtaining a safe admin session cookie without logging it:

AUTH_COOKIE='REDACTED_SESSION_COOKIE' \
BASE_URL='http://127.0.0.1:3000' \
TENANT_ID='tenant_smoke' \
PROJECT_ID='project_smoke' \
./bin/acceptance_agnet_local.sh

Do not commit or report the real cookie.

7. Rollback checklist

If the health gate fails or post-deploy checks fail:

1. Keep the failed container logs for local VM inspection only; redact before sharing.
2. Inspect .last_success_image on the VM.
3. Re-run compose with the last known good image or revert the repo commit, then:

ENV_FILE=.env docker compose -f docker-compose.azure-vm.yml --env-file .env up -d --force-recreate heicode
curl -fsS http://127.0.0.1:3000/api/status

4. If database migrations were applied by the app, do not manually mutate production data without an explicit backup and approval.

8. Known risks / open items

• go test ./... can be blocked by missing embedded frontend build artifacts or pre-existing relay tests; use focused checks for the changed slice and a full Docker image build before production.
• Production PostgreSQL and Redis are external managed services; the compose file intentionally does not start local database/cache containers for Azure VM deployment.
• NewAPI remains an independent model gateway and should not be exposed as a normal SaaS user backend.
• Long-lived secrets must move toward Secret Store / Secret Broker; current VM .env usage is an operational bridge, not the final P2 target.