feat(heicode): 客户端错误遥测上报端点(默认关闭)(#24) (#27)

* feat(heicode): 客户端错误遥测上报端点(默认关闭)(#24)

按客户端 winos#23 契约 + 权威 schema 实现,结合 HM 入库逻辑:

- POST /api/heicode/telemetry/events,挂 UserOrV2DeviceAuth(设备配对鉴权)。
- 接收顶层 JSON 数组(非包裹),批量 1-20、<=256KB;校验 body client_id 等于已验签
  设备(X-Heicode-Device-Id),不一致 403;无设备身份拒绝。
- 真实 4xx/5xx 码(400 非数组、413 超限、403 设备、410 关闭),让客户端"4xx 丢弃"
  语义生效;2xx 返回 {accepted:n}。
- 独立表 telemetry_events,与计费完全隔离:不写 consume log、不碰 quota。
- 宽松入库(最大化采集):未知枚举 / 哨兵 app_version(0.0.0-heicode-local)/ 缺字段
  原样入库;schema_version 缺省 1;stack_top/context 存 TEXT(JSON);记 user_id 作
  device 到 account 关联 + 服务端 received_at。
- 默认 HEICODE_TELEMETRY_ENABLED=false 时返回 410(kill switch);隐私政策更新 +
  端点下发形态确认前不开启外发。

测试用客户端仿真夹具:parseTelemetryBatch / toModel 映射与默认 / 拒绝非数组,全过。

Refs #24(上线门槛:隐私政策 §2 如实披露 + 下发形态 + 去重;见工单评论)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(heicode): GET /api/heicode/config 下发 telemetry 配置/kill-switch (#24)

按客户端 #24 拍板:选独立 config 端点(而非塞登录响应),便于 kill switch 在
会话内传导、不依赖重登录。返回 telemetry 块 {enabled, endpoint, max_batch,
flush_interval_sec};enabled 取 HEICODE_TELEMETRY_ENABLED(默认 false)。
未鉴权全局只读(同 capabilities 姿态)。

测试 heicode_config_test.go:enabled 反映 env、endpoint 与摄入路由一致、缺省 false。

Refs #24

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(telemetry): 服务端二次脱敏 stack_top/context + 明确生产门槛 (#24 review)

按 Fasthei 复审意见补隐私门槛:

- 服务端纵深防御脱敏:新增导出 model.RedactText(包装已上线的 redactAuditSecrets,
  #11),在 telemetry 入库前对 stack_top / context 再脱敏一遍(sk-/Bearer/URL
  token/JSON 密钥字段),即便客户端漏脱敏也不会把明文密钥落库。
- 测试 TestTelemetryToModel_RedactsSecrets:stack_top 里的 sk-、context 里的
  Bearer token 被打码,非密钥内容保留。
- 端点默认 HEICODE_TELEMETRY_ENABLED=false,关时 410;隐私政策披露完成前生产
  不得开启外发(见 #24 评论记录产品/法务状态)。

go build / vet 干净;controller 测试通过。

Refs #24

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: chenchen <chenchen@xinghanlab.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
zsbgnw12
2026-06-09 22:16:29 +08:00
committed by GitHub
co-authored by Claude Opus 4.8 chenchen
parent 758e2d0342
commit 818f1412d5
8 changed files with 354 additions and 0 deletions
+29
View File
@@ -0,0 +1,29 @@
package controller
import (
"github.com/gin-gonic/gin"
"github.com/heicode/manager/common"
)
// HeicodeConfig: GET /api/heicode/config — client runtime config (issue #24).
//
// Dedicated config endpoint (client's chosen delivery, #24 §1) so kill switches
// propagate within a session WITHOUT re-login: the client polls this and obeys
// the latest telemetry.enabled / endpoint. Unauthenticated, non-sensitive global
// config — same posture as /api/heicode/capabilities; a natural home for future
// client config (feature flags, model-list pointer, …).
//
// telemetry.enabled defaults FALSE — telemetry stays off (the ingest endpoint
// also answers 410) until the privacy policy discloses account-linkable device
// IDs and ops flips HEICODE_TELEMETRY_ENABLED=true.
func HeicodeConfig(c *gin.Context) {
common.ApiSuccess(c, gin.H{
"telemetry": gin.H{
"enabled": common.GetEnvOrDefaultBool("HEICODE_TELEMETRY_ENABLED", false),
"endpoint": "/api/heicode/telemetry/events",
"max_batch": common.GetEnvOrDefault("HEICODE_TELEMETRY_MAX_BATCH", telemetryMaxBatch),
"flush_interval_sec": common.GetEnvOrDefault("HEICODE_TELEMETRY_FLUSH_INTERVAL_SEC", 30),
},
})
}
+37
View File
@@ -0,0 +1,37 @@
package controller
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/require"
)
// /api/heicode/config carries the telemetry kill-switch block (#24). enabled
// must reflect HEICODE_TELEMETRY_ENABLED and the endpoint must match the ingest
// route so the client polls the right place.
func TestHeicodeConfig_TelemetryBlock(t *testing.T) {
gin.SetMode(gin.TestMode)
t.Setenv("HEICODE_TELEMETRY_ENABLED", "true")
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
ctx.Request = httptest.NewRequest(http.MethodGet, "/api/heicode/config", nil)
HeicodeConfig(ctx)
require.Equal(t, http.StatusOK, rec.Code)
body := rec.Body.String()
require.Contains(t, body, `"telemetry"`)
require.Contains(t, body, `"endpoint":"/api/heicode/telemetry/events"`)
require.Contains(t, body, `"enabled":true`)
// default OFF when the flag is unset
t.Setenv("HEICODE_TELEMETRY_ENABLED", "")
rec2 := httptest.NewRecorder()
ctx2, _ := gin.CreateTestContext(rec2)
ctx2.Request = httptest.NewRequest(http.MethodGet, "/api/heicode/config", nil)
HeicodeConfig(ctx2)
require.Contains(t, rec2.Body.String(), `"enabled":false`)
}
+156
View File
@@ -0,0 +1,156 @@
package controller
import (
"encoding/json"
"io"
"net/http"
"strings"
"github.com/gin-gonic/gin"
"github.com/heicode/manager/common"
"github.com/heicode/manager/model"
)
// Client error-telemetry ingest (issue #24). Contract: winos#23
// docs/integration/telemetry-ingest-endpoint.md. Body is a TOP-LEVEL JSON array
// of events (NOT wrapped). Auth reuses device-pairing (UserOrV2DeviceAuth). It
// is diagnostic traffic: NEVER bills (separate table, no consume log / quota).
//
// Default-OFF: until the privacy policy discloses account-linkable device IDs
// and endpoint delivery is agreed, HEICODE_TELEMETRY_ENABLED stays false and the
// endpoint answers 410 (kill switch) so the client stops sending.
const (
telemetryMaxBatch = 20
telemetryMaxBodySize = 256 * 1024
headerDeviceID = "X-Heicode-Device-Id"
)
type telemetryEventIn struct {
ClientId string `json:"client_id"`
SchemaVersion int `json:"schema_version"`
AppVersion string `json:"app_version"`
Platform string `json:"platform"`
OsVersion string `json:"os_version"`
Arch string `json:"arch"`
Locale string `json:"locale"`
ErrorCategory string `json:"error_category"`
ErrorCode string `json:"error_code"`
ErrorMessageHash string `json:"error_message_hash"`
StackHash string `json:"stack_hash"`
StackTop []string `json:"stack_top"`
Context json.RawMessage `json:"context"`
Timestamp string `json:"timestamp"`
SessionSeq int `json:"session_seq"`
}
// parseTelemetryBatch unmarshals the top-level JSON array of events. Pure.
func parseTelemetryBatch(body []byte) ([]telemetryEventIn, error) {
var events []telemetryEventIn
if err := common.Unmarshal(body, &events); err != nil {
return nil, err
}
return events, nil
}
// toModel maps an ingested event to its row. Deliberately tolerant (maximize
// capture): unknown enums / sentinel app_version / missing fields are stored
// as-is; schema_version defaults to 1; stack_top/context are kept as JSON text.
func (e telemetryEventIn) toModel(userID int, deviceID string, now int64) model.TelemetryEvent {
clientID := strings.TrimSpace(e.ClientId)
if clientID == "" {
clientID = deviceID
}
sv := e.SchemaVersion
if sv <= 0 {
sv = 1
}
// Defense-in-depth (#24 review): the client already sanitizes, but we also
// redact stack_top / context server-side — strip sk-/Bearer/URL tokens/JSON
// secret fields before persistence, so a producer bug can't land plaintext
// secrets in the telemetry store.
stackTopJSON := ""
if len(e.StackTop) > 0 {
if b, err := common.Marshal(e.StackTop); err == nil {
stackTopJSON = model.RedactText(string(b))
}
}
ctxJSON := ""
if len(e.Context) > 0 {
ctxJSON = model.RedactText(string(e.Context))
}
return model.TelemetryEvent{
ReceivedAt: now,
UserId: userID,
ClientId: clientID,
SchemaVersion: sv,
AppVersion: e.AppVersion,
Platform: e.Platform,
OsVersion: e.OsVersion,
Arch: e.Arch,
Locale: e.Locale,
ErrorCategory: e.ErrorCategory,
ErrorCode: e.ErrorCode,
ErrorMessageHash: e.ErrorMessageHash,
StackHash: e.StackHash,
StackTopJSON: stackTopJSON,
ContextJSON: ctxJSON,
EventTimestamp: e.Timestamp,
SessionSeq: e.SessionSeq,
}
}
// HeicodeTelemetryEvents: POST /api/heicode/telemetry/events (issue #24).
func HeicodeTelemetryEvents(c *gin.Context) {
// Kill switch / not-live gate (default OFF). 410 => client stops sending.
if !common.GetEnvOrDefaultBool("HEICODE_TELEMETRY_ENABLED", false) {
c.JSON(http.StatusGone, gin.H{"success": false, "message": "telemetry ingest disabled"})
return
}
userID := c.GetInt("id")
if userID <= 0 {
c.JSON(http.StatusUnauthorized, gin.H{"success": false, "message": "authentication required"})
return
}
deviceID := strings.TrimSpace(c.GetHeader(headerDeviceID))
if deviceID == "" {
// Telemetry is desktop-only (device-paired); reject session-only callers.
c.JSON(http.StatusForbidden, gin.H{"success": false, "message": "telemetry requires a paired device"})
return
}
body, _ := io.ReadAll(io.LimitReader(c.Request.Body, telemetryMaxBodySize+1))
if len(body) > telemetryMaxBodySize {
c.JSON(http.StatusRequestEntityTooLarge, gin.H{"success": false, "message": "telemetry batch too large"})
return
}
events, err := parseTelemetryBatch(body)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"success": false, "message": "invalid telemetry body (expect a JSON array of events)"})
return
}
if len(events) == 0 {
c.JSON(http.StatusOK, gin.H{"success": true, "accepted": 0})
return
}
if len(events) > telemetryMaxBatch {
c.JSON(http.StatusRequestEntityTooLarge, gin.H{"success": false, "message": "telemetry batch exceeds 20 events"})
return
}
now := common.GetTimestamp()
rows := make([]model.TelemetryEvent, 0, len(events))
for _, e := range events {
// Anti-spoof: a present client_id must equal the verified paired device.
if cid := strings.TrimSpace(e.ClientId); cid != "" && cid != deviceID {
c.JSON(http.StatusForbidden, gin.H{"success": false, "message": "client_id does not match paired device"})
return
}
rows = append(rows, e.toModel(userID, deviceID, now))
}
if err := model.InsertTelemetryEvents(rows); err != nil {
common.SysLog("telemetry ingest persist failed: " + err.Error())
c.JSON(http.StatusInternalServerError, gin.H{"success": false, "message": "telemetry persist failed"})
return
}
c.JSON(http.StatusOK, gin.H{"success": true, "accepted": len(rows)})
}
@@ -0,0 +1,76 @@
package controller
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/require"
)
// Uses the client's authoritative simulation batch (winos#23 §4) verbatim as the
// ingest fixture, so HM parsing/mapping stays in lock-step with what the client
// actually sends (#24).
const telemetryClientFixtureBatch = `[
{"client_id":"d3f1c0a2-7b44-4e19-9a8c-2f6b1e0c5a77","app_version":"0.5.0","platform":"win32","os_version":"10.0.26100","arch":"x64","locale":"zh-CN","error_category":"ui_crash","error_code":"RENDERER_ERROR","error_message_hash":"9f2a7c1b4e8d","stack_hash":"a1b2c3d4e5f6","stack_top":["at MessageList (MessageList.tsx:212:9)","at renderWithHooks (react-dom.production.min.js:0:0)"],"context":{"route":"chat"},"timestamp":"2026-06-09T07:21:33.123Z","session_seq":1},
{"client_id":"d3f1c0a2-7b44-4e19-9a8c-2f6b1e0c5a77","app_version":"0.0.0-heicode-local","platform":"win32","os_version":"10.0.26100","arch":"x64","locale":"zh-CN","error_category":"cli_startup_failed","error_code":"CLI_STARTUP_TIMEOUT","error_message_hash":"5c8e1f0a9b2d","stack_hash":"000000000000","stack_top":[],"context":{"retryable":true},"timestamp":"2026-06-09T07:22:01.880Z","session_seq":2}
]`
func TestParseTelemetryBatch_ClientFixture(t *testing.T) {
events, err := parseTelemetryBatch([]byte(telemetryClientFixtureBatch))
require.NoError(t, err)
require.Len(t, events, 2)
require.Equal(t, "d3f1c0a2-7b44-4e19-9a8c-2f6b1e0c5a77", events[0].ClientId)
require.Equal(t, "ui_crash", events[0].ErrorCategory)
require.Equal(t, "9f2a7c1b4e8d", events[0].ErrorMessageHash)
require.Len(t, events[0].StackTop, 2)
require.Equal(t, 1, events[0].SessionSeq)
// sentinel app_version is parsed as-is (must be tolerated on ingest)
require.Equal(t, "0.0.0-heicode-local", events[1].AppVersion)
}
func TestTelemetryToModel_MappingAndDefaults(t *testing.T) {
events, err := parseTelemetryBatch([]byte(telemetryClientFixtureBatch))
require.NoError(t, err)
const uid, dev, now = 4242, "d3f1c0a2-7b44-4e19-9a8c-2f6b1e0c5a77", int64(1700000000)
m0 := events[0].toModel(uid, dev, now)
require.Equal(t, uid, m0.UserId) // device -> account link recorded
require.Equal(t, dev, m0.ClientId)
require.EqualValues(t, now, m0.ReceivedAt) // server time, not client
require.Equal(t, "ui_crash", m0.ErrorCategory)
require.Equal(t, 1, m0.SchemaVersion) // missing schema_version defaults to 1
require.JSONEq(t, `{"route":"chat"}`, m0.ContextJSON)
require.JSONEq(t, `["at MessageList (MessageList.tsx:212:9)","at renderWithHooks (react-dom.production.min.js:0:0)"]`, m0.StackTopJSON)
require.Equal(t, "2026-06-09T07:21:33.123Z", m0.EventTimestamp)
// empty client_id falls back to the verified device id
ev := telemetryEventIn{ClientId: "", SchemaVersion: 0}
m := ev.toModel(uid, dev, now)
require.Equal(t, dev, m.ClientId)
require.Equal(t, 1, m.SchemaVersion)
require.Equal(t, "", m.StackTopJSON)
require.Equal(t, "", m.ContextJSON)
}
func TestParseTelemetryBatch_RejectsNonArray(t *testing.T) {
_, err := parseTelemetryBatch([]byte(`{"events":[]}`)) // wrapped object, not the contract
require.Error(t, err)
}
// Defense-in-depth (#24 review §3): even if the client failed to sanitize, the
// server must strip plaintext secrets from stack_top / context before storing.
func TestTelemetryToModel_RedactsSecrets(t *testing.T) {
ev := telemetryEventIn{
ClientId: "dev-1",
StackTop: []string{"at boom (auth.ts) key=sk-abcDEF1234567890"},
Context: json.RawMessage(`{"hdr":"Authorization: Bearer aZ09tokenVALUE","ok":true}`),
}
m := ev.toModel(7, "dev-1", 1700000000)
require.NotContains(t, m.StackTopJSON, "sk-abcDEF1234567890", "sk- secret must be redacted in stack_top")
require.Contains(t, m.StackTopJSON, "REDACTED")
require.NotContains(t, m.ContextJSON, "aZ09tokenVALUE", "Bearer token must be redacted in context")
require.Contains(t, m.ContextJSON, "ok") // non-secret content preserved
}
+8
View File
@@ -118,6 +118,14 @@ func redactAuditSecrets(s string) string {
return s
}
// RedactText is an exported wrapper over the audit secret-redactor so other
// ingest paths (e.g. client telemetry #24) can defense-in-depth strip plaintext
// secrets (sk-/Bearer/URL tokens/JSON secret fields) before persistence, even
// when the producer claims the payload is already sanitized.
func RedactText(s string) string {
return redactAuditSecrets(s)
}
// ListAgentAuditEventsFilter narrows the audit query to a slice of
// dashboard relevant rows. Zero-value fields are ignored — callers
// pass only the filters they care about.
+1
View File
@@ -333,6 +333,7 @@ func migrateDB() error {
// in-process `agentEvents map` that was wiped on every container
// restart. See model/agent_audit.go for the rationale.
&AgentAuditEvent{},
&TelemetryEvent{},
)
if err != nil {
return err
+41
View File
@@ -0,0 +1,41 @@
package model
// TelemetryEvent stores client error-telemetry (issue #24). It is deliberately
// isolated from billing: ingest never writes a consume Log nor touches
// user.Quota. Event payload carries NO user content — only hashes, enums,
// counts and sanitized stack frames (the client sanitizes before sending). The
// device_id (ClientId) is account-linkable via UserId — this association is the
// reason the privacy policy must disclose "device ID (account-linkable)".
type TelemetryEvent struct {
Id int `json:"id"`
ReceivedAt int64 `json:"received_at" gorm:"index"` // server unix seconds
UserId int `json:"user_id" gorm:"index"` // device -> account link
ClientId string `json:"client_id" gorm:"type:varchar(64);index"` // device_id
SchemaVersion int `json:"schema_version" gorm:"default:1"`
AppVersion string `json:"app_version" gorm:"type:varchar(64);default:''"`
Platform string `json:"platform" gorm:"type:varchar(32);default:''"`
OsVersion string `json:"os_version" gorm:"type:varchar(64);default:''"`
Arch string `json:"arch" gorm:"type:varchar(32);default:''"`
Locale string `json:"locale" gorm:"type:varchar(32);default:''"`
ErrorCategory string `json:"error_category" gorm:"type:varchar(40);index;default:''"`
ErrorCode string `json:"error_code" gorm:"type:varchar(128);default:''"`
ErrorMessageHash string `json:"error_message_hash" gorm:"type:varchar(32);default:''"`
StackHash string `json:"stack_hash" gorm:"type:varchar(32);default:''"`
StackTopJSON string `json:"stack_top" gorm:"type:text"` // JSON array, sanitized frames
ContextJSON string `json:"context" gorm:"type:text"` // JSON object, non-content
EventTimestamp string `json:"timestamp" gorm:"type:varchar(40);default:''"` // client ISO8601
SessionSeq int `json:"session_seq" gorm:"default:0"` // process-local
}
// InsertTelemetryEvents batch-inserts ingested telemetry. Uses LOG_DB (the same
// store as Log) since this is diagnostic, append-only, non-billing data.
func InsertTelemetryEvents(events []TelemetryEvent) error {
if len(events) == 0 {
return nil
}
return LOG_DB.Create(&events).Error
}
+6
View File
@@ -63,6 +63,9 @@ func SetApiRouter(router *gin.Engine) {
apiRouter.POST("/agent/callbacks/runtime-events", controller.AgentReceiveRuntimeEventCallback)
// Client-facing capability discovery (unified spec §6). Catalog data only.
apiRouter.GET("/heicode/capabilities", controller.HeicodeCapabilities)
// Client runtime config (#24): telemetry enable/endpoint/kill-switch etc.
// Pollable within a session so kill switches propagate without re-login.
apiRouter.GET("/heicode/config", controller.HeicodeConfig)
apiRouter.POST("/swarms", middleware.UserOrV2DeviceAuth(), controller.AgentCreateUserSwarm)
//apiRouter.POST("/waffo-pancake/webhook", controller.WaffoPancakeWebhook)
@@ -539,6 +542,9 @@ func SetApiRouter(router *gin.Engine) {
heicodeAgentRoute.GET("/agents/:deployment_id/usage", controller.HeicodeGetAgentUsage)
heicodeAgentRoute.POST("/agents/:deployment_id/stop", controller.HeicodeStopAgent)
heicodeAgentRoute.DELETE("/agents/:deployment_id", controller.HeicodeDeleteAgent)
// Client error-telemetry ingest (#24). Device-paired; never bills.
// Gated by HEICODE_TELEMETRY_ENABLED (default off -> 410 kill switch).
heicodeAgentRoute.POST("/telemetry/events", controller.HeicodeTelemetryEvents)
}
// Client↔agent access control (HM-provided, AM-optional). Called by the