Files
heicode-mananger/heicode/model/agent_audit.go
T
818f1412d5 feat(heicode): 客户端错误遥测上报端点(默认关闭)(#24) (#27)
* feat(heicode): 客户端错误遥测上报端点(默认关闭)(#24)

按客户端 winos#23 契约 + 权威 schema 实现,结合 HM 入库逻辑:

- POST /api/heicode/telemetry/events,挂 UserOrV2DeviceAuth(设备配对鉴权)。
- 接收顶层 JSON 数组(非包裹),批量 1-20、<=256KB;校验 body client_id 等于已验签
  设备(X-Heicode-Device-Id),不一致 403;无设备身份拒绝。
- 真实 4xx/5xx 码(400 非数组、413 超限、403 设备、410 关闭),让客户端"4xx 丢弃"
  语义生效;2xx 返回 {accepted:n}。
- 独立表 telemetry_events,与计费完全隔离:不写 consume log、不碰 quota。
- 宽松入库(最大化采集):未知枚举 / 哨兵 app_version(0.0.0-heicode-local)/ 缺字段
  原样入库;schema_version 缺省 1;stack_top/context 存 TEXT(JSON);记 user_id 作
  device 到 account 关联 + 服务端 received_at。
- 默认 HEICODE_TELEMETRY_ENABLED=false 时返回 410(kill switch);隐私政策更新 +
  端点下发形态确认前不开启外发。

测试用客户端仿真夹具:parseTelemetryBatch / toModel 映射与默认 / 拒绝非数组,全过。

Refs #24(上线门槛:隐私政策 §2 如实披露 + 下发形态 + 去重;见工单评论)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(heicode): GET /api/heicode/config 下发 telemetry 配置/kill-switch (#24)

按客户端 #24 拍板:选独立 config 端点(而非塞登录响应),便于 kill switch 在
会话内传导、不依赖重登录。返回 telemetry 块 {enabled, endpoint, max_batch,
flush_interval_sec};enabled 取 HEICODE_TELEMETRY_ENABLED(默认 false)。
未鉴权全局只读(同 capabilities 姿态)。

测试 heicode_config_test.go:enabled 反映 env、endpoint 与摄入路由一致、缺省 false。

Refs #24

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(telemetry): 服务端二次脱敏 stack_top/context + 明确生产门槛 (#24 review)

按 Fasthei 复审意见补隐私门槛:

- 服务端纵深防御脱敏:新增导出 model.RedactText(包装已上线的 redactAuditSecrets,
  #11),在 telemetry 入库前对 stack_top / context 再脱敏一遍(sk-/Bearer/URL
  token/JSON 密钥字段),即便客户端漏脱敏也不会把明文密钥落库。
- 测试 TestTelemetryToModel_RedactsSecrets:stack_top 里的 sk-、context 里的
  Bearer token 被打码,非密钥内容保留。
- 端点默认 HEICODE_TELEMETRY_ENABLED=false,关时 410;隐私政策披露完成前生产
  不得开启外发(见 #24 评论记录产品/法务状态)。

go build / vet 干净;controller 测试通过。

Refs #24

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: chenchen <chenchen@xinghanlab.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 22:16:29 +08:00

195 lines
7.6 KiB
Go

package model
import (
"regexp"
"time"
"github.com/heicode/manager/common"
)
// AgentAuditEvent is the persistent audit-trail row for the Agent
// control-plane. Each row records one observable transition in the
// Agent lifecycle — deployment accepted, instance phase changed, SK
// snapshot refreshed, etc. — so admins can answer the "who / when /
// for which task / against which resource / with what result" set of
// questions even after the Manager container restarts.
//
// Before this table existed the control-plane stashed events in an
// in-process `map[string][]agentEvent` (controller/agent_control_plane
// .go:219). Every restart wiped audit history — unacceptable for a
// product where the audit page is part of the security story.
//
// Schema notes
// - All string columns are bounded varchar to keep MySQL happy.
// - `OccurredAt` is unix milliseconds so cross-DB ORDER BY / range
// queries are trivial without a TIMESTAMP-with-timezone dance.
// - `DetailsJSON` is a free-form payload bucket for fields we don't
// want to promote to first-class columns yet (e.g. failure_reason,
// phase, callback_url). Add a real column when a field is queried
// enough to need an index.
// - Indexes are picked for the dashboard queries: filter by user_id,
// filter by deployment, time-range scan.
//
// Cross-DB compatibility (CLAUDE.md Rule 2): GORM AutoMigrate maps the
// tags to the correct types on SQLite / MySQL / PostgreSQL. No raw
// SQL. No DB-specific column types.
type AgentAuditEvent struct {
Id int `gorm:"primaryKey" json:"id"`
EventID string `gorm:"type:varchar(64);uniqueIndex" json:"event_id"`
Event string `gorm:"type:varchar(64);index" json:"event"`
Actor string `gorm:"type:varchar(64)" json:"actor"`
Resource string `gorm:"type:varchar(128);index" json:"resource"`
UserID string `gorm:"type:varchar(64);index" json:"user_id"`
ChannelID string `gorm:"type:varchar(64)" json:"channel_id"`
BindingScope string `gorm:"type:varchar(64);index" json:"binding_scope"`
DeploymentID string `gorm:"type:varchar(64);index" json:"deployment_id"`
CorrelationID string `gorm:"type:varchar(64)" json:"correlation_id"`
RequestID string `gorm:"type:varchar(64)" json:"request_id"`
Result string `gorm:"type:varchar(16)" json:"result"`
SchemaVersion int `gorm:"default:1" json:"schema_version"`
OccurredAt int64 `gorm:"bigint;index" json:"occurred_at"`
DetailsJSON string `gorm:"type:text" json:"details_json,omitempty"`
}
// TableName pins the migration target so we can rename the Go struct
// later without breaking the deployed schema.
func (AgentAuditEvent) TableName() string {
return "agent_audit_events"
}
// InsertAgentAuditEvent best-effort persists one audit row. Callers
// invoke this in a hot path (right after mutating a deployment), so:
// - errors are logged but never returned — the audit write must NOT
// fail the user-facing API
// - DB is nil-guarded so unit tests / partial-init binaries don't
// panic on a missing connection
func InsertAgentAuditEvent(evt *AgentAuditEvent) {
if DB == nil || evt == nil {
return
}
if evt.OccurredAt == 0 {
evt.OccurredAt = time.Now().UnixMilli()
}
if evt.SchemaVersion == 0 {
evt.SchemaVersion = 1
}
if evt.Result == "" {
evt.Result = "ok"
}
// Defence-in-depth: scrub plaintext secrets out of the free-form payload
// before it lands in the audit table. DetailsJSON is the catch-all bucket
// (failure_reason / phase / callback_url …) and a callback_url can carry a
// query-string token; product policy forbids plaintext token/password/
// private-key/access-key in logs. Callers should still avoid putting secrets
// here — this is a backstop, not a licence to log them.
evt.DetailsJSON = redactAuditSecrets(evt.DetailsJSON)
if err := DB.Create(evt).Error; err != nil {
common.SysLog("InsertAgentAuditEvent: " + err.Error())
}
}
// secretRedactors strip well-known secret shapes from free-form audit payloads.
// Targeted (not a generic high-entropy scan) to avoid mangling normal text;
// over-redacting a rare false positive in audit details is preferable to
// leaking a credential.
var secretRedactors = []struct {
re *regexp.Regexp
repl string
}{
// sk- style API keys (NewAPI / OpenAI-compatible gateway keys)
{regexp.MustCompile(`sk-[A-Za-z0-9_-]{8,}`), "sk-***REDACTED***"},
// Authorization: Bearer <token>
{regexp.MustCompile(`(?i)(bearer\s+)[A-Za-z0-9._\-]{8,}`), "${1}***REDACTED***"},
// URL / form query secrets: ?token=.. &access_key=.. (keep the key, drop value)
{regexp.MustCompile(`(?i)([?&](?:access_token|refresh_token|token|api_?key|access_?key|secret|password|passwd|pwd|sig|signature)=)[^&\s"']+`), "${1}***REDACTED***"},
// JSON string fields: "password":"..", "token":"..", "access_key":".."
{regexp.MustCompile(`(?i)("(?:password|passwd|pwd|secret|api_?key|access_?key|private_?key|token|refresh_token)"\s*:\s*")[^"]*(")`), "${1}***REDACTED***${2}"},
}
// redactAuditSecrets removes plaintext secrets from a free-form audit payload
// before persistence. Returns the input unchanged when empty.
func redactAuditSecrets(s string) string {
if s == "" {
return s
}
for _, r := range secretRedactors {
s = r.re.ReplaceAllString(s, r.repl)
}
return s
}
// RedactText is an exported wrapper over the audit secret-redactor so other
// ingest paths (e.g. client telemetry #24) can defense-in-depth strip plaintext
// secrets (sk-/Bearer/URL tokens/JSON secret fields) before persistence, even
// when the producer claims the payload is already sanitized.
func RedactText(s string) string {
return redactAuditSecrets(s)
}
// ListAgentAuditEventsFilter narrows the audit query to a slice of
// dashboard relevant rows. Zero-value fields are ignored — callers
// pass only the filters they care about.
type ListAgentAuditEventsFilter struct {
UserID string
BindingScope string
DeploymentID string
EventLike string // prefix filter on Event column, e.g. "deployment."
FromMs int64 // inclusive
ToMs int64 // exclusive; zero == no upper bound
Limit int // defaults to 200
Offset int
}
// ListAgentAuditEvents pages over audit rows ordered newest-first.
// Used by the /api/agent/audit-logs endpoint and the future task-
// scoped audit drawer.
func ListAgentAuditEvents(f ListAgentAuditEventsFilter) ([]AgentAuditEvent, int64, error) {
if DB == nil {
return nil, 0, nil
}
q := DB.Model(&AgentAuditEvent{})
if f.UserID != "" {
q = q.Where("user_id = ?", f.UserID)
}
if f.BindingScope != "" {
q = q.Where("binding_scope = ?", f.BindingScope)
}
if f.DeploymentID != "" {
q = q.Where("deployment_id = ?", f.DeploymentID)
}
if f.EventLike != "" {
q = q.Where("event LIKE ?", f.EventLike+"%")
}
if f.FromMs > 0 {
q = q.Where("occurred_at >= ?", f.FromMs)
}
if f.ToMs > 0 {
q = q.Where("occurred_at < ?", f.ToMs)
}
var total int64
if err := q.Count(&total).Error; err != nil {
return nil, 0, err
}
limit := f.Limit
if limit <= 0 || limit > 1000 {
limit = 200
}
var items []AgentAuditEvent
err := q.Order("occurred_at desc, id desc").Limit(limit).Offset(f.Offset).Find(&items).Error
return items, total, err
}
// ListAgentAuditEventsByDeployment is the hot path for the deployment
// detail drawer — returns all events for one deployment in chronological
// order so the timeline reads top-to-bottom.
func ListAgentAuditEventsByDeployment(deploymentID string) ([]AgentAuditEvent, error) {
if DB == nil || deploymentID == "" {
return nil, nil
}
var items []AgentAuditEvent
err := DB.Where("deployment_id = ?", deploymentID).
Order("occurred_at asc, id asc").
Find(&items).Error
return items, err
}