* feat(heicode): 客户端错误遥测上报端点(默认关闭)(#24) 按客户端 winos#23 契约 + 权威 schema 实现,结合 HM 入库逻辑: - POST /api/heicode/telemetry/events,挂 UserOrV2DeviceAuth(设备配对鉴权)。 - 接收顶层 JSON 数组(非包裹),批量 1-20、<=256KB;校验 body client_id 等于已验签 设备(X-Heicode-Device-Id),不一致 403;无设备身份拒绝。 - 真实 4xx/5xx 码(400 非数组、413 超限、403 设备、410 关闭),让客户端"4xx 丢弃" 语义生效;2xx 返回 {accepted:n}。 - 独立表 telemetry_events,与计费完全隔离:不写 consume log、不碰 quota。 - 宽松入库(最大化采集):未知枚举 / 哨兵 app_version(0.0.0-heicode-local)/ 缺字段 原样入库;schema_version 缺省 1;stack_top/context 存 TEXT(JSON);记 user_id 作 device 到 account 关联 + 服务端 received_at。 - 默认 HEICODE_TELEMETRY_ENABLED=false 时返回 410(kill switch);隐私政策更新 + 端点下发形态确认前不开启外发。 测试用客户端仿真夹具:parseTelemetryBatch / toModel 映射与默认 / 拒绝非数组,全过。 Refs #24(上线门槛:隐私政策 §2 如实披露 + 下发形态 + 去重;见工单评论) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(heicode): GET /api/heicode/config 下发 telemetry 配置/kill-switch (#24) 按客户端 #24 拍板:选独立 config 端点(而非塞登录响应),便于 kill switch 在 会话内传导、不依赖重登录。返回 telemetry 块 {enabled, endpoint, max_batch, flush_interval_sec};enabled 取 HEICODE_TELEMETRY_ENABLED(默认 false)。 未鉴权全局只读(同 capabilities 姿态)。 测试 heicode_config_test.go:enabled 反映 env、endpoint 与摄入路由一致、缺省 false。 Refs #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(telemetry): 服务端二次脱敏 stack_top/context + 明确生产门槛 (#24 review) 按 Fasthei 复审意见补隐私门槛: - 服务端纵深防御脱敏:新增导出 model.RedactText(包装已上线的 redactAuditSecrets, #11),在 telemetry 入库前对 stack_top / context 再脱敏一遍(sk-/Bearer/URL token/JSON 密钥字段),即便客户端漏脱敏也不会把明文密钥落库。 - 测试 TestTelemetryToModel_RedactsSecrets:stack_top 里的 sk-、context 里的 Bearer token 被打码,非密钥内容保留。 - 端点默认 HEICODE_TELEMETRY_ENABLED=false,关时 410;隐私政策披露完成前生产 不得开启外发(见 #24 评论记录产品/法务状态)。 go build / vet 干净;controller 测试通过。 Refs #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: chenchen <chenchen@xinghanlab.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
195 lines
7.6 KiB
Go
195 lines
7.6 KiB
Go
package model
|
|
|
|
import (
|
|
"regexp"
|
|
"time"
|
|
|
|
"github.com/heicode/manager/common"
|
|
)
|
|
|
|
// AgentAuditEvent is the persistent audit-trail row for the Agent
|
|
// control-plane. Each row records one observable transition in the
|
|
// Agent lifecycle — deployment accepted, instance phase changed, SK
|
|
// snapshot refreshed, etc. — so admins can answer the "who / when /
|
|
// for which task / against which resource / with what result" set of
|
|
// questions even after the Manager container restarts.
|
|
//
|
|
// Before this table existed the control-plane stashed events in an
|
|
// in-process `map[string][]agentEvent` (controller/agent_control_plane
|
|
// .go:219). Every restart wiped audit history — unacceptable for a
|
|
// product where the audit page is part of the security story.
|
|
//
|
|
// Schema notes
|
|
// - All string columns are bounded varchar to keep MySQL happy.
|
|
// - `OccurredAt` is unix milliseconds so cross-DB ORDER BY / range
|
|
// queries are trivial without a TIMESTAMP-with-timezone dance.
|
|
// - `DetailsJSON` is a free-form payload bucket for fields we don't
|
|
// want to promote to first-class columns yet (e.g. failure_reason,
|
|
// phase, callback_url). Add a real column when a field is queried
|
|
// enough to need an index.
|
|
// - Indexes are picked for the dashboard queries: filter by user_id,
|
|
// filter by deployment, time-range scan.
|
|
//
|
|
// Cross-DB compatibility (CLAUDE.md Rule 2): GORM AutoMigrate maps the
|
|
// tags to the correct types on SQLite / MySQL / PostgreSQL. No raw
|
|
// SQL. No DB-specific column types.
|
|
type AgentAuditEvent struct {
|
|
Id int `gorm:"primaryKey" json:"id"`
|
|
EventID string `gorm:"type:varchar(64);uniqueIndex" json:"event_id"`
|
|
Event string `gorm:"type:varchar(64);index" json:"event"`
|
|
Actor string `gorm:"type:varchar(64)" json:"actor"`
|
|
Resource string `gorm:"type:varchar(128);index" json:"resource"`
|
|
UserID string `gorm:"type:varchar(64);index" json:"user_id"`
|
|
ChannelID string `gorm:"type:varchar(64)" json:"channel_id"`
|
|
BindingScope string `gorm:"type:varchar(64);index" json:"binding_scope"`
|
|
DeploymentID string `gorm:"type:varchar(64);index" json:"deployment_id"`
|
|
CorrelationID string `gorm:"type:varchar(64)" json:"correlation_id"`
|
|
RequestID string `gorm:"type:varchar(64)" json:"request_id"`
|
|
Result string `gorm:"type:varchar(16)" json:"result"`
|
|
SchemaVersion int `gorm:"default:1" json:"schema_version"`
|
|
OccurredAt int64 `gorm:"bigint;index" json:"occurred_at"`
|
|
DetailsJSON string `gorm:"type:text" json:"details_json,omitempty"`
|
|
}
|
|
|
|
// TableName pins the migration target so we can rename the Go struct
|
|
// later without breaking the deployed schema.
|
|
func (AgentAuditEvent) TableName() string {
|
|
return "agent_audit_events"
|
|
}
|
|
|
|
// InsertAgentAuditEvent best-effort persists one audit row. Callers
|
|
// invoke this in a hot path (right after mutating a deployment), so:
|
|
// - errors are logged but never returned — the audit write must NOT
|
|
// fail the user-facing API
|
|
// - DB is nil-guarded so unit tests / partial-init binaries don't
|
|
// panic on a missing connection
|
|
func InsertAgentAuditEvent(evt *AgentAuditEvent) {
|
|
if DB == nil || evt == nil {
|
|
return
|
|
}
|
|
if evt.OccurredAt == 0 {
|
|
evt.OccurredAt = time.Now().UnixMilli()
|
|
}
|
|
if evt.SchemaVersion == 0 {
|
|
evt.SchemaVersion = 1
|
|
}
|
|
if evt.Result == "" {
|
|
evt.Result = "ok"
|
|
}
|
|
// Defence-in-depth: scrub plaintext secrets out of the free-form payload
|
|
// before it lands in the audit table. DetailsJSON is the catch-all bucket
|
|
// (failure_reason / phase / callback_url …) and a callback_url can carry a
|
|
// query-string token; product policy forbids plaintext token/password/
|
|
// private-key/access-key in logs. Callers should still avoid putting secrets
|
|
// here — this is a backstop, not a licence to log them.
|
|
evt.DetailsJSON = redactAuditSecrets(evt.DetailsJSON)
|
|
if err := DB.Create(evt).Error; err != nil {
|
|
common.SysLog("InsertAgentAuditEvent: " + err.Error())
|
|
}
|
|
}
|
|
|
|
// secretRedactors strip well-known secret shapes from free-form audit payloads.
|
|
// Targeted (not a generic high-entropy scan) to avoid mangling normal text;
|
|
// over-redacting a rare false positive in audit details is preferable to
|
|
// leaking a credential.
|
|
var secretRedactors = []struct {
|
|
re *regexp.Regexp
|
|
repl string
|
|
}{
|
|
// sk- style API keys (NewAPI / OpenAI-compatible gateway keys)
|
|
{regexp.MustCompile(`sk-[A-Za-z0-9_-]{8,}`), "sk-***REDACTED***"},
|
|
// Authorization: Bearer <token>
|
|
{regexp.MustCompile(`(?i)(bearer\s+)[A-Za-z0-9._\-]{8,}`), "${1}***REDACTED***"},
|
|
// URL / form query secrets: ?token=.. &access_key=.. (keep the key, drop value)
|
|
{regexp.MustCompile(`(?i)([?&](?:access_token|refresh_token|token|api_?key|access_?key|secret|password|passwd|pwd|sig|signature)=)[^&\s"']+`), "${1}***REDACTED***"},
|
|
// JSON string fields: "password":"..", "token":"..", "access_key":".."
|
|
{regexp.MustCompile(`(?i)("(?:password|passwd|pwd|secret|api_?key|access_?key|private_?key|token|refresh_token)"\s*:\s*")[^"]*(")`), "${1}***REDACTED***${2}"},
|
|
}
|
|
|
|
// redactAuditSecrets removes plaintext secrets from a free-form audit payload
|
|
// before persistence. Returns the input unchanged when empty.
|
|
func redactAuditSecrets(s string) string {
|
|
if s == "" {
|
|
return s
|
|
}
|
|
for _, r := range secretRedactors {
|
|
s = r.re.ReplaceAllString(s, r.repl)
|
|
}
|
|
return s
|
|
}
|
|
|
|
// RedactText is an exported wrapper over the audit secret-redactor so other
|
|
// ingest paths (e.g. client telemetry #24) can defense-in-depth strip plaintext
|
|
// secrets (sk-/Bearer/URL tokens/JSON secret fields) before persistence, even
|
|
// when the producer claims the payload is already sanitized.
|
|
func RedactText(s string) string {
|
|
return redactAuditSecrets(s)
|
|
}
|
|
|
|
// ListAgentAuditEventsFilter narrows the audit query to a slice of
|
|
// dashboard relevant rows. Zero-value fields are ignored — callers
|
|
// pass only the filters they care about.
|
|
type ListAgentAuditEventsFilter struct {
|
|
UserID string
|
|
BindingScope string
|
|
DeploymentID string
|
|
EventLike string // prefix filter on Event column, e.g. "deployment."
|
|
FromMs int64 // inclusive
|
|
ToMs int64 // exclusive; zero == no upper bound
|
|
Limit int // defaults to 200
|
|
Offset int
|
|
}
|
|
|
|
// ListAgentAuditEvents pages over audit rows ordered newest-first.
|
|
// Used by the /api/agent/audit-logs endpoint and the future task-
|
|
// scoped audit drawer.
|
|
func ListAgentAuditEvents(f ListAgentAuditEventsFilter) ([]AgentAuditEvent, int64, error) {
|
|
if DB == nil {
|
|
return nil, 0, nil
|
|
}
|
|
q := DB.Model(&AgentAuditEvent{})
|
|
if f.UserID != "" {
|
|
q = q.Where("user_id = ?", f.UserID)
|
|
}
|
|
if f.BindingScope != "" {
|
|
q = q.Where("binding_scope = ?", f.BindingScope)
|
|
}
|
|
if f.DeploymentID != "" {
|
|
q = q.Where("deployment_id = ?", f.DeploymentID)
|
|
}
|
|
if f.EventLike != "" {
|
|
q = q.Where("event LIKE ?", f.EventLike+"%")
|
|
}
|
|
if f.FromMs > 0 {
|
|
q = q.Where("occurred_at >= ?", f.FromMs)
|
|
}
|
|
if f.ToMs > 0 {
|
|
q = q.Where("occurred_at < ?", f.ToMs)
|
|
}
|
|
var total int64
|
|
if err := q.Count(&total).Error; err != nil {
|
|
return nil, 0, err
|
|
}
|
|
limit := f.Limit
|
|
if limit <= 0 || limit > 1000 {
|
|
limit = 200
|
|
}
|
|
var items []AgentAuditEvent
|
|
err := q.Order("occurred_at desc, id desc").Limit(limit).Offset(f.Offset).Find(&items).Error
|
|
return items, total, err
|
|
}
|
|
|
|
// ListAgentAuditEventsByDeployment is the hot path for the deployment
|
|
// detail drawer — returns all events for one deployment in chronological
|
|
// order so the timeline reads top-to-bottom.
|
|
func ListAgentAuditEventsByDeployment(deploymentID string) ([]AgentAuditEvent, error) {
|
|
if DB == nil || deploymentID == "" {
|
|
return nil, nil
|
|
}
|
|
var items []AgentAuditEvent
|
|
err := DB.Where("deployment_id = ?", deploymentID).
|
|
Order("occurred_at asc, id asc").
|
|
Find(&items).Error
|
|
return items, err
|
|
}
|