重构代码

This commit is contained in:
2026-03-19 16:37:57 +08:00
parent 51608601cf
commit 8023bfe328
75 changed files with 13325 additions and 394 deletions
+72
View File
@@ -0,0 +1,72 @@
# Development Guide
This file defines the local test matrix and troubleshooting notes for `starlog`.
## Local Test Matrix
Run these commands from repository root:
```powershell
go test ./...
go test -race ./...
go test . -run '^$' -bench Benchmark -benchmem -benchtime=100x
go test . -run '^$' -fuzz=FuzzTextAndJSONFormatter -fuzztime=2s
go test . -run '^$' -fuzz=FuzzKeywordHighlight -fuzztime=2s
```
## One-Command Local Check
Use the helper script:
```powershell
powershell -ExecutionPolicy Bypass -File scripts/test-local.ps1
```
The script runs:
1. Unit tests.
2. Race tests (with a precheck).
3. Benchmark smoke.
4. Fuzz smoke.
## Race Troubleshooting (Windows)
If this command fails:
```powershell
go test -race fmt
```
with:
```text
runtime/race: package testmain: cannot find package
```
the problem is the local Go toolchain/runtime environment, not `starlog` code.
Recommended steps:
1. Verify toolchain:
```powershell
where.exe go
go version
go env GOROOT GOPATH GOOS GOARCH CGO_ENABLED
```
2. Clear caches:
```powershell
go clean -cache -testcache -fuzzcache
```
3. Reinstall an official Go distribution for your platform.
4. Re-run:
```powershell
go test -race fmt
```
## CI Mapping
The same matrix is mirrored in:
- `.github/workflows/quality.yml`
Linux executes `-race` and fuzz smoke; Windows keeps unit test coverage.
+139
View File
@@ -0,0 +1,139 @@
# MIGRATION (V1)
本文档说明从旧版本/旧用法迁移到当前 `starlog` 的推荐路径。
## 先说结论
- 当前版本是 **V1 兼容迁移阶段**
- **旧 API 仍可继续使用,不需要立即重写。**
- 推荐逐步迁移到 `Config` 快照方式和新能力(结构化、多 sink、脱敏、测试 hook)。
## 兼容性承诺(V1
1. 不移除旧 API。
2. 旧调用方式保持可运行。
3. 新能力以“增量引入”为主,不要求一次性改造。
## 推荐迁移顺序
### 第 0 步:先不改业务逻辑
如果你当前线上稳定,可以先升级依赖并保持原写法。
### 第 1 步:把“多项配置”改为一次性更新
旧写法(仍可用):
```go
log.SetShowColor(false)
log.SetShowLevel(true)
log.SetShowFlag(false)
log.SetLevel(starlog.LvInfo)
```
推荐写法(V1 新增):
```go
log.UpdateConfig(func(cfg *starlog.Config) {
cfg.ShowColor = false
cfg.ShowLevel = true
cfg.ShowFlag = false
cfg.Level = starlog.LvInfo
})
```
收益:
- 多项配置原子生效,减少并发中间态。
- 配置修改更集中,便于审计和 review。
### 第 2 步:逐步引入结构化字段与 Context
```go
log.WithField("trace_id", traceID).Info("request done")
log.WithContext(ctx).Error("call downstream failed")
```
### 第 3 步:输出链路升级
- 单路写入升级到 `SetSinks(...)` / `MultiSink`
- 需要分级分文件时使用 `RouteHandler`
- 归档建议走 `RotatePolicy` 主路径,并叠加 `RotateManageOptions`
### 第 4 步:安全与测试
- 脱敏:`SetRedactor` / `AddRedactRule`
- 测试断言:`Observer` / `TestHook`
### 第 5 步:生命周期收口(推荐)
旧写法(仍可用):
```go
defer starlog.CloseStd()
```
推荐写法:
```go
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
defer starlog.Shutdown(ctx)
```
说明:
- `Shutdown(ctx)` 会等待异步 handler drain,并统一关闭资源。
- `Close()` 仍可用,但不等待异步 handler 队列。
## 常见旧写法到推荐写法
1. 配置项逐条 setter
推荐:`UpdateConfig` 批量改
2. 只打印字符串
推荐:关键路径增加 `WithField(s)``WithError`
3. 仅 writer 输出
推荐:`MultiSink` + `RouteHandler` 做按级别拆分
4. 原生 `log.Logger` 接入
推荐:`AsStdlibLoggerWithOptions`prefix/flags/level-mapper
5. 手工配置大量 setter
推荐:`NewProductionConfig/NewDevelopmentConfig` + `ApplyProductionConfig/ApplyDevelopmentConfig`
6.`StartArchive` 轮转启动
推荐:`StartRotatePolicy` / `StartManagedRotatePolicy`,模板场景可直接用 `StartRotateByTime/BySize/ByTimeSize`
## 旧 API 仍可用(重要)
以下旧/兼容路径当前仍可用:
- 旧 setter/getter 全部可用
- 历史错拼兼容名仍可用(如 `EnbaleWrite``IsWriteStoed``HookBeforArchive`
- `Archive` 旧模型仍可用(建议新代码优先 `RotatePolicy`
- 旧生命周期入口仍可用(如 `CloseStd``Close`),但建议新代码优先 `Shutdown(ctx)` / `CloseLogFile`
## 升级后验证建议
1. 跑单元测试:`go test ./...`
2. 跑并发检测:`go test -race ./...`
3. 跑规则检查:`sentrux check .`
4. 重点验证:
- 级别过滤是否符合预期
- 归档文件数量/保留策略
- 异步丢弃计数和写入错误计数
## 何时需要进一步迁移
如果你符合以下场景,建议优先推进推荐写法:
- 多 goroutine 同时改日志配置
- 需要跨模块统一日志字段规范
- 需要审计日志(脱敏、分级路由、归档治理)
---
如果你希望,我可以基于你当前项目的实际日志初始化代码,给出一份“一次改完可落地”的迁移 patch 方案。
+686
View File
@@ -0,0 +1,686 @@
# starlog 使用指南
本文档覆盖从接入到生产落地的常见场景。
- 模块:`b612.me/starlog`
- Go 版本:`1.16+`
## 1. 快速开始
```go
package main
import (
"os"
"b612.me/starlog"
)
func main() {
log := starlog.NewStarlog(os.Stdout)
log.SetName("demo")
log.SetShowStd(true)
log.SetColorMode(starlog.ColorModeLevelOnly)
log.Info("service start")
log.WithField("user_id", 42).Info("login ok")
}
```
## 2. 推荐初始化方式
### 2.1 开发环境
```go
log := starlog.NewDevelopment(os.Stdout)
log.SetName("my-service")
log.SetShowStd(true)
```
特点:
- 默认 `Debug` 级别
- 颜色模式:`ColorModeLevelOnly`
- 显示源码位置
### 2.2 生产环境
```go
log := starlog.NewProduction(nil)
log.SetName("my-service")
if err := starlog.SetLogFile("./logs/app.log", log, true); err != nil {
panic(err)
}
```
特点:
- 默认 `Info` 级别
- 默认 `JSONFormatter`
- 默认不直出控制台
### 2.3 对现有 logger 套用预设
```go
log.ApplyProductionConfig()
// 或
log.ApplyDevelopmentConfig()
```
## 3. 输出 API 与换行规则
### 3.1 输出函数
- `Info/Error/...``fmt.Sprint` 语义
- `Infof/Errorf/...``fmt.Sprintf` 语义
- `Infoln/Errorln/...``fmt.Sprintln` 语义
默认不会自动补 `\n``ln` 系列除外)。
### 3.2 自动补换行
```go
log.SetAutoAppendNewline(true)
```
行为:
- 末尾无 `\n`:自动补一个
- 末尾有 `\n`:保持不变
## 4. 结构化日志
### 4.1 字段与错误
```go
log.WithField("order_id", "O-1001").Info("create order")
log.WithFields(starlog.Fields{
"trace_id": "t-001",
"module": "payment",
}).WithError(err).Error("charge failed")
```
### 4.2 Context 注入
```go
log.SetContextFieldExtractor(func(ctx context.Context) starlog.Fields {
traceID, _ := ctx.Value("trace_id").(string)
if traceID == "" {
return nil
}
return starlog.Fields{"trace_id": traceID}
})
log.WithContext(ctx).Info("request done")
log.InfoContext(ctx, "request done")
```
## 5. 级别过滤
```go
log.SetLevel(starlog.LvInfo)
if log.IsLevelEnabled(starlog.LvDebug) {
log.Debug("expensive debug")
}
lv, err := starlog.ParseLevel("warn") // LvWarning
_ = lv
_ = err
```
`SetStdErrLevel` 只控制 stdout/stderr 分流阈值,不是日志过滤阈值。
## 6. 显示、颜色与格式化
### 6.1 颜色模式
```go
log.SetColorMode(starlog.ColorModeOff)
log.SetColorMode(starlog.ColorModeFullLine)
log.SetColorMode(starlog.ColorModeLevelOnly)
```
### 6.2 关键词和字段着色
```go
log.SetKeywordColor("timeout", []starlog.Attr{starlog.FgRed, starlog.Bold})
log.SetShowFieldColor(true)
log.SetFieldKeyColor([]starlog.Attr{starlog.FgHiBlue})
log.SetFieldTypeColor(starlog.FieldTypeNumber, []starlog.Attr{starlog.FgYellow})
log.SetFieldValueColor("user_id", []starlog.Attr{starlog.FgCyan})
```
### 6.3 关键词预设(Moba 风格)
```go
log.ApplyKeywordPreset(starlog.KeywordPresetMobaLite) // 覆盖现有关键词映射
// 或
log.MergeKeywordPreset(starlog.KeywordPresetMobaFull) // 在现有映射上合并
// 可选匹配模式(默认关闭,兼容旧行为)
log.SetKeywordMatchOptions(starlog.KeywordMatchOptions{
IgnoreCase: true, // 忽略大小写
WholeWord: true, // 仅匹配完整单词
})
```
说明:
- `MobaLite`:常用词高亮(如 `error/warn/true/false/success`
- `MobaFull`:在 Lite 基础上增加更多运行态词汇
- 预设后仍可继续 `SetKeywordColor(...)` 覆盖单个关键词颜色
### 6.4 Formatter
```go
log.SetFormatter(starlog.NewTextFormatter())
log.SetFormatter(starlog.NewJSONFormatter())
```
自定义 formatter
```go
type Formatter interface { Format(*Entry) ([]byte, error) }
```
## 7. 输出管道(Writer / Sink / MultiSink
### 7.1 Writer
```go
log.SetWriter(os.Stdout)
```
### 7.2 Sink
```go
type Sink interface {
Write([]byte) error
Close() error
}
log.SetSink(mySink)
```
### 7.3 MultiSink
```go
multi := starlog.NewMultiSink(fileSink, networkSink)
multi.SetContinueOnError(true)
log.SetSink(multi)
stats := multi.GetStats()
_ = stats
```
## 8. 按级别分流(RouteHandler
示例:`info+notice -> brief.log``error+ -> err.log`
```go
brief, _ := os.OpenFile("./logs/brief.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
errf, _ := os.OpenFile("./logs/err.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
route := starlog.NewRouteHandler(
starlog.Route{
Name: "brief",
Match: starlog.MatchLevels(starlog.LvInfo, starlog.LvNotice),
Formatter: starlog.NewTextFormatter(),
Sink: starlog.NewWriterSink(brief),
},
starlog.Route{
Name: "error",
Match: starlog.MatchAtLeast(starlog.LvError),
Formatter: starlog.NewJSONFormatter(),
Sink: starlog.NewWriterSink(errf),
},
)
log.SetEntryHandler(route)
```
如果要“主日志 + 分流日志”同时保留,用 `ChainHandler` 组合 handler。
## 9. 轮转与归档
### 9.1 `RotatePolicy` 语义
```go
type RotatePolicy interface {
ShouldRotate(starlog.FileInfo, *starlog.Entry) bool
NextPath(string, time.Time) string
}
// 可选扩展接口
type RotateArchivePathProvider interface {
ArchivePath(string, time.Time) string
}
```
语义说明:
- `ShouldRotate`:判定是否触发轮转
- `NextPath`:返回“被切走旧日志”的归档路径
- 若策略实现了 `ArchivePath`,框架优先使用 `ArchivePath`
-`ArchivePath` 返回空字符串,回退到 `NextPath`
### 9.2 主日志轮转(推荐入口)
```go
if err := starlog.StartRotatePolicy(log, policy, 1); err != nil {
panic(err)
}
```
内置模板:
```go
_ = starlog.StartRotateByTime(log, 24*time.Hour, 10)
_ = starlog.StartRotateBySize(log, 200*1024*1024, 5)
_ = starlog.StartRotateByTimeSize(log, 24*time.Hour, 200*1024*1024, 5)
```
### 9.3 管理增强(保留、压缩)
```go
opts := starlog.RotateManageOptions{
MaxBackups: 7,
MaxAge: 7 * 24 * time.Hour,
Compress: true,
Pattern: "20060102-150405",
}
if err := starlog.StartManagedRotatePolicy(log, policy, 1, opts); err != nil {
panic(err)
}
```
模板增强版:
```go
_ = starlog.StartManagedRotateByTime(log, 24*time.Hour, 10, opts)
_ = starlog.StartManagedRotateBySize(log, 200*1024*1024, 5, opts)
_ = starlog.StartManagedRotateByTimeSize(log, 24*time.Hour, 200*1024*1024, 5, opts)
```
### 9.4 RouteHandler 分流文件轮转
分流 sink 可直接使用轮转 sink:
```go
debugSink, err := starlog.NewManagedRotateByTimeSink(
"./logs/debug.log",
true,
24*time.Hour,
30*time.Second,
starlog.RotateManageOptions{MaxBackups: 30, Compress: true, Pattern: "20060102"},
)
if err != nil {
panic(err)
}
route := starlog.NewRouteHandler(
starlog.Route{
Name: "debug-info",
Match: starlog.MatchLevels(starlog.LvDebug, starlog.LvInfo),
Formatter: starlog.NewTextFormatter(),
Sink: debugSink,
},
)
log.SetEntryHandler(route)
```
### 9.5 自定义轮转策略,同时应用到两种文件
```go
type HourOrSizePolicy struct {
HourInterval time.Duration
MaxBytes int64
}
func (p HourOrSizePolicy) ShouldRotate(fi starlog.FileInfo, _ *starlog.Entry) bool {
if p.MaxBytes > 0 && fi.Size >= p.MaxBytes {
return true
}
if p.HourInterval > 0 && fi.ModTime.Add(p.HourInterval).Before(time.Now()) {
return true
}
return false
}
func (p HourOrSizePolicy) NextPath(path string, now time.Time) string {
ext := filepath.Ext(path)
base := strings.TrimSuffix(path, ext)
return base + "." + now.Format("20060102-150405") + ext
}
```
应用到主日志:
```go
mainLog := starlog.NewProduction(nil)
_ = starlog.SetLogFile("./logs/app.log", mainLog, true)
policy := HourOrSizePolicy{HourInterval: 6 * time.Hour, MaxBytes: 300 * 1024 * 1024}
_ = starlog.StartManagedRotatePolicy(mainLog, policy, 1, starlog.RotateManageOptions{
MaxBackups: 20,
MaxAge: 14 * 24 * time.Hour,
Compress: true,
Pattern: "20060102-150405",
})
```
应用到分流文件:
```go
debugSink, _ := starlog.NewManagedRotatePolicySink(
"./logs/debug.log",
true,
policy,
30*time.Second,
starlog.RotateManageOptions{
MaxBackups: 30,
MaxAge: 30 * 24 * time.Hour,
Compress: true,
Pattern: "20060102-150405",
},
)
route := starlog.NewRouteHandler(
starlog.Route{
Name: "debug-info",
Match: starlog.MatchLevels(starlog.LvDebug, starlog.LvInfo),
Formatter: starlog.NewTextFormatter(),
Sink: debugSink,
},
)
mainLog.SetEntryHandler(route)
```
### 9.6 兼容入口
`StartArchive` 仍可使用,但新代码建议优先 `StartRotatePolicy` / `StartManagedRotatePolicy`
## 10. 异步、回调与 pending
```go
log.SetHandler(func(data starlog.LogData) {
// 异步显示回调
})
log.SetAsyncFallbackToSync(true)
log.SetAsyncHandlerTimeout(100 * time.Millisecond)
log.SetEntryHandler(myHandler)
log.SetEntryHandlerTimeout(200 * time.Millisecond)
log.SetPendingWriteLimit(1024)
log.SetPendingDropPolicy(starlog.PendingDropOldest)
```
可观测项:
- `GetAsyncDropCount()`
- `GetPendingStats()`
## 11. 高频防爆(去重 / 采样 / 限流)
主链路顺序:`去重 -> 采样 -> 限流`
```go
log.SetDedupConfig(starlog.DedupConfig{
Enable: true,
Levels: []int{starlog.LvInfo, starlog.LvNotice},
Window: 2 * time.Second,
Scope: starlog.DedupScopeByKey,
})
log.SetSamplingConfig(starlog.SamplingConfig{
Enable: true,
Levels: []int{starlog.LvInfo},
Rate: 0.2,
Scope: starlog.SamplingScopeByKey,
})
log.SetRateLimitConfig(starlog.RateLimitConfig{
Enable: true,
Levels: []int{starlog.LvInfo},
Rate: 50,
Burst: 100,
Scope: starlog.RateLimitScopeByKey,
})
```
## 12. 脱敏
```go
log.AddRedactRule(starlog.NewSensitiveFieldRule("[MASK]", "password", "token"))
log.AddRedactRule(starlog.NewMessageRegexRule(regexp.MustCompile(`\b1\d{10}\b`), "[PHONE]"))
log.SetRedactFailMode(starlog.RedactFailMaskAll)
```
失败策略:
- `RedactFailMaskAll`(推荐)
- `RedactFailOpen`
- `RedactFailDrop`
## 13. 指标观测
```go
snapshot := log.GetMetricsSnapshot()
_ = snapshot
```
常用统计:
- `GetPendingStats()`
- `GetSamplingStats()`
- `GetDedupStats()`
- `GetRateLimitStats()`
- `GetWriteErrorCount()`
- `GetAsyncDropCount()`
## 14. 生命周期与退出
语义:
- `Flush()`:刷写 pending
- `Sync()``Flush + 底层 Sync()`(若支持)
- `Close()`:关闭资源,不等待异步队列 drain
- `Shutdown(ctx)`:等待异步 drain,再关闭资源
推荐:
```go
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
defer log.Shutdown(ctx)
```
全局 logger 可使用:`starlog.Shutdown(ctx)`
## 15. 标准库桥接
```go
stdLogger := log.AsStdlibLoggerWithOptions(starlog.LvInfo,
starlog.WithStdlibPrefix("http "),
starlog.WithStdlibFlags(0),
starlog.WithStdlibTrimNewline(true),
)
stdLogger.Println("server started")
```
也可以桥接为 `io.Writer``AsWriter(level)` / `AsWriterWithOptions(...)`
## 16. 测试工具
### 16.1 Observer
```go
obs := starlog.NewObserverWithLimit(200)
log.SetEntryHandler(obs)
entries := obs.Entries()
_ = entries
```
### 16.2 TestHook
```go
hook := starlog.NewTestHook(log)
defer hook.Close()
log.Info("hello")
if hook.Count() == 0 {
panic("expect log")
}
```
## 17. 配置快照 API(并发配置推荐)
```go
log.UpdateConfig(func(cfg *starlog.Config) {
cfg.Level = starlog.LvInfo
cfg.ShowColor = false
cfg.AutoAppendNewline = true
cfg.Sampling = starlog.SamplingConfig{Enable: true, Rate: 0.3}
cfg.Dedup = starlog.DedupConfig{Enable: true, Window: time.Second}
})
```
配套:
- `GetConfig()`
- `ApplyConfig(cfg)`
- `DefaultConfig()`
## 18. 生产最小模板
```go
func NewProdLogger(path string) *starlog.StarLogger {
log := starlog.NewProduction(nil)
log.SetName("svc")
log.SetAutoAppendNewline(true)
if err := starlog.SetLogFile(path, log, true); err != nil {
panic(err)
}
_ = starlog.StartManagedRotateBySize(log, 200*1024*1024, 5, starlog.RotateManageOptions{
MaxBackups: 14,
MaxAge: 14 * 24 * time.Hour,
Compress: true,
Pattern: "20060102-150405",
})
return log
}
```
## 19. 完整实战示例(主日志 + 协程分隔 + 双文件轮转)
覆盖目标:
1. 使用 `starlog`
2. 协程中使用新 `LOG` 分隔符(`NewFlag()` 自动随机)
3. 设置主日志文件
4. 主日志使用压缩轮转
5. `debug/info` 额外保存到 `debug.log`
6. `debug.log` 使用独立轮转策略
```go
package main
import (
"context"
"os"
"path/filepath"
"sync"
"time"
"b612.me/starlog"
)
func main() {
_ = os.MkdirAll("./logs", 0755)
mainLog := starlog.NewProduction(nil)
mainLog.SetName("app-main")
mainLog.SetAutoAppendNewline(true)
mainPath := filepath.Clean("./logs/app.log")
if err := starlog.SetLogFile(mainPath, mainLog, true); err != nil {
panic(err)
}
if err := starlog.StartManagedRotateBySize(mainLog, 200*1024*1024, 5, starlog.RotateManageOptions{
MaxBackups: 14,
MaxAge: 14 * 24 * time.Hour,
Compress: true,
Pattern: "20060102-150405",
}); err != nil {
panic(err)
}
debugPath := filepath.Clean("./logs/debug.log")
debugSink, err := starlog.NewManagedRotateByTimeSink(debugPath, true, 24*time.Hour, 30*time.Second, starlog.RotateManageOptions{
MaxBackups: 30,
MaxAge: 30 * 24 * time.Hour,
Compress: true,
Pattern: "20060102",
})
if err != nil {
panic(err)
}
route := starlog.NewRouteHandler(
starlog.Route{
Name: "debug-info-to-debug-log",
Match: starlog.MatchLevels(starlog.LvDebug, starlog.LvInfo),
Formatter: starlog.NewTextFormatter(),
Sink: debugSink,
},
)
mainLog.SetEntryHandler(route)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func(worker int) {
defer wg.Done()
workerLog := mainLog.NewFlag()
workerLog.WithField("worker", worker).Debug("worker start")
workerLog.WithField("worker", worker).Info("worker running")
if worker == 1 {
workerLog.WithField("worker", worker).Error("worker failed")
}
}(i)
}
wg.Wait()
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
_ = mainLog.Shutdown(ctx)
}
```
## 20. 常见问题
### Q1`Infof/Errorf` 需要手动加 `\n` 吗?
默认需要。若开启 `SetAutoAppendNewline(true)`,末尾无 `\n` 会自动补齐。
### Q2:异步模式会丢日志吗?
- 显示回调链路:可能因超时或队列满发生丢弃,可通过计数观测
- writer/sink 主写链路:只要未被过滤(级别/去重/采样/限流)且写入正常,会落盘
### Q3:新项目应该选 `Archive` 还是 `RotatePolicy`
优先 `RotatePolicy``Archive` 作为兼容路径保留。