Compare commits

..

No commits in common. "main" and "v1.1.13" have entirely different histories.

23 changed files with 311 additions and 658 deletions

7
.gitignore vendored
View File

@ -1,9 +1,2 @@
.log.meta.json .log.meta.json
.test.meta.json .test.meta.json
.ai/
.geminiignore
.gemini
env.json
env.yml
env.yaml
/CODE-FULL.md

View File

@ -1,39 +1,5 @@
# Changelog # Changelog
## [1.3.2] - 2026-05-13
- **功能增强: 引入摩擦消除工具 `As`**:
- **泛型支持**: 新增全局泛型函数 `log.As[T](v T, err error) T`,仿照 `cast.As` 设计,自动记录错误并返回零值,极大简化了带 error 返回值的函数链式调用。
- **Logger 扩展**: `Logger` 结构体新增 `As(v any, err error) any` 方法,支持实例级别的错误捕获与自动记录。
- **调用栈优化**: 优化了 `GetCallStacks` 逻辑,自动跳过 `default_logger.go` 中的内部帧,确保 `log.As` 记录的错误位置精准指向业务代码。
## [1.3.1] - 2026-05-12
- **架构升级: 引入 LoggerService**:
- **解耦重构**: 重构全局变量管理,引入 `loggerService` 结构体集中化管理异步写入协程、Writers 对象池、文件句柄与丢弃计数。
- **生命周期管理**: 实现了 `Start(ctx, logger)`, `Stop(ctx)`, `Health()` 接口,完美支持与 `apigo.cc/go/starter` 的基础设施集成。
- **安全性增强**: 优化了平滑停止逻辑,确保 `Stop` 调用时能完整 Flush 缓冲区数据。
- **功能增强**:
- **动态应用名**: 新增 `SetDefaultName(name)` 全局方法,支持在微服务启动后动态设置应用名称,并自动同步至 `DefaultLogger`
- **配置与忽略规则**:
- 更新 `.gitignore`,增加了对 `.gemini`, `env.json`, `env.yml` 以及 `/CODE-FULL.md` 的忽略支持。
- **文档与测试同步**: 全面更新了 `README.md`, `CHANGELOG.md``TEST.md`
## [1.3.0] - 2026-05-10
- **全栈基础设施对齐**:
- 将所有内部依赖(`cast`, `config`, `file`, `id`, `shell` 等)统一升级至 `v1.3.0` 语义版本。
- 优化项目忽略规则,在 `.gitignore` 中增加了对 `.geminiignore` 的支持。
## [1.1.16] - 2026-05-10
- **依赖对齐**: 同步更新 `golang.org/x/crypto``golang.org/x/sys` 至最新版本,确保底层安全性。
## [1.1.15] - 2026-05-10
- **安全与性能双重增强**:
- **敏感字段扩展**: `authorization` 现在默认纳入脱敏规则 (`LogDefaultSensitive`),自动过滤 HTTP 认证凭证。
- **TraceId 吞吐优化**: 切换至 `id.Get10Bytes14MPerSecond()` 算法,在保持 10 字节唯一性的同时,极大提升了超高并发下的 ID 生成性能。
- **应用名称识别对齐**: 重构 `GetDefaultName` 逻辑,优先通过 `debug.ReadBuildInfo()` 识别 Module 路径,并支持自动裁剪 Windows 环境下的 `.exe` 后缀。
## [1.1.14] - 2026-05-09
- **可视化能力调整**: 调整优化了 `viewer` 模块相关的可视化能力,提升了日志的可读性与调试体验。
## [1.1.13] - 2026-05-09 ## [1.1.13] - 2026-05-09
- **绝对索引优化与零空洞**: - **绝对索引优化与零空洞**:
- 彻底消除 `BaseLog` 与业务字段之间的索引空洞。字段位置调整为:`BaseLog` (0-5),标准消息字段 (`Info`, `Error` 等) 与业务日志字段从 `pos: 6` 起始。 - 彻底消除 `BaseLog` 与业务字段之间的索引空洞。字段位置调整为:`BaseLog` (0-5),标准消息字段 (`Info`, `Error` 等) 与业务日志字段从 `pos: 6` 起始。

View File

@ -25,9 +25,6 @@ import "apigo.cc/go/log"
// 默认 logger (通过 log.json 或环境变量配置) // 默认 logger (通过 log.json 或环境变量配置)
func main() { func main() {
// 在微服务场景下动态设置应用名称
log.SetDefaultName("my-microservice")
log.Info("服务启动", "port", 8080) log.Info("服务启动", "port", 8080)
log.Error("数据库连接失败", "db", "mysql") log.Error("数据库连接失败", "db", "mysql")
@ -54,7 +51,7 @@ func main() {
"level": "info", "level": "info",
"file": "logs/app.log", "file": "logs/app.log",
"splitTag": ".2006-01-02", "splitTag": ".2006-01-02",
"sensitive": "phone,password,secret,token,accessToken,authorization" "sensitive": "phone,password,secret,token,key"
} }
``` ```
@ -71,7 +68,7 @@ export LOG_FILE=console
### 配置项说明 ### 配置项说明
* `name`: 应用名称 (默认通过 `debug.ReadBuildInfo()``os.Args[0]` 自动识别)。 * `name`: 应用名称 (默认读取 DISCOVER_APP 或从 `go.mod` 自动识别)。
* `level`: 日志级别 (`debug`, `info`, `warning`, `error`)。 * `level`: 日志级别 (`debug`, `info`, `warning`, `error`)。
* `file`: 输出目标。 * `file`: 输出目标。
* `console`: 直接输出到控制台(默认)。 * `console`: 直接输出到控制台(默认)。
@ -80,7 +77,7 @@ export LOG_FILE=console
* `splitTag`: 文件切分格式,仅当 `file` 为文件路径时有效。 * `splitTag`: 文件切分格式,仅当 `file` 为文件路径时有效。
* 语法遵循 Go 标准的 `time.Format` 布局,如 `".2006-01-02"` (按天切分)`".2006-01-02-15"` (按小时切分)。 * 语法遵循 Go 标准的 `time.Format` 布局,如 `".2006-01-02"` (按天切分)`".2006-01-02-15"` (按小时切分)。
* `truncations`: 堆栈信息截断前缀(多个以逗号分隔,默认截断 `github.com/`, `golang.org/`, `/apigo.cc/`)。 * `truncations`: 堆栈信息截断前缀(多个以逗号分隔,默认截断 `github.com/`, `golang.org/`, `/apigo.cc/`)。
* `sensitive`: 需要自动脱敏的字段名(多个以逗号分隔,不区分大小写),默认处理 `phone,password,secret,token,accessToken,authorization`。 * `sensitive`: 需要自动脱敏的字段名(多个以逗号分隔,不区分大小写),默认处理 `phone,password,secret,token,key`。
## 🛠 API 指南 ## 🛠 API 指南
@ -89,14 +86,10 @@ export LOG_FILE=console
1. **分级记录** 1. **分级记录**
* `Debug`, `Info`, `Warning`, `Error` —— 标准日志方法,支持 `message` + 变长 `extra` 参数。 * `Debug`, `Info`, `Warning`, `Error` —— 标准日志方法,支持 `message` + 变长 `extra` 参数。
2. **摩擦消除 (`As`)** 2. **通用记录 (`Log`)**
* `As(v, err)` —— 仿照 `cast.As`,忽略错误并返回零值,但会自动将错误记录到日志中。支持全局调用 (`log.As`) 或实例调用 (`logger.As`)。
* **优势**: 在类型转换或快速赋值场景下,无需繁琐的 `if err != nil` 判断,同时确保异常被记录。
3. **通用记录 (`Log`)**
* `Log(LogEntry)` —— 记录自定义结构的日志。 * `Log(LogEntry)` —— 记录自定义结构的日志。
4. **独立可视化工具 (`logv`)** 3. **独立可视化工具 (`logv`)**
* **安装**: `go install apigo.cc/go/log/logv@latest` * **安装**: `go install apigo.cc/go/log/logv@latest`
* **使用**: `tail -f app.log | logv``tail -f app.log | logv -json` * **使用**: `tail -f app.log | logv``tail -f app.log | logv -json`

86
TEST.md
View File

@ -1,62 +1,30 @@
# Test Results # 日志性能测试报告
## 单元测试报告 ## 测试环境
- 操作系统: darwin
- 架构: amd64
- CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
## 基准测试结果 (v1.1.10)
| 测试用例 | 迭代次数 | 耗时 (ns/op) | 内存分配 (B/op) | 分配次数 (allocs/op) |
| :--- | :--- | :--- | :--- | :--- |
| `BenchmarkLogger_RequestLog_Realistic` | 344,300 | 3,338 | 1,331 | 19 |
| `BenchmarkLoggerInfo` | 291,952 | 4,083 | - | - |
| `BenchmarkLoggerAsyncConcurrent` | 784,453 | 1,466 | - | - |
## 版本对比评估
| 版本 | 机制 | 存储格式 | 可视化 | 性能 (Async) |
| :--- | :--- | :--- | :--- | :--- |
| **v1.0.3** | Map 序列化 | JSON Object | 内置 | ~8,773 ns/op |
| **v1.1.7** | Dead Code Removal | JSON Array | 独立工具/Meta | ~1,059 ns/op |
| **v1.1.10** | Stability & Infrastructure | JSON Array | 独立工具/Meta | ~919 ns/op |
| **v1.1.11** | **Absolute Indexing (Schema)** | **Fixed Array** | **LogType Opt** | **~1,466 ns/op** |
## 总结
- **Schema 兼容性**: v1.1.11 实现了 `pos` 绝对索引。虽然因数组稀疏化(填充 0导致序列化开销略微增加~1.4µs但换取了极强的 Schema 稳定性,适配各类数仓接入。
- **可观测性**: 引入 `droppedLogs` 监控,解决了高并发场景下日志丢弃“黑盒”的问题。
- **鲁棒性**: 替换为 UDP 拨号法获取 IP消除了在 K8s 等复杂网络环境下的识别摩擦。
```text
=== RUN TestLoggerCore_Initialization
--- PASS: TestLoggerCore_Initialization (0.00s)
=== RUN TestLoggerCore_Concurrency
--- PASS: TestLoggerCore_Concurrency (0.00s)
=== RUN TestMetaExtraction
--- PASS: TestMetaExtraction (0.00s)
=== RUN TestWithEntry
--- PASS: TestWithEntry (0.00s)
=== RUN TestLoggerReliability
--- PASS: TestLoggerReliability (0.01s)
=== RUN TestToArrayBytes
serializer_test.go:64: Raw log: ["test-app","mock_info_test",1620000000,"abc-123","Hello, World!",{"user_id":42}]
--- PASS: TestToArrayBytes (0.00s)
=== RUN TestToArrayBytes_Desensitize
--- PASS: TestToArrayBytes_Desensitize (0.00s)
=== RUN TestSplitTag
--- PASS: TestSplitTag (1.80s)
=== RUN TestSensitiveDetailed
--- PASS: TestSensitiveDetailed (0.00s)
=== RUN TestDeepDesensitization
--- PASS: TestDeepDesensitization (0.00s)
=== RUN TestLogger
--- PASS: TestLogger (0.00s)
=== RUN TestDesensitization
--- PASS: TestDesensitization (0.00s)
=== RUN TestDBLog
--- PASS: TestDBLog (0.00s)
=== RUN TestRequestLog
--- PASS: TestRequestLog (0.00s)
=== RUN TestExtraLogs
--- PASS: TestExtraLogs (0.00s)
=== RUN TestViewable
--- PASS: TestViewable (0.00s)
=== RUN TestToJSON
--- PASS: TestToJSON (0.00s)
=== RUN TestLoadMeta
--- PASS: TestLoadMeta (0.00s)
=== RUN TestEnhancedViewable
--- PASS: TestEnhancedViewable (0.00s)
=== RUN TestEnhancedToJSON
--- PASS: TestEnhancedToJSON (0.00s)
=== RUN TestCallStacksViewable
--- PASS: TestCallStacksViewable (0.00s)
=== RUN TestPrecisionViewable
--- PASS: TestPrecisionViewable (0.00s)
PASS
ok apigo.cc/go/log 2.246s
```
## 核心指标验证
- **初始化安全性**: `TestLoggerCore_Initialization` 确保 Logger 实例配置正确加载。
- **高并发稳定性**: `TestLoggerCore_Concurrency` 验证了在多协程竞争环境下日志写入的线程安全。
- **元数据驱动验证**: `TestMetaExtraction``TestLoadMeta` 确保 `.log.meta.json` 协议的解析与应用。
- **序列化性能**: `TestToArrayBytes` 验证了 Positional Array 格式的正确性。
- **深度脱敏能力**: `TestDeepDesensitization` 闭环验证了对复杂嵌套结构的脱敏逻辑。
- **可靠性边界**: `TestLoggerReliability` 模拟了极高压力下的日志丢弃与缓冲策略。
- **文件切分**: `TestSplitTag` 实测了基于时间滚动的文件切分能力。

View File

@ -7,8 +7,8 @@ import (
var DefaultLogger *Logger var DefaultLogger *Logger
func init() { func init() {
RegisterWriterMaker("es", newESWriter) RegisterWriterMaker("es", NewESWriter)
RegisterWriterMaker("ess", newESWriter) RegisterWriterMaker("ess", NewESWriter)
var conf Config var conf Config
_ = config.Load(&conf, "log") _ = config.Load(&conf, "log")
@ -19,46 +19,3 @@ func init() {
func New(traceId string) *Logger { func New(traceId string) *Logger {
return DefaultLogger.New(traceId) return DefaultLogger.New(traceId)
} }
// SetDefaultName 设置全局默认应用名称,并同步更新 DefaultLogger
func SetDefaultName(name string) {
if name == "" {
return
}
globalDefaultName = name
if DefaultLogger != nil {
DefaultLogger.SetName(name)
}
}
// As 仿照 cast.As使用 DefaultLogger 记录错误并返回零值 (消除摩擦)
func As[T any](v T, err error) T {
if err != nil {
if DefaultLogger != nil {
DefaultLogger.Error(err.Error())
}
var zero T
return zero
}
return v
}
// Debug 记录一条调试级别日志
func Debug(message string, extra ...any) {
DefaultLogger.Debug(message, extra...)
}
// Info 记录一条信息级别日志
func Info(message string, extra ...any) {
DefaultLogger.Info(message, extra...)
}
// Warning 记录一条警告级别日志
func Warning(message string, extra ...any) {
DefaultLogger.Warning(message, extra...)
}
// Error 记录一条错误级别日志
func Error(message string, extra ...any) {
DefaultLogger.Error(message, extra...)
}

View File

@ -14,7 +14,7 @@ import (
"apigo.cc/go/cast" "apigo.cc/go/cast"
) )
type esWriter struct { type ESWriter struct {
config *Config config *Config
url string url string
user string user string
@ -27,8 +27,8 @@ type esWriter struct {
prefix string prefix string
} }
func newESWriter(conf *Config) Writer { func NewESWriter(conf *Config) Writer {
w := &esWriter{ w := &ESWriter{
config: conf, config: conf,
queue: make([]string, 0), queue: make([]string, 0),
client: &http.Client{}, client: &http.Client{},
@ -76,7 +76,7 @@ func newESWriter(conf *Config) Writer {
return w return w
} }
func (w *esWriter) Log(entry LogEntry, data []byte) { func (w *ESWriter) Log(entry LogEntry, data []byte) {
objBytes, err := cast.ToJSONBytes(entry) objBytes, err := cast.ToJSONBytes(entry)
if err != nil || len(objBytes) == 0 { if err != nil || len(objBytes) == 0 {
return return
@ -90,14 +90,14 @@ func (w *esWriter) Log(entry LogEntry, data []byte) {
var responseOkBytes = []byte("\"errors\":false") var responseOkBytes = []byte("\"errors\":false")
func (w *esWriter) Run() { func (w *ESWriter) Run() {
now := time.Now().Unix() now := time.Now().Unix()
w.lock.Lock() w.lock.Lock()
queueLen := len(w.queue) queueLen := len(w.queue)
w.lock.Unlock() w.lock.Unlock()
// 超过100条数据 或 过了1秒 发送数据 // 超过100条数据 或 过了1秒 发送数据
if queueLen > 100 || (queueLen > 0 && (now > w.last || !WriterService.Running.Load())) { if queueLen > 100 || (queueLen > 0 && (now > w.last || !writerRunning.Load())) {
w.lock.Lock() w.lock.Lock()
sendings := w.queue sendings := w.queue
w.queue = make([]string, 0) w.queue = make([]string, 0)

View File

@ -4,6 +4,7 @@ import (
"bufio" "bufio"
"fmt" "fmt"
"os" "os"
"sync"
"time" "time"
) )
@ -16,6 +17,11 @@ type FileWriter struct {
bufWriter *bufio.Writer bufWriter *bufio.Writer
} }
var (
files = make(map[string]*FileWriter)
filesLock sync.RWMutex
)
// Write 由外层的 writerRunner 单协程调用,绝对并发安全,无需加锁 // Write 由外层的 writerRunner 单协程调用,绝对并发安全,无需加锁
func (f *FileWriter) Write(tm time.Time, data []byte) { func (f *FileWriter) Write(tm time.Time, data []byte) {
nowSplit := tm.Format(f.splitTag) nowSplit := tm.Format(f.splitTag)

View File

@ -20,7 +20,6 @@ func TestSplitTag(t *testing.T) {
File: logFile, File: logFile,
SplitTag: splitTag, SplitTag: splitTag,
} }
log.WriterService.Start(nil, nil)
logger := log.NewLogger(conf) logger := log.NewLogger(conf)
// 1. 记录第一条日志 // 1. 记录第一条日志
@ -79,7 +78,7 @@ func TestSensitiveDetailed(t *testing.T) {
// 直接测试 ToArrayBytes // 直接测试 ToArrayBytes
// 注意passed to ToArrayBytes 的 keys 应该是已经过 fixField 处理的 // 注意passed to ToArrayBytes 的 keys 应该是已经过 fixField 处理的
sensitiveKeys := []string{"password", "secretkey"} sensitiveKeys := []string{"password", "secretkey"}
buf := log.Marshal(entry, sensitiveKeys) buf := log.ToArrayBytes(entry, sensitiveKeys)
result := string(buf) result := string(buf)
if strings.Contains(result, "my_password") { if strings.Contains(result, "my_password") {
@ -116,7 +115,7 @@ func TestDeepDesensitization(t *testing.T) {
} }
sensitiveKeys := []string{"password", "token"} sensitiveKeys := []string{"password", "token"}
buf := log.Marshal(entry, sensitiveKeys) buf := log.ToArrayBytes(entry, sensitiveKeys)
result := string(buf) result := string(buf)
// Check deep desensitization in map // Check deep desensitization in map
@ -134,3 +133,4 @@ func TestDeepDesensitization(t *testing.T) {
t.Errorf("Safe data 'data_safe' should be present in: %s", result) t.Errorf("Safe data 'data_safe' should be present in: %s", result)
} }
} }

20
go.mod
View File

@ -3,18 +3,18 @@ module apigo.cc/go/log
go 1.25.0 go 1.25.0
require ( require (
apigo.cc/go/cast v1.3.3 apigo.cc/go/cast v1.2.8
apigo.cc/go/config v1.3.1 apigo.cc/go/config v1.0.6
apigo.cc/go/file v1.3.2 apigo.cc/go/file v1.0.6
apigo.cc/go/id v1.3.1 apigo.cc/go/id v1.0.5
apigo.cc/go/shell v1.3.1 apigo.cc/go/shell v1.0.5
) )
require ( require (
apigo.cc/go/encoding v1.3.1 // indirect apigo.cc/go/encoding v1.0.5 // indirect
apigo.cc/go/rand v1.3.1 // indirect apigo.cc/go/rand v1.0.5 // indirect
apigo.cc/go/safe v1.3.1 // indirect apigo.cc/go/safe v1.0.5 // indirect
golang.org/x/crypto v0.51.0 // indirect golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.44.0 // indirect golang.org/x/sys v0.43.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

40
go.sum
View File

@ -1,29 +1,29 @@
apigo.cc/go/cast v1.3.3 h1:aln5eDR5DZVWVzZ/y5SJh1gQNgWv2sT82I25NaO9g34= apigo.cc/go/cast v1.2.8 h1:plb676DH2TjYljzf8OEMGT6lIhmZ/xaxEFfs0kDOiSI=
apigo.cc/go/cast v1.3.3/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk= apigo.cc/go/cast v1.2.8/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk=
apigo.cc/go/config v1.3.1 h1:wZzUh4oL+fGD6SayVgX6prLPMsniM25etWFcEH8XzIE= apigo.cc/go/config v1.0.6 h1:32nOCr+8AkGFnKuythCjHPOjxilg6SOlSWXKTkNtx6I=
apigo.cc/go/config v1.3.1/go.mod h1:7KHz/1WmtBLM762Lln/TaXh2dmlMvJTLhnlk33zbS3U= apigo.cc/go/config v1.0.6/go.mod h1:nX+nLKZTP6Xton9Gt/9XsTh0d1sQ+Qkwysgyjq/k4R0=
apigo.cc/go/encoding v1.3.1 h1:y8O58KYAyulkThg1O2ji2BqjnFoSvk42sit9I3z+K7Y= apigo.cc/go/encoding v1.0.5 h1:a2XbXyd8D2gKo1ekXn/pt5adltWbIfdJCMhaF2uvzF0=
apigo.cc/go/encoding v1.3.1/go.mod h1:xAJk5b83VZ31mXMTnyp0dfMoBKfT/AHDn0u+cQfojgY= apigo.cc/go/encoding v1.0.5/go.mod h1:V5CgT7rBbCxy+uCU20q0ptcNNRSgMtpA8cNOs6r8IeI=
apigo.cc/go/file v1.3.2 h1:pu4oiDyiqgj3/eykfnJf+/6+A9v/Z0b3ClP5XK+lwG4= apigo.cc/go/file v1.0.6 h1:kyrPJ+oqC0DtYubX2aI+3QIVoDAPkRiYyBwd1F0cBlA=
apigo.cc/go/file v1.3.2/go.mod h1:vci4h0Pz94mV6dkniQkuyBYERVYeq7/LX4jJVuCg9hs= apigo.cc/go/file v1.0.6/go.mod h1:AOw8+3q1fmCZpBWpBfUSSb+Q6Li3W9jH1EktQXmFhVg=
apigo.cc/go/id v1.3.1 h1:pkqi6VeWyQoHuIu0Zbx/RRxIAdM61Js0j6cY1M9XVCk= apigo.cc/go/id v1.0.5 h1:23YkR7oklSA69gthYlu8zl/kpIkeIoEYxi1f1Sz5l3A=
apigo.cc/go/id v1.3.1/go.mod h1:P2/vl3tyW3US+ayOFSMoPIOCulNLBngNYPhXJC/Z7J4= apigo.cc/go/id v1.0.5/go.mod h1:ZaYLIyrJvkf3j7J8a0lnKywSAHljaczWxU0x2HmQDzg=
apigo.cc/go/rand v1.3.1 h1:7FvsI6PtQ5XrWER0dTiLVo0p7GIxRidT/TBKhVy93j8= apigo.cc/go/rand v1.0.5 h1:AkUoWr0SELgeDmRjLEDjOIp29nXdzqQQvmGRIHpTN7U=
apigo.cc/go/rand v1.3.1/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk= apigo.cc/go/rand v1.0.5/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk=
apigo.cc/go/safe v1.3.1 h1:irTCqPAC97gGsX/Lw5AzLelDt1xXLEZIAaVhLELWe9Q= apigo.cc/go/safe v1.0.5 h1:yZJLhpMntJrtqU/ev0UlyOoHu/cLrnnGUO4aHyIZcwE=
apigo.cc/go/safe v1.3.1/go.mod h1:XdOpBhN2vkImalaykYXXmEpczqWa1y3ah6/Q72cdRqE= apigo.cc/go/safe v1.0.5/go.mod h1:i9xnh7reJIFPauLnlzuIDgvrQvhjxpFlpVh3O6ulWd0=
apigo.cc/go/shell v1.3.1 h1:M8oD0b2HcJuCC6frQFx11b3UTcTx3lATX8XK+YXSVm8= apigo.cc/go/shell v1.0.5 h1:bmvUTJGe1GwsHAy42v3iaoK40PoBC7Xq1aMCYxUZmtg=
apigo.cc/go/shell v1.3.1/go.mod h1:ZMdJjpCpWdvsHKUXlelh/AxsV/nWdkH/k3lISfzMdUw= apigo.cc/go/shell v1.0.5/go.mod h1:sx/nYw5CihHWmo5JHkaZUbmMYXNHx8swzArbQCUGHjc=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI= golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8= golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ= golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

View File

@ -1,7 +1,6 @@
package log_test package log_test
import ( import (
"strconv"
"testing" "testing"
"apigo.cc/go/log" "apigo.cc/go/log"
@ -87,28 +86,3 @@ func TestExtraLogs(t *testing.T) {
logger.Info("Extra log test", "key", "value") logger.Info("Extra log test", "key", "value")
} }
func TestAs(t *testing.T) {
// 1. 测试 log.As (使用 DefaultLogger)
val1 := log.As(strconv.Atoi("123"))
if val1 != 123 {
t.Errorf("log.As expected 123, got %v", val1)
}
val2 := log.As(strconv.Atoi("abc"))
if val2 != 0 {
t.Errorf("log.As expected 0, got %v", val2)
}
// 2. 测试 logger.As (方法)
logger := log.NewLogger(log.Config{Level: "debug"})
val3 := logger.As(strconv.Atoi("456")).(int)
if val3 != 456 {
t.Errorf("logger.As expected 456, got %v", val3)
}
val4 := logger.As(strconv.Atoi("def")).(int)
if val4 != 0 {
t.Errorf("logger.As expected 0, got %v", val4)
}
}

View File

@ -43,12 +43,12 @@ func NewLogger(conf Config) *Logger {
} }
if conf.Name == "" { if conf.Name == "" {
conf.Name = getDefaultName() conf.Name = GetDefaultName()
} }
logger := Logger{ logger := Logger{
truncations: cast.Split(conf.Truncations, ","), truncations: cast.Split(conf.Truncations, ","),
traceId: id.Get10Bytes14MPerSecond(), traceId: id.MakeID(10),
} }
if len(conf.Sensitive) > 0 { if len(conf.Sensitive) > 0 {
@ -78,27 +78,29 @@ func NewLogger(conf Config) *Logger {
if m, ok := writerMakers[writerName]; ok { if m, ok := writerMakers[writerName]; ok {
if w := m(&conf); w != nil { if w := m(&conf); w != nil {
logger.writer = w logger.writer = w
WriterService.WriterLock.Lock() writerLock.Lock()
cur := WriterService.Writers.Load().([]Writer) cur := writers.Load().([]Writer)
newW := append(cur, w) newW := append(cur, w)
WriterService.Writers.Store(newW) writers.Store(newW)
WriterService.WriterLock.Unlock() writerLock.Unlock()
Start()
} }
} }
} else { } else {
if conf.SplitTag != "" { if conf.SplitTag != "" {
WriterService.FilesLock.RLock() filesLock.RLock()
logger.file = WriterService.Files[conf.File+conf.SplitTag] logger.file = files[conf.File+conf.SplitTag]
WriterService.FilesLock.RUnlock() filesLock.RUnlock()
if logger.file == nil { if logger.file == nil {
logger.file = &FileWriter{ logger.file = &FileWriter{
fileName: conf.File, fileName: conf.File,
splitTag: conf.SplitTag, splitTag: conf.SplitTag,
} }
WriterService.FilesLock.Lock() filesLock.Lock()
WriterService.Files[conf.File+conf.SplitTag] = logger.file files[conf.File+conf.SplitTag] = logger.file
WriterService.FilesLock.Unlock() filesLock.Unlock()
} }
Start()
} else { } else {
fp, err := os.OpenFile(conf.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644) fp, err := os.OpenFile(conf.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err == nil { if err == nil {
@ -116,14 +118,14 @@ func (logger *Logger) Log(entry LogEntry) {
} }
func (logger *Logger) asyncWrite(entry LogEntry) { func (logger *Logger) asyncWrite(entry LogEntry) {
buf := Marshal(entry, logger.sensitiveKeys) buf := ToArrayBytes(entry, logger.sensitiveKeys)
logger.writeBuf(entry, buf) logger.writeBuf(entry, buf)
putEntry(entry) PutEntry(entry)
} }
func (logger *Logger) writeBuf(entry LogEntry, buf []byte) { func (logger *Logger) writeBuf(entry LogEntry, buf []byte) {
if WriterService.Running.Load() { if writerRunning.Load() {
writeAsync(logPayload{ WriteAsync(logPayload{
entry: entry, entry: entry,
buf: buf, buf: buf,
writer: logger.writer, writer: logger.writer,
@ -236,10 +238,6 @@ func (logger *Logger) Error(message string, extra ...any) {
} }
} }
func (logger *Logger) SetName(name string) {
logger.config.Name = name
}
func (logger *Logger) SetLevel(level LevelType) { func (logger *Logger) SetLevel(level LevelType) {
logger.level = level logger.level = level
} }
@ -254,14 +252,6 @@ func (logger *Logger) GetTraceId() string {
return logger.traceId return logger.traceId
} }
// As 仿照 cast.As忽略错误并返回零值但会将错误记录到日志中 (消除摩擦)
func (logger *Logger) As(v any, err error) any {
if err != nil {
logger.Error(err.Error())
}
return v
}
func (logger *Logger) CheckLevel(logLevel LevelType) bool { func (logger *Logger) CheckLevel(logLevel LevelType) bool {
settedLevel := logger.level settedLevel := logger.level
if settedLevel == 0 { if settedLevel == 0 {

26
meta.go
View File

@ -12,15 +12,12 @@ import (
// MetaField describes the serialization and visualization metadata for a single log field. // MetaField describes the serialization and visualization metadata for a single log field.
type MetaField struct { type MetaField struct {
Index int Index int `json:"index"`
Name string Name string `json:"name"`
KeyName string Color string `json:"color,omitempty"`
AttachBefore bool Format string `json:"format,omitempty"`
Color string WithoutKey bool `json:"withoutKey,omitempty"`
Format string Hide bool `json:"hide,omitempty"`
Precision int
WithoutKey bool
Hide bool
} }
var ( var (
@ -147,11 +144,6 @@ func extractMetaFields(model any) []MetaField {
if tag != "" { if tag != "" {
parts := strings.Split(tag, ",") parts := strings.Split(tag, ",")
for _, part := range parts { for _, part := range parts {
part = strings.TrimSpace(part)
if part == "attachBefore" {
meta.AttachBefore = true
continue
}
kv := strings.SplitN(part, ":", 2) kv := strings.SplitN(part, ":", 2)
if len(kv) == 2 { if len(kv) == 2 {
key := strings.TrimSpace(kv[0]) key := strings.TrimSpace(kv[0])
@ -165,12 +157,6 @@ func extractMetaFields(model any) []MetaField {
meta.WithoutKey = (val == "true") meta.WithoutKey = (val == "true")
case "hide": case "hide":
meta.Hide = (val == "true") meta.Hide = (val == "true")
case "keyname":
meta.KeyName = val
case "attachBefore":
meta.AttachBefore = (val == "true")
case "precision":
meta.Precision = cast.To[int](val)
} }
} }
} }

View File

@ -1,21 +0,0 @@
package log
import (
"testing"
)
func TestSetDefaultName(t *testing.T) {
oldName := getDefaultName()
defer SetDefaultName(oldName)
newName := "test-service-name"
SetDefaultName(newName)
if getDefaultName() != newName {
t.Errorf("GetDefaultName() = %v, want %v", getDefaultName(), newName)
}
if DefaultLogger.config.Name != newName {
t.Errorf("DefaultLogger.config.Name = %v, want %v", DefaultLogger.config.Name, newName)
}
}

View File

@ -33,8 +33,8 @@ func GetEntry[T any]() *T {
return entry return entry
} }
// putEntry 将日志对象归还到池中 // PutEntry 将日志对象归还到池中
func putEntry(entry any) { func PutEntry(entry any) {
t := reflect.TypeOf(entry) t := reflect.TypeOf(entry)
if pool, ok := globalPools.pools.Load(t); ok { if pool, ok := globalPools.pools.Load(t); ok {
pool.(*sync.Pool).Put(entry) pool.(*sync.Pool).Put(entry)
@ -44,7 +44,7 @@ func putEntry(entry any) {
// WithEntry 执行闭包并在结束后自动回收对象 // WithEntry 执行闭包并在结束后自动回收对象
func WithEntry[T any](fn func(*T)) { func WithEntry[T any](fn func(*T)) {
entry := GetEntry[T]() entry := GetEntry[T]()
defer putEntry(entry) defer PutEntry(entry)
fn(entry) fn(entry)
} }

View File

@ -28,7 +28,8 @@ func TestLoggerReliability(t *testing.T) {
} }
wg.Wait() wg.Wait()
WriterService.Stop(nil) Stop()
Wait()
file, err := os.Open(logFile) file, err := os.Open(logFile)
if err != nil { if err != nil {

View File

@ -90,8 +90,7 @@ func getAccessors(logType string, model any) []fieldAccessor {
return accessors return accessors
} }
// donot export this function func ToArrayBytes(entry LogEntry, sensitiveKeys []string) []byte {
func Marshal(entry LogEntry, sensitiveKeys []string) []byte {
var buf bytes.Buffer var buf bytes.Buffer
buf.WriteByte('[') buf.WriteByte('[')

View File

@ -59,7 +59,7 @@ func TestToArrayBytes(t *testing.T) {
RegisterType("mock_info_test", entry) // trigger meta generation RegisterType("mock_info_test", entry) // trigger meta generation
bytes := Marshal(entry, nil) bytes := ToArrayBytes(entry, nil)
str := string(bytes) str := string(bytes)
t.Logf("Raw log: %s", str) t.Logf("Raw log: %s", str)
@ -113,7 +113,7 @@ func TestToArrayBytes_Desensitize(t *testing.T) {
RegisterType("mock_info_test2", entry) RegisterType("mock_info_test2", entry)
bytes := Marshal(entry, []string{"password"}) bytes := ToArrayBytes(entry, []string{"password"})
str := string(bytes) str := string(bytes)
var arr []any var arr []any

View File

@ -14,7 +14,7 @@ const LogTypeMonitor = "monitor"
const LogTypeStatistic = "statistic" const LogTypeStatistic = "statistic"
const LogTypeRequest = "request" const LogTypeRequest = "request"
const LogDefaultSensitive = "phone,password,secret,token,accessToken,authorization" const LogDefaultSensitive = "phone,password,secret,token,accessToken"
const LogEnvLevel = "LOG_LEVEL" const LogEnvLevel = "LOG_LEVEL"
const LogEnvFile = "LOG_FILE" const LogEnvFile = "LOG_FILE"
const LogEnvSensitive = "LOG_SENSITIVE" const LogEnvSensitive = "LOG_SENSITIVE"
@ -30,9 +30,9 @@ type BaseLog struct {
LogName string `log:"pos:0,color:cyan,hide:true"` LogName string `log:"pos:0,color:cyan,hide:true"`
LogType string `log:"pos:1,color:magenta,hide:true"` LogType string `log:"pos:1,color:magenta,hide:true"`
LogTime int64 `log:"pos:2,format:time"` LogTime int64 `log:"pos:2,format:time"`
TraceId string `log:"pos:3,color:gray,withoutkey:true"` TraceId string `log:"pos:3,color:blue"`
Image string `log:"pos:4,hide:true"` Image string `log:"pos:4,color:darkGray,hide:true"`
Server string `log:"pos:5,hide:true"` Server string `log:"pos:5,color:darkGray,hide:true"`
Extra map[string]any `log:"pos:1000"` Extra map[string]any `log:"pos:1000"`
} }

View File

@ -1,6 +1,7 @@
package log package log
import ( import (
"encoding/json"
"fmt" "fmt"
"net" "net"
"os" "os"
@ -8,6 +9,9 @@ import (
"runtime" "runtime"
"runtime/debug" "runtime/debug"
"strings" "strings"
"time"
"apigo.cc/go/cast"
) )
var ( var (
@ -45,6 +49,103 @@ func init() {
} }
} }
// MakeTime 解析纳秒时间戳或 RFC3339 字符串
func MakeTime(v any) time.Time {
if ts, ok := cast.ToInt64E(v); ok == nil {
return time.Unix(0, ts)
}
tm, _ := time.Parse(time.RFC3339Nano, cast.String(v))
return tm
}
// MakeUsedTime 计算消耗时间(毫秒)
func MakeUsedTime(startTime, endTime time.Time) float32 {
return float32(endTime.UnixNano()-startTime.UnixNano()) / 1e6
}
// ParseBaseLog 解析基础日志行
func ParseBaseLog(line string) *BaseLog {
pos := strings.IndexByte(line, '{')
if pos == -1 {
return ParseBadLog(line)
}
l := make(map[string]any)
err := json.Unmarshal([]byte(line[pos:]), &l)
if err != nil {
return ParseBadLog(line)
}
baseLog := BaseLog{Extra: make(map[string]any)}
for k, v := range l {
lk := strings.ToLower(k)
switch lk {
case "logname":
baseLog.LogName = cast.String(v)
case "logtype":
baseLog.LogType = cast.String(v)
case "logtime":
baseLog.LogTime = cast.Int64(v)
case "traceid":
baseLog.TraceId = cast.String(v)
case "imagename":
if baseLog.Image != "" {
baseLog.Image = cast.String(v) + ":" + baseLog.Image
} else {
baseLog.Image = cast.String(v)
}
case "imagetag":
if baseLog.Image != "" {
baseLog.Image = baseLog.Image + ":" + cast.String(v)
} else {
baseLog.Image = cast.String(v)
}
case "servername":
if baseLog.Server != "" {
baseLog.Server = cast.String(v) + ":" + baseLog.Server
} else {
baseLog.Server = cast.String(v)
}
case "serverip":
if baseLog.Server != "" {
baseLog.Server = baseLog.Server + ":" + cast.String(v)
} else {
baseLog.Server = cast.String(v)
}
default:
baseLog.Extra[lk] = v
}
}
return &baseLog
}
// ParseBadLog 解析非 JSON 格式的日志
func ParseBadLog(line string) *BaseLog {
baseLog := BaseLog{Extra: make(map[string]any)}
baseLog.LogType = LogTypeUndefined
if len(line) > 19 && line[19] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05", line[0:19])
if err == nil {
baseLog.LogTime = tm.UnixNano()
line = line[20:]
} else {
return nil
}
} else if len(line) > 26 && line[26] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05.000000", line[0:26])
if err == nil {
baseLog.LogTime = tm.UnixNano()
line = line[27:]
} else {
return nil
}
} else {
return nil
}
baseLog.Extra["info"] = line
return &baseLog
}
// fixField 格式化字段名(去横线、下划线,小写) // fixField 格式化字段名(去横线、下划线,小写)
func fixField(s string) string { func fixField(s string) string {
s = strings.ReplaceAll(s, "-", "") s = strings.ReplaceAll(s, "-", "")
@ -69,7 +170,6 @@ func getCallStacks(truncations []string) []string {
isLogInternal := (strings.Contains(file, "/log/logger.go") || isLogInternal := (strings.Contains(file, "/log/logger.go") ||
strings.Contains(file, "/log/utility.go") || strings.Contains(file, "/log/utility.go") ||
strings.Contains(file, "/log/standard.go") || strings.Contains(file, "/log/standard.go") ||
strings.Contains(file, "/log/default_logger.go") ||
strings.Contains(file, "/log/extra.go")) strings.Contains(file, "/log/extra.go"))
if isLogInternal { if isLogInternal {
@ -92,21 +192,19 @@ func getCallStacks(truncations []string) []string {
return callStacks return callStacks
} }
var globalDefaultName string // GetDefaultName 获取默认应用名称
func GetDefaultName() string {
// getDefaultName 获取默认应用名称 name := os.Getenv("DISCOVER_APP")
func getDefaultName() string { if name == "" {
if globalDefaultName != "" { name = os.Getenv("discover_app")
return globalDefaultName
} }
name := "" if name == "" {
if info, ok := debug.ReadBuildInfo(); ok && info.Path != "" && info.Path != "command-line-arguments" { if info, ok := debug.ReadBuildInfo(); ok && info.Path != "" && info.Path != "command-line-arguments" {
name = path.Base(info.Path) name = path.Base(info.Path)
} }
}
if name == "" { if name == "" {
name = path.Base(os.Args[0]) name = path.Base(os.Args[0])
} }
// 处理 Windows 下的 .exe 后缀
name = strings.TrimSuffix(name, ".exe")
return name return name
} }

131
viewer.go
View File

@ -2,9 +2,7 @@ package log
import ( import (
"fmt" "fmt"
"os"
"regexp" "regexp"
"sort"
"strings" "strings"
"time" "time"
@ -14,7 +12,6 @@ import (
var errorLineMatcher = regexp.MustCompile(`(\w+\.go:\d+)`) var errorLineMatcher = regexp.MustCompile(`(\w+\.go:\d+)`)
var codeFileMatcher = regexp.MustCompile(`(\w+?\.)(go|js)`) var codeFileMatcher = regexp.MustCompile(`(\w+?\.)(go|js)`)
var workspaceRoot, _ = os.Getwd()
func Viewable(line string) string { func Viewable(line string) string {
line = strings.TrimSpace(line) line = strings.TrimSpace(line)
@ -79,16 +76,17 @@ func Viewable(line string) string {
if m.Name == "Extra" { if m.Name == "Extra" {
extraMap, ok := v.(map[string]any) extraMap, ok := v.(map[string]any)
if ok && len(extraMap) > 0 { if ok && len(extraMap) > 0 {
keys := make([]string, 0, len(extraMap)) for k, ev := range extraMap {
for k := range extraMap {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
ev := extraMap[k]
builder.WriteString(" ") builder.WriteString(" ")
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":")) builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":"))
builder.WriteString(renderValue(ev, 0, "")) vStr := ""
switch ev.(type) {
case map[string]any, []any:
vStr, _ = cast.ToJSON(ev)
default:
vStr = cast.String(ev)
}
builder.WriteString(vStr)
} }
} }
continue continue
@ -97,18 +95,9 @@ func Viewable(line string) string {
if m.Name == "CallStacks" { if m.Name == "CallStacks" {
callStacksList, ok := v.([]any) callStacksList, ok := v.([]any)
if ok && len(callStacksList) > 0 { if ok && len(callStacksList) > 0 {
stackColor := shell.TextRed
if strings.Contains(strings.ToLower(logType), "warn") {
stackColor = shell.TextYellow
}
builder.WriteString("\n") builder.WriteString("\n")
for _, vi := range callStacksList { for _, vi := range callStacksList {
vStr := cast.String(vi) vStr := cast.String(vi)
if workspaceRoot != "" {
vStr = strings.TrimPrefix(vStr, workspaceRoot)
vStr = strings.TrimPrefix(vStr, "/")
}
postfix := "" postfix := ""
if pos := strings.LastIndexByte(vStr, '/'); pos != -1 { if pos := strings.LastIndexByte(vStr, '/'); pos != -1 {
postfix = vStr[pos+1:] postfix = vStr[pos+1:]
@ -119,7 +108,7 @@ func Viewable(line string) string {
} }
builder.WriteString(" ") builder.WriteString(" ")
builder.WriteString(shell.Style(shell.Dim, vStr)) builder.WriteString(shell.Style(shell.Dim, vStr))
builder.WriteString(shell.Style(stackColor, postfix)) builder.WriteString(shell.Style(shell.TextWhite, postfix))
builder.WriteString("\n") builder.WriteString("\n")
} }
} }
@ -131,42 +120,28 @@ func Viewable(line string) string {
if m.Format == "time" { if m.Format == "time" {
// Convert int64 ns to time string // Convert int64 ns to time string
logTime := time.Unix(0, cast.Int64(v)) logTime := time.Unix(0, cast.Int64(v))
dateStr := logTime.Format("01-02") vStr = logTime.Format("01-02 15:04:05.000")
timeStr := logTime.Format("15:04:05") if m.Color == "" {
milliStr := logTime.Format(".000") builder.WriteString(shell.White(shell.Bold, vStr))
if builder.Len() > 0 {
builder.WriteString(" ") builder.WriteString(" ")
}
builder.WriteString(shell.Style(shell.Dim, dateStr))
builder.WriteString(" ")
builder.WriteString(shell.White(shell.Bold, timeStr))
builder.WriteString(shell.Style(shell.Dim, milliStr))
continue continue
}
} else { } else {
vStr = renderValue(v, m.Precision, m.Color) vStr = cast.String(v)
if vStr == "" { if vStr == "" {
continue continue
} }
} }
if builder.Len() > 0 { if builder.Len() > 0 {
if m.AttachBefore {
builder.WriteString(":")
} else {
builder.WriteString(" ") builder.WriteString(" ")
} }
if !m.WithoutKey {
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, m.Name+":"))
} }
if !m.WithoutKey && !m.AttachBefore { builder.WriteString(applyColor(vStr, m.Color))
name := m.KeyName
if name == "" {
name = m.Name
}
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, name+":"))
}
builder.WriteString(vStr)
} }
return builder.String() return builder.String()
@ -209,12 +184,6 @@ func ToJSON(line string) string {
if m.Name == "" { if m.Name == "" {
continue continue
} }
name := m.KeyName
if name == "" {
name = m.Name
}
if m.Name == "Extra" { if m.Name == "Extra" {
if extraMap, ok := v.(map[string]any); ok { if extraMap, ok := v.(map[string]any); ok {
for k, ev := range extraMap { for k, ev := range extraMap {
@ -222,7 +191,7 @@ func ToJSON(line string) string {
} }
} }
} else { } else {
result[name] = v result[m.Name] = v
} }
} else if cast.String(v) != "0" { } else if cast.String(v) != "0" {
result[fmt.Sprintf("Extra%d", i)] = v result[fmt.Sprintf("Extra%d", i)] = v
@ -247,7 +216,7 @@ func applyColor(text string, color string) string {
return shell.Yellow(text) return shell.Yellow(text)
case "green": case "green":
return shell.Green(text) return shell.Green(text)
case "gray": case "gray", "darkGray":
return shell.Style(shell.Dim, text) return shell.Style(shell.Dim, text)
default: default:
return text return text
@ -264,61 +233,3 @@ func fallbackRenderArray(arr []any) string {
} }
return builder.String() return builder.String()
} }
func renderValue(v any, precision int, color string) string {
if v == nil {
return ""
}
switch val := v.(type) {
case float32, float64:
vStr := ""
if precision > 0 {
vStr = fmt.Sprintf("%.*f", precision, cast.To[float64](v))
} else {
vStr = cast.String(v)
}
return applyColor(vStr, color)
case map[string]any:
if len(val) == 0 {
return ""
}
var parts []string
keys := make([]string, 0, len(val))
for k := range val {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
// Key is always dim, value is colored
vStr := renderValue(val[k], precision, color)
if vStr != "" {
parts = append(parts, fmt.Sprintf("%s:%s", shell.Style(shell.Dim, k), vStr))
}
}
if len(parts) == 0 {
return ""
}
return "[ " + strings.Join(parts, " ") + " ]"
case []any:
if len(val) == 0 {
return ""
}
var parts []string
for _, iv := range val {
vStr := renderValue(iv, precision, color)
if vStr != "" {
parts = append(parts, vStr)
}
}
if len(parts) == 0 {
return ""
}
return "[ " + strings.Join(parts, " ") + " ]"
default:
s := cast.String(v)
if s == "" {
return ""
}
return applyColor(s, color)
}
}

View File

@ -20,7 +20,7 @@ func TestViewable(t *testing.T) {
} }
log.RegisterType("info", entry) log.RegisterType("info", entry)
line := string(log.Marshal(entry, nil)) line := string(log.ToArrayBytes(entry, nil))
out := log.Viewable(line) out := log.Viewable(line)
if !strings.Contains(out, "hello world") { if !strings.Contains(out, "hello world") {
@ -44,7 +44,7 @@ func TestToJSON(t *testing.T) {
entry.Extra = map[string]any{"key": "value"} entry.Extra = map[string]any{"key": "value"}
log.RegisterType("info", entry) log.RegisterType("info", entry)
line := string(log.Marshal(entry, nil)) line := string(log.ToArrayBytes(entry, nil))
jsonStr := log.ToJSON(line) jsonStr := log.ToJSON(line)
if !strings.Contains(jsonStr, `"Info":"hello world"`) { if !strings.Contains(jsonStr, `"Info":"hello world"`) {
@ -77,154 +77,3 @@ func TestLoadMeta(t *testing.T) {
t.Errorf("expected Field1, got %s", meta[0].Name) t.Errorf("expected Field1, got %s", meta[0].Name)
} }
} }
type EnhancedLog struct {
log.BaseLog
App string `log:"pos:10,withoutkey:true"`
Node string `log:"pos:11,attachBefore,withoutkey:true"`
RequestHeaders map[string]string `log:"pos:13,keyname:reqH"`
ClientIP string `log:"pos:12,keyname:ip"`
Tags []string `log:"pos:14"`
}
func (l *EnhancedLog) Reset() {
l.BaseLog.Reset()
l.App = ""
l.Node = ""
l.ClientIP = ""
l.RequestHeaders = nil
l.Tags = nil
}
func TestEnhancedViewable(t *testing.T) {
entry := &EnhancedLog{
BaseLog: log.BaseLog{
LogType: "enhanced",
LogTime: 1714896000000000000,
},
App: "MyApp",
Node: "Node1",
ClientIP: "127.0.0.1",
RequestHeaders: map[string]string{
"User-Agent": "Go-http-cli",
},
Tags: []string{"tag1", "tag2"},
}
log.RegisterType("enhanced", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
// Check attachBefore: MyApp:Node1 (since both are withoutkey)
if !strings.Contains(out, "MyApp:Node1") {
t.Errorf("expected MyApp:Node1, got: %s", out)
}
// Check pos ordering and keyname: ip:127.0.0.1 should come before reqH
if !strings.Contains(out, "ip:") || !strings.Contains(out, "127.0.0.1") {
t.Errorf("expected ip:127.0.0.1, got: %s", out)
}
if !strings.Contains(out, "reqH:") || !strings.Contains(out, "User-Agent") || !strings.Contains(out, "Go-http-cli") {
t.Errorf("expected reqH:[ User-Agent:Go-http-cli ], got: %s", out)
}
ipIdx := strings.Index(out, "ip:")
reqHIdx := strings.Index(out, "reqH:")
if ipIdx > reqHIdx {
t.Errorf("expected ip to come before reqH, but ipIdx=%d, reqHIdx=%d", ipIdx, reqHIdx)
}
// Check array rendering: Tags:[ tag1 tag2 ]
if !strings.Contains(out, "Tags:") || !strings.Contains(out, "[ tag1 tag2 ]") {
t.Errorf("expected Tags:[ tag1 tag2 ], got: %s", out)
}
}
func TestEnhancedToJSON(t *testing.T) {
entry := &EnhancedLog{
BaseLog: log.BaseLog{
LogType: "enhanced",
LogTime: 1714896000000000000,
},
App: "MyApp",
Node: "Node1",
ClientIP: "127.0.0.1",
RequestHeaders: map[string]string{
"User-Agent": "Go-http-cli",
},
}
log.RegisterType("enhanced", entry)
line := string(log.Marshal(entry, nil))
jsonStr := log.ToJSON(line)
// Check keyname in JSON
if !strings.Contains(jsonStr, `"ip":"127.0.0.1"`) {
t.Errorf("expected ip field in JSON, got: %s", jsonStr)
}
if !strings.Contains(jsonStr, `"reqH":{"User-Agent":"Go-http-cli"}`) {
t.Errorf("expected reqH field in JSON, got: %s", jsonStr)
}
}
type CallStackLog struct {
log.BaseLog
CallStacks []string `log:"pos:6"`
}
func (l *CallStackLog) Reset() {
l.BaseLog.Reset()
l.CallStacks = nil
}
func TestCallStacksViewable(t *testing.T) {
wd, _ := os.Getwd()
entry := &CallStackLog{
BaseLog: log.BaseLog{
LogType: "test_error",
LogTime: 1714896000000000000,
},
CallStacks: []string{wd + "/main.go:10", "/usr/local/go/src/runtime/panic.go:100"},
}
log.RegisterType("test_error", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
// Check path truncation (should contain relative "main.go:10")
if !strings.Contains(out, "main.go:10") {
t.Errorf("expected relative path main.go:10, got: %s", out)
}
// Absolute path should be removed if it matches wd
if strings.Contains(out, wd) {
t.Errorf("absolute path should be truncated, but still found: %s", out)
}
}
type PrecisionLog struct {
log.BaseLog
Value float64 `log:"pos:6,precision:2"`
}
func (l *PrecisionLog) Reset() {
l.BaseLog.Reset()
l.Value = 0
}
func TestPrecisionViewable(t *testing.T) {
entry := &PrecisionLog{
BaseLog: log.BaseLog{
LogType: "precision",
LogTime: 1714896000000000000,
},
Value: 3.14159,
}
log.RegisterType("precision", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
if !strings.Contains(out, "3.14") || strings.Contains(out, "3.141") {
t.Errorf("expected 3.14 (precision 2), got: %s", out)
}
}

109
writer.go
View File

@ -1,7 +1,6 @@
package log package log
import ( import (
"context"
"fmt" "fmt"
"sync" "sync"
"sync/atomic" "sync/atomic"
@ -22,22 +21,13 @@ type logPayload struct {
file *FileWriter // 目标文件 Writer file *FileWriter // 目标文件 Writer
} }
// writerService manages the background writing of log entries.
type writerService struct {
Running atomic.Bool
StopChan chan bool
LogChannel chan logPayload
Dropped atomic.Uint64
Writers atomic.Value // []Writer
WriterLock sync.Mutex
Files map[string]*FileWriter
FilesLock sync.RWMutex
}
var ( var (
// WriterService is the global instance of defaultService. writerRunning atomic.Bool
WriterService = &writerService{} writerLock sync.Mutex // 仅用于注册时锁定
writerStopChan chan bool
writers atomic.Value // 存储 []Writer
logChannel chan logPayload
droppedLogs atomic.Uint64
) )
// ConsoleWriter 控制台写入器 // ConsoleWriter 控制台写入器
@ -52,28 +42,26 @@ func (w *ConsoleWriter) Run() {
} }
func init() { func init() {
WriterService.LogChannel = make(chan logPayload, 10000) logChannel = make(chan logPayload, 10000)
WriterService.Writers.Store([]Writer{}) writers.Store([]Writer{})
WriterService.Files = make(map[string]*FileWriter)
RegisterWriterMaker("console", func(conf *Config) Writer { RegisterWriterMaker("console", func(conf *Config) Writer {
return &ConsoleWriter{} return &ConsoleWriter{}
}) })
} }
// writeAsync 异步写入日志 // WriteAsync 异步写入日志
func writeAsync(payload logPayload) { func WriteAsync(payload logPayload) {
defer func() { defer func() {
recover() recover()
}() }()
if !WriterService.Running.Load() { if !writerRunning.Load() {
return return
} }
select { select {
case WriterService.LogChannel <- payload: case logChannel <- payload:
default: default:
// 丢弃或处理过载 // 丢弃或处理过载
dropped := WriterService.Dropped.Add(1) dropped := droppedLogs.Add(1)
if dropped%1000 == 1 { if dropped%1000 == 1 {
if DefaultLogger != nil { if DefaultLogger != nil {
// 注意:这里可能会产生递归调用,但 select default 保证了不会死锁 // 注意:这里可能会产生递归调用,但 select default 保证了不会死锁
@ -85,79 +73,74 @@ func writeAsync(payload logPayload) {
// GetDroppedLogs 获取被丢弃的日志数量 // GetDroppedLogs 获取被丢弃的日志数量
func GetDroppedLogs() uint64 { func GetDroppedLogs() uint64 {
return WriterService.Dropped.Load() return droppedLogs.Load()
} }
// Start implements starter.Service interface. // Start 启动写入器
func (s *writerService) Start(_ context.Context, _ *Logger) error { func Start() {
if !s.Running.CompareAndSwap(false, true) { if !writerRunning.CompareAndSwap(false, true) {
return nil return
} }
s.StopChan = make(chan bool) writerStopChan = make(chan bool)
go s.writerRunner() go writerRunner()
return nil
} }
// Stop implements starter.Service interface. // Stop 停止写入器
func (s *writerService) Stop(_ context.Context) error { func Stop() {
if s.Running.CompareAndSwap(true, false) { if writerRunning.CompareAndSwap(true, false) {
close(s.LogChannel) close(logChannel)
if s.StopChan != nil {
<-s.StopChan
s.StopChan = nil
} }
} }
return nil
}
// Status implements starter.Service interface. // Wait 等待写入器停止
func (s *writerService) Status() (string, error) { func Wait() {
if s.Running.Load() { if writerStopChan != nil {
return fmt.Sprintf("queue: %d, dropped: %d", len(s.LogChannel), s.Dropped.Load()), nil <-writerStopChan
writerStopChan = nil
} }
return "stopped", nil
} }
func (s *writerService) writerRunner() {
func writerRunner() {
ticker := time.NewTicker(200 * time.Millisecond) ticker := time.NewTicker(200 * time.Millisecond)
defer ticker.Stop() defer ticker.Stop()
defer func() { defer func() {
if s.StopChan != nil { if writerStopChan != nil {
close(s.StopChan) close(writerStopChan)
} }
}() }()
for { for {
select { select {
case payload, ok := <-s.LogChannel: case payload, ok := <-logChannel:
if !ok { if !ok {
s.flushWriters() flushWriters()
return return
} }
s.processLog(payload) processLog(payload)
// 尝试批量处理更多日志 // 尝试批量处理更多日志
batchCount := 0 batchCount := 0
for batchCount < 100 { for batchCount < 100 {
select { select {
case nextPayload, nextOk := <-s.LogChannel: case nextPayload, nextOk := <-logChannel:
if !nextOk { if !nextOk {
s.flushWriters() flushWriters()
return return
} }
s.processLog(nextPayload) processLog(nextPayload)
batchCount++ batchCount++
default: default:
batchCount = 100 // break outer loop batchCount = 100 // break outer loop
} }
} }
case <-ticker.C: case <-ticker.C:
s.flushWriters() flushWriters()
} }
} }
} }
func (s *writerService) processLog(payload logPayload) { func processLog(payload logPayload) {
// 精准路由:根据包裹信息决定写入目标 // 精准路由:根据包裹信息决定写入目标
if payload.writer != nil { if payload.writer != nil {
payload.writer.Log(payload.entry, payload.buf) payload.writer.Log(payload.entry, payload.buf)
@ -166,15 +149,15 @@ func (s *writerService) processLog(payload logPayload) {
} }
} }
func (s *writerService) flushWriters() { func flushWriters() {
curWriters, _ := s.Writers.Load().([]Writer) curWriters, _ := writers.Load().([]Writer)
for _, w := range curWriters { for _, w := range curWriters {
w.Run() w.Run()
} }
s.FilesLock.RLock() filesLock.RLock()
for _, f := range s.Files { for _, f := range files {
f.Run() f.Run()
} }
s.FilesLock.RUnlock() filesLock.RUnlock()
} }