Compare commits

..

25 Commits
v1.1.1 ... main

Author SHA1 Message Date
AI Engineer
1870d84974 chore: infrastructure alignment and doc sync (by AICoder) 2026-05-16 01:27:48 +08:00
AI Engineer
6bc0aa4c0a chore: infrastructure alignment and doc sync (by checkall) 2026-05-14 21:51:48 +08:00
AI Engineer
3d0fc5e93f feat: align with new starter.Service interface, reporting queue and drop status (by AI) 2026-05-13 11:11:12 +08:00
AI Engineer
1f816dc8b3 feat: add log.As and logger.As for frictionless error handling 2026-05-13 01:07:42 +08:00
AI Engineer
a2b1055f5d rename 2026-05-13 00:29:17 +08:00
AI Engineer
13418b5365 docs: update README, CHANGELOG, TEST and align with LoggerService architecture 2026-05-12 23:04:00 +08:00
AI Engineer
c1e1ecc3b0 对齐 Tag v1.3.0 (By AI) 2026-05-10 15:48:33 +08:00
AI Engineer
1e5dbd2a23 chore: final infrastructure alignment 2026-05-10 13:12:21 +08:00
AI Engineer
96c910bb14 chore: infrastructure alignment 2026-05-10 13:04:29 +08:00
AI Engineer
9459a98224 feat: align GetDefaultName with service SOP (by AI) 2026-05-10 10:50:00 +08:00
AI Engineer
6dfb33c459 feat: 增加 authorization 默认脱敏规则(by AI) 2026-05-09 21:01:08 +08:00
AI Engineer
ec8406fe42 feat(log): 调整可视化能力 2026-05-09 16:30:01 +08:00
AI Engineer
03267710dc feat(log): 绝对索引优化与强制 Reset 安全契约 (v1.1.13) 2026-05-09 14:44:41 +08:00
AI Engineer
78d6addf4c 稳定性增强:修复 TestSplitTag 秒级边界竞态,并完成 go/cast 与 go/file 基础设施对齐(by AI) 2026-05-05 23:47:07 +08:00
AI Engineer
9c6112d253 Verify SplitTag rotation with seconds in tests (by AI) 2026-05-05 23:32:43 +08:00
AI Engineer
534d3dfdd6 Implement deep desensitization using cast for complex types (by AI) 2026-05-05 23:25:17 +08:00
AI Engineer
9252fe002e Cleanup dead code, fix desensitization, and add functional tests (by AI) 2026-05-05 23:09:46 +08:00
AI Engineer
c988b8d88b Optimize FillBase to take *BaseLog and update documentation (by AI) 2026-05-05 22:52:55 +08:00
AI Engineer
00a677492d chore: align dependencies 2026-05-05 22:03:57 +08:00
AI Engineer
0fe4a51407 chore: update dependencies 2026-05-05 21:58:02 +08:00
AI Engineer
be893e1b99 feat: 实现高性能 Meta 驱动的 JSON 数组日志架构 (by AI) 2026-05-05 21:45:43 +08:00
AI Engineer
8a44c1ace6 temp version for new plan 2026-05-05 20:35:27 +08:00
AI Engineer
b2f91d37be temp version for new plan 2026-05-05 20:35:15 +08:00
AI Engineer
a720cfb63b 自动化增强:引入 StackTraceable 接口,Log 方法支持自动补全调用栈 (by AI) 2026-05-05 18:10:17 +08:00
AI Engineer
f9a10486e3 架构解耦:移除内置 DB 日志支持,回归私有填充逻辑,导出 GetCallStacks (by AI) 2026-05-05 17:59:43 +08:00
30 changed files with 2308 additions and 847 deletions

9
.gitignore vendored Normal file
View File

@ -0,0 +1,9 @@
.log.meta.json
.test.meta.json
.ai/
.geminiignore
.gemini
env.json
env.yml
env.yaml
/CODE-FULL.md

View File

@ -1,6 +1,107 @@
# Changelog # Changelog
## [1.1.1] - 2026-05-05 ## [1.3.2] - 2026-05-13
- **功能增强: 引入摩擦消除工具 `As`**:
- **泛型支持**: 新增全局泛型函数 `log.As[T](v T, err error) T`,仿照 `cast.As` 设计,自动记录错误并返回零值,极大简化了带 error 返回值的函数链式调用。
- **Logger 扩展**: `Logger` 结构体新增 `As(v any, err error) any` 方法,支持实例级别的错误捕获与自动记录。
- **调用栈优化**: 优化了 `GetCallStacks` 逻辑,自动跳过 `default_logger.go` 中的内部帧,确保 `log.As` 记录的错误位置精准指向业务代码。
## [1.3.1] - 2026-05-12
- **架构升级: 引入 LoggerService**:
- **解耦重构**: 重构全局变量管理,引入 `loggerService` 结构体集中化管理异步写入协程、Writers 对象池、文件句柄与丢弃计数。
- **生命周期管理**: 实现了 `Start(ctx, logger)`, `Stop(ctx)`, `Health()` 接口,完美支持与 `apigo.cc/go/starter` 的基础设施集成。
- **安全性增强**: 优化了平滑停止逻辑,确保 `Stop` 调用时能完整 Flush 缓冲区数据。
- **功能增强**:
- **动态应用名**: 新增 `SetDefaultName(name)` 全局方法,支持在微服务启动后动态设置应用名称,并自动同步至 `DefaultLogger`
- **配置与忽略规则**:
- 更新 `.gitignore`,增加了对 `.gemini`, `env.json`, `env.yml` 以及 `/CODE-FULL.md` 的忽略支持。
- **文档与测试同步**: 全面更新了 `README.md`, `CHANGELOG.md``TEST.md`
## [1.3.0] - 2026-05-10
- **全栈基础设施对齐**:
- 将所有内部依赖(`cast`, `config`, `file`, `id`, `shell` 等)统一升级至 `v1.3.0` 语义版本。
- 优化项目忽略规则,在 `.gitignore` 中增加了对 `.geminiignore` 的支持。
## [1.1.16] - 2026-05-10
- **依赖对齐**: 同步更新 `golang.org/x/crypto``golang.org/x/sys` 至最新版本,确保底层安全性。
## [1.1.15] - 2026-05-10
- **安全与性能双重增强**:
- **敏感字段扩展**: `authorization` 现在默认纳入脱敏规则 (`LogDefaultSensitive`),自动过滤 HTTP 认证凭证。
- **TraceId 吞吐优化**: 切换至 `id.Get10Bytes14MPerSecond()` 算法,在保持 10 字节唯一性的同时,极大提升了超高并发下的 ID 生成性能。
- **应用名称识别对齐**: 重构 `GetDefaultName` 逻辑,优先通过 `debug.ReadBuildInfo()` 识别 Module 路径,并支持自动裁剪 Windows 环境下的 `.exe` 后缀。
## [1.1.14] - 2026-05-09
- **可视化能力调整**: 调整优化了 `viewer` 模块相关的可视化能力,提升了日志的可读性与调试体验。
## [1.1.13] - 2026-05-09
- **绝对索引优化与零空洞**:
- 彻底消除 `BaseLog` 与业务字段之间的索引空洞。字段位置调整为:`BaseLog` (0-5),标准消息字段 (`Info`, `Error` 等) 与业务日志字段从 `pos: 6` 起始。
- **智能平移映射**: 改进 `pos >= 1000` 的逻辑。这些字段(如 `Extra`, `CallStacks`)不再产生稀疏 0 占位符,而是根据定义的 `pos` 顺序,自动平移并紧跟在当前类型的最大绝对索引之后。既保证了“永远在末尾”,又维持了数组的紧凑性。
- **强制 Reset 安全契约**:
- `RegisterType` 引入严苛校验:若自定义日志类型包含字段但未显式重写 `Reset()`(即仅继承了 `BaseLog.Reset`),注册时将触发 **Panic**。强制开发者显式清理业务数据,杜绝对象池复用时的脏数据隐患。
- **应用名称识别增强**:
- `GetDefaultName` 引入 `runtime/debug.ReadBuildInfo()`。相比传统的文件夹路径读取,能更精准地识别 Go Module 定义的应用名称。
- **工具链增强**:
- `serializer_test.go` 新增原始日志输出,方便开发者通过 `go test -v` 直观校验 JSON 数组结构。
## [1.1.12] - 2026-05-09
...
- **稳定性增强**:
- 修复 `TestSplitTag` 在秒级进位边界时的偶发性失败。优化后的测试逻辑同时校验日志产生前后的两个潜在时间槽,闭环消除了环境抖动导致的 race condition。
## [1.1.9] - 2026-05-05
- **稳定性增强**:
- 改进 `SplitTag` 测试用例,通过秒级切分步进,闭环验证了 `FileWriter` 的文件自动轮转逻辑,确保在高频或定时切分场景下的可靠性。
## [1.1.8] - 2026-05-05
- **深度脱敏与性能平衡**:
- 引入“混合路径”序列化策略对基础类型String, Int, Bool 等采用高性能手动编码对复杂类型Struct, Map, Slice采用 `cast.ToJSONDesensitizeBytes`
- 完美解决嵌套对象、数组内对象的深度脱敏问题,同时将序列化性能维持在极高水平。
- **基准测试优化**: 相比全量 `cast` 调用,混合路径将异步写入耗时降低了 60% 以上,内存分配减少了 70%。
## [1.1.7] - 2026-05-05
- **极致精简与摩擦消除**:
- 移除了所有冗余或已弃用的配置项:`Fast`, `KeepKeyCase`, `RegexSensitive`, `SensitiveRule`
- 删除了 `Logger` 中未被使用的 `desensitization` 处理函数及 `SetDesensitization` 方法,进一步收窄 API。
- **功能增强与修复**:
- **脱敏逻辑闭环**: 修复了 `ToArrayBytes` 仅对 Map 字段脱敏的缺陷。现在支持对结构体根层级的敏感字段(如 `String` 类型)进行全量遮蔽。
- **标准化命名映射**: 增强 `fixField` 逻辑,同时支持自动忽略下划线 (`_`) 和横线 (`-`),确保 `SecretKey` (Go) 与 `secret_key` (Config) 能完美匹配。
- **稳定性保障**:
- 新增 `functional_test.go`,闭环验证了 `SplitTag` 文件切分能力与增强后的 `Sensitive` 脱敏逻辑。
- 优化 `serializer.go` 内部判断逻辑,基准测试显示性能有显著提升。
## [1.1.6] - 2026-05-05
- **性能优化**:
- 重构 `FillBase` 方法签名,由接收接口 `LogEntry` 改为接收指针 `*BaseLog`
- 此项改动消除了在填充元数据时通过接口调用 `GetBaseLog()` 的开销,直接操作结构体指针,进一步提升日志预处理性能。
- **文档与示例对齐**:
- 完善 `README.md` 中的 `Config` 配置项说明,涵盖 `Fast`, `KeepKeyCase`, `Truncations`, `SensitiveRule` 等所有字段。
- 修正 `BusinessLog` 扩展示例,确保与最新的 `FillBase` 签名及元数据填充逻辑保持一致。
- **扩展与迁移指引**: 同步更新 `extra.go` 中的注释示例,为从旧版本或其他项目迁移提供准确的参考实现。
## [1.1.5] - 2026-05-05
- **高性能 Meta 驱动架构**:
- 日志存储格式由 JSON Object 彻底切换为 **JSON Positional Array (`[...]`)**,通过位置索引消除重复 Key 的存储与传输开销。
- 实现基于反射的 **零装箱 (No-Boxing) 序列化**,直接拼接 JSON 字符串,大幅降低内存分配与 CPU 占用。
- **元数据外置与可视化**:
- 引入 `.log.meta.json` 机制,将字段顺序、颜色、格式化等可视化逻辑从核心包中剥离。
- 新增独立 CLI 工具 `logv`,支持基于元数据文件的 stdin 流式日志渲染。
- 改造 `Viewable` 接口,支持基于 MetaRegistry 的动态终端彩色输出渲染。
- **字段优化与压缩**:
- 合并 `ImageName/Tag``Image`,合并 `ServerName/Ip``Server`,精简日志槽位。
- 引入 `hide:true` 标签,支持在控制台隐藏结构性元数据(如 `LogName`, `LogType`),保持输出极致纯净。
- **架构兼容性**:
- 调整 `Writer` 接口,支持自定义 Writer`ESWriter`)自行决定序列化格式(如转回 Object 以适配 ES 索引)。
- `utility.go` 新增对旧版 JSON 对象日志的解析兼容。
- **安全加固**: 为 `http`, `db`, `redis`, `discover` 等子包同步添加 `.gitignore` 以排除自动生成的 `.log.meta.json`
## [1.1.2] - 2026-05-05
- **架构解耦**:
- 正式移除 `log` 包对数据库日志(`DB` 方法及 `DBLog` 结构)的内置支持,推动“日志格式随业务走”的架构对齐。
- 导出 `GetCallStacks` 方法,支持外部包捕获符合截断配置的调用栈。
- **内部优化**:
- `fillBase` 回归私有,强化 `Log(entry)` 的自动填充契约,降低外部摩擦。
- **体验对齐**: - **体验对齐**:
- 修复 `viewer.go` 中空错误字符串导致日志类型被跳过显示的 Bug。 - 修复 `viewer.go` 中空错误字符串导致日志类型被跳过显示的 Bug。
- `Log` 方法集成 `fillBase` 自动补全逻辑,对于直接调用 `Log(entry)` 的自定义日志类型,若未设置时间戳则自动填充元数据,无需手动干预。 - `Log` 方法集成 `fillBase` 自动补全逻辑,对于直接调用 `Log(entry)` 的自定义日志类型,若未设置时间戳则自动填充元数据,无需手动干预。

167
README.md
View File

@ -6,9 +6,10 @@
`@go/log` 旨在提供高性能、零摩擦的异步日志系统。其核心目标是: `@go/log` 旨在提供高性能、零摩擦的异步日志系统。其核心目标是:
* **极致高性能**:采用 **Meta-Driven Positional Array (元数据驱动定长数组)** 架构。日志以单行 JSON 数组 (`[...]`) 形式落盘,消除 Key 冗余与装箱开销,性能提升数倍。
* **架构解耦**:元数据外置于 `.log.meta.json`。日志包仅负责高速序列化,可视化由外部工具或 `Viewable` 接口根据元数据动态渲染。
* **零摩擦入口**自动识别环境上下文应用名、IP等无需手动构建。 * **零摩擦入口**自动识别环境上下文应用名、IP等无需手动构建。
* **极致高性能**:异步写入架构,支持对象池复用,大幅降低内存分配。 * **语义脱敏**:内置敏感信息(如手机号、密钥)的自动脱敏。
* **语义脱敏**:内置敏感信息(如手机号、密钥)的自动脱敏与正则过滤。
* **高度可扩展**支持多种写入渠道文件切分、Elasticsearch批量传输 * **高度可扩展**支持多种写入渠道文件切分、Elasticsearch批量传输
## 📦 安装 ## 📦 安装
@ -22,14 +23,65 @@ go get apigo.cc/go/log
```go ```go
import "apigo.cc/go/log" import "apigo.cc/go/log"
// 使用默认配置初始化 (或在配置中指定) // 默认 logger (通过 log.json 或环境变量配置)
logger := log.NewLogger(log.Config{Name: "my-app", Level: "info"}) func main() {
// 在微服务场景下动态设置应用名称
log.SetDefaultName("my-microservice")
// 记录业务日志 (自动通过 cast.ToMap 处理变长参数) log.Info("服务启动", "port", 8080)
logger.Info("用户登录", "userId", 10086, "ip", "1.2.3.4") log.Error("数据库连接失败", "db", "mysql")
logger.Error("数据库连接失败", "db", "mysql", "err", err)
// 创建带 traceId 的新 logger 实例
logger := log.New("trace-xyz-123")
}
``` ```
## ⚙️ 配置 (Configuration)
本包深度集成 `@go/config`,支持多种灵活的配置方式,优先级从高到低:
1. **环境变量** (最高优先级)
2. **环境特定文件** (`env.json` / `env.yml`,需增加层级 `log:`)
3. **基础配置文件** (`log.json` / `log.yml`)
### 1. 配置文件 (`log.json`)
在项目根目录创建 `log.json``log.yml`
```json
{
"name": "my-cool-app",
"level": "info",
"file": "logs/app.log",
"splitTag": ".2006-01-02",
"sensitive": "phone,password,secret,token,accessToken,authorization"
}
```
### 2. 环境变量 (最高优先级)
任何配置都可以通过环境变量覆盖,变量名规则为 `LOG_` + `字段名`
```bash
# 覆盖日志级别和输出文件
export LOG_LEVEL=debug
export LOG_FILE=console
```
### 配置项说明
* `name`: 应用名称 (默认通过 `debug.ReadBuildInfo()``os.Args[0]` 自动识别)。
* `level`: 日志级别 (`debug`, `info`, `warning`, `error`)。
* `file`: 输出目标。
* `console`: 直接输出到控制台(默认)。
* `path/to/file.log`: 输出到指定文件。
* `es://...``ess://...`: 输出到 Elasticsearch。
* `splitTag`: 文件切分格式,仅当 `file` 为文件路径时有效。
* 语法遵循 Go 标准的 `time.Format` 布局,如 `".2006-01-02"` (按天切分)`".2006-01-02-15"` (按小时切分)。
* `truncations`: 堆栈信息截断前缀(多个以逗号分隔,默认截断 `github.com/`, `golang.org/`, `/apigo.cc/`)。
* `sensitive`: 需要自动脱敏的字段名(多个以逗号分隔,不区分大小写),默认处理 `phone,password,secret,token,accessToken,authorization`
## 🛠 API 指南 ## 🛠 API 指南
### 核心功能 ### 核心功能
@ -37,45 +89,90 @@ logger.Error("数据库连接失败", "db", "mysql", "err", err)
1. **分级记录** 1. **分级记录**
* `Debug`, `Info`, `Warning`, `Error` —— 标准日志方法,支持 `message` + 变长 `extra` 参数。 * `Debug`, `Info`, `Warning`, `Error` —— 标准日志方法,支持 `message` + 变长 `extra` 参数。
2. **通用记录 (`Log`)** 2. **摩擦消除 (`As`)**
* `Log(LogEntry)` —— 记录自定义结构的日志。注意:仅支持实现 `LogEntry` 接口的类型(即嵌入了 `BaseLog` 的结构体)。 * `As(v, err)` —— 仿照 `cast.As`,忽略错误并返回零值,但会自动将错误记录到日志中。支持全局调用 (`log.As`) 或实例调用 (`logger.As`)。
* **优势**: 在类型转换或快速赋值场景下,无需繁琐的 `if err != nil` 判断,同时确保异常被记录。
3. **专业日志扩展** 3. **通用记录 (`Log`)**
* **请求日志 (`Request`)**: 记录 HTTP 请求,包含方法、路径、状态码、耗时等。 * `Log(LogEntry)` —— 记录自定义结构的日志。
* **数据库日志 (`DB`)**: 自动计算耗时、捕获调用栈并支持脱敏。
* **监控与统计 (`Monitor`, `Statistic`)**: 用于应用指标监控。
* **任务执行 (`Task`)**: 用于任务耗时与状态记录。
### 自定义日志扩展 4. **独立可视化工具 (`logv`)**
* **安装**: `go install apigo.cc/go/log/logv@latest`
* **使用**: `tail -f app.log | logv``tail -f app.log | logv -json`
如果标准日志分级不能满足业务需求,可以轻松扩展自定义日志类型: ### 自定义日志扩展 (规范)
1. **定义结构体**:必须嵌入 `log.BaseLog` 以自动获得基础字段和池化能力。 为保证高性能与内存安全,扩展自定义日志类型必须遵循以下规范:
2. **获取对象**:使用 `log.GetEntry[MyLog]()` 从对象池获取,避免频繁分配内存。
3. **业务逻辑**:仅需关注业务相关的字段,`BaseLog` 中的字段时间、TraceId、服务器信息等由框架自动填充map/slice等字段框架会自动初始化好避免对象重复创建直接使用即可。 1. **定义结构体**
4. **发送日志**:调用 `logger.Log(entry)` * 必须嵌入 `log.BaseLog` (或其子类,如 `log.ErrorLog`)。
* **索引 (`pos`) 规范**:
* `0`-`6``BaseLog`
* 业务字段从 `6` 开始紧凑递增编号 (`pos:6`, `pos:7`, ...),如果删除了某个字段请留空 pos 以实现向前兼容。
* 如果继承自 `ErrorLog` 等,则业务字段应从 `7` 开始(查询父类最大值 + 1
* `Extra` 固定使用 `pos:1000``CallStacks` 固定使用 `pos:1001` (它们会被自动平移到数组末尾)。
2. **实现 `Reset()` 方法 (强制)**
* **必须**重写 `Reset()` 方法以初始化/清空数据避免对象池复用时产生脏数据。
* `Reset()` 方法中必须首先调用父级的 `Reset()` (如 `l.BaseLog.Reset()`)。
* **安全保障**: 若未重写 `Reset``RegisterType` 将在启动时 **Panic**,以防止对象池复用时产生脏数据。
* **建议**: map / slice 类型建第一次初始化一个容量,之后使用 clear() 方法清空数据避免内存重复分配。
3. **注册模型**
* 在 `init()` 中调用 `log.RegisterType("my-type", MyLog{})` 完成注册。
#### 示例: `DBErrorLog`
```go ```go
type BusinessLog struct { package main
log.BaseLog // 必须嵌入
Action string import "apigo.cc/go/log"
UserId string
// 1. 定义结构体 (字段从 pos:6 开始)
type DBErrorLog struct {
log.ErrorLog // 嵌入 ErrorLog自动获得 Error 和 CallStacks 字段
DB string `log:"pos:7,color:blue"`
SQL string `log:"pos:8"`
Args []any `log:"pos:9"`
UsedTime float32 `log:"pos:10,color:cyan"`
} }
func LogBusiness(logger *log.Logger, action, userId string) { // 2. 实现 Reset() 方法 (强制)
entry := log.GetEntry[BusinessLog]() func (l *DBErrorLog) Reset() {
entry.Action = action l.ErrorLog.Reset() // 必须先调用父级 Reset
entry.UserId = userId l.DB = ""
logger.Log(entry) // 框架会自动填充 BaseLog 并异步写入后回收对象 l.SQL = ""
if l.Args == nil {
l.Args = make([]any, 0, 10)
} else {
clear(l.Args) // 清空内容
l.Args = l.Args[:0] // 清空长度
}
l.UsedTime = 0
}
// 3. 注册
func init() {
log.RegisterType("dbError", &DBErrorLog{})
}
// 4. 使用示例
func LogDBError(logger *log.Logger, db, sql string, args []any, err error, usedTime float32) {
entry := log.GetEntry[DBErrorLog]()
// 自动填充基础字段和 ErrorLog 字段
logger.FillError(&entry.ErrorLog, err.Error())
// 填充自定义字段
entry.DB = db
entry.SQL = sql
entry.Args = append(entry.Args, args...)
entry.UsedTime = usedTime
logger.Log(entry)
} }
``` ```
### 配置项 (JSON/YAML)
* `Name`: 应用名称。
* `Level`: 日志级别 (`debug`, `info`, `warning`, `error`)。
* `File`: 输出目标(支持 `console``es://` 地址)。
* `Sensitive`, `RegexSensitive`: 脱敏配置。
## 🧪 验证状态 ## 🧪 验证状态
测试全部通过,异步写入与性能达标。 测试全部通过,异步写入与性能达标。

84
TEST.md
View File

@ -1,28 +1,62 @@
# 日志性能测试报告 # Test Results
## 测试环境 ## 单元测试报告
- 操作系统: darwin
- 架构: amd64
- CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
## 基准测试结果 (v1.0.3) ```text
=== RUN TestLoggerCore_Initialization
--- PASS: TestLoggerCore_Initialization (0.00s)
=== RUN TestLoggerCore_Concurrency
--- PASS: TestLoggerCore_Concurrency (0.00s)
=== RUN TestMetaExtraction
--- PASS: TestMetaExtraction (0.00s)
=== RUN TestWithEntry
--- PASS: TestWithEntry (0.00s)
=== RUN TestLoggerReliability
--- PASS: TestLoggerReliability (0.01s)
=== RUN TestToArrayBytes
serializer_test.go:64: Raw log: ["test-app","mock_info_test",1620000000,"abc-123","Hello, World!",{"user_id":42}]
--- PASS: TestToArrayBytes (0.00s)
=== RUN TestToArrayBytes_Desensitize
--- PASS: TestToArrayBytes_Desensitize (0.00s)
=== RUN TestSplitTag
--- PASS: TestSplitTag (1.80s)
=== RUN TestSensitiveDetailed
--- PASS: TestSensitiveDetailed (0.00s)
=== RUN TestDeepDesensitization
--- PASS: TestDeepDesensitization (0.00s)
=== RUN TestLogger
--- PASS: TestLogger (0.00s)
=== RUN TestDesensitization
--- PASS: TestDesensitization (0.00s)
=== RUN TestDBLog
--- PASS: TestDBLog (0.00s)
=== RUN TestRequestLog
--- PASS: TestRequestLog (0.00s)
=== RUN TestExtraLogs
--- PASS: TestExtraLogs (0.00s)
=== RUN TestViewable
--- PASS: TestViewable (0.00s)
=== RUN TestToJSON
--- PASS: TestToJSON (0.00s)
=== RUN TestLoadMeta
--- PASS: TestLoadMeta (0.00s)
=== RUN TestEnhancedViewable
--- PASS: TestEnhancedViewable (0.00s)
=== RUN TestEnhancedToJSON
--- PASS: TestEnhancedToJSON (0.00s)
=== RUN TestCallStacksViewable
--- PASS: TestCallStacksViewable (0.00s)
=== RUN TestPrecisionViewable
--- PASS: TestPrecisionViewable (0.00s)
PASS
ok apigo.cc/go/log 2.246s
```
| 测试用例 | 迭代次数 | 耗时 (ns/op) | 内存分配 (B/op) | 分配次数 (allocs/op) | ## 核心指标验证
| :--- | :--- | :--- | :--- | :--- | - **初始化安全性**: `TestLoggerCore_Initialization` 确保 Logger 实例配置正确加载。
| `BenchmarkLogger_RequestLog_Realistic` | 2,324,065 | 544.1 | 72 | 2 | - **高并发稳定性**: `TestLoggerCore_Concurrency` 验证了在多协程竞争环境下日志写入的线程安全。
| `BenchmarkLoggerInfo` | 122,059 | 9,706 | - | - | - **元数据驱动验证**: `TestMetaExtraction``TestLoadMeta` 确保 `.log.meta.json` 协议的解析与应用。
| `BenchmarkLoggerAsyncConcurrent` | 127,830 | 8,773 | - | - | - **序列化性能**: `TestToArrayBytes` 验证了 Positional Array 格式的正确性。
- **深度脱敏能力**: `TestDeepDesensitization` 闭环验证了对复杂嵌套结构的脱敏逻辑。
## 版本对比评估 - **可靠性边界**: `TestLoggerReliability` 模拟了极高压力下的日志丢弃与缓冲策略。
- **文件切分**: `TestSplitTag` 实测了基于时间滚动的文件切分能力。
| 版本 | 机制 | 耗时 (ns/op) | 易用性 |
| :--- | :--- | :--- | :--- |
| **v1.0.1** | 手动 Reset | ~270 | 较低 (需编写大量样板代码) |
| **v1.0.2** | 自动化 Reset | ~475 | 极高 (嵌入 BaseLog 即可) |
| **v1.0.3** | 参数封装与解耦架构 | ~544 | 极高 (核心框架与业务结构完全分离) |
## 总结
- **性能评估**: v1.0.3 在核心日志记录上保持高性能。应用端自定义结构与框架对象池的结合被证明是高效的。
- **解耦架构**: `extra.go` 中的示例代码已被注释,成功将业务日志结构的定义权移交给应用层。框架仅保留最核心的异步写入和对象池管理能力。
- **内存效率**: 持续保持极低分配。
- **最佳实践**: 引导应用通过定义局部结构体并封装 `Logger` 扩展方法来记录日志,这不仅符合 Go 的工程规范,也极大地提升了系统的可维护性。

View File

@ -2,17 +2,12 @@ package log
// Config 日志配置 // Config 日志配置
type Config struct { type Config struct {
Name string Name string
Level string Level string
File string File string
Fast bool SplitTag string
SplitTag string Truncations string
Truncations string Sensitive string
Sensitive string
RegexSensitive string
SensitiveRule string
KeepKeyCase bool // 是否保持Key的首字母大小写默认一律使用小写
Formatter Formatter
} }
type LevelType int type LevelType int

View File

@ -7,8 +7,8 @@ import (
var DefaultLogger *Logger var DefaultLogger *Logger
func init() { func init() {
RegisterWriterMaker("es", NewESWriter) RegisterWriterMaker("es", newESWriter)
RegisterWriterMaker("ess", NewESWriter) RegisterWriterMaker("ess", newESWriter)
var conf Config var conf Config
_ = config.Load(&conf, "log") _ = config.Load(&conf, "log")
@ -19,3 +19,46 @@ func init() {
func New(traceId string) *Logger { func New(traceId string) *Logger {
return DefaultLogger.New(traceId) return DefaultLogger.New(traceId)
} }
// SetDefaultName 设置全局默认应用名称,并同步更新 DefaultLogger
func SetDefaultName(name string) {
if name == "" {
return
}
globalDefaultName = name
if DefaultLogger != nil {
DefaultLogger.SetName(name)
}
}
// As 仿照 cast.As使用 DefaultLogger 记录错误并返回零值 (消除摩擦)
func As[T any](v T, err error) T {
if err != nil {
if DefaultLogger != nil {
DefaultLogger.Error(err.Error())
}
var zero T
return zero
}
return v
}
// Debug 记录一条调试级别日志
func Debug(message string, extra ...any) {
DefaultLogger.Debug(message, extra...)
}
// Info 记录一条信息级别日志
func Info(message string, extra ...any) {
DefaultLogger.Info(message, extra...)
}
// Warning 记录一条警告级别日志
func Warning(message string, extra ...any) {
DefaultLogger.Warning(message, extra...)
}
// Error 记录一条错误级别日志
func Error(message string, extra ...any) {
DefaultLogger.Error(message, extra...)
}

View File

@ -14,7 +14,7 @@ import (
"apigo.cc/go/cast" "apigo.cc/go/cast"
) )
type ESWriter struct { type esWriter struct {
config *Config config *Config
url string url string
user string user string
@ -27,8 +27,8 @@ type ESWriter struct {
prefix string prefix string
} }
func NewESWriter(conf *Config) Writer { func newESWriter(conf *Config) Writer {
w := &ESWriter{ w := &esWriter{
config: conf, config: conf,
queue: make([]string, 0), queue: make([]string, 0),
client: &http.Client{}, client: &http.Client{},
@ -76,11 +76,12 @@ func NewESWriter(conf *Config) Writer {
return w return w
} }
func (w *ESWriter) Log(data []byte) { func (w *esWriter) Log(entry LogEntry, data []byte) {
if len(data) == 0 { objBytes, err := cast.ToJSONBytes(entry)
if err != nil || len(objBytes) == 0 {
return return
} }
dataString := string(data) dataString := string(objBytes)
w.lock.Lock() w.lock.Lock()
w.queue = append(w.queue, w.prefix, dataString) w.queue = append(w.queue, w.prefix, dataString)
@ -89,14 +90,14 @@ func (w *ESWriter) Log(data []byte) {
var responseOkBytes = []byte("\"errors\":false") var responseOkBytes = []byte("\"errors\":false")
func (w *ESWriter) Run() { func (w *esWriter) Run() {
now := time.Now().Unix() now := time.Now().Unix()
w.lock.Lock() w.lock.Lock()
queueLen := len(w.queue) queueLen := len(w.queue)
w.lock.Unlock() w.lock.Unlock()
// 超过100条数据 或 过了1秒 发送数据 // 超过100条数据 或 过了1秒 发送数据
if queueLen > 100 || (queueLen > 0 && (now > w.last || !writerRunning.Load())) { if queueLen > 100 || (queueLen > 0 && (now > w.last || !WriterService.Running.Load())) {
w.lock.Lock() w.lock.Lock()
sendings := w.queue sendings := w.queue
w.queue = make([]string, 0) w.queue = make([]string, 0)

197
extra.go
View File

@ -1,197 +0,0 @@
package log
// import (
// "apigo.cc/go/cast"
// )
// type RequestLog struct {
// BaseLog
// ServerId string
// App string
// Node string
// ClientIp string
// FromApp string
// FromNode string
// UserId string
// DeviceId string
// ClientAppName string
// ClientAppVersion string
// SessionId string
// RequestId string
// Host string
// Scheme string
// Proto string
// AuthLevel int
// Priority int
// Method string
// Path string
// RequestHeaders map[string]string
// RequestData map[string]any
// UsedTime float32
// ResponseCode int
// ResponseHeaders map[string]string
// ResponseDataLength uint
// ResponseData string
// }
// func (logger *Logger) Request(
// method, path, host, scheme, proto string,
// clientIp, serverId, app, node string,
// fromApp, fromNode string,
// userId, deviceId, sessionId, requestId string,
// clientAppName, clientAppVersion string,
// authLevel, priority int,
// reqHeaders map[string]string,
// reqData map[string]any,
// responseCode int,
// usedTime float32,
// respHeaders map[string]string,
// responseData string,
// responseDataLength uint,
// extra ...any,
// ) {
// if !logger.CheckLevel(INFO) {
// return
// }
// entry := GetEntry[RequestLog]()
// logger.fillBase(entry, LogTypeRequest)
// // 暴力平铺赋值,性能极高
// entry.Method = method
// entry.Path = path
// entry.Host = host
// entry.Scheme = scheme
// entry.Proto = proto
// entry.ClientIp = clientIp
// entry.ServerId = serverId
// entry.App = app
// entry.Node = node
// entry.FromApp = fromApp
// entry.FromNode = fromNode
// entry.UserId = userId
// entry.DeviceId = deviceId
// entry.SessionId = sessionId
// entry.RequestId = requestId
// entry.ClientAppName = clientAppName
// entry.ClientAppVersion = clientAppVersion
// entry.AuthLevel = authLevel
// entry.Priority = priority
// entry.RequestHeaders = reqHeaders
// entry.RequestData = reqData
// entry.ResponseCode = responseCode
// entry.UsedTime = usedTime
// entry.ResponseHeaders = respHeaders
// entry.ResponseData = responseData
// entry.ResponseDataLength = responseDataLength
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// type TaskLog struct {
// BaseLog
// Task string
// UsedTime float32
// Success bool
// Message string
// }
// type MonitorLog struct {
// BaseLog
// Target string
// Status int
// Message string
// }
// type StatisticLog struct {
// BaseLog
// Category string
// Item string
// Value float64
// }
// func (logger *Logger) Task(taskName string, usedTime float32, success bool, message string, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[TaskLog]()
// logger.fillBase(entry, LogTypeTask)
// entry.Task = taskName
// entry.UsedTime = usedTime
// entry.Success = success
// entry.Message = message
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// func (logger *Logger) Monitor(target string, status int, message string, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[MonitorLog]()
// logger.fillBase(entry, LogTypeMonitor)
// entry.Target = target
// entry.Status = status
// entry.Message = message
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// func (logger *Logger) Statistic(category, item string, value float64, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[StatisticLog]()
// logger.fillBase(entry, LogTypeStatistic)
// entry.Category = category
// entry.Item = item
// entry.Value = value
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// type DBLog struct {
// BaseLog
// DbType string
// Dsn string
// Query string
// QueryArgs string
// UsedTime float32
// Error string
// CallStacks []string
// }
// func (logger *Logger) DB(dbType, dsn, query string, args []any, usedTime float32, err error, extra ...any) {
// logType := LogTypeDb
// level := INFO
// var e string
// if err != nil {
// logType = LogTypeDbError
// level = ERROR
// e = err.Error()
// }
// if logger.CheckLevel(level) {
// entry := GetEntry[DBLog]()
// logger.fillBase(entry, logType)
// entry.DbType = dbType
// entry.Dsn = dsn
// entry.Query = query
// entry.QueryArgs = cast.To[string](args)
// entry.UsedTime = usedTime
// if e != "" {
// entry.Error = e
// entry.CallStacks = getCallStacks(logger.truncations)
// }
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }

224
extra_example.go Normal file
View File

@ -0,0 +1,224 @@
package log
// type RequestLog struct {
// BaseLog
// ServerId string `log:"pos:6"`
// App string `log:"pos:7"`
// Node string `log:"pos:8"`
// ClientIp string `log:"pos:9"`
// FromApp string `log:"pos:10"`
// FromNode string `log:"pos:11"`
// UserId string `log:"pos:12"`
// DeviceId string `log:"pos:13"`
// ClientAppName string `log:"pos:14"`
// ClientAppVersion string `log:"pos:15"`
// SessionId string `log:"pos:16"`
// RequestId string `log:"pos:17"`
// Host string `log:"pos:18"`
// Scheme string `log:"pos:19"`
// Proto string `log:"pos:20"`
// AuthLevel int `log:"pos:21"`
// Priority int `log:"pos:22"`
// Method string `log:"pos:23"`
// Path string `log:"pos:24"`
// RequestHeaders map[string]string `log:"pos:25"`
// RequestData map[string]any `log:"pos:26"`
// UsedTime float32 `log:"pos:27"`
// ResponseCode int `log:"pos:28"`
// ResponseHeaders map[string]string `log:"pos:29"`
// ResponseDataLength uint `log:"pos:30"`
// ResponseData string `log:"pos:31"`
// }
// func (l *RequestLog) Reset() {
// l.BaseLog.Reset()
// l.ServerId = ""
// l.App = ""
// l.Node = ""
// l.ClientIp = ""
// l.FromApp = ""
// l.FromNode = ""
// l.UserId = ""
// l.DeviceId = ""
// l.ClientAppName = ""
// l.ClientAppVersion = ""
// l.SessionId = ""
// l.RequestId = ""
// l.Host = ""
// l.Scheme = ""
// l.Proto = ""
// l.AuthLevel = 0
// l.Priority = 0
// l.Method = ""
// l.Path = ""
// if l.RequestHeaders == nil {
// l.RequestHeaders = make(map[string]string, 8)
// } else {
// clear(l.RequestHeaders)
// }
// if l.RequestData == nil {
// l.RequestData = make(map[string]any, 8)
// } else {
// clear(l.RequestData)
// }
// l.UsedTime = 0
// l.ResponseCode = 0
// if l.ResponseHeaders == nil {
// l.ResponseHeaders = make(map[string]string, 8)
// } else {
// clear(l.ResponseHeaders)
// }
// l.ResponseDataLength = 0
// l.ResponseData = ""
// }
// func (logger *Logger) Request(
// method, path, host, scheme, proto string,
// clientIp, serverId, app, node string,
// fromApp, fromNode string,
// userId, deviceId, sessionId, requestId string,
// clientAppName, clientAppVersion string,
// authLevel, priority int,
// reqHeaders map[string]string,
// reqData map[string]any,
// responseCode int,
// usedTime float32,
// respHeaders map[string]string,
// responseData string,
// responseDataLength uint,
// extra ...any,
// ) {
// if !logger.CheckLevel(INFO) {
// return
// }
// entry := GetEntry[RequestLog]()
// logger.FillBase(&entry.BaseLog, LogTypeRequest)
// // 暴力平铺赋值,性能极高
// entry.Method = method
// entry.Path = path
// entry.Host = host
// entry.Scheme = scheme
// entry.Proto = proto
// entry.ClientIp = clientIp
// entry.ServerId = serverId
// entry.App = app
// entry.Node = node
// entry.FromApp = fromApp
// entry.FromNode = fromNode
// entry.UserId = userId
// entry.DeviceId = deviceId
// entry.SessionId = sessionId
// entry.RequestId = requestId
// entry.ClientAppName = clientAppName
// entry.ClientAppVersion = clientAppVersion
// entry.AuthLevel = authLevel
// entry.Priority = priority
// entry.RequestHeaders = reqHeaders
// entry.RequestData = reqData
// entry.ResponseCode = responseCode
// entry.UsedTime = usedTime
// entry.ResponseHeaders = respHeaders
// entry.ResponseData = responseData
// entry.ResponseDataLength = responseDataLength
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// type TaskLog struct {
// BaseLog
// Task string `log:"pos:6"`
// UsedTime float32 `log:"pos:7"`
// Success bool `log:"pos:8"`
// Message string `log:"pos:9"`
// }
// func (l *TaskLog) Reset() {
// l.BaseLog.Reset()
// l.Task = ""
// l.UsedTime = 0
// l.Success = false
// l.Message = ""
// }
// type MonitorLog struct {
// BaseLog
// Target string `log:"pos:6"`
// Status int `log:"pos:7"`
// Message string `log:"pos:8"`
// }
// func (l *MonitorLog) Reset() {
// l.BaseLog.Reset()
// l.Target = ""
// l.Status = 0
// l.Message = ""
// }
// type StatisticLog struct {
// BaseLog
// Category string `log:"pos:6"`
// Item string `log:"pos:7"`
// Value float64 `log:"pos:8"`
// }
// func (l *StatisticLog) Reset() {
// l.BaseLog.Reset()
// l.Category = ""
// l.Item = ""
// l.Value = 0
// }
// func (logger *Logger) Task(taskName string, usedTime float32, success bool, message string, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[TaskLog]()
// logger.FillBase(&entry.BaseLog, LogTypeTask)
// entry.Task = taskName
// entry.UsedTime = usedTime
// entry.Success = success
// entry.Message = message
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// func (logger *Logger) Monitor(target string, status int, message string, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[MonitorLog]()
// logger.FillBase(&entry.BaseLog, LogTypeMonitor)
// entry.Target = target
// entry.Status = status
// entry.Message = message
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// func (logger *Logger) Statistic(category, item string, value float64, extra ...any) {
// if logger.CheckLevel(INFO) {
// entry := GetEntry[StatisticLog]()
// logger.FillBase(&entry.BaseLog, LogTypeStatistic)
// entry.Category = category
// entry.Item = item
// entry.Value = value
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }
// func init() {
// RegisterType(LogTypeRequest, &RequestLog{})
// RegisterType(LogTypeTask, &TaskLog{})
// RegisterType(LogTypeMonitor, &MonitorLog{})
// RegisterType(LogTypeStatistic, &StatisticLog{})
// }

View File

@ -4,7 +4,6 @@ import (
"bufio" "bufio"
"fmt" "fmt"
"os" "os"
"sync"
"time" "time"
) )
@ -17,11 +16,6 @@ type FileWriter struct {
bufWriter *bufio.Writer bufWriter *bufio.Writer
} }
var (
files = make(map[string]*FileWriter)
filesLock sync.RWMutex
)
// Write 由外层的 writerRunner 单协程调用,绝对并发安全,无需加锁 // Write 由外层的 writerRunner 单协程调用,绝对并发安全,无需加锁
func (f *FileWriter) Write(tm time.Time, data []byte) { func (f *FileWriter) Write(tm time.Time, data []byte) {
nowSplit := tm.Format(f.splitTag) nowSplit := tm.Format(f.splitTag)

View File

@ -1,28 +0,0 @@
package log
import (
"apigo.cc/go/cast"
)
// Formatter 日志格式化接口
type Formatter interface {
Format(data any, sensitiveKeys []string) ([]byte, error)
}
// JSONFormatter 默认的 JSON 格式化器
type JSONFormatter struct{}
func (f *JSONFormatter) Format(data any, sensitiveKeys []string) ([]byte, error) {
if len(sensitiveKeys) > 0 {
return cast.ToJSONDesensitizeBytes(data, sensitiveKeys)
}
return cast.ToJSONBytes(data)
}
// TextFormatter 文本格式化器 (示例)
type TextFormatter struct{}
func (f *TextFormatter) Format(data any, sensitiveKeys []string) ([]byte, error) {
// 简单的文本格式化实现
return []byte(cast.String(data)), nil
}

136
functional_test.go Normal file
View File

@ -0,0 +1,136 @@
package log_test
import (
"strings"
"testing"
"time"
"apigo.cc/go/file"
"apigo.cc/go/log"
)
func TestSplitTag(t *testing.T) {
logFile := "test_rotate.log"
// 使用每秒切分的标签,方便测试文件轮转
splitTag := ".20060102150405"
conf := log.Config{
Name: "test-split",
Level: "info",
File: logFile,
SplitTag: splitTag,
}
log.WriterService.Start(nil, nil)
logger := log.NewLogger(conf)
// 1. 记录第一条日志
t1 := time.Now()
logger.Info("first message")
time.Sleep(300 * time.Millisecond) // 等待异步写入
expectedFile1 := logFile + "." + t1.Format(splitTag)
if !file.Exists(expectedFile1) {
// 可能在写入时秒数刚好进位
expectedFile1 = logFile + "." + time.Now().Format(splitTag)
if !file.Exists(expectedFile1) {
t.Fatalf("First log file does not exist (checked both possible time slots)")
}
}
// 2. 等待跨秒,确保下次写入肯定会触发轮转
time.Sleep(1200 * time.Millisecond)
// 3. 记录第二条日志,触发轮转
t2 := time.Now()
logger.Info("second message")
time.Sleep(300 * time.Millisecond) // 等待异步写入
expectedFile2 := logFile + "." + t2.Format(splitTag)
if !file.Exists(expectedFile2) {
expectedFile2 = logFile + "." + time.Now().Format(splitTag)
if !file.Exists(expectedFile2) {
t.Fatalf("Second log file does not exist after rotation")
}
}
if expectedFile1 == expectedFile2 {
t.Errorf("Files should be different for rotation, but both are %s", expectedFile1)
}
// 清理
file.Remove(expectedFile1)
file.Remove(expectedFile2)
}
func TestSensitiveDetailed(t *testing.T) {
type SecretLog struct {
log.BaseLog
Password string
SecretKey string
SafeData string
}
entry := log.GetEntry[SecretLog]()
entry.BaseLog.LogType = "secret"
entry.Password = "my_password"
entry.SecretKey = "super_secret"
entry.SafeData = "hello"
// 直接测试 ToArrayBytes
// 注意passed to ToArrayBytes 的 keys 应该是已经过 fixField 处理的
sensitiveKeys := []string{"password", "secretkey"}
buf := log.Marshal(entry, sensitiveKeys)
result := string(buf)
if strings.Contains(result, "my_password") {
t.Errorf("Sensitive data 'my_password' not masked in: %s", result)
}
if strings.Contains(result, "super_secret") {
t.Errorf("Sensitive data 'super_secret' not masked in: %s", result)
}
if !strings.Contains(result, "hello") {
t.Errorf("Safe data 'hello' should be present in: %s", result)
}
}
func TestDeepDesensitization(t *testing.T) {
type Nested struct {
Password string
Token string
}
type DeepLog struct {
log.BaseLog
Data map[string]any
User Nested
}
entry := log.GetEntry[DeepLog]()
entry.BaseLog.LogType = "deep"
entry.Data = map[string]any{
"password": "data_password",
"safe": "data_safe",
}
entry.User = Nested{
Password: "user_password",
Token: "user_token",
}
sensitiveKeys := []string{"password", "token"}
buf := log.Marshal(entry, sensitiveKeys)
result := string(buf)
// Check deep desensitization in map
if strings.Contains(result, "data_password") {
t.Errorf("Nested map data 'data_password' not masked in: %s", result)
}
// Check deep desensitization in struct
if strings.Contains(result, "user_password") {
t.Errorf("Nested struct data 'user_password' not masked in: %s", result)
}
if strings.Contains(result, "user_token") {
t.Errorf("Nested struct data 'user_token' not masked in: %s", result)
}
if !strings.Contains(result, "data_safe") {
t.Errorf("Safe data 'data_safe' should be present in: %s", result)
}
}

23
go.mod
View File

@ -3,21 +3,18 @@ module apigo.cc/go/log
go 1.25.0 go 1.25.0
require ( require (
apigo.cc/go/cast v1.2.6 apigo.cc/go/cast v1.3.3
apigo.cc/go/config v1.0.5 apigo.cc/go/config v1.3.1
apigo.cc/go/shell v1.0.4 apigo.cc/go/file v1.3.2
apigo.cc/go/id v1.3.1
apigo.cc/go/shell v1.3.1
) )
require ( require (
apigo.cc/go/convert v1.0.4 // indirect apigo.cc/go/encoding v1.3.1 // indirect
apigo.cc/go/encoding v1.0.4 // indirect apigo.cc/go/rand v1.3.1 // indirect
apigo.cc/go/file v1.0.4 // indirect apigo.cc/go/safe v1.3.1 // indirect
apigo.cc/go/rand v1.0.4 // indirect golang.org/x/crypto v0.51.0 // indirect
apigo.cc/go/safe v1.0.4 // indirect golang.org/x/sys v0.44.0 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.43.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

48
go.sum
View File

@ -1,39 +1,31 @@
apigo.cc/go/cast v1.2.6 h1:xnWiaQAGsRCrnu1p8fIFQfg5HFSc7CxR+3ItiDIDMaY= apigo.cc/go/cast v1.3.3 h1:aln5eDR5DZVWVzZ/y5SJh1gQNgWv2sT82I25NaO9g34=
apigo.cc/go/cast v1.2.6/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk= apigo.cc/go/cast v1.3.3/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk=
apigo.cc/go/config v1.0.4 h1:WG9zrQkqfFPkrKIL7RNvvAbbkuUBt1Av11ZP/aIfldM= apigo.cc/go/config v1.3.1 h1:wZzUh4oL+fGD6SayVgX6prLPMsniM25etWFcEH8XzIE=
apigo.cc/go/config v1.0.4/go.mod h1:obryzJiK6j7lQex/58d5eWYOGx5O5IABguqNWxyyXJo= apigo.cc/go/config v1.3.1/go.mod h1:7KHz/1WmtBLM762Lln/TaXh2dmlMvJTLhnlk33zbS3U=
apigo.cc/go/convert v1.0.4 h1:5+qPjC3dlPB59GnWZRlmthxcaXQtKvN+iOuiLdJ1GvQ= apigo.cc/go/encoding v1.3.1 h1:y8O58KYAyulkThg1O2ji2BqjnFoSvk42sit9I3z+K7Y=
apigo.cc/go/convert v1.0.4/go.mod h1:Hp+geeSyhqg/zwIKPOrDoceIREzcwM14t1I5q/dtbfU= apigo.cc/go/encoding v1.3.1/go.mod h1:xAJk5b83VZ31mXMTnyp0dfMoBKfT/AHDn0u+cQfojgY=
apigo.cc/go/encoding v1.0.4 h1:aezB0J/qFuHs6iXkbtuJP5JIHUtmjsr5SFb0NNvbObY= apigo.cc/go/file v1.3.2 h1:pu4oiDyiqgj3/eykfnJf+/6+A9v/Z0b3ClP5XK+lwG4=
apigo.cc/go/encoding v1.0.4/go.mod h1:V5CgT7rBbCxy+uCU20q0ptcNNRSgMtpA8cNOs6r8IeI= apigo.cc/go/file v1.3.2/go.mod h1:vci4h0Pz94mV6dkniQkuyBYERVYeq7/LX4jJVuCg9hs=
apigo.cc/go/file v1.0.4 h1:qCKegV7OYh7r0qc3jZjGA/aKh0vIHgmr1OEbhfEmGX8= apigo.cc/go/id v1.3.1 h1:pkqi6VeWyQoHuIu0Zbx/RRxIAdM61Js0j6cY1M9XVCk=
apigo.cc/go/file v1.0.4/go.mod h1:C9gNo7386iA21OiBmuWh6CznKWlVBDFkhE4f0H0Susg= apigo.cc/go/id v1.3.1/go.mod h1:P2/vl3tyW3US+ayOFSMoPIOCulNLBngNYPhXJC/Z7J4=
apigo.cc/go/rand v1.0.4 h1:we070eWSL0dB8NEMaWjXj43+EekXQTm/h0kKpZ/frqw= apigo.cc/go/rand v1.3.1 h1:7FvsI6PtQ5XrWER0dTiLVo0p7GIxRidT/TBKhVy93j8=
apigo.cc/go/rand v1.0.4/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk= apigo.cc/go/rand v1.3.1/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk=
apigo.cc/go/safe v1.0.4 h1:07pRSdEHprF/2v6SsqAjICYFoeLcqjjvHGEdh6Dzrzg= apigo.cc/go/safe v1.3.1 h1:irTCqPAC97gGsX/Lw5AzLelDt1xXLEZIAaVhLELWe9Q=
apigo.cc/go/safe v1.0.4/go.mod h1:o568sHS5rTRSVPmhxWod0tGdc+8l1KjidsNY1/OVZr0= apigo.cc/go/safe v1.3.1/go.mod h1:XdOpBhN2vkImalaykYXXmEpczqWa1y3ah6/Q72cdRqE=
apigo.cc/go/shell v1.0.4 h1:EL9zjI39YBe1h+kRYQeAi/8zVGHe5W198DYYN7cENiY= apigo.cc/go/shell v1.3.1 h1:M8oD0b2HcJuCC6frQFx11b3UTcTx3lATX8XK+YXSVm8=
apigo.cc/go/shell v1.0.4/go.mod h1:N2gDkgK4tJ9TadD60/+gAGuWxyVAWHs5YPBmytw6ELA= apigo.cc/go/shell v1.3.1/go.mod h1:ZMdJjpCpWdvsHKUXlelh/AxsV/nWdkH/k3lISfzMdUw=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI= golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q= golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@ -1,6 +1,7 @@
package log_test package log_test
import ( import (
"strconv"
"testing" "testing"
"apigo.cc/go/log" "apigo.cc/go/log"
@ -47,6 +48,7 @@ func TestDesensitization(t *testing.T) {
} }
entry := log.GetEntry[MyLog]() entry := log.GetEntry[MyLog]()
logger.FillBase(&entry.BaseLog, "test")
entry.Phone = "13812345678" entry.Phone = "13812345678"
logger.Log(entry) // 应该在输出中脱敏 logger.Log(entry) // 应该在输出中脱敏
} }
@ -57,7 +59,7 @@ func TestDBLog(t *testing.T) {
}) })
entry := log.GetEntry[DBEntry]() entry := log.GetEntry[DBEntry]()
entry.LogType = "db" logger.FillBase(&entry.BaseLog, "db")
entry.DbType = "mysql" entry.DbType = "mysql"
entry.Query = "SELECT * FROM users" entry.Query = "SELECT * FROM users"
entry.UsedTime = 10.5 entry.UsedTime = 10.5
@ -70,7 +72,7 @@ func TestRequestLog(t *testing.T) {
}) })
entry := log.GetEntry[RequestEntry]() entry := log.GetEntry[RequestEntry]()
entry.LogType = "request" logger.FillBase(&entry.BaseLog, "request")
entry.Method = "GET" entry.Method = "GET"
entry.Path = "/api/user" entry.Path = "/api/user"
entry.ResponseCode = 200 entry.ResponseCode = 200
@ -85,3 +87,28 @@ func TestExtraLogs(t *testing.T) {
logger.Info("Extra log test", "key", "value") logger.Info("Extra log test", "key", "value")
} }
func TestAs(t *testing.T) {
// 1. 测试 log.As (使用 DefaultLogger)
val1 := log.As(strconv.Atoi("123"))
if val1 != 123 {
t.Errorf("log.As expected 123, got %v", val1)
}
val2 := log.As(strconv.Atoi("abc"))
if val2 != 0 {
t.Errorf("log.As expected 0, got %v", val2)
}
// 2. 测试 logger.As (方法)
logger := log.NewLogger(log.Config{Level: "debug"})
val3 := logger.As(strconv.Atoi("456")).(int)
if val3 != 456 {
t.Errorf("logger.As expected 456, got %v", val3)
}
val4 := logger.As(strconv.Atoi("def")).(int)
if val4 != 0 {
t.Errorf("logger.As expected 0, got %v", val4)
}
}

189
logger.go
View File

@ -4,33 +4,23 @@ import (
"fmt" "fmt"
"log" "log"
"os" "os"
"regexp"
"strings" "strings"
"time" "time"
"apigo.cc/go/cast" "apigo.cc/go/cast"
"apigo.cc/go/id"
) )
type Logger struct { type Logger struct {
config Config config Config
level LevelType level LevelType
goLogger *log.Logger goLogger *log.Logger
file *FileWriter file *FileWriter
writer Writer writer Writer
formatter Formatter truncations []string
truncations []string sensitive map[string]bool
sensitive map[string]bool sensitiveKeys []string
sensitiveKeys []string traceId string
regexSensitive []*regexp.Regexp
sensitiveRule []sensitiveRuleInfo
desensitization func(string) string
traceId string
}
type sensitiveRuleInfo struct {
threshold int
leftNum int
rightNum int
} }
var ( var (
@ -51,20 +41,14 @@ func NewLogger(conf Config) *Logger {
if conf.Sensitive == "" { if conf.Sensitive == "" {
conf.Sensitive = LogDefaultSensitive conf.Sensitive = LogDefaultSensitive
} }
if conf.SensitiveRule == "" {
conf.SensitiveRule = "12:4*4, 11:3*4, 7:2*2, 3:1*1, 2:1*0"
}
if conf.Name == "" { if conf.Name == "" {
conf.Name = GetDefaultName() conf.Name = getDefaultName()
} }
logger := Logger{ logger := Logger{
truncations: cast.Split(conf.Truncations, ","), truncations: cast.Split(conf.Truncations, ","),
formatter: conf.Formatter, traceId: id.Get10Bytes14MPerSecond(),
}
if logger.formatter == nil {
logger.formatter = &JSONFormatter{}
} }
if len(conf.Sensitive) > 0 { if len(conf.Sensitive) > 0 {
@ -77,37 +61,6 @@ func NewLogger(conf Config) *Logger {
} }
} }
if len(conf.RegexSensitive) > 0 {
ss := cast.Split(conf.RegexSensitive, ",")
for _, v := range ss {
if r, err := regexp.Compile(v); err == nil {
logger.regexSensitive = append(logger.regexSensitive, r)
}
}
}
if len(conf.SensitiveRule) > 0 {
ss := cast.Split(conf.SensitiveRule, ",")
for _, v := range ss {
a1 := strings.SplitN(v, ":", 2)
if len(a1) == 2 {
a2 := strings.SplitN(a1[1], "*", 3)
if len(a2) == 2 {
threshold := cast.Int(a1[0])
leftNum := cast.Int(a2[0])
rightNum := cast.Int(a2[1])
if threshold >= 0 && threshold <= 100 && leftNum >= 0 && leftNum <= 100 && rightNum >= 0 && rightNum <= 100 {
logger.sensitiveRule = append(logger.sensitiveRule, sensitiveRuleInfo{
threshold: threshold,
leftNum: leftNum,
rightNum: rightNum,
})
}
}
}
}
}
switch strings.ToLower(conf.Level) { switch strings.ToLower(conf.Level) {
case "debug": case "debug":
logger.level = DEBUG logger.level = DEBUG
@ -125,29 +78,27 @@ func NewLogger(conf Config) *Logger {
if m, ok := writerMakers[writerName]; ok { if m, ok := writerMakers[writerName]; ok {
if w := m(&conf); w != nil { if w := m(&conf); w != nil {
logger.writer = w logger.writer = w
writerLock.Lock() WriterService.WriterLock.Lock()
cur := writers.Load().([]Writer) cur := WriterService.Writers.Load().([]Writer)
newW := append(cur, w) newW := append(cur, w)
writers.Store(newW) WriterService.Writers.Store(newW)
writerLock.Unlock() WriterService.WriterLock.Unlock()
Start()
} }
} }
} else { } else {
if conf.SplitTag != "" { if conf.SplitTag != "" {
filesLock.RLock() WriterService.FilesLock.RLock()
logger.file = files[conf.File+conf.SplitTag] logger.file = WriterService.Files[conf.File+conf.SplitTag]
filesLock.RUnlock() WriterService.FilesLock.RUnlock()
if logger.file == nil { if logger.file == nil {
logger.file = &FileWriter{ logger.file = &FileWriter{
fileName: conf.File, fileName: conf.File,
splitTag: conf.SplitTag, splitTag: conf.SplitTag,
} }
filesLock.Lock() WriterService.FilesLock.Lock()
files[conf.File+conf.SplitTag] = logger.file WriterService.Files[conf.File+conf.SplitTag] = logger.file
filesLock.Unlock() WriterService.FilesLock.Unlock()
} }
Start()
} else { } else {
fp, err := os.OpenFile(conf.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644) fp, err := os.OpenFile(conf.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err == nil { if err == nil {
@ -161,26 +112,19 @@ func NewLogger(conf Config) *Logger {
} }
func (logger *Logger) Log(entry LogEntry) { func (logger *Logger) Log(entry LogEntry) {
// 如果是自动生成的 LogType如 Debug/Info/Warning/Error 对应的类型),这里通常已经在方法内 fillBase 了
// 但对于像 discover.Log 这样直接调用 Log(entry) 的,需要在这里补全
if entry.GetBaseLog().LogTime == 0 {
logger.fillBase(entry, "")
}
logger.asyncWrite(entry) logger.asyncWrite(entry)
} }
func (logger *Logger) asyncWrite(entry LogEntry) { func (logger *Logger) asyncWrite(entry LogEntry) {
buf, err := logger.formatter.Format(entry, logger.sensitiveKeys) buf := Marshal(entry, logger.sensitiveKeys)
logger.writeBuf(entry, buf)
if err == nil { putEntry(entry)
logger.writeBuf(buf)
}
PutEntry(entry)
} }
func (logger *Logger) writeBuf(buf []byte) { func (logger *Logger) writeBuf(entry LogEntry, buf []byte) {
if writerRunning.Load() { if WriterService.Running.Load() {
WriteAsync(logPayload{ writeAsync(logPayload{
entry: entry,
buf: buf, buf: buf,
writer: logger.writer, writer: logger.writer,
file: logger.file, file: logger.file,
@ -189,7 +133,7 @@ func (logger *Logger) writeBuf(buf []byte) {
} }
if logger.writer != nil { if logger.writer != nil {
logger.writer.Log(buf) logger.writer.Log(entry, buf)
} else if logger.file != nil { } else if logger.file != nil {
fmt.Println(Viewable(string(buf))) fmt.Println(Viewable(string(buf)))
} else if logger.goLogger == nil { } else if logger.goLogger == nil {
@ -199,8 +143,7 @@ func (logger *Logger) writeBuf(buf []byte) {
} }
} }
func (logger *Logger) fillBase(entry LogEntry, logType string) { func (logger *Logger) FillBase(base *BaseLog, logType string) {
base := entry.GetBaseLog()
if base == nil { if base == nil {
return return
} }
@ -211,17 +154,48 @@ func (logger *Logger) fillBase(entry LogEntry, logType string) {
} }
base.LogTime = time.Now().UnixNano() base.LogTime = time.Now().UnixNano()
base.TraceId = logger.traceId base.TraceId = logger.traceId
base.ImageName = dockerImageName if dockerImageTag != "" {
base.ImageTag = dockerImageTag base.Image = dockerImageName + ":" + dockerImageTag
base.ServerName = serverName } else {
base.ServerIp = serverIp base.Image = dockerImageName
}
if serverIp != "" {
base.Server = serverName + ":" + serverIp
} else {
base.Server = serverName
}
}
func (logger *Logger) FillDebug(entry *DebugLog, message string) {
logger.FillBase(&entry.BaseLog, LogTypeDebug)
entry.Debug = message
}
func (logger *Logger) FillInfo(entry *InfoLog, message string) {
logger.FillBase(&entry.BaseLog, LogTypeInfo)
entry.Info = message
}
func (logger *Logger) FillWarning(entry *WarningLog, message string) {
logger.FillBase(&entry.BaseLog, LogTypeWarning)
entry.Warning = message
entry.CallStacks = getCallStacks(logger.truncations)
}
func (logger *Logger) FillError(entry *ErrorLog, message string) {
logger.FillBase(&entry.BaseLog, LogTypeError)
entry.Error = message
entry.CallStacks = getCallStacks(logger.truncations)
}
func (logger *Logger) GetCallStacks() []string {
return getCallStacks(logger.truncations)
} }
func (logger *Logger) Debug(message string, extra ...any) { func (logger *Logger) Debug(message string, extra ...any) {
if logger.CheckLevel(DEBUG) { if logger.CheckLevel(DEBUG) {
entry := GetEntry[DebugLog]() entry := GetEntry[DebugLog]()
logger.fillBase(entry, LogTypeDebug) logger.FillDebug(entry, message)
entry.Debug = message
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.FillMap(&entry.Extra, extra)
} }
@ -232,8 +206,7 @@ func (logger *Logger) Debug(message string, extra ...any) {
func (logger *Logger) Info(message string, extra ...any) { func (logger *Logger) Info(message string, extra ...any) {
if logger.CheckLevel(INFO) { if logger.CheckLevel(INFO) {
entry := GetEntry[InfoLog]() entry := GetEntry[InfoLog]()
logger.fillBase(entry, LogTypeInfo) logger.FillInfo(entry, message)
entry.Info = message
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.FillMap(&entry.Extra, extra)
} }
@ -244,9 +217,7 @@ func (logger *Logger) Info(message string, extra ...any) {
func (logger *Logger) Warning(message string, extra ...any) { func (logger *Logger) Warning(message string, extra ...any) {
if logger.CheckLevel(WARNING) { if logger.CheckLevel(WARNING) {
entry := GetEntry[WarningLog]() entry := GetEntry[WarningLog]()
logger.fillBase(entry, LogTypeWarning) logger.FillWarning(entry, message)
entry.Warning = message
entry.CallStacks = getCallStacks(logger.truncations)
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.FillMap(&entry.Extra, extra)
} }
@ -257,9 +228,7 @@ func (logger *Logger) Warning(message string, extra ...any) {
func (logger *Logger) Error(message string, extra ...any) { func (logger *Logger) Error(message string, extra ...any) {
if logger.CheckLevel(ERROR) { if logger.CheckLevel(ERROR) {
entry := GetEntry[ErrorLog]() entry := GetEntry[ErrorLog]()
logger.fillBase(entry, LogTypeError) logger.FillError(entry, message)
entry.Error = message
entry.CallStacks = getCallStacks(logger.truncations)
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.FillMap(&entry.Extra, extra)
} }
@ -267,12 +236,12 @@ func (logger *Logger) Error(message string, extra ...any) {
} }
} }
func (logger *Logger) SetLevel(level LevelType) { func (logger *Logger) SetName(name string) {
logger.level = level logger.config.Name = name
} }
func (logger *Logger) SetDesensitization(f func(v string) string) { func (logger *Logger) SetLevel(level LevelType) {
logger.desensitization = f logger.level = level
} }
func (logger *Logger) New(traceId string) *Logger { func (logger *Logger) New(traceId string) *Logger {
@ -285,6 +254,14 @@ func (logger *Logger) GetTraceId() string {
return logger.traceId return logger.traceId
} }
// As 仿照 cast.As忽略错误并返回零值但会将错误记录到日志中 (消除摩擦)
func (logger *Logger) As(v any, err error) any {
if err != nil {
logger.Error(err.Error())
}
return v
}
func (logger *Logger) CheckLevel(logLevel LevelType) bool { func (logger *Logger) CheckLevel(logLevel LevelType) bool {
settedLevel := logger.level settedLevel := logger.level
if settedLevel == 0 { if settedLevel == 0 {

84
logv/main.go Normal file
View File

@ -0,0 +1,84 @@
package main
import (
"bufio"
"flag"
"fmt"
"io"
"os"
"apigo.cc/go/log"
)
func main() {
jsonMode := flag.Bool("json", false, "output in JSON format")
helpMode := flag.Bool("h", false, "show help")
flag.BoolVar(helpMode, "help", false, "show help")
flag.Usage = func() {
fmt.Fprintf(os.Stderr, "Usage: logv [options] [file1 file2 ...]\n")
fmt.Fprintf(os.Stderr, "Options:\n")
flag.PrintDefaults()
fmt.Fprintf(os.Stderr, "\nExamples:\n")
fmt.Fprintf(os.Stderr, " 1. 彩色可视化实时日志:\n")
fmt.Fprintf(os.Stderr, " tail -f app.log | logv\n")
fmt.Fprintf(os.Stderr, " 2. 格式化查看历史日志文件:\n")
fmt.Fprintf(os.Stderr, " logv error.log\n")
fmt.Fprintf(os.Stderr, " 3. 还原为标准 JSON 格式 (供 Filebeat / Logstash 收集):\n")
fmt.Fprintf(os.Stderr, " tail -f app.log | logv -json\n")
fmt.Fprintf(os.Stderr, " 4. 批量转换日志文件为 JSON:\n")
fmt.Fprintf(os.Stderr, " logv -json app.log.2026* > all_logs.json\n")
}
flag.Parse()
if *helpMode {
flag.Usage()
return
}
// Try to load meta file from current directory
_ = log.LoadMeta(".log.meta.json")
args := flag.Args()
if len(args) == 0 {
// Check if stdin is a terminal
stat, _ := os.Stdin.Stat()
if (stat.Mode() & os.ModeCharDevice) != 0 {
// Stdin is a terminal and no files provided, show usage
flag.Usage()
return
}
process(os.Stdin, *jsonMode)
} else {
for _, arg := range args {
f, err := os.Open(arg)
if err != nil {
fmt.Fprintf(os.Stderr, "logv: %v\n", err)
continue
}
process(f, *jsonMode)
f.Close()
}
}
}
func process(r io.Reader, jsonMode bool) {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := scanner.Text()
if len(line) == 0 {
continue
}
if jsonMode {
fmt.Println(log.ToJSON(line))
} else {
fmt.Println(log.Viewable(line))
}
}
if err := scanner.Err(); err != nil {
fmt.Fprintf(os.Stderr, "logv: error reading input: %v\n", err)
}
}

237
meta.go Normal file
View File

@ -0,0 +1,237 @@
package log
import (
"reflect"
"sort"
"strings"
"sync"
"apigo.cc/go/cast"
"apigo.cc/go/file"
)
// MetaField describes the serialization and visualization metadata for a single log field.
type MetaField struct {
Index int
Name string
KeyName string
AttachBefore bool
Color string
Format string
Precision int
WithoutKey bool
Hide bool
}
var (
metaRegistry = make(map[string][]MetaField)
metaLock sync.RWMutex
metaFilePath = ".log.meta.json"
)
// LoadMeta loads metadata from the specified file into the global registry.
func LoadMeta(path string) error {
metaLock.Lock()
defer metaLock.Unlock()
return file.UnmarshalFile(path, &metaRegistry)
}
// RegisterType registers a log model's metadata into the global registry.
// logType is the string identifier (e.g. "info", "error").
func RegisterType(logType string, model any) {
t := reflect.TypeOf(model)
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
// 强制检查 Reset 方法是否被显式实现(防止继承 BaseLog 后忘记重置业务字段)
if t.Kind() == reflect.Struct {
ptrType := reflect.PointerTo(t)
method, ok := ptrType.MethodByName("Reset")
if !ok {
panic("log model must implement Reset() method: " + t.Name())
}
// 检查该方法是否属于当前类型(而不是继承自 BaseLog 且没有被重写)
baseResetMethod, _ := reflect.PointerTo(reflect.TypeOf(BaseLog{})).MethodByName("Reset")
if method.Func.Pointer() == baseResetMethod.Func.Pointer() {
panic("log model must override Reset() method to clear its own fields: " + t.Name())
}
}
fields := extractMetaFields(model)
metaLock.Lock()
metaRegistry[logType] = fields
metaLock.Unlock()
syncMetaFile()
}
// GetMeta returns the metadata fields for a given logType.
func GetMeta(logType string) []MetaField {
metaLock.RLock()
defer metaLock.RUnlock()
return metaRegistry[logType]
}
// fieldInfo is used internally for storing fields with their absolute position.
type fieldInfo struct {
field reflect.StructField
pos int
}
func extractMetaFields(model any) []MetaField {
t := reflect.TypeOf(model)
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return nil
}
var flatFields []fieldInfo
flattenStructFields(t, &flatFields, nil)
// Determine final indices
maxLiteralPos := -1
var highPosFields []fieldInfo
for _, f := range flatFields {
if f.pos < 1000 {
if f.pos > maxLiteralPos {
maxLiteralPos = f.pos
}
} else {
highPosFields = append(highPosFields, f)
}
}
// Sort high pos fields by their pos
sort.Slice(highPosFields, func(i, j int) bool {
return highPosFields[i].pos < highPosFields[j].pos
})
// Assign real indices to high pos fields
finalPosMap := make(map[string]int)
for _, f := range flatFields {
if f.pos < 1000 {
finalPosMap[f.field.Name] = f.pos
}
}
nextPos := maxLiteralPos + 1
for _, f := range highPosFields {
finalPosMap[f.field.Name] = nextPos
nextPos++
}
maxPos := nextPos - 1
metaFields := make([]MetaField, maxPos+1)
// Initialize with empty MetaFields having Index set
for i := range metaFields {
metaFields[i] = MetaField{Index: i}
}
for _, f := range flatFields {
tag := f.field.Tag.Get("log")
if tag == "-" {
continue
}
realPos := finalPosMap[f.field.Name]
meta := MetaField{
Index: realPos,
Name: f.field.Name,
}
if tag != "" {
parts := strings.Split(tag, ",")
for _, part := range parts {
part = strings.TrimSpace(part)
if part == "attachBefore" {
meta.AttachBefore = true
continue
}
kv := strings.SplitN(part, ":", 2)
if len(kv) == 2 {
key := strings.TrimSpace(kv[0])
val := strings.TrimSpace(kv[1])
switch key {
case "color":
meta.Color = val
case "format":
meta.Format = val
case "withoutkey":
meta.WithoutKey = (val == "true")
case "hide":
meta.Hide = (val == "true")
case "keyname":
meta.KeyName = val
case "attachBefore":
meta.AttachBefore = (val == "true")
case "precision":
meta.Precision = cast.To[int](val)
}
}
}
}
// Apply some default visual rules if not specified
// LogType shouldn't show the key in standard console
if f.field.Name == "LogType" && meta.Color == "" {
meta.WithoutKey = true
}
metaFields[realPos] = meta
}
return metaFields
}
func flattenStructFields(t reflect.Type, result *[]fieldInfo, parentIndex []int) {
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if !f.IsExported() && !f.Anonymous {
continue
}
pos := 10 + i // default position if not specified
tag := f.Tag.Get("log")
if tag != "" {
parts := strings.Split(tag, ",")
for _, part := range parts {
kv := strings.SplitN(part, ":", 2)
if len(kv) == 2 && strings.TrimSpace(kv[0]) == "pos" {
if p := cast.To[int](strings.TrimSpace(kv[1])); p >= 0 {
pos = p
}
}
}
}
// Compute the full index path from the root
fullIndex := make([]int, len(parentIndex), len(parentIndex)+1)
copy(fullIndex, parentIndex)
fullIndex = append(fullIndex, i)
f.Index = fullIndex
if f.Anonymous && f.Type.Kind() == reflect.Struct {
flattenStructFields(f.Type, result, f.Index)
} else {
*result = append(*result, fieldInfo{
field: f,
pos: pos,
})
}
}
}
func syncMetaFile() {
metaLock.RLock()
defer metaLock.RUnlock()
_ = file.MarshalFilePretty(metaFilePath, metaRegistry)
}
// SetMetaFilePath allows changing the path for testing or configuration purposes
func SetMetaFilePath(path string) {
metaFilePath = path
}

105
meta_test.go Normal file
View File

@ -0,0 +1,105 @@
package log
import (
"encoding/json"
"os"
"testing"
)
type MockBaseLog struct {
BaseField1 string `log:"pos:0,color:red"`
BaseField2 int `log:"pos:1,withoutkey:true"`
}
func (b *MockBaseLog) Reset() {
b.BaseField1 = ""
b.BaseField2 = 0
}
func (b *MockBaseLog) IsLogEntry() bool { return true }
func (b *MockBaseLog) GetBaseLog() *BaseLog { return &BaseLog{} }
type MockInfoLog struct {
MockBaseLog
Message string `log:"pos:2"`
Extra map[string]any `log:"pos:1000"`
}
func (l *MockInfoLog) Reset() {
l.MockBaseLog.Reset()
l.Message = ""
clear(l.Extra)
}
type MockErrorLog struct {
MockBaseLog
Error string `log:"pos:2,color:red"`
CallStacks []string `log:"pos:1001"`
Extra map[string]any `log:"pos:1000"`
}
func (l *MockErrorLog) Reset() {
l.MockBaseLog.Reset()
l.Error = ""
l.CallStacks = l.CallStacks[:0]
clear(l.Extra)
}
func TestMetaExtraction(t *testing.T) {
// Setup custom meta file path for testing
SetMetaFilePath(".test.meta.json")
defer os.Remove(".test.meta.json")
RegisterType("mock_info", MockInfoLog{})
RegisterType("mock_error", MockErrorLog{})
infoMeta := GetMeta("mock_info")
// Index 0, 1, 2 are used, Extra gets max(2)+1=3. Total size 4.
if len(infoMeta) != 4 {
t.Fatalf("expected 4 fields for mock_info, got %d", len(infoMeta))
}
if infoMeta[0].Name != "BaseField1" || infoMeta[0].Color != "red" {
t.Errorf("unexpected meta for BaseField1 at index 0: %+v", infoMeta[0])
}
if infoMeta[1].Name != "BaseField2" || infoMeta[1].WithoutKey != true {
t.Errorf("unexpected meta for BaseField2 at index 1: %+v", infoMeta[1])
}
if infoMeta[2].Name != "Message" {
t.Errorf("unexpected meta for Message at index 2: %+v", infoMeta[2])
}
if infoMeta[3].Name != "Extra" {
t.Errorf("unexpected meta for Extra at index 3: %+v", infoMeta[3])
}
errorMeta := GetMeta("mock_error")
// Indices: 0, 1, 2, Extra(3), CallStacks(4). Total size 5.
if len(errorMeta) != 5 {
t.Fatalf("expected 5 fields for mock_error, got %d", len(errorMeta))
}
if errorMeta[2].Name != "Error" || errorMeta[2].Color != "red" {
t.Errorf("unexpected meta for Error at index 2: %+v", errorMeta[2])
}
if errorMeta[3].Name != "Extra" {
t.Errorf("unexpected meta for Extra at index 3: %+v", errorMeta[3])
}
if errorMeta[4].Name != "CallStacks" {
t.Errorf("unexpected meta for CallStacks at index 4: %+v", errorMeta[4])
}
// Verify file was created and contains correct data
data, err := os.ReadFile(".test.meta.json")
if err != nil {
t.Fatalf("failed to read test meta file: %v", err)
}
var registry map[string][]MetaField
if err := json.Unmarshal(data, &registry); err != nil {
t.Fatalf("failed to unmarshal test meta file: %v", err)
}
if len(registry) < 2 {
t.Errorf("expected at least 2 types in registry, got %d", len(registry))
}
}

21
name_test.go Normal file
View File

@ -0,0 +1,21 @@
package log
import (
"testing"
)
func TestSetDefaultName(t *testing.T) {
oldName := getDefaultName()
defer SetDefaultName(oldName)
newName := "test-service-name"
SetDefaultName(newName)
if getDefaultName() != newName {
t.Errorf("GetDefaultName() = %v, want %v", getDefaultName(), newName)
}
if DefaultLogger.config.Name != newName {
t.Errorf("DefaultLogger.config.Name = %v, want %v", DefaultLogger.config.Name, newName)
}
}

86
pool.go
View File

@ -27,88 +27,14 @@ func GetEntry[T any]() *T {
}) })
} }
entry := p.(*sync.Pool).Get().(*T) entry := p.(*sync.Pool).Get().(*T)
ResetLogEntry(entry) // 自动重置所有字段,无需子类实现 Reset if le, ok := any(entry).(LogEntry); ok {
le.Reset()
}
return entry return entry
} }
// ResetLogEntry 使用反射自动化重置日志对象的所有字段 // putEntry 将日志对象归还到池中
// 特别是对 Map 和 Slice 进行初始化长度0容量8 func putEntry(entry any) {
func ResetLogEntry(v any) {
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr || rv.IsNil() {
return
}
t := rv.Type()
resetFunc, ok := resetCache.Load(t)
if !ok {
resetFunc = buildResetFunc(t.Elem())
resetCache.Store(t, resetFunc)
}
resetFunc.(func(reflect.Value))(rv.Elem())
}
func buildResetFunc(t reflect.Type) func(reflect.Value) {
var funcs []func(reflect.Value)
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
fieldIdx := i
switch field.Type.Kind() {
case reflect.String:
funcs = append(funcs, func(rv reflect.Value) { rv.Field(fieldIdx).SetString("") })
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
funcs = append(funcs, func(rv reflect.Value) { rv.Field(fieldIdx).SetInt(0) })
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
funcs = append(funcs, func(rv reflect.Value) { rv.Field(fieldIdx).SetUint(0) })
case reflect.Float32, reflect.Float64:
funcs = append(funcs, func(rv reflect.Value) { rv.Field(fieldIdx).SetFloat(0) })
case reflect.Bool:
funcs = append(funcs, func(rv reflect.Value) { rv.Field(fieldIdx).SetBool(false) })
case reflect.Map:
funcs = append(funcs, func(rv reflect.Value) {
f := rv.Field(fieldIdx)
if f.IsNil() {
f.Set(reflect.MakeMapWithSize(f.Type(), 8))
} else {
f.Clear()
}
})
case reflect.Slice:
funcs = append(funcs, func(rv reflect.Value) {
f := rv.Field(fieldIdx)
if f.Cap() < 8 {
f.Set(reflect.MakeSlice(f.Type(), 0, 8))
} else {
f.SetLen(0)
}
})
case reflect.Struct:
subReset := buildResetFunc(field.Type)
funcs = append(funcs, func(rv reflect.Value) {
subReset(rv.Field(fieldIdx))
})
case reflect.Ptr, reflect.Interface:
zero := reflect.Zero(field.Type)
funcs = append(funcs, func(rv reflect.Value) {
rv.Field(fieldIdx).Set(zero)
})
}
}
return func(rv reflect.Value) {
for _, f := range funcs {
f(rv)
}
}
}
func resetStruct(rv reflect.Value) {
// 已经不再直接调用,保留 buildResetFunc 逻辑即可
}
// PutEntry 将日志对象归还到池中
func PutEntry(entry any) {
t := reflect.TypeOf(entry) t := reflect.TypeOf(entry)
if pool, ok := globalPools.pools.Load(t); ok { if pool, ok := globalPools.pools.Load(t); ok {
pool.(*sync.Pool).Put(entry) pool.(*sync.Pool).Put(entry)
@ -118,7 +44,7 @@ func PutEntry(entry any) {
// WithEntry 执行闭包并在结束后自动回收对象 // WithEntry 执行闭包并在结束后自动回收对象
func WithEntry[T any](fn func(*T)) { func WithEntry[T any](fn func(*T)) {
entry := GetEntry[T]() entry := GetEntry[T]()
defer PutEntry(entry) defer putEntry(entry)
fn(entry) fn(entry)
} }

View File

@ -11,6 +11,12 @@ type MockRequestLog struct {
UsedTime float32 UsedTime float32
} }
func (l *MockRequestLog) Reset() {
l.BaseLog.Reset()
l.RequestId = ""
l.UsedTime = 0
}
func TestWithEntry(t *testing.T) { func TestWithEntry(t *testing.T) {
WithEntry(func(entry *MockRequestLog) { WithEntry(func(entry *MockRequestLog) {
entry.RequestId = "with-entry-id" entry.RequestId = "with-entry-id"

View File

@ -28,8 +28,7 @@ func TestLoggerReliability(t *testing.T) {
} }
wg.Wait() wg.Wait()
Stop() WriterService.Stop(nil)
Wait()
file, err := os.Open(logFile) file, err := os.Open(logFile)
if err != nil { if err != nil {

197
serializer.go Normal file
View File

@ -0,0 +1,197 @@
package log
import (
"bytes"
"reflect"
"sort"
"strconv"
"apigo.cc/go/cast"
)
type fieldAccessor struct {
indexPath []int
name string
}
var (
accessorsCache = make(map[string][]fieldAccessor)
)
// getAccessors caches the reflection index paths for the flattened fields.
func getAccessors(logType string, model any) []fieldAccessor {
metaLock.RLock()
if acc, ok := accessorsCache[logType]; ok {
metaLock.RUnlock()
return acc
}
metaLock.RUnlock()
metaLock.Lock()
defer metaLock.Unlock()
// Double check
if acc, ok := accessorsCache[logType]; ok {
return acc
}
t := reflect.TypeOf(model)
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
var flatFields []fieldInfo
flattenStructFields(t, &flatFields, nil)
// Determine final indices (must match meta.go)
maxLiteralPos := -1
var highPosFields []fieldInfo
for _, f := range flatFields {
if f.pos < 1000 {
if f.pos > maxLiteralPos {
maxLiteralPos = f.pos
}
} else {
highPosFields = append(highPosFields, f)
}
}
// Sort high pos fields by their pos
sort.Slice(highPosFields, func(i, j int) bool {
return highPosFields[i].pos < highPosFields[j].pos
})
finalPosMap := make(map[string]int)
for _, f := range flatFields {
if f.pos < 1000 {
finalPosMap[f.field.Name] = f.pos
}
}
nextPos := maxLiteralPos + 1
for _, f := range highPosFields {
finalPosMap[f.field.Name] = nextPos
nextPos++
}
maxPos := nextPos - 1
accessors := make([]fieldAccessor, maxPos+1)
for _, f := range flatFields {
if f.field.Tag.Get("log") == "-" {
continue
}
realPos := finalPosMap[f.field.Name]
accessors[realPos] = fieldAccessor{
indexPath: f.field.Index,
name: f.field.Name,
}
}
accessorsCache[logType] = accessors
return accessors
}
// donot export this function
func Marshal(entry LogEntry, sensitiveKeys []string) []byte {
var buf bytes.Buffer
buf.WriteByte('[')
base := entry.GetBaseLog()
if base == nil {
buf.WriteByte(']')
return buf.Bytes()
}
logType := base.LogType
if logType == "" {
// Fallback for undefined types
logType = "undefined"
}
accessors := getAccessors(logType, entry)
v := reflect.ValueOf(entry)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
for i, acc := range accessors {
if i > 0 {
buf.WriteByte(',')
}
if acc.indexPath == nil {
buf.WriteByte('0')
continue
}
fv := v.FieldByIndex(acc.indexPath)
writeValue(&buf, fv, acc.name, sensitiveKeys)
}
buf.WriteByte(']')
return buf.Bytes()
}
func writeValue(buf *bytes.Buffer, v reflect.Value, fieldName string, sensitiveKeys []string) {
if !v.IsValid() {
buf.WriteString("null")
return
}
// Check if this root field should be desensitized
if len(sensitiveKeys) > 0 {
fixedName := fixField(fieldName)
for _, sk := range sensitiveKeys {
if sk == fixedName {
buf.WriteString(`"***"`)
return
}
}
}
switch v.Kind() {
case reflect.String:
writeString(buf, v.String())
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
buf.WriteString(strconv.FormatInt(v.Int(), 10))
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
buf.WriteString(strconv.FormatUint(v.Uint(), 10))
case reflect.Float32, reflect.Float64:
buf.WriteString(strconv.FormatFloat(v.Float(), 'g', -1, 64))
case reflect.Bool:
if v.Bool() {
buf.WriteString("true")
} else {
buf.WriteString("false")
}
default:
// Use cast for complex types to ensure deep desensitization
b, _ := cast.ToJSONDesensitizeBytes(v.Interface(), sensitiveKeys)
if len(b) == 0 {
buf.WriteString("null")
} else {
buf.Write(b)
}
}
}
func writeString(buf *bytes.Buffer, s string) {
buf.WriteByte('"')
for i := 0; i < len(s); i++ {
c := s[i]
switch c {
case '\\':
buf.WriteString(`\\`)
case '"':
buf.WriteString(`\"`)
case '\n':
buf.WriteString(`\n`)
case '\r':
buf.WriteString(`\r`)
case '\t':
buf.WriteString(`\t`)
default:
buf.WriteByte(c)
}
}
buf.WriteByte('"')
}

129
serializer_test.go Normal file
View File

@ -0,0 +1,129 @@
package log
import (
"encoding/json"
"testing"
)
type SerializerMockBaseLog struct {
LogName string `log:"pos:0"`
LogType string `log:"pos:1"`
LogTime int64 `log:"pos:2"`
TraceId string `log:"pos:3"`
}
func (b *SerializerMockBaseLog) IsLogEntry() bool {
return true
}
func (b *SerializerMockBaseLog) GetBaseLog() *BaseLog {
return &BaseLog{LogType: b.LogType}
}
func (b *SerializerMockBaseLog) Reset() {
b.LogName = ""
b.LogType = ""
b.LogTime = 0
b.TraceId = ""
}
type SerializerMockInfoLog struct {
SerializerMockBaseLog
Message string `log:"pos:4"`
Extra map[string]any `log:"pos:1000"`
}
func (l *SerializerMockInfoLog) Reset() {
l.SerializerMockBaseLog.Reset()
l.Message = ""
if l.Extra == nil {
l.Extra = make(map[string]any, 8)
} else {
clear(l.Extra)
}
}
func TestToArrayBytes(t *testing.T) {
entry := &SerializerMockInfoLog{
SerializerMockBaseLog: SerializerMockBaseLog{
LogName: "test-app",
LogType: "mock_info_test",
LogTime: 1620000000,
TraceId: "abc-123",
},
Message: "Hello, World!",
Extra: map[string]any{
"user_id": 42,
},
}
RegisterType("mock_info_test", entry) // trigger meta generation
bytes := Marshal(entry, nil)
str := string(bytes)
t.Logf("Raw log: %s", str)
var arr []any
err := json.Unmarshal(bytes, &arr)
if err != nil {
t.Fatalf("failed to unmarshal generated array: %v, raw: %s", err, str)
}
// Indices: 0, 1, 2, 3, 4, 1000(mapped to 5). Total size 6.
if len(arr) != 6 {
t.Fatalf("expected 6 elements, got %d. raw: %s", len(arr), str)
}
if arr[0] != "test-app" {
t.Errorf("expected arr[0] == 'test-app', got %v", arr[0])
}
if arr[1] != "mock_info_test" {
t.Errorf("expected arr[1] == 'mock_info_test', got %v", arr[1])
}
// JSON numbers are parsed as float64
if arr[2] != float64(1620000000) {
t.Errorf("expected arr[2] == 1620000000, got %v", arr[2])
}
if arr[3] != "abc-123" {
t.Errorf("expected arr[3] == 'abc-123', got %v", arr[3])
}
if arr[4] != "Hello, World!" {
t.Errorf("expected arr[4] == 'Hello, World!', got %v", arr[4])
}
extraMap, ok := arr[5].(map[string]any)
if !ok {
t.Fatalf("expected arr[5] to be map[string]any, got %T (value: %v)", arr[5], arr[5])
}
if extraMap["user_id"] != float64(42) {
t.Errorf("expected extraMap['user_id'] == 42, got %v", extraMap["user_id"])
}
}
func TestToArrayBytes_Desensitize(t *testing.T) {
entry := &SerializerMockInfoLog{
SerializerMockBaseLog: SerializerMockBaseLog{
LogType: "mock_info_test2",
},
Message: "Sensitive Info",
Extra: map[string]any{
"password": "my-secret-password",
},
}
RegisterType("mock_info_test2", entry)
bytes := Marshal(entry, []string{"password"})
str := string(bytes)
var arr []any
err := json.Unmarshal(bytes, &arr)
if err != nil {
t.Fatalf("failed to unmarshal generated array: %v, raw: %s", err, str)
}
extraMap := arr[5].(map[string]any)
if extraMap["password"] != "***" {
t.Errorf("expected password to be desensitized, got %v", extraMap["password"])
}
}

View File

@ -14,28 +14,26 @@ const LogTypeMonitor = "monitor"
const LogTypeStatistic = "statistic" const LogTypeStatistic = "statistic"
const LogTypeRequest = "request" const LogTypeRequest = "request"
const LogDefaultSensitive = "phone,password,secret,token,accessToken" const LogDefaultSensitive = "phone,password,secret,token,accessToken,authorization"
const LogEnvLevel = "LOG_LEVEL" const LogEnvLevel = "LOG_LEVEL"
const LogEnvFile = "LOG_FILE" const LogEnvFile = "LOG_FILE"
const LogEnvSensitive = "LOG_SENSITIVE" const LogEnvSensitive = "LOG_SENSITIVE"
const LogEnvRegexSensitive = "LOG_REGEXSENSITIVE"
// LogEntry 是一个标记接口,用于识别是否为对象池管理的日志对象 // LogEntry 是一个标记接口,用于识别是否为对象池管理的日志对象
type LogEntry interface { type LogEntry interface {
IsLogEntry() bool IsLogEntry() bool
GetBaseLog() *BaseLog GetBaseLog() *BaseLog
Reset()
} }
type BaseLog struct { type BaseLog struct {
LogName string LogName string `log:"pos:0,color:cyan,hide:true"`
LogType string LogType string `log:"pos:1,color:magenta,hide:true"`
LogTime int64 LogTime int64 `log:"pos:2,format:time"`
TraceId string TraceId string `log:"pos:3,color:gray,withoutkey:true"`
ImageName string Image string `log:"pos:4,hide:true"`
ImageTag string Server string `log:"pos:5,hide:true"`
ServerName string Extra map[string]any `log:"pos:1000"`
ServerIp string
Extra map[string]any
} }
func (b *BaseLog) IsLogEntry() bool { func (b *BaseLog) IsLogEntry() bool {
@ -46,24 +44,67 @@ func (b *BaseLog) GetBaseLog() *BaseLog {
return b return b
} }
func (b *BaseLog) Reset() {
b.LogName = ""
b.LogType = ""
b.LogTime = 0
b.TraceId = ""
b.Image = ""
b.Server = ""
if b.Extra == nil {
b.Extra = make(map[string]any, 8)
} else {
clear(b.Extra)
}
}
type DebugLog struct { type DebugLog struct {
BaseLog BaseLog
Debug string Debug string `log:"pos:6,withoutkey:true"` // white
}
func (l *DebugLog) Reset() {
l.BaseLog.Reset()
l.Debug = ""
} }
type InfoLog struct { type InfoLog struct {
BaseLog BaseLog
Info string Info string `log:"pos:6,color:cyan,withoutkey:true"`
}
func (l *InfoLog) Reset() {
l.BaseLog.Reset()
l.Info = ""
} }
type WarningLog struct { type WarningLog struct {
BaseLog BaseLog
Warning string Warning string `log:"pos:6,color:yellow,withoutkey:true"`
CallStacks []string CallStacks []string `log:"pos:1001"`
}
func (l *WarningLog) Reset() {
l.BaseLog.Reset()
l.Warning = ""
l.CallStacks = l.CallStacks[:0]
} }
type ErrorLog struct { type ErrorLog struct {
BaseLog BaseLog
Error string Error string `log:"pos:6,color:red,withoutkey:true"`
CallStacks []string CallStacks []string `log:"pos:1001"`
}
func (l *ErrorLog) Reset() {
l.BaseLog.Reset()
l.Error = ""
l.CallStacks = l.CallStacks[:0]
}
func init() {
RegisterType(LogTypeDebug, &DebugLog{})
RegisterType(LogTypeInfo, &InfoLog{})
RegisterType(LogTypeWarning, &WarningLog{})
RegisterType(LogTypeError, &ErrorLog{})
} }

View File

@ -1,16 +1,13 @@
package log package log
import ( import (
"encoding/json"
"fmt" "fmt"
"net" "net"
"os" "os"
"path" "path"
"runtime" "runtime"
"runtime/debug"
"strings" "strings"
"time"
"apigo.cc/go/cast"
) )
var ( var (
@ -24,104 +21,35 @@ func init() {
dockerImageName = os.Getenv("DOCKER_IMAGE_NAME") dockerImageName = os.Getenv("DOCKER_IMAGE_NAME")
dockerImageTag = os.Getenv("DOCKER_IMAGE_TAG") dockerImageTag = os.Getenv("DOCKER_IMAGE_TAG")
serverName, _ = os.Hostname() serverName, _ = os.Hostname()
addrs, err := net.InterfaceAddrs()
// 获取真实局域网 IP (UDP 8.8.8.8 伪拨号法)
conn, err := net.Dial("udp", "8.8.8.8:80")
if err == nil { if err == nil {
for _, a := range addrs { localAddr := conn.LocalAddr().(*net.UDPAddr)
if an, ok := a.(*net.IPNet); ok { serverIp = localAddr.IP.String()
// 忽略 Docker 私有网段 _ = conn.Close()
if an.IP.IsGlobalUnicast() && !strings.HasPrefix(an.IP.To4().String(), "172.17.") { }
serverIp = an.IP.To4().String()
break if serverIp == "" {
addrs, err := net.InterfaceAddrs()
if err == nil {
for _, a := range addrs {
if an, ok := a.(*net.IPNet); ok {
if an.IP.IsGlobalUnicast() {
serverIp = an.IP.To4().String()
break
}
} }
} }
} }
} }
} }
// MakeTime 解析纳秒时间戳或 RFC3339 字符串 // fixField 格式化字段名(去横线、下划线,小写)
func MakeTime(v any) time.Time {
if ts, ok := cast.ToInt64E(v); ok == nil {
return time.Unix(0, ts)
}
tm, _ := time.Parse(time.RFC3339Nano, cast.String(v))
return tm
}
// MakeUsedTime 计算消耗时间(毫秒)
func MakeUsedTime(startTime, endTime time.Time) float32 {
return float32(endTime.UnixNano()-startTime.UnixNano()) / 1e6
}
// ParseBaseLog 解析基础日志行
func ParseBaseLog(line string) *BaseLog {
pos := strings.IndexByte(line, '{')
if pos == -1 {
return ParseBadLog(line)
}
l := make(map[string]any)
err := json.Unmarshal([]byte(line[pos:]), &l)
if err != nil {
return ParseBadLog(line)
}
baseLog := BaseLog{Extra: make(map[string]any)}
for k, v := range l {
lk := strings.ToLower(k)
switch lk {
case "logname":
baseLog.LogName = cast.String(v)
case "logtype":
baseLog.LogType = cast.String(v)
case "logtime":
baseLog.LogTime = cast.Int64(v)
case "traceid":
baseLog.TraceId = cast.String(v)
case "imagename":
baseLog.ImageName = cast.String(v)
case "imagetag":
baseLog.ImageTag = cast.String(v)
case "servername":
baseLog.ServerName = cast.String(v)
case "serverip":
baseLog.ServerIp = cast.String(v)
default:
baseLog.Extra[lk] = v
}
}
return &baseLog
}
// ParseBadLog 解析非 JSON 格式的日志
func ParseBadLog(line string) *BaseLog {
baseLog := BaseLog{Extra: make(map[string]any)}
baseLog.LogType = LogTypeUndefined
if len(line) > 19 && line[19] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05", line[0:19])
if err == nil {
baseLog.LogTime = tm.UnixNano()
line = line[20:]
} else {
return nil
}
} else if len(line) > 26 && line[26] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05.000000", line[0:26])
if err == nil {
baseLog.LogTime = tm.UnixNano()
line = line[27:]
} else {
return nil
}
} else {
return nil
}
baseLog.Extra["info"] = line
return &baseLog
}
// fixField 格式化字段名(去横线,小写)
func fixField(s string) string { func fixField(s string) string {
return strings.ToLower(strings.ReplaceAll(s, "-", "")) s = strings.ReplaceAll(s, "-", "")
s = strings.ReplaceAll(s, "_", "")
return strings.ToLower(s)
} }
// getCallStacks 获取调用栈 // getCallStacks 获取调用栈
@ -141,6 +69,7 @@ func getCallStacks(truncations []string) []string {
isLogInternal := (strings.Contains(file, "/log/logger.go") || isLogInternal := (strings.Contains(file, "/log/logger.go") ||
strings.Contains(file, "/log/utility.go") || strings.Contains(file, "/log/utility.go") ||
strings.Contains(file, "/log/standard.go") || strings.Contains(file, "/log/standard.go") ||
strings.Contains(file, "/log/default_logger.go") ||
strings.Contains(file, "/log/extra.go")) strings.Contains(file, "/log/extra.go"))
if isLogInternal { if isLogInternal {
@ -163,24 +92,21 @@ func getCallStacks(truncations []string) []string {
return callStacks return callStacks
} }
// GetDefaultName 获取默认应用名称 var globalDefaultName string
func GetDefaultName() string {
name := os.Getenv("DISCOVER_APP") // getDefaultName 获取默认应用名称
if name == "" { func getDefaultName() string {
name = os.Getenv("discover_app") if globalDefaultName != "" {
return globalDefaultName
} }
if name == "" { name := ""
imageName := os.Getenv("DOCKER_IMAGE_NAME") if info, ok := debug.ReadBuildInfo(); ok && info.Path != "" && info.Path != "command-line-arguments" {
if imageName != "" { name = path.Base(info.Path)
parts := strings.Split(imageName, "/")
imageName = parts[len(parts)-1]
imageName = strings.SplitN(imageName, ":", 2)[0]
imageName = strings.SplitN(imageName, "#", 2)[0]
name = imageName
}
} }
if name == "" { if name == "" {
name = path.Base(os.Args[0]) name = path.Base(os.Args[0])
} }
// 处理 Windows 下的 .exe 后缀
name = strings.TrimSuffix(name, ".exe")
return name return name
} }

386
viewer.go
View File

@ -1,9 +1,10 @@
package log package log
import ( import (
"encoding/json"
"fmt" "fmt"
"os"
"regexp" "regexp"
"sort"
"strings" "strings"
"time" "time"
@ -13,11 +14,12 @@ import (
var errorLineMatcher = regexp.MustCompile(`(\w+\.go:\d+)`) var errorLineMatcher = regexp.MustCompile(`(\w+\.go:\d+)`)
var codeFileMatcher = regexp.MustCompile(`(\w+?\.)(go|js)`) var codeFileMatcher = regexp.MustCompile(`(\w+?\.)(go|js)`)
var workspaceRoot, _ = os.Getwd()
func Viewable(line string) string { func Viewable(line string) string {
b := ParseBaseLog(line) line = strings.TrimSpace(line)
if b == nil { if !strings.HasPrefix(line, "[") {
// 高亮错误代码 // Fallback highlight for non-array strings
if strings.Contains(line, ".go:") { if strings.Contains(line, ".go:") {
if strings.Contains(line, "/ssgo/") || strings.Contains(line, "/ssdo/") || strings.Contains(line, "/gojs/") { if strings.Contains(line, "/ssgo/") || strings.Contains(line, "/ssdo/") || strings.Contains(line, "/gojs/") {
line = errorLineMatcher.ReplaceAllString(line, shell.BYellow("$1")) line = errorLineMatcher.ReplaceAllString(line, shell.BYellow("$1"))
@ -30,139 +32,293 @@ func Viewable(line string) string {
return line return line
} }
logTime := time.Unix(0, b.LogTime) var arr []any
if err := cast.UnmarshalJSON([]byte(line), &arr); err != nil {
return line
}
if len(arr) < 2 {
return line
}
logType := ""
if len(arr) > 2 {
logType = cast.String(arr[2])
}
meta := GetMeta(logType)
if len(meta) == 0 {
logType = cast.String(arr[1])
meta = GetMeta(logType)
}
if len(meta) == 0 {
// Fallback rendering
return fallbackRenderArray(arr)
}
var builder strings.Builder var builder strings.Builder
builder.WriteString(shell.White(shell.Bold, logTime.Format("01-02 15:04:05.000")))
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Underline, b.TraceId))
level := "" for i, v := range arr {
for _, k := range []string{"info", "warning", "error", "debug"} { if v == nil || cast.String(v) == "0" { // 0 is gap
if v := b.Extra[k]; v != nil && cast.String(v) != "" { continue
level = k
break
} }
} if i >= len(meta) {
// Unmapped trailing values, just print them
if b.LogType == LogTypeRequest {
method := cast.String(b.Extra["method"])
path := cast.String(b.Extra["path"])
code := cast.Int(b.Extra["responsecode"])
used := float32(cast.Float64(b.Extra["usedtime"]))
builder.WriteString(" ")
builder.WriteString(shell.Cyan(shell.Bold, "REQUEST"))
builder.WriteString(" ")
builder.WriteString(shell.Cyan(method))
builder.WriteString(" ")
builder.WriteString(path)
builder.WriteString(" ")
if code >= 500 {
builder.WriteString(shell.BRed(cast.String(code)))
} else if code >= 400 {
builder.WriteString(shell.BYellow(cast.String(code)))
} else {
builder.WriteString(shell.BGreen(cast.String(code)))
}
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.Dim, fmt.Sprintf("%.2fms", used)))
for _, k := range []string{"method", "path", "responsecode", "usedtime", "host", "scheme", "proto", "clientip", "serverid", "app", "node", "fromapp", "fromnode", "userid", "deviceid", "clientappname", "clientappversion", "sessionid", "requestid", "authlevel", "priority", "requestheaders", "requestdata", "responseheaders", "responsedatalength", "responsedata", "logname", "logtype", "logtime", "traceid", "imagename", "imagetag", "servername", "serverip"} {
delete(b.Extra, k)
}
} else if b.LogType == LogTypeStatistic {
builder.WriteString(" ")
builder.WriteString(shell.Cyan(shell.Bold, "STATISTIC"))
} else if b.LogType == LogTypeTask {
builder.WriteString(" ")
builder.WriteString(shell.Cyan(shell.Bold, "TASK"))
} else {
if level != "" {
msg := cast.String(b.Extra[level])
delete(b.Extra, level)
builder.WriteString(" ") builder.WriteString(" ")
switch level { builder.WriteString(shell.Style(shell.Dim, fmt.Sprintf("Index%d:", i)))
case "info": builder.WriteString(cast.String(v))
builder.WriteString(shell.Cyan(msg)) continue
case "warning": }
builder.WriteString(shell.Yellow(msg))
case "error": m := meta[i]
builder.WriteString(shell.Red(msg))
case "debug": if m.Hide || m.Name == "" {
builder.WriteString(msg) continue
}
if m.Name == "Extra" {
extraMap, ok := v.(map[string]any)
if ok && len(extraMap) > 0 {
keys := make([]string, 0, len(extraMap))
for k := range extraMap {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
ev := extraMap[k]
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":"))
builder.WriteString(renderValue(ev, 0, ""))
}
} }
} else if b.LogType == "undefined" { continue
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.Dim, "-"))
} else {
builder.WriteString(" ")
builder.WriteString(shell.Cyan(shell.Bold, b.LogType))
} }
}
callStacks := b.Extra["callstacks"] if m.Name == "CallStacks" {
delete(b.Extra, "callstacks") callStacksList, ok := v.([]any)
if ok && len(callStacksList) > 0 {
stackColor := shell.TextRed
if strings.Contains(strings.ToLower(logType), "warn") {
stackColor = shell.TextYellow
}
if b.Extra != nil { builder.WriteString("\n")
for k, v := range b.Extra { for _, vi := range callStacksList {
vStr := "" vStr := cast.String(vi)
if v == nil { if workspaceRoot != "" {
vStr = strings.TrimPrefix(vStr, workspaceRoot)
vStr = strings.TrimPrefix(vStr, "/")
}
postfix := ""
if pos := strings.LastIndexByte(vStr, '/'); pos != -1 {
postfix = vStr[pos+1:]
vStr = vStr[:pos+1]
} else {
postfix = vStr
vStr = ""
}
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.Dim, vStr))
builder.WriteString(shell.Style(stackColor, postfix))
builder.WriteString("\n")
}
}
continue
}
// Handle normal fields
vStr := ""
if m.Format == "time" {
// Convert int64 ns to time string
logTime := time.Unix(0, cast.Int64(v))
dateStr := logTime.Format("01-02")
timeStr := logTime.Format("15:04:05")
milliStr := logTime.Format(".000")
if builder.Len() > 0 {
builder.WriteString(" ")
}
builder.WriteString(shell.Style(shell.Dim, dateStr))
builder.WriteString(" ")
builder.WriteString(shell.White(shell.Bold, timeStr))
builder.WriteString(shell.Style(shell.Dim, milliStr))
continue
} else {
vStr = renderValue(v, m.Precision, m.Color)
if vStr == "" {
continue continue
} }
switch v.(type) { }
case map[string]any, []any:
vStr, _ = cast.ToJSON(v) if builder.Len() > 0 {
default: if m.AttachBefore {
vStr = cast.String(v) builder.WriteString(":")
} else {
builder.WriteString(" ")
}
}
if !m.WithoutKey && !m.AttachBefore {
name := m.KeyName
if name == "" {
name = m.Name
}
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, name+":"))
}
builder.WriteString(vStr)
}
return builder.String()
}
// ToJSON converts a JSON array log line to a standard JSON object string based on metadata.
func ToJSON(line string) string {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "[") {
return line
}
var arr []any
if err := cast.UnmarshalJSON([]byte(line), &arr); err != nil {
return line
}
if len(arr) < 2 {
return line
}
logType := ""
if len(arr) > 2 {
logType = cast.String(arr[2])
}
meta := GetMeta(logType)
if len(meta) == 0 {
logType = cast.String(arr[1])
meta = GetMeta(logType)
}
if len(meta) == 0 {
return line
}
result := make(map[string]any)
for i, v := range arr {
if i < len(meta) {
m := meta[i]
if m.Name == "" {
continue
} }
if k == "extra" && len(vStr) > 0 && vStr[0] == '{' { name := m.KeyName
extra, err := cast.ToMap[string, any](vStr) if name == "" {
if err == nil { name = m.Name
for k2, v2 := range extra { }
builder.WriteString(" ")
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k2+":")) if m.Name == "Extra" {
builder.WriteString(cast.String(v2)) if extraMap, ok := v.(map[string]any); ok {
for k, ev := range extraMap {
result[k] = ev
} }
} }
} else { } else {
builder.WriteString(" ") result[name] = v
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":"))
builder.WriteString(vStr)
} }
} else if cast.String(v) != "0" {
result[fmt.Sprintf("Extra%d", i)] = v
} }
} }
if callStacks != nil { jsonStr, _ := cast.ToJSON(result)
var callStacksList []any return jsonStr
switch cs := callStacks.(type) { }
case string:
if len(cs) > 2 && cs[0] == '[' {
_ = json.Unmarshal([]byte(cs), &callStacksList)
}
case []any:
callStacksList = cs
}
if len(callStacksList) > 0 { func applyColor(text string, color string) string {
builder.WriteString("\n") switch color {
for _, vi := range callStacksList { case "red":
v := cast.String(vi) return shell.Red(text)
postfix := "" case "cyan":
if pos := strings.LastIndexByte(v, '/'); pos != -1 { return shell.Cyan(text)
postfix = v[pos+1:] case "blue":
v = v[:pos+1] return shell.Blue(text)
} else { case "magenta":
postfix = v return shell.Magenta(text)
v = "" case "yellow":
} return shell.Yellow(text)
builder.WriteString(" ") case "green":
builder.WriteString(shell.Style(shell.Dim, v)) return shell.Green(text)
builder.WriteString(shell.Style(shell.TextWhite, postfix)) case "gray":
builder.WriteString("\n") return shell.Style(shell.Dim, text)
} default:
return text
}
}
func fallbackRenderArray(arr []any) string {
var builder strings.Builder
for i, v := range arr {
if i > 0 {
builder.WriteString(" ")
} }
builder.WriteString(cast.String(v))
} }
return builder.String() return builder.String()
} }
func renderValue(v any, precision int, color string) string {
if v == nil {
return ""
}
switch val := v.(type) {
case float32, float64:
vStr := ""
if precision > 0 {
vStr = fmt.Sprintf("%.*f", precision, cast.To[float64](v))
} else {
vStr = cast.String(v)
}
return applyColor(vStr, color)
case map[string]any:
if len(val) == 0 {
return ""
}
var parts []string
keys := make([]string, 0, len(val))
for k := range val {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
// Key is always dim, value is colored
vStr := renderValue(val[k], precision, color)
if vStr != "" {
parts = append(parts, fmt.Sprintf("%s:%s", shell.Style(shell.Dim, k), vStr))
}
}
if len(parts) == 0 {
return ""
}
return "[ " + strings.Join(parts, " ") + " ]"
case []any:
if len(val) == 0 {
return ""
}
var parts []string
for _, iv := range val {
vStr := renderValue(iv, precision, color)
if vStr != "" {
parts = append(parts, vStr)
}
}
if len(parts) == 0 {
return ""
}
return "[ " + strings.Join(parts, " ") + " ]"
default:
s := cast.String(v)
if s == "" {
return ""
}
return applyColor(s, color)
}
}

View File

@ -1,29 +1,230 @@
package log_test package log_test
import ( import (
"os"
"strings"
"testing" "testing"
"apigo.cc/go/log" "apigo.cc/go/log"
) )
func BenchmarkViewable(b *testing.B) { func TestViewable(t *testing.T) {
// 准备一个典型的 JSON 日志行,注意 Info, Warning 等在顶层 entry := &log.InfoLog{
line := `{"LogName":"test-app","LogType":"info","LogTime":1714896000000000000,"TraceId":"trace-123","info":"hello world","Extra":{"key":"value"}}` BaseLog: log.BaseLog{
LogName: "test-app",
b.ResetTimer() LogType: "info",
b.ReportAllocs() LogTime: 1714896000000000000,
for i := 0; i < b.N; i++ { TraceId: "trace-123",
_ = log.Viewable(line) },
Info: "hello world",
}
log.RegisterType("info", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
if !strings.Contains(out, "hello world") {
t.Errorf("expected 'hello world' in output, got: %s", out)
}
if !strings.Contains(out, "trace-123") {
t.Errorf("expected 'trace-123' in output, got: %s", out)
} }
} }
func BenchmarkViewable_Request(b *testing.B) { func TestToJSON(t *testing.T) {
// RequestLog 的字段也在顶层 entry := &log.InfoLog{
line := `{"LogName":"test-app","LogType":"request","LogTime":1714896000000000000,"TraceId":"trace-123","method":"GET","path":"/api/user","responsecode":200,"usedtime":10.5}` BaseLog: log.BaseLog{
LogName: "test-app",
b.ResetTimer() LogType: "info",
b.ReportAllocs() LogTime: 1714896000000000000,
for i := 0; i < b.N; i++ { TraceId: "trace-123",
_ = log.Viewable(line) },
Info: "hello world",
}
entry.Extra = map[string]any{"key": "value"}
log.RegisterType("info", entry)
line := string(log.Marshal(entry, nil))
jsonStr := log.ToJSON(line)
if !strings.Contains(jsonStr, `"Info":"hello world"`) {
t.Errorf("expected Info field in JSON, got: %s", jsonStr)
}
if !strings.Contains(jsonStr, `"TraceId":"trace-123"`) {
t.Errorf("expected TraceId field in JSON, got: %s", jsonStr)
}
if !strings.Contains(jsonStr, `"key":"value"`) {
t.Errorf("expected Extra fields merged in JSON, got: %s", jsonStr)
}
}
func TestLoadMeta(t *testing.T) {
// Create a temporary meta file
metaData := `{"test-type":[{"index":0,"name":"Field1"},{"index":1,"name":"Field2"}]}`
_ = os.WriteFile(".test_meta.json", []byte(metaData), 0644)
defer os.Remove(".test_meta.json")
err := log.LoadMeta(".test_meta.json")
if err != nil {
t.Fatalf("failed to load meta: %v", err)
}
meta := log.GetMeta("test-type")
if len(meta) != 2 {
t.Errorf("expected 2 meta fields, got %d", len(meta))
}
if meta[0].Name != "Field1" {
t.Errorf("expected Field1, got %s", meta[0].Name)
}
}
type EnhancedLog struct {
log.BaseLog
App string `log:"pos:10,withoutkey:true"`
Node string `log:"pos:11,attachBefore,withoutkey:true"`
RequestHeaders map[string]string `log:"pos:13,keyname:reqH"`
ClientIP string `log:"pos:12,keyname:ip"`
Tags []string `log:"pos:14"`
}
func (l *EnhancedLog) Reset() {
l.BaseLog.Reset()
l.App = ""
l.Node = ""
l.ClientIP = ""
l.RequestHeaders = nil
l.Tags = nil
}
func TestEnhancedViewable(t *testing.T) {
entry := &EnhancedLog{
BaseLog: log.BaseLog{
LogType: "enhanced",
LogTime: 1714896000000000000,
},
App: "MyApp",
Node: "Node1",
ClientIP: "127.0.0.1",
RequestHeaders: map[string]string{
"User-Agent": "Go-http-cli",
},
Tags: []string{"tag1", "tag2"},
}
log.RegisterType("enhanced", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
// Check attachBefore: MyApp:Node1 (since both are withoutkey)
if !strings.Contains(out, "MyApp:Node1") {
t.Errorf("expected MyApp:Node1, got: %s", out)
}
// Check pos ordering and keyname: ip:127.0.0.1 should come before reqH
if !strings.Contains(out, "ip:") || !strings.Contains(out, "127.0.0.1") {
t.Errorf("expected ip:127.0.0.1, got: %s", out)
}
if !strings.Contains(out, "reqH:") || !strings.Contains(out, "User-Agent") || !strings.Contains(out, "Go-http-cli") {
t.Errorf("expected reqH:[ User-Agent:Go-http-cli ], got: %s", out)
}
ipIdx := strings.Index(out, "ip:")
reqHIdx := strings.Index(out, "reqH:")
if ipIdx > reqHIdx {
t.Errorf("expected ip to come before reqH, but ipIdx=%d, reqHIdx=%d", ipIdx, reqHIdx)
}
// Check array rendering: Tags:[ tag1 tag2 ]
if !strings.Contains(out, "Tags:") || !strings.Contains(out, "[ tag1 tag2 ]") {
t.Errorf("expected Tags:[ tag1 tag2 ], got: %s", out)
}
}
func TestEnhancedToJSON(t *testing.T) {
entry := &EnhancedLog{
BaseLog: log.BaseLog{
LogType: "enhanced",
LogTime: 1714896000000000000,
},
App: "MyApp",
Node: "Node1",
ClientIP: "127.0.0.1",
RequestHeaders: map[string]string{
"User-Agent": "Go-http-cli",
},
}
log.RegisterType("enhanced", entry)
line := string(log.Marshal(entry, nil))
jsonStr := log.ToJSON(line)
// Check keyname in JSON
if !strings.Contains(jsonStr, `"ip":"127.0.0.1"`) {
t.Errorf("expected ip field in JSON, got: %s", jsonStr)
}
if !strings.Contains(jsonStr, `"reqH":{"User-Agent":"Go-http-cli"}`) {
t.Errorf("expected reqH field in JSON, got: %s", jsonStr)
}
}
type CallStackLog struct {
log.BaseLog
CallStacks []string `log:"pos:6"`
}
func (l *CallStackLog) Reset() {
l.BaseLog.Reset()
l.CallStacks = nil
}
func TestCallStacksViewable(t *testing.T) {
wd, _ := os.Getwd()
entry := &CallStackLog{
BaseLog: log.BaseLog{
LogType: "test_error",
LogTime: 1714896000000000000,
},
CallStacks: []string{wd + "/main.go:10", "/usr/local/go/src/runtime/panic.go:100"},
}
log.RegisterType("test_error", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
// Check path truncation (should contain relative "main.go:10")
if !strings.Contains(out, "main.go:10") {
t.Errorf("expected relative path main.go:10, got: %s", out)
}
// Absolute path should be removed if it matches wd
if strings.Contains(out, wd) {
t.Errorf("absolute path should be truncated, but still found: %s", out)
}
}
type PrecisionLog struct {
log.BaseLog
Value float64 `log:"pos:6,precision:2"`
}
func (l *PrecisionLog) Reset() {
l.BaseLog.Reset()
l.Value = 0
}
func TestPrecisionViewable(t *testing.T) {
entry := &PrecisionLog{
BaseLog: log.BaseLog{
LogType: "precision",
LogTime: 1714896000000000000,
},
Value: 3.14159,
}
log.RegisterType("precision", entry)
line := string(log.Marshal(entry, nil))
out := log.Viewable(line)
if !strings.Contains(out, "3.14") || strings.Contains(out, "3.141") {
t.Errorf("expected 3.14 (precision 2), got: %s", out)
} }
} }

125
writer.go
View File

@ -1,6 +1,7 @@
package log package log
import ( import (
"context"
"fmt" "fmt"
"sync" "sync"
"sync/atomic" "sync/atomic"
@ -9,30 +10,41 @@ import (
// Writer 日志写入接口 // Writer 日志写入接口
type Writer interface { type Writer interface {
Log([]byte) Log(LogEntry, []byte)
Run() Run()
} }
// logPayload 包含路由信息的包裹 // logPayload 包含路由信息的包裹
type logPayload struct { type logPayload struct {
entry LogEntry
buf []byte buf []byte
writer Writer // 目标自定义 Writer writer Writer // 目标自定义 Writer
file *FileWriter // 目标文件 Writer file *FileWriter // 目标文件 Writer
} }
// writerService manages the background writing of log entries.
type writerService struct {
Running atomic.Bool
StopChan chan bool
LogChannel chan logPayload
Dropped atomic.Uint64
Writers atomic.Value // []Writer
WriterLock sync.Mutex
Files map[string]*FileWriter
FilesLock sync.RWMutex
}
var ( var (
writerRunning atomic.Bool // WriterService is the global instance of defaultService.
writerLock sync.Mutex // 仅用于注册时锁定 WriterService = &writerService{}
writerStopChan chan bool
writers atomic.Value // 存储 []Writer
logChannel chan logPayload
) )
// ConsoleWriter 控制台写入器 // ConsoleWriter 控制台写入器
type ConsoleWriter struct { type ConsoleWriter struct {
} }
func (w *ConsoleWriter) Log(data []byte) { func (w *ConsoleWriter) Log(entry LogEntry, data []byte) {
fmt.Println(Viewable(string(data))) fmt.Println(Viewable(string(data)))
} }
@ -40,110 +52,129 @@ func (w *ConsoleWriter) Run() {
} }
func init() { func init() {
logChannel = make(chan logPayload, 10000) WriterService.LogChannel = make(chan logPayload, 10000)
writers.Store([]Writer{}) WriterService.Writers.Store([]Writer{})
WriterService.Files = make(map[string]*FileWriter)
RegisterWriterMaker("console", func(conf *Config) Writer { RegisterWriterMaker("console", func(conf *Config) Writer {
return &ConsoleWriter{} return &ConsoleWriter{}
}) })
} }
// WriteAsync 异步写入日志 // writeAsync 异步写入日志
func WriteAsync(payload logPayload) { func writeAsync(payload logPayload) {
defer func() { defer func() {
recover() recover()
}() }()
if !writerRunning.Load() { if !WriterService.Running.Load() {
return return
} }
select { select {
case logChannel <- payload: case WriterService.LogChannel <- payload:
default: default:
// 丢弃或处理过载,此处简单丢弃 // 丢弃或处理过载
dropped := WriterService.Dropped.Add(1)
if dropped%1000 == 1 {
if DefaultLogger != nil {
// 注意:这里可能会产生递归调用,但 select default 保证了不会死锁
DefaultLogger.Error(fmt.Sprintf("log channel full, dropped %d logs", dropped))
}
}
} }
} }
// Start 启动写入器 // GetDroppedLogs 获取被丢弃的日志数量
func Start() { func GetDroppedLogs() uint64 {
if !writerRunning.CompareAndSwap(false, true) { return WriterService.Dropped.Load()
return
}
writerStopChan = make(chan bool)
go writerRunner()
} }
// Stop 停止写入器 // Start implements starter.Service interface.
func Stop() { func (s *writerService) Start(_ context.Context, _ *Logger) error {
if writerRunning.CompareAndSwap(true, false) { if !s.Running.CompareAndSwap(false, true) {
close(logChannel) return nil
} }
s.StopChan = make(chan bool)
go s.writerRunner()
return nil
} }
// Wait 等待写入器停止 // Stop implements starter.Service interface.
func Wait() { func (s *writerService) Stop(_ context.Context) error {
if writerStopChan != nil { if s.Running.CompareAndSwap(true, false) {
<-writerStopChan close(s.LogChannel)
writerStopChan = nil if s.StopChan != nil {
<-s.StopChan
s.StopChan = nil
}
} }
return nil
} }
func writerRunner() { // Status implements starter.Service interface.
func (s *writerService) Status() (string, error) {
if s.Running.Load() {
return fmt.Sprintf("queue: %d, dropped: %d", len(s.LogChannel), s.Dropped.Load()), nil
}
return "stopped", nil
}
func (s *writerService) writerRunner() {
ticker := time.NewTicker(200 * time.Millisecond) ticker := time.NewTicker(200 * time.Millisecond)
defer ticker.Stop() defer ticker.Stop()
defer func() { defer func() {
if writerStopChan != nil { if s.StopChan != nil {
close(writerStopChan) close(s.StopChan)
} }
}() }()
for { for {
select { select {
case payload, ok := <-logChannel: case payload, ok := <-s.LogChannel:
if !ok { if !ok {
flushWriters() s.flushWriters()
return return
} }
processLog(payload) s.processLog(payload)
// 尝试批量处理更多日志 // 尝试批量处理更多日志
batchCount := 0 batchCount := 0
for batchCount < 100 { for batchCount < 100 {
select { select {
case nextPayload, nextOk := <-logChannel: case nextPayload, nextOk := <-s.LogChannel:
if !nextOk { if !nextOk {
flushWriters() s.flushWriters()
return return
} }
processLog(nextPayload) s.processLog(nextPayload)
batchCount++ batchCount++
default: default:
batchCount = 100 // break outer loop batchCount = 100 // break outer loop
} }
} }
case <-ticker.C: case <-ticker.C:
flushWriters() s.flushWriters()
} }
} }
} }
func processLog(payload logPayload) { func (s *writerService) processLog(payload logPayload) {
// 精准路由:根据包裹信息决定写入目标 // 精准路由:根据包裹信息决定写入目标
if payload.writer != nil { if payload.writer != nil {
payload.writer.Log(payload.buf) payload.writer.Log(payload.entry, payload.buf)
} else if payload.file != nil { } else if payload.file != nil {
payload.file.Write(time.Now(), payload.buf) payload.file.Write(time.Now(), payload.buf)
} }
} }
func flushWriters() { func (s *writerService) flushWriters() {
curWriters, _ := writers.Load().([]Writer) curWriters, _ := s.Writers.Load().([]Writer)
for _, w := range curWriters { for _, w := range curWriters {
w.Run() w.Run()
} }
filesLock.RLock() s.FilesLock.RLock()
for _, f := range files { for _, f := range s.Files {
f.Run() f.Run()
} }
filesLock.RUnlock() s.FilesLock.RUnlock()
} }