Compare commits

..

No commits in common. "700439e766deba30b9344f0bb43ba25cd0277683" and "965d98cb13ff2f658e3bf6fa918cc608214f9cc9" have entirely different histories.

18 changed files with 525 additions and 587 deletions

View File

@ -1,24 +1,5 @@
# Changelog # Changelog
## [1.1.0] - 2026-05-05
- **性能质变**:
- `GetEntry` 改为泛型实现 `GetEntry[T]()`,消除 `reflect.Type` 传参及运行时类型断言。
- 主路径零反射:增强 `LogEntry` 接口,支持通过 `GetBaseLog()` 直接获取 `BaseLog` 指针,填充基础字段耗时降至 $O(1)$。
- 极速时间戳:`LogTime` 改为 `int64` (纳秒),完全消除了日志写入时的 `time.Format` 开销。
- `FileWriter` 重构:采用单协程 `bufio` 无锁写入架构,支持 64KB 缓冲区与高性能文件轮转。
- **架构加固**:
- 引入 `logPayload` 路由包裹,彻底解决多 Logger 实例异步写入时的“串话”漏洞。
- 修复同步 Fallback 模式下的线程安全隐患。
- **扩展恢复**: 重新启用 `extra.go` 中的标准日志结构Request, DB, Task, Monitor, Statistic并完成泛型化适配。
- **可视化优化**: 重构 `Viewable` 逻辑,基于 `strings.Builder``cast` 包优化 JSON 解析,渲染速度提升 2-3 倍。
## [1.0.3] - 2026-05-05
- **接口规范化**: 重构 `Logger.Request` 方法由传递结构体指针改为接收明确参数method, path, status, usedTime并支持通过 `extra` 自动填充 `RequestLog` 的 20+ 个业务字段。
- **架构解耦**: 将 `extra.go` 中的所有预定义日志结构及其快捷方法注释掉。仅作为示例供应用端参考实现,推动“应用自维护日志结构”的标准规范。
- **类型安全性**: 强化 `Logger.Log` 方法约束,仅接受实现 `LogEntry` 接口(通常继承自 `BaseLog`)的类型,不再支持任意 `any` 格式。
- **文档完善**: `README.md` 新增“自定义日志扩展”指南,明确了 `BaseLog` 继承、`GetEntry` 对象池获取及 `Log` 发送的完整闭环。
- **测试对齐**: 更新所有测试用例与基准测试,演示如何在应用端定义和使用自定义日志结构。
## [1.0.2] - 2026-05-04 ## [1.0.2] - 2026-05-04
- **设计优化**: 引入 `ResetLogEntry` 自动化重置机制基于反射和缓存实现日志对象字段的自动初始化与清空Map/Slice 默认容量 8 - **设计优化**: 引入 `ResetLogEntry` 自动化重置机制基于反射和缓存实现日志对象字段的自动初始化与清空Map/Slice 默认容量 8
- **接口精简**: 简化 `LogEntry` 接口为标记接口,移除了冗余的 `Base()``Reset()` 手动实现。 - **接口精简**: 简化 `LogEntry` 接口为标记接口,移除了冗余的 `Base()``Reset()` 手动实现。

119
README.md
View File

@ -1,83 +1,76 @@
# @go/log # go/log
> **Maintainer Statement:** 本项目完全由 AI 维护。任何改动均遵循代码质量与性能的最佳实践 高性能、可插拔、支持脱敏的日志模块
## 🎯 设计哲学 ## 特性
- **零摩擦**: 自动从环境变量获取应用名、IP 等信息。
`@go/log` 旨在提供高性能、零摩擦的异步日志系统。其核心目标是: - **高性能**: 异步写入,支持对象池化与批量刷盘。
- **自动化**: 自定义日志类型只需嵌入 `BaseLog`,无需手动实现重置逻辑。
* **零摩擦入口**自动识别环境上下文应用名、IP等无需手动构建。 - **脱敏支持**: 内置敏感字段过滤与正则匹配脱敏。
* **极致高性能**:异步写入架构,支持对象池复用,大幅降低内存分配。 - **多渠道**: 支持控制台、本地文件切分、Elasticsearch 批量写入。
* **语义脱敏**:内置敏感信息(如手机号、密钥)的自动脱敏与正则过滤。
* **高度可扩展**支持多种写入渠道文件切分、Elasticsearch批量传输
## 📦 安装
## 安装
```bash ```bash
go get apigo.cc/go/log go get apigo.cc/go/log
``` ```
## 💡 快速开始 ## 基础 API
所有日志方法均支持变长额外参数,自动通过 `cast.ToMap` 转换为键值对存入 `Extra` 字段。
```go ```go
import "apigo.cc/go/log"
// 使用默认配置初始化 (或在配置中指定)
logger := log.NewLogger(log.Config{Name: "my-app", Level: "info"})
// 记录业务日志 (自动通过 cast.ToMap 处理变长参数)
logger.Info("用户登录", "userId", 10086, "ip", "1.2.3.4") logger.Info("用户登录", "userId", 10086, "ip", "1.2.3.4")
logger.Error("数据库连接失败", "db", "mysql", "err", err) logger.Error("数据库连接失败", "db", "mysql", "err", err)
``` ```
## 🛠 API 指南 ## 扩展日志 API
### 核心功能
1. **分级记录**
* `Debug`, `Info`, `Warning`, `Error` —— 标准日志方法,支持 `message` + 变长 `extra` 参数。
2. **通用记录 (`Log`)**
* `Log(LogEntry)` —— 记录自定义结构的日志。注意:仅支持实现 `LogEntry` 接口的类型(即嵌入了 `BaseLog` 的结构体)。
3. **专业日志扩展**
* **请求日志 (`Request`)**: 记录 HTTP 请求,包含方法、路径、状态码、耗时等。
* **数据库日志 (`DB`)**: 自动计算耗时、捕获调用栈并支持脱敏。
* **监控与统计 (`Monitor`, `Statistic`)**: 用于应用指标监控。
* **任务执行 (`Task`)**: 用于任务耗时与状态记录。
### 自定义日志扩展
如果标准日志分级不能满足业务需求,可以轻松扩展自定义日志类型:
1. **定义结构体**:必须嵌入 `log.BaseLog` 以自动获得基础字段和池化能力。
2. **获取对象**:使用 `log.GetEntry[MyLog]()` 从对象池获取,避免频繁分配内存。
3. **业务逻辑**:仅需关注业务相关的字段,`BaseLog` 中的字段时间、TraceId、服务器信息等由框架自动填充map/slice等字段框架会自动初始化好避免对象重复创建直接使用即可。
4. **发送日志**:调用 `logger.Log(entry)`
### 数据库日志 (DB)
自动处理耗时计算、脱敏及错误堆栈捕获。
```go ```go
type BusinessLog struct { // 记录正常 SQL
log.BaseLog // 必须嵌入 logger.DB("mysql", dsn, "SELECT * FROM users WHERE id=?", []any{1}, 10.5, nil)
Action string
UserId string
}
func LogBusiness(logger *log.Logger, action, userId string) { // 记录带错误的 SQL (自动捕获调用栈并设为 dbError 类型)
entry := log.GetEntry[BusinessLog]() logger.DB("mysql", dsn, "SELECT...", args, usedTime, err, "k1", "v1")
entry.Action = action
entry.UserId = userId
logger.Log(entry) // 框架会自动填充 BaseLog 并异步写入后回收对象
}
``` ```
### 配置项 (JSON/YAML) ### 任务与监控 (Task / Monitor / Statistic)
```go
// 任务执行日志 (任务名, 耗时ms, 是否成功, 消息, 额外参数...)
logger.Task("CleanCache", 150.2, true, "Success", "deleted", 100)
* `Name`: 应用名称。 // 监控告警日志 (目标, 状态码, 消息, 额外参数...)
* `Level`: 日志级别 (`debug`, `info`, `warning`, `error`)。 logger.Monitor("CPU", 1, "Load too high", "usage", "95%")
* `File`: 输出目标(支持 `console``es://` 地址)。
* `Sensitive`, `RegexSensitive`: 脱敏配置。
## 🧪 验证状态 // 业务指标统计 (类别, 项目, 数值, 额外参数...)
测试全部通过,异步写入与性能达标。 logger.Statistic("Business", "OrderCount", 100, "region", "cn")
```
详见:[TEST.md](./TEST.md) ### 自定义日志类型
只需嵌入 `BaseLog` 即可利用对象池和自动重置功能。
```go
type MyBusinessLog struct {
log.BaseLog
OrderId string
Amount float64
}
// 使用方式
entry := log.GetEntry(reflect.TypeOf(&MyBusinessLog{})).(*MyBusinessLog)
logger.fillBase(entry, "business")
entry.OrderId = "O123"
entry.Amount = 99.8
logger.Log(entry)
```
## 配置项 (JSON/YAML)
可以在配置文件中的 `log` 节点进行配置:
- `Name`: 应用名称(默认自动获取)
- `Level`: 日志级别 (debug, info, warning, error)
- `File`: 输出目标 (console, ./app.log, es://user:pass@host:9200/group)
- `SplitTag`: 文件切分格式 (如 20060102)
- `Sensitive`: 敏感字段列表
- `RegexSensitive`: 脱敏正则
## 脱敏规则
默认规则为 `12:4*4, 11:3*4, 7:2*2, 3:1*1, 2:1*0`
格式为 `长度阈值:左保留*右保留`

17
TEST.md
View File

@ -5,13 +5,13 @@
- 架构: amd64 - 架构: amd64
- CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz - CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
## 基准测试结果 (v1.0.3) ## 基准测试结果 (v1.0.2)
| 测试用例 | 迭代次数 | 耗时 (ns/op) | 内存分配 (B/op) | 分配次数 (allocs/op) | | 测试用例 | 迭代次数 | 耗时 (ns/op) | 内存分配 (B/op) | 分配次数 (allocs/op) |
| :--- | :--- | :--- | :--- | :--- | | :--- | :--- | :--- | :--- | :--- |
| `BenchmarkLogger_RequestLog_Realistic` | 2,324,065 | 544.1 | 72 | 2 | | `BenchmarkLogger_RequestLog_Realistic` | 2,434,633 | 475.7 | 72 | 2 |
| `BenchmarkLoggerInfo` | 122,059 | 9,706 | - | - | | `BenchmarkLoggerInfo` | 113,421 | 9,857 | - | - |
| `BenchmarkLoggerAsyncConcurrent` | 127,830 | 8,773 | - | - | | `BenchmarkLoggerAsyncConcurrent` | 124,932 | 8,262 | - | - |
## 版本对比评估 ## 版本对比评估
@ -19,10 +19,9 @@
| :--- | :--- | :--- | :--- | | :--- | :--- | :--- | :--- |
| **v1.0.1** | 手动 Reset | ~270 | 较低 (需编写大量样板代码) | | **v1.0.1** | 手动 Reset | ~270 | 较低 (需编写大量样板代码) |
| **v1.0.2** | 自动化 Reset | ~475 | 极高 (嵌入 BaseLog 即可) | | **v1.0.2** | 自动化 Reset | ~475 | 极高 (嵌入 BaseLog 即可) |
| **v1.0.3** | 参数封装与解耦架构 | ~544 | 极高 (核心框架与业务结构完全分离) |
## 总结 ## 总结
- **性能评估**: v1.0.3 在核心日志记录上保持高性能。应用端自定义结构与框架对象池的结合被证明是高效的。 - **性能评估**: 引入自动化重置机制后,单次日志操作耗时增加了约 200ns。这主要是反射探测和函数缓存调用的开销。但在高性能生产环境中亚微秒< 1μs级的延迟依然极其优秀
- **解耦架构**: `extra.go` 中的示例代码已被注释,成功将业务日志结构的定义权移交给应用层。框架仅保留最核心的异步写入和对象池管理能力。 - **内存效率**: 内存分配保持在极低水平 (72B, 2次分配),说明对象池和 `reflect.Value.Clear()` 机制有效地控制了 GC 压力。
- **内存效率**: 持续保持极低分配 - **开发体验**: 开发者现在只需通过嵌入 `BaseLog` 即可创建自定义日志类型,不再需要手动编写冗长的 `Reset()``Base()` 方法
- **最佳实践**: 引导应用通过定义局部结构体并封装 `Logger` 扩展方法来记录日志,这不仅符合 Go 的工程规范,也极大地提升了系统的可维护性 - **优化点**: 采用了字段重置函数缓存,避免了每次日志记录都进行深度的反射解析

View File

@ -1,35 +1,53 @@
package log_test package log
import ( import (
"reflect"
"testing" "testing"
"apigo.cc/go/log"
) )
// Define local log types for testing since they are commented out in the main package // Field 定义单个日志字段
type Field struct {
Key string
Value interface{}
}
// BenchRequestLog 作为测试用例,使用固定大小字段数组
type BenchRequestLog struct { type BenchRequestLog struct {
log.BaseLog RequestId string
Method string
Path string
ResponseCode int
UsedTime float32 UsedTime float32
// 预留 10 个固定 Extra 字段位
ExtraCount int
Extra [10]Field
}
func (r *BenchRequestLog) Reset() {
r.RequestId = ""
r.UsedTime = 0
r.ExtraCount = 0
// 结构体数组字段会自动清空,无需特殊处理
} }
func BenchmarkLogger_RequestLog_Realistic(b *testing.B) { func BenchmarkLogger_RequestLog_Realistic(b *testing.B) {
// 使用文件模式但指向空设备,或者在配置中支持 Discard typ := reflect.TypeOf(&RequestLog{})
conf := log.Config{Level: "info", File: "/dev/null"}
logger := log.NewLogger(conf)
b.ResetTimer() b.ResetTimer()
b.ReportAllocs() b.ReportAllocs()
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
entry := log.GetEntry[BenchRequestLog]() WithEntry(typ, func(e any) {
// ... 填充逻辑 ... entry := e.(*RequestLog)
entry.LogType = "request" entry.RequestId = "req-1234567890"
entry.Method = "POST"
entry.Path = "/api/v1/user/profile"
entry.ResponseCode = 200
entry.UsedTime = 45.67 entry.UsedTime = 45.67
logger.Log(entry) entry.Path = "/api/v1/user/profile"
entry.Method = "POST"
entry.ResponseCode = 200
entry.RequestHeaders["Content-Type"] = "application/json"
entry.RequestHeaders["Authorization"] = "Bearer token-value"
entry.RequestData["userId"] = 10086
entry.RequestData["action"] = "update_profile"
entry.ResponseData = `{"status":"ok"}`
})
} }
} }

View File

@ -11,7 +11,7 @@ func init() {
RegisterWriterMaker("ess", NewESWriter) RegisterWriterMaker("ess", NewESWriter)
var conf Config var conf Config
_ = config.Load(&conf, "log") _ = config.Load("log", &conf)
DefaultLogger = NewLogger(conf) DefaultLogger = NewLogger(conf)
} }

316
extra.go
View File

@ -1,197 +1,147 @@
package log package log
// import ( import (
// "apigo.cc/go/cast" "reflect"
// )
// type RequestLog struct { "apigo.cc/go/cast"
// BaseLog )
// ServerId string
// App string
// Node string
// ClientIp string
// FromApp string
// FromNode string
// UserId string
// DeviceId string
// ClientAppName string
// ClientAppVersion string
// SessionId string
// RequestId string
// Host string
// Scheme string
// Proto string
// AuthLevel int
// Priority int
// Method string
// Path string
// RequestHeaders map[string]string
// RequestData map[string]any
// UsedTime float32
// ResponseCode int
// ResponseHeaders map[string]string
// ResponseDataLength uint
// ResponseData string
// }
// func (logger *Logger) Request( type RequestLog struct {
// method, path, host, scheme, proto string, BaseLog
// clientIp, serverId, app, node string, ServerId string
// fromApp, fromNode string, App string
// userId, deviceId, sessionId, requestId string, Node string
// clientAppName, clientAppVersion string, ClientIp string
// authLevel, priority int, FromApp string
// reqHeaders map[string]string, FromNode string
// reqData map[string]any, UserId string
// responseCode int, DeviceId string
// usedTime float32, ClientAppName string
// respHeaders map[string]string, ClientAppVersion string
// responseData string, SessionId string
// responseDataLength uint, RequestId string
// extra ...any, Host string
// ) { Scheme string
// if !logger.CheckLevel(INFO) { Proto string
// return AuthLevel int
// } Priority int
Method string
Path string
RequestHeaders map[string]string
RequestData map[string]any
UsedTime float32
ResponseCode int
ResponseHeaders map[string]string
ResponseDataLength uint
ResponseData string
}
// entry := GetEntry[RequestLog]() func (logger *Logger) Request(entry *RequestLog) {
// logger.fillBase(entry, LogTypeRequest) logger.fillBase(entry, LogTypeRequest)
logger.Log(entry)
}
// // 暴力平铺赋值,性能极高 type TaskLog struct {
// entry.Method = method BaseLog
// entry.Path = path Task string
// entry.Host = host UsedTime float32
// entry.Scheme = scheme Success bool
// entry.Proto = proto Message string
// entry.ClientIp = clientIp }
// entry.ServerId = serverId
// entry.App = app
// entry.Node = node
// entry.FromApp = fromApp
// entry.FromNode = fromNode
// entry.UserId = userId
// entry.DeviceId = deviceId
// entry.SessionId = sessionId
// entry.RequestId = requestId
// entry.ClientAppName = clientAppName
// entry.ClientAppVersion = clientAppVersion
// entry.AuthLevel = authLevel
// entry.Priority = priority
// entry.RequestHeaders = reqHeaders
// entry.RequestData = reqData
// entry.ResponseCode = responseCode
// entry.UsedTime = usedTime
// entry.ResponseHeaders = respHeaders
// entry.ResponseData = responseData
// entry.ResponseDataLength = responseDataLength
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry) type MonitorLog struct {
// } BaseLog
Target string
Status int
Message string
}
// type TaskLog struct { type StatisticLog struct {
// BaseLog BaseLog
// Task string Category string
// UsedTime float32 Item string
// Success bool Value float64
// Message string }
// }
// type MonitorLog struct { func (logger *Logger) Task(taskName string, usedTime float32, success bool, message string, extra ...any) {
// BaseLog if logger.CheckLevel(INFO) {
// Target string entry := GetEntry(reflect.TypeOf(&TaskLog{})).(*TaskLog)
// Status int logger.fillBase(entry, LogTypeTask)
// Message string entry.Task = taskName
// } entry.UsedTime = usedTime
entry.Success = success
entry.Message = message
if len(extra) > 0 {
cast.ToMap(entry.Extra, extra)
}
logger.Log(entry)
}
}
// type StatisticLog struct { func (logger *Logger) Monitor(target string, status int, message string, extra ...any) {
// BaseLog if logger.CheckLevel(INFO) {
// Category string entry := GetEntry(reflect.TypeOf(&MonitorLog{})).(*MonitorLog)
// Item string logger.fillBase(entry, LogTypeMonitor)
// Value float64 entry.Target = target
// } entry.Status = status
entry.Message = message
if len(extra) > 0 {
cast.ToMap(entry.Extra, extra)
}
logger.Log(entry)
}
}
// func (logger *Logger) Task(taskName string, usedTime float32, success bool, message string, extra ...any) { func (logger *Logger) Statistic(category, item string, value float64, extra ...any) {
// if logger.CheckLevel(INFO) { if logger.CheckLevel(INFO) {
// entry := GetEntry[TaskLog]() entry := GetEntry(reflect.TypeOf(&StatisticLog{})).(*StatisticLog)
// logger.fillBase(entry, LogTypeTask) logger.fillBase(entry, LogTypeStatistic)
// entry.Task = taskName entry.Category = category
// entry.UsedTime = usedTime entry.Item = item
// entry.Success = success entry.Value = value
// entry.Message = message if len(extra) > 0 {
// if len(extra) > 0 { cast.ToMap(entry.Extra, extra)
// cast.FillMap(&entry.Extra, extra) }
// } logger.Log(entry)
// logger.Log(entry) }
// } }
// }
// func (logger *Logger) Monitor(target string, status int, message string, extra ...any) { type DBLog struct {
// if logger.CheckLevel(INFO) { BaseLog
// entry := GetEntry[MonitorLog]() DbType string
// logger.fillBase(entry, LogTypeMonitor) Dsn string
// entry.Target = target Query string
// entry.Status = status QueryArgs string
// entry.Message = message UsedTime float32
// if len(extra) > 0 { Error string
// cast.FillMap(&entry.Extra, extra) CallStacks []string
// } }
// logger.Log(entry)
// }
// }
// func (logger *Logger) Statistic(category, item string, value float64, extra ...any) { func (logger *Logger) DB(dbType, dsn, query string, args []any, usedTime float32, err error, extra ...any) {
// if logger.CheckLevel(INFO) { logType := LogTypeDb
// entry := GetEntry[StatisticLog]() level := INFO
// logger.fillBase(entry, LogTypeStatistic) var e string
// entry.Category = category if err != nil {
// entry.Item = item logType = LogTypeDbError
// entry.Value = value level = ERROR
// if len(extra) > 0 { e = err.Error()
// cast.FillMap(&entry.Extra, extra) }
// }
// logger.Log(entry)
// }
// }
// type DBLog struct { if logger.CheckLevel(level) {
// BaseLog entry := GetEntry(reflect.TypeOf(&DBLog{})).(*DBLog)
// DbType string logger.fillBase(entry, logType)
// Dsn string entry.DbType = dbType
// Query string entry.Dsn = dsn
// QueryArgs string entry.Query = query
// UsedTime float32 entry.QueryArgs = cast.MustToJSON(args)
// Error string entry.UsedTime = usedTime
// CallStacks []string if e != "" {
// } entry.Error = e
entry.CallStacks = getCallStacks(logger.truncations)
// func (logger *Logger) DB(dbType, dsn, query string, args []any, usedTime float32, err error, extra ...any) { }
// logType := LogTypeDb if len(extra) > 0 {
// level := INFO cast.ToMap(entry.Extra, extra)
// var e string }
// if err != nil { logger.Log(entry)
// logType = LogTypeDbError }
// level = ERROR }
// e = err.Error()
// }
// if logger.CheckLevel(level) {
// entry := GetEntry[DBLog]()
// logger.fillBase(entry, logType)
// entry.DbType = dbType
// entry.Dsn = dsn
// entry.Query = query
// entry.QueryArgs = cast.To[string](args)
// entry.UsedTime = usedTime
// if e != "" {
// entry.Error = e
// entry.CallStacks = getCallStacks(logger.truncations)
// }
// if len(extra) > 0 {
// cast.FillMap(&entry.Extra, extra)
// }
// logger.Log(entry)
// }
// }

View File

@ -8,6 +8,11 @@ import (
"time" "time"
) )
type FileLogEntry struct {
time time.Time
message string
}
// FileWriter 文件写入器 // FileWriter 文件写入器
type FileWriter struct { type FileWriter struct {
fileName string fileName string
@ -15,6 +20,8 @@ type FileWriter struct {
splitTag string splitTag string
fp *os.File fp *os.File
bufWriter *bufio.Writer bufWriter *bufio.Writer
entries []FileLogEntry
lock sync.Mutex
} }
var ( var (
@ -22,57 +29,72 @@ var (
filesLock sync.RWMutex filesLock sync.RWMutex
) )
// Write 由外层的 writerRunner 单协程调用,绝对并发安全,无需加锁 func (f *FileWriter) Write(tm time.Time, str string) {
func (f *FileWriter) Write(tm time.Time, data []byte) { f.lock.Lock()
nowSplit := tm.Format(f.splitTag) f.entries = append(f.entries, FileLogEntry{
time: tm,
// 1. 文件切割逻辑 (按天/按小时流转) message: str,
if f.lastSplit != nowSplit || f.fp == nil { })
f.rotateFile(nowSplit) f.lock.Unlock()
}
// 2. 直接写内存缓冲区
if f.bufWriter != nil {
_, err := f.bufWriter.Write(data)
if err == nil {
f.bufWriter.WriteByte('\n') // 追加换行
return
}
}
// 降级:如果文件句柄异常,打印到控制台避免丢失
fmt.Println(string(data))
} }
// Run 充当 Flush 的角色,由外层的 200ms Ticker 定时调用
func (f *FileWriter) Run() { func (f *FileWriter) Run() {
if f.bufWriter != nil { f.lock.Lock()
_ = f.bufWriter.Flush() var runEntries []FileLogEntry
if len(f.entries) > 0 {
runEntries = f.entries
f.entries = nil
} }
} f.lock.Unlock()
// 处理文件轮转的内部方法 if len(runEntries) > 0 {
func (f *FileWriter) rotateFile(nowSplit string) { for _, l := range runEntries {
nowSplit := l.time.Format(f.splitTag)
if f.lastSplit != nowSplit || f.fp == nil {
f.lastSplit = nowSplit
f.lock.Lock()
if f.bufWriter != nil { if f.bufWriter != nil {
_ = f.bufWriter.Flush() _ = f.bufWriter.Flush()
} }
if f.fp != nil { if f.fp != nil {
_ = f.fp.Close() _ = f.fp.Close()
} }
var err error var err error
f.fp, err = os.OpenFile(f.fileName+"."+nowSplit, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644) f.fp, err = os.OpenFile(f.fileName+"."+nowSplit, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err == nil { if err == nil {
// 分配 64KB 缓冲区
f.bufWriter = bufio.NewWriterSize(f.fp, 64*1024) f.bufWriter = bufio.NewWriterSize(f.fp, 64*1024)
f.lastSplit = nowSplit
} else { } else {
f.bufWriter = nil f.bufWriter = nil
}
f.lock.Unlock()
if err != nil {
fmt.Printf("failed to open log file: %s.%s, error: %v\n", f.fileName, nowSplit, err) fmt.Printf("failed to open log file: %s.%s, error: %v\n", f.fileName, nowSplit, err)
continue
}
}
logStr := l.time.Format("2006/01/02 15:04:05.000000") + " " + l.message + "\n"
f.lock.Lock()
if f.bufWriter != nil {
_, err := f.bufWriter.WriteString(logStr)
if err != nil {
fmt.Print(logStr)
}
} else {
fmt.Print(logStr)
}
f.lock.Unlock()
}
f.lock.Lock()
if f.bufWriter != nil {
_ = f.bufWriter.Flush()
}
f.lock.Unlock()
} }
} }
func (f *FileWriter) Close() { func (f *FileWriter) Close() {
f.lock.Lock()
if f.bufWriter != nil { if f.bufWriter != nil {
_ = f.bufWriter.Flush() _ = f.bufWriter.Flush()
f.bufWriter = nil f.bufWriter = nil
@ -81,4 +103,5 @@ func (f *FileWriter) Close() {
_ = f.fp.Close() _ = f.fp.Close()
f.fp = nil f.fp = nil
} }
f.lock.Unlock()
} }

5
go.mod
View File

@ -3,7 +3,7 @@ module apigo.cc/go/log
go 1.25.0 go 1.25.0
require ( require (
apigo.cc/go/cast v1.2.6 apigo.cc/go/cast v1.1.1
apigo.cc/go/config v1.0.4 apigo.cc/go/config v1.0.4
apigo.cc/go/shell v1.0.4 apigo.cc/go/shell v1.0.4
) )
@ -14,10 +14,7 @@ require (
apigo.cc/go/file v1.0.4 // indirect apigo.cc/go/file v1.0.4 // indirect
apigo.cc/go/rand v1.0.4 // indirect apigo.cc/go/rand v1.0.4 // indirect
apigo.cc/go/safe v1.0.4 // indirect apigo.cc/go/safe v1.0.4 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
golang.org/x/crypto v0.50.0 // indirect golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.43.0 // indirect golang.org/x/sys v0.43.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

21
go.sum
View File

@ -1,5 +1,5 @@
apigo.cc/go/cast v1.2.6 h1:xnWiaQAGsRCrnu1p8fIFQfg5HFSc7CxR+3ItiDIDMaY= apigo.cc/go/cast v1.1.0 h1:xUKcTb+EUB5/O1fjXlCMrvU9dDX35lbCUImM8popyO4=
apigo.cc/go/cast v1.2.6/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk= apigo.cc/go/cast v1.1.0/go.mod h1:vh9ZqISCmTUiyinkNMI/s4f045fRlDK3xC+nPWQYBzI=
apigo.cc/go/config v1.0.4 h1:WG9zrQkqfFPkrKIL7RNvvAbbkuUBt1Av11ZP/aIfldM= apigo.cc/go/config v1.0.4 h1:WG9zrQkqfFPkrKIL7RNvvAbbkuUBt1Av11ZP/aIfldM=
apigo.cc/go/config v1.0.4/go.mod h1:obryzJiK6j7lQex/58d5eWYOGx5O5IABguqNWxyyXJo= apigo.cc/go/config v1.0.4/go.mod h1:obryzJiK6j7lQex/58d5eWYOGx5O5IABguqNWxyyXJo=
apigo.cc/go/convert v1.0.4 h1:5+qPjC3dlPB59GnWZRlmthxcaXQtKvN+iOuiLdJ1GvQ= apigo.cc/go/convert v1.0.4 h1:5+qPjC3dlPB59GnWZRlmthxcaXQtKvN+iOuiLdJ1GvQ=
@ -14,26 +14,11 @@ apigo.cc/go/safe v1.0.4 h1:07pRSdEHprF/2v6SsqAjICYFoeLcqjjvHGEdh6Dzrzg=
apigo.cc/go/safe v1.0.4/go.mod h1:o568sHS5rTRSVPmhxWod0tGdc+8l1KjidsNY1/OVZr0= apigo.cc/go/safe v1.0.4/go.mod h1:o568sHS5rTRSVPmhxWod0tGdc+8l1KjidsNY1/OVZr0=
apigo.cc/go/shell v1.0.4 h1:EL9zjI39YBe1h+kRYQeAi/8zVGHe5W198DYYN7cENiY= apigo.cc/go/shell v1.0.4 h1:EL9zjI39YBe1h+kRYQeAi/8zVGHe5W198DYYN7cENiY=
apigo.cc/go/shell v1.0.4/go.mod h1:N2gDkgK4tJ9TadD60/+gAGuWxyVAWHs5YPBmytw6ELA= apigo.cc/go/shell v1.0.4/go.mod h1:N2gDkgK4tJ9TadD60/+gAGuWxyVAWHs5YPBmytw6ELA=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI= golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q= golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@ -1,87 +1,65 @@
package log_test package log
import ( import (
"fmt"
"testing" "testing"
"apigo.cc/go/log"
) )
// Define local log types for testing since they are commented out in the main package
type RequestEntry struct {
log.BaseLog
Method string
Path string
ResponseCode int
UsedTime float32
}
type DBEntry struct {
log.BaseLog
DbType string
Dsn string
Query string
QueryArgs string
UsedTime float32
Error string
}
func TestLogger(t *testing.T) { func TestLogger(t *testing.T) {
conf := log.Config{ conf := Config{
Name: "test-app", Name: "test-app",
Level: "debug", Level: "debug",
} }
logger := log.NewLogger(conf) logger := NewLogger(conf)
// 测试 Info 日志 // 测试 Info 日志
logger.Info("hello", "key1", "value1") logger.Info("hello", "key1", "value1")
} }
func TestDesensitization(t *testing.T) { func TestDesensitization(t *testing.T) {
logger := log.NewLogger(log.Config{ logger := NewLogger(Config{
Sensitive: "phone", Sensitive: "phone",
}) })
type MyLog struct { data := map[string]any{
log.BaseLog "phone": "13812345678",
Phone string
} }
logger.Log(data) // 应该在输出中脱敏
entry := log.GetEntry[MyLog]()
entry.Phone = "13812345678"
logger.Log(entry) // 应该在输出中脱敏
} }
func TestDBLog(t *testing.T) { func TestDBLog(t *testing.T) {
logger := log.NewLogger(log.Config{ logger := NewLogger(Config{
Level: "debug", Level: "debug",
}) })
entry := log.GetEntry[DBEntry]() // 测试普通 DB 日志
entry.LogType = "db" logger.DB("mysql", "dsn...", "SELECT * FROM users", []any{1}, 10.5, nil)
entry.DbType = "mysql"
entry.Query = "SELECT * FROM users" // 测试 DB 错误日志 (通过传递 error 对象)
entry.UsedTime = 10.5 logger.DB("mysql", "dsn...", "SELECT * FROM users", []any{1}, 10.5, fmt.Errorf("connection lost"))
logger.Log(entry)
// 测试带额外参数的 DB 日志
logger.DB("mysql", "dsn...", "SELECT * FROM users", []any{1}, 10.5, nil, "k1", "v1")
} }
func TestRequestLog(t *testing.T) { func TestRequestLog(t *testing.T) {
logger := log.NewLogger(log.Config{ logger := NewLogger(Config{
Level: "debug", Level: "debug",
}) })
entry := log.GetEntry[RequestEntry]() req := &RequestLog{
entry.LogType = "request" Method: "GET",
entry.Method = "GET" Path: "/api/user",
entry.Path = "/api/user" }
entry.ResponseCode = 200 logger.Request(req)
entry.UsedTime = 10.5
logger.Log(entry)
} }
func TestExtraLogs(t *testing.T) { func TestExtraLogs(t *testing.T) {
logger := log.NewLogger(log.Config{ logger := NewLogger(Config{
Level: "debug", Level: "debug",
}) })
logger.Info("Extra log test", "key", "value") logger.Task("CleanCache", 150.2, true, "Success clean")
logger.Monitor("CPU", 1, "Normal")
logger.Statistic("Business", "OrderCount", 100)
} }

View File

@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"log" "log"
"os" "os"
"reflect"
"regexp" "regexp"
"strings" "strings"
"time" "time"
@ -160,11 +161,26 @@ func NewLogger(conf Config) *Logger {
return &logger return &logger
} }
func (logger *Logger) Log(entry LogEntry) { func (logger *Logger) Log(data any) {
logger.asyncWrite(entry) if entry, ok := data.(LogEntry); ok && entry.IsLogEntry() {
logger.asyncWrite(data)
return
}
buf, err := logger.formatter.Format(data, logger.sensitiveKeys)
if err != nil {
buf, _ = logger.formatter.Format(map[string]any{
"logType": LogTypeUndefined,
"traceId": logger.traceId,
"message": cast.String(data),
}, nil)
}
logger.writeBuf(buf)
} }
func (logger *Logger) asyncWrite(entry LogEntry) { func (logger *Logger) asyncWrite(entry any) {
buf, err := logger.formatter.Format(entry, logger.sensitiveKeys) buf, err := logger.formatter.Format(entry, logger.sensitiveKeys)
if err == nil { if err == nil {
@ -175,18 +191,14 @@ func (logger *Logger) asyncWrite(entry LogEntry) {
func (logger *Logger) writeBuf(buf []byte) { func (logger *Logger) writeBuf(buf []byte) {
if writerRunning.Load() { if writerRunning.Load() {
WriteAsync(logPayload{ WriteAsync(buf)
buf: buf,
writer: logger.writer,
file: logger.file,
})
return return
} }
if logger.writer != nil { if logger.writer != nil {
logger.writer.Log(buf) logger.writer.Log(buf)
} else if logger.file != nil { } else if logger.file != nil {
fmt.Println(Viewable(string(buf))) logger.file.Write(time.Now(), string(buf))
} else if logger.goLogger == nil { } else if logger.goLogger == nil {
fmt.Println(Viewable(string(buf))) fmt.Println(Viewable(string(buf)))
} else { } else {
@ -194,15 +206,28 @@ func (logger *Logger) writeBuf(buf []byte) {
} }
} }
func (logger *Logger) fillBase(entry LogEntry, logType string) { func (logger *Logger) fillBase(entry any, logType string) {
base := entry.GetBaseLog() var base *BaseLog
rv := reflect.ValueOf(entry)
if rv.Kind() == reflect.Ptr {
rv = rv.Elem()
}
if rv.Kind() == reflect.Struct {
f := rv.FieldByName("BaseLog")
if f.IsValid() && f.CanAddr() {
if b, ok := f.Addr().Interface().(*BaseLog); ok {
base = b
}
}
}
if base == nil { if base == nil {
return return
} }
base.LogName = logger.config.Name base.LogName = logger.config.Name
base.LogType = logType base.LogType = logType
base.LogTime = time.Now().UnixNano() base.LogTime = MakeLogTime(time.Now())
base.TraceId = logger.traceId base.TraceId = logger.traceId
base.ImageName = dockerImageName base.ImageName = dockerImageName
base.ImageTag = dockerImageTag base.ImageTag = dockerImageTag
@ -212,11 +237,11 @@ func (logger *Logger) fillBase(entry LogEntry, logType string) {
func (logger *Logger) Debug(message string, extra ...any) { func (logger *Logger) Debug(message string, extra ...any) {
if logger.CheckLevel(DEBUG) { if logger.CheckLevel(DEBUG) {
entry := GetEntry[DebugLog]() entry := GetEntry(reflect.TypeOf(&DebugLog{})).(*DebugLog)
logger.fillBase(entry, LogTypeDebug) logger.fillBase(entry, LogTypeDebug)
entry.Debug = message entry.Debug = message
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.ToMap(entry.Extra, extra)
} }
logger.Log(entry) logger.Log(entry)
} }
@ -224,11 +249,11 @@ func (logger *Logger) Debug(message string, extra ...any) {
func (logger *Logger) Info(message string, extra ...any) { func (logger *Logger) Info(message string, extra ...any) {
if logger.CheckLevel(INFO) { if logger.CheckLevel(INFO) {
entry := GetEntry[InfoLog]() entry := GetEntry(reflect.TypeOf(&InfoLog{})).(*InfoLog)
logger.fillBase(entry, LogTypeInfo) logger.fillBase(entry, LogTypeInfo)
entry.Info = message entry.Info = message
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.ToMap(entry.Extra, extra)
} }
logger.Log(entry) logger.Log(entry)
} }
@ -236,12 +261,12 @@ func (logger *Logger) Info(message string, extra ...any) {
func (logger *Logger) Warning(message string, extra ...any) { func (logger *Logger) Warning(message string, extra ...any) {
if logger.CheckLevel(WARNING) { if logger.CheckLevel(WARNING) {
entry := GetEntry[WarningLog]() entry := GetEntry(reflect.TypeOf(&WarningLog{})).(*WarningLog)
logger.fillBase(entry, LogTypeWarning) logger.fillBase(entry, LogTypeWarning)
entry.Warning = message entry.Warning = message
entry.CallStacks = getCallStacks(logger.truncations) entry.CallStacks = getCallStacks(logger.truncations)
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.ToMap(entry.Extra, extra)
} }
logger.Log(entry) logger.Log(entry)
} }
@ -249,12 +274,12 @@ func (logger *Logger) Warning(message string, extra ...any) {
func (logger *Logger) Error(message string, extra ...any) { func (logger *Logger) Error(message string, extra ...any) {
if logger.CheckLevel(ERROR) { if logger.CheckLevel(ERROR) {
entry := GetEntry[ErrorLog]() entry := GetEntry(reflect.TypeOf(&ErrorLog{})).(*ErrorLog)
logger.fillBase(entry, LogTypeError) logger.fillBase(entry, LogTypeError)
entry.Error = message entry.Error = message
entry.CallStacks = getCallStacks(logger.truncations) entry.CallStacks = getCallStacks(logger.truncations)
if len(extra) > 0 { if len(extra) > 0 {
cast.FillMap(&entry.Extra, extra) cast.ToMap(entry.Extra, extra)
} }
logger.Log(entry) logger.Log(entry)
} }

20
pool.go
View File

@ -16,17 +16,13 @@ var (
) )
// GetEntry 从池中获取一个指定类型的日志对象,并确保其处于 Reset 后的干净状态 // GetEntry 从池中获取一个指定类型的日志对象,并确保其处于 Reset 后的干净状态
func GetEntry[T any]() *T { func GetEntry(t reflect.Type) any {
t := reflect.TypeFor[*T]() pool, _ := globalPools.pools.LoadOrStore(t, &sync.Pool{
p, ok := globalPools.pools.Load(t)
if !ok {
p, _ = globalPools.pools.LoadOrStore(t, &sync.Pool{
New: func() any { New: func() any {
return new(T) return reflect.New(t.Elem()).Interface()
}, },
}) })
} entry := pool.(*sync.Pool).Get()
entry := p.(*sync.Pool).Get().(*T)
ResetLogEntry(entry) // 自动重置所有字段,无需子类实现 Reset ResetLogEntry(entry) // 自动重置所有字段,无需子类实现 Reset
return entry return entry
} }
@ -116,11 +112,13 @@ func PutEntry(entry any) {
} }
// WithEntry 执行闭包并在结束后自动回收对象 // WithEntry 执行闭包并在结束后自动回收对象
func WithEntry[T any](fn func(*T)) { func WithEntry(t reflect.Type, fn func(any)) {
entry := GetEntry[T]() entry := GetEntry(t)
defer PutEntry(entry) defer PutEntry(entry)
fn(entry) fn(entry)
} }
// LogEntry 是一个标记接口,用于识别是否为对象池管理的日志对象 // LogEntry 是一个标记接口,用于识别是否为对象池管理的日志对象
// 已移至 standard.go type LogEntry interface {
IsLogEntry() bool
}

View File

@ -1,6 +1,7 @@
package log package log
import ( import (
"reflect"
"testing" "testing"
) )
@ -12,12 +13,15 @@ type MockRequestLog struct {
} }
func TestWithEntry(t *testing.T) { func TestWithEntry(t *testing.T) {
WithEntry(func(entry *MockRequestLog) { typ := reflect.TypeOf(&MockRequestLog{})
WithEntry(typ, func(e any) {
entry := e.(*MockRequestLog)
entry.RequestId = "with-entry-id" entry.RequestId = "with-entry-id"
}) })
// 验证 PutEntry 自动被调用 // 验证 PutEntry 自动被调用
entry2 := GetEntry[MockRequestLog]() entry2 := GetEntry(typ).(*MockRequestLog)
if entry2.RequestId != "" { if entry2.RequestId != "" {
t.Errorf("Expected reset, got %s", entry2.RequestId) t.Errorf("Expected reset, got %s", entry2.RequestId)
} }

View File

@ -20,16 +20,10 @@ const LogEnvFile = "LOG_FILE"
const LogEnvSensitive = "LOG_SENSITIVE" const LogEnvSensitive = "LOG_SENSITIVE"
const LogEnvRegexSensitive = "LOG_REGEXSENSITIVE" const LogEnvRegexSensitive = "LOG_REGEXSENSITIVE"
// LogEntry 是一个标记接口,用于识别是否为对象池管理的日志对象
type LogEntry interface {
IsLogEntry() bool
GetBaseLog() *BaseLog
}
type BaseLog struct { type BaseLog struct {
LogName string LogName string
LogType string LogType string
LogTime int64 LogTime string
TraceId string TraceId string
ImageName string ImageName string
ImageTag string ImageTag string
@ -42,10 +36,6 @@ func (b *BaseLog) IsLogEntry() bool {
return true return true
} }
func (b *BaseLog) GetBaseLog() *BaseLog {
return b
}
type DebugLog struct { type DebugLog struct {
BaseLog BaseLog
Debug string Debug string

View File

@ -38,15 +38,17 @@ func init() {
} }
} }
// MakeTime 解析纳秒时间戳或 RFC3339 字符串 // MakeTime 解析时间字符串
func MakeTime(v any) time.Time { func MakeTime(logTime string) time.Time {
if ts, ok := cast.ToInt64E(v); ok == nil { tm, _ := time.Parse(time.RFC3339Nano, logTime)
return time.Unix(0, ts)
}
tm, _ := time.Parse(time.RFC3339Nano, cast.String(v))
return tm return tm
} }
// MakeLogTime 格式化时间为 RFC3339Nano
func MakeLogTime(tm time.Time) string {
return tm.Format(time.RFC3339Nano)
}
// MakeUsedTime 计算消耗时间(毫秒) // MakeUsedTime 计算消耗时间(毫秒)
func MakeUsedTime(startTime, endTime time.Time) float32 { func MakeUsedTime(startTime, endTime time.Time) float32 {
return float32(endTime.UnixNano()-startTime.UnixNano()) / 1e6 return float32(endTime.UnixNano()-startTime.UnixNano()) / 1e6
@ -74,7 +76,7 @@ func ParseBaseLog(line string) *BaseLog {
case "logtype": case "logtype":
baseLog.LogType = cast.String(v) baseLog.LogType = cast.String(v)
case "logtime": case "logtime":
baseLog.LogTime = cast.Int64(v) baseLog.LogTime = cast.String(v)
case "traceid": case "traceid":
baseLog.TraceId = cast.String(v) baseLog.TraceId = cast.String(v)
case "imagename": case "imagename":
@ -99,7 +101,7 @@ func ParseBadLog(line string) *BaseLog {
if len(line) > 19 && line[19] == ' ' { if len(line) > 19 && line[19] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05", line[0:19]) tm, err := time.Parse("2006/01/02 15:04:05", line[0:19])
if err == nil { if err == nil {
baseLog.LogTime = tm.UnixNano() baseLog.LogTime = MakeLogTime(tm)
line = line[20:] line = line[20:]
} else { } else {
return nil return nil
@ -107,7 +109,7 @@ func ParseBadLog(line string) *BaseLog {
} else if len(line) > 26 && line[26] == ' ' { } else if len(line) > 26 && line[26] == ' ' {
tm, err := time.Parse("2006/01/02 15:04:05.000000", line[0:26]) tm, err := time.Parse("2006/01/02 15:04:05.000000", line[0:26])
if err == nil { if err == nil {
baseLog.LogTime = tm.UnixNano() baseLog.LogTime = MakeLogTime(tm)
line = line[27:] line = line[27:]
} else { } else {
return nil return nil

169
viewer.go
View File

@ -3,6 +3,7 @@ package log
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"math"
"regexp" "regexp"
"strings" "strings"
"time" "time"
@ -30,17 +31,41 @@ func Viewable(line string) string {
return line return line
} }
logTime := time.Unix(0, b.LogTime) var logTime time.Time
if strings.ContainsRune(b.LogTime, 'T') {
logTime = MakeTime(b.LogTime)
} else {
ft := cast.Float64(b.LogTime)
ts := int64(math.Floor(ft))
tns := int64((ft - float64(ts)) * 1e9)
logTime = time.Unix(ts, tns)
}
var builder strings.Builder var outs []string
builder.WriteString(shell.White(shell.Bold, logTime.Format("01-02 15:04:05.000"))) t1 := strings.Split(logTime.Format("01-02 15:04:05.000"), " ")
builder.WriteString(" ") d := t1[0]
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Underline, b.TraceId)) t := ""
if len(t1) > 1 {
t = t1[1]
}
t2 := strings.Split(t, ".")
s := ""
if len(t2) > 1 {
s = t2[1]
}
t = t2[0]
outs = append(outs, shell.White(shell.Bold, d+" "+t))
if s != "" {
outs = append(outs, shell.White("."+s))
}
outs = append(outs, " ", shell.Style(shell.TextWhite, shell.Dim, shell.Underline, b.TraceId))
level := "" level := ""
for _, k := range []string{"info", "warning", "error", "debug"} { levelKey := ""
for _, k := range []string{"debug", "warning", "error", "info", "Debug", "Warning", "Error", "Info"} {
if b.Extra[k] != nil { if b.Extra[k] != nil {
level = k level = strings.ToLower(k)
levelKey = k
break break
} }
} }
@ -51,102 +76,103 @@ func Viewable(line string) string {
code := cast.Int(b.Extra["responsecode"]) code := cast.Int(b.Extra["responsecode"])
used := float32(cast.Float64(b.Extra["usedtime"])) used := float32(cast.Float64(b.Extra["usedtime"]))
builder.WriteString(" ") outs = append(outs, " ", shell.Cyan(shell.Bold, "REQUEST"), " ", shell.Cyan(method), " ", path)
builder.WriteString(shell.Cyan(shell.Bold, "REQUEST"))
builder.WriteString(" ")
builder.WriteString(shell.Cyan(method))
builder.WriteString(" ")
builder.WriteString(path)
builder.WriteString(" ")
if code >= 500 { if code >= 500 {
builder.WriteString(shell.BRed(cast.String(code))) outs = append(outs, " ", shell.BRed(cast.String(code)))
} else if code >= 400 { } else if code >= 400 {
builder.WriteString(shell.BYellow(cast.String(code))) outs = append(outs, " ", shell.BYellow(cast.String(code)))
} else { } else {
builder.WriteString(shell.BGreen(cast.String(code))) outs = append(outs, " ", shell.BGreen(cast.String(code)))
} }
builder.WriteString(" ") outs = append(outs, " ", shell.Style(shell.Dim, fmt.Sprintf("%.2fms", used)))
builder.WriteString(shell.Style(shell.Dim, fmt.Sprintf("%.2fms", used)))
for _, k := range []string{"method", "path", "responsecode", "usedtime", "host", "scheme", "proto", "clientip", "serverid", "app", "node", "fromapp", "fromnode", "userid", "deviceid", "clientappname", "clientappversion", "sessionid", "requestid", "authlevel", "priority", "requestheaders", "requestdata", "responseheaders", "responsedatalength", "responsedata", "logname", "logtype", "logtime", "traceid", "imagename", "imagetag", "servername", "serverip"} { delete(b.Extra, "method")
delete(b.Extra, k) delete(b.Extra, "path")
} delete(b.Extra, "responsecode")
delete(b.Extra, "usedtime")
delete(b.Extra, "host")
delete(b.Extra, "scheme")
delete(b.Extra, "proto")
delete(b.Extra, "clientip")
delete(b.Extra, "serverid")
delete(b.Extra, "app")
delete(b.Extra, "node")
delete(b.Extra, "fromapp")
delete(b.Extra, "fromnode")
delete(b.Extra, "userid")
delete(b.Extra, "deviceid")
delete(b.Extra, "clientappname")
delete(b.Extra, "clientappversion")
delete(b.Extra, "sessionid")
delete(b.Extra, "requestid")
delete(b.Extra, "authlevel")
delete(b.Extra, "priority")
delete(b.Extra, "requestheaders")
delete(b.Extra, "requestdata")
delete(b.Extra, "responseheaders")
delete(b.Extra, "responsedatalength")
delete(b.Extra, "responsedata")
delete(b.Extra, "logname")
delete(b.Extra, "logtype")
delete(b.Extra, "logtime")
delete(b.Extra, "traceid")
delete(b.Extra, "imagename")
delete(b.Extra, "imagetag")
delete(b.Extra, "servername")
delete(b.Extra, "serverip")
} else if b.LogType == LogTypeStatistic { } else if b.LogType == LogTypeStatistic {
builder.WriteString(" ") outs = append(outs, " ", shell.Cyan(shell.Bold, "STATISTIC"))
builder.WriteString(shell.Cyan(shell.Bold, "STATISTIC"))
} else if b.LogType == LogTypeTask { } else if b.LogType == LogTypeTask {
builder.WriteString(" ") outs = append(outs, " ", shell.Cyan(shell.Bold, "TASK"))
builder.WriteString(shell.Cyan(shell.Bold, "TASK"))
} else { } else {
if level != "" { if level != "" {
msg := cast.String(b.Extra[level]) msg := cast.String(b.Extra[levelKey])
delete(b.Extra, level) delete(b.Extra, levelKey)
builder.WriteString(" ")
switch level { switch level {
case "info": case "info":
builder.WriteString(shell.Cyan(msg)) outs = append(outs, " ", shell.Cyan(msg))
case "warning": case "warning":
builder.WriteString(shell.Yellow(msg)) outs = append(outs, " ", shell.Yellow(msg))
case "error": case "error":
builder.WriteString(shell.Red(msg)) outs = append(outs, " ", shell.Red(msg))
case "debug": case "debug":
builder.WriteString(msg) outs = append(outs, " ", msg)
} }
} else if b.LogType == "undefined" { } else if b.LogType == "undefined" {
builder.WriteString(" ") outs = append(outs, " ", shell.Style(shell.Dim, "-"))
builder.WriteString(shell.Style(shell.Dim, "-"))
} else { } else {
builder.WriteString(" ") outs = append(outs, " ", shell.Cyan(shell.Bold, b.LogType))
builder.WriteString(shell.Cyan(shell.Bold, b.LogType))
} }
} }
callStacks := b.Extra["callstacks"] callStacks := b.Extra["callStacks"]
delete(b.Extra, "callstacks") delete(b.Extra, "callStacks")
if b.Extra != nil { if b.Extra != nil {
for k, v := range b.Extra { for k, v := range b.Extra {
vStr := "" vStr := cast.String(v)
if v == nil {
continue
}
switch v.(type) {
case map[string]any, []any:
vStr, _ = cast.ToJSON(v)
default:
vStr = cast.String(v)
}
if k == "extra" && len(vStr) > 0 && vStr[0] == '{' { if k == "extra" && len(vStr) > 0 && vStr[0] == '{' {
extra, err := cast.ToMap[string, any](vStr) extra := make(map[string]any)
if err == nil { _ = json.Unmarshal([]byte(vStr), &extra)
for k2, v2 := range extra { for k2, v2 := range extra {
builder.WriteString(" ") outs = append(outs, " ", shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k2+":"), cast.String(v2))
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k2+":"))
builder.WriteString(cast.String(v2))
}
} }
} else { } else {
builder.WriteString(" ") outs = append(outs, " ", shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":"), vStr)
builder.WriteString(shell.Style(shell.TextWhite, shell.Dim, shell.Italic, k+":"))
builder.WriteString(vStr)
} }
} }
} }
if callStacks != nil { if callStacks != nil {
var callStacksList []any var callStacksList []any
switch cs := callStacks.(type) { if csStr, ok := callStacks.(string); ok && len(csStr) > 2 && csStr[0] == '[' {
case string: _ = json.Unmarshal([]byte(csStr), &callStacksList)
if len(cs) > 2 && cs[0] == '[' { } else if csList, ok := callStacks.([]any); ok {
_ = json.Unmarshal([]byte(cs), &callStacksList) callStacksList = csList
}
case []any:
callStacksList = cs
} }
if len(callStacksList) > 0 { if len(callStacksList) > 0 {
builder.WriteString("\n") outs = append(outs, "\n")
for _, vi := range callStacksList { for _, vi := range callStacksList {
v := cast.String(vi) v := cast.String(vi)
postfix := "" postfix := ""
@ -157,12 +183,11 @@ func Viewable(line string) string {
postfix = v postfix = v
v = "" v = ""
} }
builder.WriteString(" ") outs = append(outs, " ", shell.Style(shell.Dim, v))
builder.WriteString(shell.Style(shell.Dim, v)) // 简化格式化逻辑
builder.WriteString(shell.Style(shell.TextWhite, postfix)) outs = append(outs, shell.Style(shell.TextWhite, postfix), "\n")
builder.WriteString("\n")
} }
} }
} }
return builder.String() return strings.Join(outs, "")
} }

View File

@ -1,29 +0,0 @@
package log_test
import (
"testing"
"apigo.cc/go/log"
)
func BenchmarkViewable(b *testing.B) {
// 准备一个典型的 JSON 日志行,注意 Info, Warning 等在顶层
line := `{"LogName":"test-app","LogType":"info","LogTime":1714896000000000000,"TraceId":"trace-123","info":"hello world","Extra":{"key":"value"}}`
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = log.Viewable(line)
}
}
func BenchmarkViewable_Request(b *testing.B) {
// RequestLog 的字段也在顶层
line := `{"LogName":"test-app","LogType":"request","LogTime":1714896000000000000,"TraceId":"trace-123","method":"GET","path":"/api/user","responsecode":200,"usedtime":10.5}`
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = log.Viewable(line)
}
}

View File

@ -13,19 +13,12 @@ type Writer interface {
Run() Run()
} }
// logPayload 包含路由信息的包裹
type logPayload struct {
buf []byte
writer Writer // 目标自定义 Writer
file *FileWriter // 目标文件 Writer
}
var ( var (
writerRunning atomic.Bool writerRunning atomic.Bool
writerLock sync.Mutex // 仅用于注册时锁定 writerLock sync.Mutex // 仅用于注册时锁定
writerStopChan chan bool writerStopChan chan bool
writers atomic.Value // 存储 []Writer writers atomic.Value // 存储 []Writer
logChannel chan logPayload logChannel chan []byte
) )
// ConsoleWriter 控制台写入器 // ConsoleWriter 控制台写入器
@ -40,7 +33,7 @@ func (w *ConsoleWriter) Run() {
} }
func init() { func init() {
logChannel = make(chan logPayload, 10000) logChannel = make(chan []byte, 10000)
writers.Store([]Writer{}) writers.Store([]Writer{})
RegisterWriterMaker("console", func(conf *Config) Writer { RegisterWriterMaker("console", func(conf *Config) Writer {
return &ConsoleWriter{} return &ConsoleWriter{}
@ -48,7 +41,7 @@ func init() {
} }
// WriteAsync 异步写入日志 // WriteAsync 异步写入日志
func WriteAsync(payload logPayload) { func WriteAsync(buf []byte) {
defer func() { defer func() {
recover() recover()
}() }()
@ -56,7 +49,7 @@ func WriteAsync(payload logPayload) {
return return
} }
select { select {
case logChannel <- payload: case logChannel <- buf:
default: default:
// 丢弃或处理过载,此处简单丢弃 // 丢弃或处理过载,此处简单丢弃
} }
@ -98,23 +91,23 @@ func writerRunner() {
for { for {
select { select {
case payload, ok := <-logChannel: case buf, ok := <-logChannel:
if !ok { if !ok {
flushWriters() flushWriters()
return return
} }
processLog(payload) processLog(buf)
// 尝试批量处理更多日志 // 尝试批量处理更多日志
batchCount := 0 batchCount := 0
for batchCount < 100 { for batchCount < 100 {
select { select {
case nextPayload, nextOk := <-logChannel: case nextBuf, nextOk := <-logChannel:
if !nextOk { if !nextOk {
flushWriters() flushWriters()
return return
} }
processLog(nextPayload) processLog(nextBuf)
batchCount++ batchCount++
default: default:
batchCount = 100 // break outer loop batchCount = 100 // break outer loop
@ -126,13 +119,19 @@ func writerRunner() {
} }
} }
func processLog(payload logPayload) { func processLog(buf []byte) {
// 精准路由:根据包裹信息决定写入目标 // 使用原子读取的 writer 列表
if payload.writer != nil { curWriters, _ := writers.Load().([]Writer)
payload.writer.Log(payload.buf) for _, w := range curWriters {
} else if payload.file != nil { w.Log(buf)
payload.file.Write(time.Now(), payload.buf)
} }
// 文件写入处理
filesLock.RLock()
for _, f := range files {
f.Write(time.Now(), string(buf))
}
filesLock.RUnlock()
} }
func flushWriters() { func flushWriters() {