go/tableDB
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: go:main
Branches Tags
go:main
go:v1.1.9
go:v1.1.8
go:v1.1.6
go:v1.1.0
..
compare: go:v1.1.0
Branches Tags
go:main
go:v1.1.9
go:v1.1.8
go:v1.1.6
go:v1.1.0

No commits in common. "main" and "v1.1.0" have entirely different histories.
main ... v1.1.0

10 changed files with 613 additions and 1321 deletions
Show Stats Expand all files Collapse all files

77
CHANGELOG.md
View File

@ -1,82 +1,5 @@
# CHANGELOG
## v1.1.8 (2026-05-16)
- **文档增强**:
- **Hook 事件指南**: 在 `README.md` 中新增了 Hook 事件的详细声明与用法说明，涵盖从表结构变更到数据 CRUD 的全生命周期钩子。
## v1.1.7 (2026-05-15)
- **底层架构重构与标准字段**:
- **强制标准字段**: 所有数据表自动携带并维护 5 个标准字段（`id`, `creator`, `createTime`, `updater`, `updateTime`），极大地规范了业务记录生命周期。
- **默认倒序排列**: `List` 与 `Query` 现默认按 `createTime DESC` 返回数据，以符合现代业务开发直觉。
- **严格的 SELECT 安全**: `buildQuery` 彻底摒弃了 `SELECT *`，基于内存缓存中的合法字段列表动态拼接 SQL，彻底杜绝已删除残余字段的泄露。
- **Hook 系统重构**:
- **增强上下文注入**: 全面重构了 `Hooks` 接口签名，现在 `OnUpdatedRows`、`OnRemovedRows` 等事件会完整返回 `rows/ids` 以及关联的 `TableSchema` 和 `FieldSchema`。
- **RAG 友好型设计**: 增强后的 Hook 系统能够让外部模块（如 RAG 索引器）在零额外查询的情况下，直接获取所需的所有元数据与数据正文。
- **基础设施对齐与 ID 自动化**:
- **移除手动 ID 生成**: 去除了 `tableDB` 内部的 `record["id"] = id.MakeID(10)` 逻辑，完全下推至 `go/db` 基础设施处理。
- **自动随机 ID 识别**: 配合底层 `db` 升级，现在只要字段名为 `id`、类型为 `char(8~16)` 且在插入时为空，系统将自动产生全局随机 ID。
- **分布式 ID 支持**: `GetDB` 增加 `redis` 参数，用于配置底层 `db` 的 Redis 实例，确保在分布式环境下生成的 ID 具备全局唯一性。
- **架构重构稳定性修复**:
- **修复 Schema 重建 bug**: 修正了 `reconstructAndSyncSchema` 在生成 DSL 时丢失 `PK` (Primary Key) 标记的问题，确保动态创建的表始终具备正确的索引结构。
- **对齐 GetDB 签名**: 同步更新所有测试用例与示例文档，对齐最新的多参数初始化模型。
## v1.1.6 (2026-05-15)
- **零开销 ABAC 鉴权引擎**:
- **内存展平技术**: 引入 `SchemaCache` 策略展平算法，支持无限深度的用户权限继承（如 `Manager -> Boss`），并将复杂权限图谱在内存中转换为 O(1) 的平滑映射，彻底消除运行时递归与 N+1 数据库查询。
- **1-RTT 动态 SQL 注入**: 重构 `checkAuth` 与 `appendAuthAndConstraint`，将行级过滤条件（RLS）直接下推至 SQL WHERE 子句中。无论是 `Get`, `List` 还是 `Remove`，所有权限校验均与业务查询合并执行，确保极致性能。
- **Opt-in 行级安全**: `_Table` 增加 `enableRLS` 开关。仅在显式开启的表上执行权限拦截，避免普通表（如字典表、日志表）产生不必要的计算开销。
- **全新策略管理 API**:
- 新增 `SetPolicy` 与 `ListPolicy` 接口，支持标准化的 ABAC 策略配置（SQL 条件、参数绑定）。
- **权限分级管理**: 实现了自主授权逻辑。普通用户仅在拥有目标表 `full` 权限时方可为他人分配该表权限；`inherit` 类高阶策略仅限 `SystemUserID` 操作。
- **极致性能与校验**:
- **O(1) 字段验证**: 增加 `ValidFieldsMap` 缓存，消除 `buildQuery` 过程中高频的 Map 构建开销。
- **防死锁与容错**: 权限继承逻辑内置环状检测，自动忽略 `A -> B -> A` 式的逻辑死循环。
- **文档与工程化**:
- 全面重写 `README.md`，以 AI 驱动开发为中心，提供了清晰的 Schema 与 Policy 使用示例。
- 同步更新 `.profile/README.md`，对齐基础设施版本索引。
## v1.1.5 (2026-05-14)
- **强制身份校验模型**: 重构 `GetDB` 返回类型为 `*TableDBUnauthorized`，在编译期强制要求必须先调用 `Auth` 后才能执行后续数据表操作。
- **自定义元数据扩展**: 在系统表 `_Table` 和 `_Field` 中增加了 `settings` (JSON) 字段，业务方可通过 `TableSchema` 和 `FieldSchema` 存储自定义的表级或字段级配置。
- **Schema API 正规化**: 引入了完整的 `TableSchema` 结构体，并将 `SetTable` 和 `RemoveTable` API 上提至 `TableDB` 层级。彻底废弃并移除了遗留的 `Table("xxx:Schema")` 魔法后缀调用方式。
- **底层行为一致性**: `SetTable`, `RemoveTable`, `SetField`, `RemoveField` 现已彻底改写为底层封装调用 `Table("_Table")` 和 `Table("_Field")` 的 `Set/Remove` 方法，确保了系统自动更新的时间戳(`createTime`)、操作人(`creator`)及各种 Hook 生命周期事件能够被 100% 一致地触发。
## v1.1.4 (2026-05-14)
- **API 语义优化**:
- `Query` 方法从 `TableDB` 移动到 `Table` 对象，更符合面向对象调用习惯；`QueryRequest` 移除 `Table` 字段。
- 移除 `App` 类型别名，统一使用 `TableDB`。
- `SyncSchema` 改为私有方法 `syncSchema`，外部调用统一通过 `SetTable` 和 `SetField` 进行结构维护。
- **结构维护增强**:
- 新增 `SetTable(memo)` 用于设置表元数据。
- `SetField` 参数从 `any` 改为强类型的 `FieldSchema` 切片，并自动忽略 `CreateTime` 等系统字段，增强类型安全性。
- `SetField` 在创建新表时会自动补全 `id` 字段的元数据，确保查询验证一致性。
- 新增 `Tables()` 方法，支持列出当前用户有权访问的所有数据表。
- **安全性完善**:
- 强化了所有公共 API 的 Auth 检查，确保未授权实例无法执行任何操作。
- 内部 `buildQuery` 现在需要明确传入表名，增强了多表操作时的上下文隔离。
- **测试回归**: 完全重构了测试用例，所有业务表创建均通过 `SetField` 接口完成，验证了公共 API 的闭环能力。
## v1.1.3 (2026-05-14)
- **API 统一与简化**: 引入 `SetField` 和 `RemoveField` 接口，支持通过统一的 Table API 动态管理表结构，实现了“操作表结构像操作数据一样简单”。
- **批量操作支持**: `Set`, `Remove`, `SetField`, `RemoveField` 全面支持变长参数（Variadic arguments），显著提升了批量数据处理效率。
- **安全性增强**:
- `GetDB` 返回实例默认未授权，强制调用 `Auth` 进行身份绑定。
- `GetRawDB` (原 `Base`) 现已增加权限校验，仅限 `_system` 身份访问。
- `Query` (App 级) 现已自动集成行级权限过滤。
- **性能与稳定性**: 优化了元数据同步流程，合并批量操作后的 Schema 重建；修复了 `Cache.Load` 在并发调用时的死锁隐患。
- **文档完善**: 重新编写了 README.md，完全抹除了实现细节，侧重于纯粹的 API 指南与最佳实践。
## v1.1.2 (2026-05-14)
- **文档重构**: 重新编写 README.md，以 AI 为中心，侧重于 API 功能说明与使用指南，移除了无关的实现细节。
- **命名规范对齐**: 消灭了所有 Struct 中的 `json` tag，全面对齐驼峰命名规范，利用 `cast` 库自动处理大小写转换。
- **API 完整性**: 基于 `getExports` 工具补全了 API 文档，涵盖了常量、全局变量及完整的 Table 操作接口。
## v1.1.1 (2026-05-14)
- **基础设施对齐**: 深度对接 `@go/db` 的自动 ID 生成器 (`NextID`) 与版本管理特性。
- **元数据模型对齐**: 重构 `FieldSchema` 结构体，对齐 `_Field` 表物理结构，并统一使用 `cast.Convert` 进行 JSON 兼容的类型转换。
- **查询安全性强化**: 增强 `BuildQuery` 的标识符验证与自动转义（Backticks），防止潜在的 SQL 注入风险。
- **Schema 同步健壮性**: 优化 `reconstructAndSyncSchema` 与 `SyncSchema` 的元数据采集逻辑，支持更复杂的 DSL 结构。
## v1.1.0
- **内置系统架构 (SystemSchema)**: `_Table`, `_Field`, `_Policy` 等核心元数据表现已内置，`SyncSchema` 会在启动时自动补齐，无需在业务 DSL 中显式定义。
- **自动化 Schema 修复**: 修复了通过 API 创建表时缺少 `id` 字段导致的同步失败问题；`reconstructAndSyncSchema` 现在会自动为所有表补齐 `id c10 PK` 主键。

182
README.md
View File

@ -1,144 +1,68 @@
# go/tableDB v1.1.8
# apigo.cc/go/tableDB
`tableDB` 是基于 `@go/db` 的高级数据库抽象引擎，支持动态 Schema 管理、身份驱动的行级隔离 (RLS) 以及 ABAC 权限策略。
`tableDB` 是建立在 `apigo.cc/go/db` 基础上的更高层抽象工具库。提供对于动态表、数据权限隔离和 Hook 的生命周期追踪管理。
## 🎯 核心设计理念
- **动态 Schema**: 像操作数据一样操作表和字段，无需手动 SQL 迁移。
- **内置标准字段**: 所有数据表强制且自动携带 5 个标准字段：`id` (全局唯一)、`creator`、`createTime`、`updater`、`updateTime`。系统会自动在插入和更新时维护它们。
- **身份驱动**: 所有操作强制绑定 `userID`，系统基于此自动拦截并填充创建者、更新者。
- **ABAC 权限模型**: 支持 SQL 条件过滤、属性权限及关系继承，替代传统的 Role 体系。
- **极速性能**: 通过内存展平策略实现零 RTT 鉴权，支持 O(1) 字段验证。
## 使用指南
## 📦 安装
```bash
go get apigo.cc/go/tableDB
```
### 1. 初始化与 Auth
## 🛠 API 指南
### 1. 初始化与授权
```go
// 获取未授权实例 (无法直接操作数据)，可传入 redis 地址用于分布式 ID 生成
unauth := tableDB.GetDB("sqlite://data.db", logger, "")
package main
// 必须绑定身份后才能使用
app := unauth.Auth("user_123")
sys := unauth.Auth(tableDB.SystemUserID) // 超管权限，绕过所有 RLS
```
### 2. 动态架构管理 (Schema)
```go
// 1. 设置表元数据
sys.SetTable(tableDB.TableSchema{
Name: "orders",
EnableRLS: true, // 开启行级安全拦截
Memo: "业务订单表",
})
// 2. 批量维护字段 (支持动态迁移)
app.Table("orders").SetField(
tableDB.FieldSchema{Name: "title", Type: "v128", IsIndex: true},
tableDB.FieldSchema{Name: "amount", Type: "i"},
tableDB.FieldSchema{Name: "status", Type: "i", IsIndex: true},
import (
"apigo.cc/go/tableDB"
"apigo.cc/go/log"
)
// 3. 查看有权访问的表
tables, _ := app.Tables()
```
func main() {
app := tableDB.GetDB("sqlite://local.db", log.DefaultLogger)
### 3. 数据操作 (CRUD)
```go
table := app.Table("orders")
// 在需要忽略权限的系统级后台执行时：
systemApp := app.Auth(tableDB.SystemUserID)
// 保存数据 (自动填充 creator)
table.Set(map[string]any{"title": "测试订单", "amount": 100})
// 获取单条 (自动进行权限校验)
order, err := table.Get("id_xxx")
// 列表查询 (自动注入过滤条件: creator = userID OR policy_conditions)
list, _ := table.List(map[string]any{"amount >": 50})
// 结构化查询 (支持 Join 和 Select 验证)
res, _ := table.Query(tableDB.QueryRequest{
Select: []string{"id", "title"},
Where: "status = ?",
Args: []any{1},
})
```
### 4. 策略与权限管理 (ABAC)
策略允许用户访问非自己创建的数据。
```go
// 1. 授权策略: 允许 user_A 读取订单，但仅限金额小于 1000 的
sys.SetPolicy(tableDB.PolicySchema{
UserID: "user_A",
Type: "table",
Targets: []string{"orders"},
Action: "read",
Condition: "amount < ?",
ConditionArgs: []any{1000},
})
// 2. 继承策略: manager 继承 clerk 的所有权限
sys.SetPolicy(tableDB.PolicySchema{
UserID: "manager",
Type: "inherit",
Targets: []string{"clerk"},
})
// 3. 自主授权: 对表拥有 'full' 权限的用户，可以给其他人授权该表
manager.SetPolicy(tableDB.PolicySchema{
UserID: "clerk",
Type: "table",
Targets: []string{"orders"},
Action: "read",
})
```
## 🛡 安全契约
1. **隐式隔离**: 只要表开启了 `EnableRLS` 且包含 `creator` 字段，查询将自动限制在用户创建的数据范围内。
2. **零开销拦截**: 鉴权逻辑已下推至 SQL WHERE 层，消除 N+1 查询隐患。
3. **特权保护**: 系统表 `_Table`, `_Field`, `_Policy` 禁止普通用户通过 Table API 直接修改，必须通过专属 API 进行管理。
4. **命名规范**: 全局强制驼峰命名，禁止使用 `json` tag。
## 🔗 Hook 事件 (Hooks)
`tableDB` 提供了一系列 Hook，允许开发者在表结构变更或数据更新时注入自定义逻辑（如：缓存同步、消息推送、业务校验等）。
### 1. 注册 Hook
Hook 需要在 `Auth` 之前，在 `TableDBUnauthorized` 实例上注册。
```go
unauth := tableDB.GetDB("sqlite://data.db", logger, "")
// 监听数据更新 (用于缓存同步)
unauth.Hooks.OnUpdatedRows = func(rows []map[string]any, table *tableDB.TableSchema, fields []tableDB.FieldSchema) {
for _, row := range rows {
fmt.Printf("表 %s 的数据已更新: %v\n", table.Name, row["id"])
}
// 在特定的业务上下文执行时，将强制携带 creator 约束
userApp := app.Auth("user_12345")
}
// 数据写入前校验 (返回 error 将中断操作)
unauth.Hooks.OnUpdatingRow = func(row map[string]any, table *tableDB.TableSchema, fields []tableDB.FieldSchema) error {
if table.Name == "orders" && cast.Int(row["amount"]) > 1000000 {
return errors.New("金额过大，需要人工审核")
}
return nil
}
app := unauth.Auth("user_123")
```
### 2. 支持的事件列表
### 2. DSL 动态表声明与同步
| 事件名 | 触发时机 | 参数说明 |
| :--- | :--- | :--- |
| `OnCreatedTable` | 新表创建后 | `table`: 表架构信息 |
| `OnRemovedTable` | 表被删除后 | `table`: 表架构信息 |
| `OnUpdatedField` | 字段新增或修改后 | `table`: 表架构，`field`: 字段架构 |
| `OnRemovedField` | 字段删除后 | `table`: 表架构，`fieldName`: 字段名 |
| `OnUpdatingRow` | **数据写入前** | `row`: 待写入数据，`table`: 表架构。**返回 error 可拦截写入**。 |
| `OnUpdatedRows` | 数据写入/更新后 | `rows`: 已更新的数据列表（包含完整字段） |
| `OnRemovedRows` | 数据删除后 | `ids`: 已删除的主键列表 |
`TableDB` 提供底层隐藏表机制自动补全如 `id` 和 `autoIndex`，可用于初始化表结构。
```go
schema := `
== UserGroup ==
users SD
name v50 U
age i
`
_ = app.SyncSchema(schema)
```
### 3. 生命周期 Hooks 订阅
通过注册 Hooks，我们可以在发生元数据变更或行更新时进行回调拦截：
```go
app.Hooks.OnUpdatingRow = func(tableName string, row map[string]any) error {
if tableName == "users" {
row["modifiedAt"] = time.Now().Unix()
}
return nil
}
```
### 4. 动态表增删改查
底层调用与 `go/db` 使用习惯对齐，不同点是操作会被权限或 Hook 钩子影响：
```go
// 设定数据，如 ID 缺省会自动生成 c10 ID
_ = userApp.Table("users").Set(map[string]any{"name": "Alice"})
// 读取
record, _ := userApp.Table("users").Get("u1xxxx")
// 统计
count, _ := userApp.Table("users").Count(map[string]any{"age >": 20})
```

27
TEST.md
View File

@ -1,33 +1,20 @@
# 测试报告
## 性能测试
| 测试项 | 性能指标 (v1.1.6) | 备注 |
| :--- | :--- | :--- |
| **BenchmarkTableSet** | ~760,000 ns/op | 包含 Schema 验证、身份注入、1-RTT 写入及自动 ID 生成 |
| **鉴权开销 (RTT)** | 1-RTT | 核心鉴权逻辑已合并至主查询/写入 SQL，无额外网络往返 |
| **内存展平查询** | $O(1)$ | 字段有效性验证与策略检索均通过内存缓存实现，无递归开销 |
## 覆盖场景
1. **零开销 ABAC 鉴权引擎**
- 验证了在开启 `enableRLS` 时，系统自动注入 `creator = ? OR (conditions)`。
- 验证了 `checkAuth` 在单条操作（Get/Remove）时的 1-RTT 性能闭环。
2. **多级权限继承与环状检测**
- 测试通过了 `Manager -> Clerk` 的权限继承，Manager 自动获得 Clerk 的所有表级访问权。
- 验证了循环继承（A -> B -> A）场景下系统依然能保持稳定运行，不发生内存溢出或死锁。
3. **自主授权体系**
- 验证了普通用户（非 `_system`）只有在对目标表拥有 `full` 权限时，才能通过 `SetPolicy` 为第三方授权。
- 验证了 `inherit` 类型的策略被严格限制在 `SystemUserID` 权限下操作。
4. **结构化数据操作与隔离**
1. **结构化数据操作与隔离**
- 测试通过了包含和排除 system userID 时的增删改查。
- `Auth` 方法通过携带不同 userID 后实现了 `creator` 的数据隔离限制。
5. **生命周期 Hook 触发机制**
2. **生命周期 Hook 触发机制**
- `OnUpdatingRow` 触发并允许修改将要入库的行数据。
- `OnUpdatedRows` 在更新结束后被触发。
- `OnRemovedRows` 在成功移除后被触发返回对应的 ID 集合。
6. **SQL注入黑盒测试**
3. **SQL注入黑盒测试**
- Table 名称注入被验证并阻止。
- Field 查询字段注入被验证并阻止。
- Join On 条件验证被拦截。
- OrderBy 查询的语句结构注入被拦截。
## 性能测试
* 待核心 DB 层修复后回归测试性能。

220
cache.go
View File

@ -8,25 +8,20 @@ import (
)
type SchemaCache struct {
Tables map[string]map[string]any // name -> table record
TableIDMap map[string]string // id -> name
Fields map[string][]FieldSchema // tableId -> fields
ValidFieldsMap map[string]map[string]bool // tableName -> fieldName -> true
FlatPolicies map[string]map[string]map[string][]FlatPolicy // userID -> tableName -> action -> []FlatPolicy
lock sync.RWMutex
lastLoad time.Time
Tables map[string]map[string]any // name -> table record
Fields map[string][]FieldSchema // tableId -> fields
Policies []map[string]any
lock sync.RWMutex
lastLoad time.Time
}
var GlobalCache = &SchemaCache{
Tables: make(map[string]map[string]any),
TableIDMap: make(map[string]string),
Fields: make(map[string][]FieldSchema),
ValidFieldsMap: make(map[string]map[string]bool),
FlatPolicies: make(map[string]map[string]map[string][]FlatPolicy),
Tables: make(map[string]map[string]any),
Fields: make(map[string][]FieldSchema),
}
func (c *SchemaCache) Load(tDB *TableDB) error {
dbInst := tDB.base
dbInst := tDB.Base()
// Check if _Table exists first
res := dbInst.Query("SELECT name FROM sqlite_master WHERE type='table' AND name='_Table'")
if dbInst.Config.Type == "mysql" {
@ -36,173 +31,45 @@ func (c *SchemaCache) Load(tDB *TableDB) error {
return nil // System tables not yet created
}
tablesRes := dbInst.Query("SELECT * FROM `_Table`")
if tablesRes.Error != nil {
return tablesRes.Error
}
tables := tablesRes.MapResults()
fieldsRes := dbInst.Query("SELECT * FROM `_Field`")
if fieldsRes.Error != nil {
return fieldsRes.Error
}
fields := fieldsRes.MapResults()
policiesRes := dbInst.Query("SELECT * FROM `_Policy`")
if policiesRes.Error != nil {
return policiesRes.Error
}
policies := policiesRes.MapResults()
newTables := make(map[string]map[string]any)
newTableIDMap := make(map[string]string)
for _, t := range tables {
name := cast.String(t["name"])
newTables[name] = t
newTableIDMap[cast.String(t["id"])] = name
}
newFields := make(map[string][]FieldSchema)
newValidFieldsMap := make(map[string]map[string]bool)
for _, f := range fields {
var fs FieldSchema
cast.Convert(&fs, f)
// If fs.TableID is empty, try to manually get it just in case
if fs.TableID == "" {
fs.TableID = cast.String(f["tableId"])
}
tid := fs.TableID
newFields[tid] = append(newFields[tid], fs)
tableName := newTableIDMap[tid]
if tableName != "" {
if newValidFieldsMap[tableName] == nil {
newValidFieldsMap[tableName] = make(map[string]bool)
}
newValidFieldsMap[tableName][fs.Name] = true
}
}
// Flatten Policies
newFlatPolicies := make(map[string]map[string]map[string][]FlatPolicy)
type rawPolicy struct {
UserID string
Type string
Targets []string
Action string
Condition string
ConditionArgs []any
}
var rawPolicies []rawPolicy
cast.Convert(&rawPolicies, policies)
userToRaw := make(map[string][]rawPolicy)
for _, p := range rawPolicies {
userToRaw[p.UserID] = append(userToRaw[p.UserID], p)
}
var flatten func(userID string, visited map[string]bool) map[string]map[string][]FlatPolicy
flatten = func(userID string, visited map[string]bool) map[string]map[string][]FlatPolicy {
if visited[userID] {
return nil
}
visited[userID] = true
defer delete(visited, userID)
result := make(map[string]map[string][]FlatPolicy) // table -> action -> []FlatPolicy
addPolicy := func(table, action string, fp FlatPolicy) {
if result[table] == nil {
result[table] = make(map[string][]FlatPolicy)
}
result[table][action] = append(result[table][action], fp)
if action == "full" {
result[table]["read"] = append(result[table]["read"], fp)
result[table]["write"] = append(result[table]["write"], fp)
}
}
for _, p := range userToRaw[userID] {
if p.Type == "table" {
cond := p.Condition
if cond == "null" {
cond = ""
}
var condArgs []any
for _, arg := range p.ConditionArgs {
if arg != nil {
condArgs = append(condArgs, arg)
}
}
for _, targetTable := range p.Targets {
addPolicy(targetTable, p.Action, FlatPolicy{
Condition: cond,
ConditionArgs: condArgs,
})
}
} else if p.Type == "inherit" {
for _, parentID := range p.Targets {
parentFlat := flatten(parentID, visited)
for table, actions := range parentFlat {
for action, fps := range actions {
if result[table] == nil {
result[table] = make(map[string][]FlatPolicy)
}
result[table][action] = append(result[table][action], fps...)
}
}
}
}
}
return result
}
for userID := range userToRaw {
newFlatPolicies[userID] = flatten(userID, make(map[string]bool))
}
c.lock.Lock()
defer c.lock.Unlock()
// Load Tables
tables, err := tDB.Table("_Table").List(nil)
if err != nil {
return err
}
newTables := make(map[string]map[string]any)
for _, t := range tables {
newTables[cast.String(t["name"])] = t
}
c.Tables = newTables
c.TableIDMap = newTableIDMap
// Load Fields
fields, err := tDB.Table("_Field").List(nil)
if err != nil {
return err
}
newFields := make(map[string][]FieldSchema)
for _, f := range fields {
var fs FieldSchema
cast.Convert(&fs, f)
tid := fs.TableID
newFields[tid] = append(newFields[tid], fs)
}
c.Fields = newFields
c.ValidFieldsMap = newValidFieldsMap
c.FlatPolicies = newFlatPolicies
// Load Policies
policies, err := tDB.Table("_Policy").List(nil)
if err != nil {
return err
}
c.Policies = policies
c.lastLoad = time.Now()
return nil
}
func (c *SchemaCache) IsValidField(tableName, fieldName string) bool {
c.lock.RLock()
defer c.lock.RUnlock()
if fieldName == "id" || fieldName == "createTime" || fieldName == "creator" || fieldName == "updateTime" || fieldName == "updater" {
return true
}
if c.ValidFieldsMap[tableName] == nil {
return false
}
return c.ValidFieldsMap[tableName][fieldName]
}
func (c *SchemaCache) GetFlatPolicies(userID, tableName, action string) []FlatPolicy {
c.lock.RLock()
defer c.lock.RUnlock()
userPol := c.FlatPolicies[userID]
if userPol == nil {
return nil
}
tablePol := userPol[tableName]
if tablePol == nil {
return nil
}
return tablePol[action]
}
func (c *SchemaCache) GetTable(name string) map[string]any {
c.lock.RLock()
defer c.lock.RUnlock()
@ -227,19 +94,8 @@ func (c *SchemaCache) GetValidFields(tableName string) []string {
tid := cast.String(table["id"])
fields := c.Fields[tid]
var names []string
standardFields := map[string]bool{
"id": true,
"createTime": true,
"creator": true,
"updateTime": true,
"updater": true,
}
for _, f := range fields {
names = append(names, f.Name)
delete(standardFields, f.Name)
}
for f := range standardFields {
names = append(names, f)
}
return names
}

397
db.go
View File

@ -3,6 +3,7 @@ package tableDB
import (
"fmt"
"strings"
"time"
"apigo.cc/go/cast"
"apigo.cc/go/db"
@ -17,13 +18,8 @@ _Table SD // 核心表：存储所有表的元数据
id c10 PK
name v64 U // 表名
memo t // 备注
enableRLS b // 是否开启行级安全
settings o // 设置
isSecret b // 是否敏感表（不被索引）
createTime bi I // 创建时间
createTime bi // 创建时间
creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
_Field SD // 核心表：存储所有字段的元数据
id c10 PK
@ -32,221 +28,123 @@ _Field SD // 核心表：存储所有字段的元数据
type v32 // 字段类型
isIndex b // 是否索引
memo t // 备注
settings o // 设置
createTime bi I // 创建时间
createTime bi // 创建时间
creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
_Policy SD // 核心表：访问策略
id c10 PK
userID c10 I // 策略拥有者
type v32 // 策略类型 (inherit, table)
targets o // 作用目标数组 (inherit: userIDs, table: tableNames)
action v16 // 动作 (read, write, full)
condition v1024 // SQL WHERE 条件片段
conditionArgs o // 条件对应的参数数组
createTime bi I // 创建时间
subject v64 I // 主体 (UserID 或 Role)
action v32 I // 动作
resource v128 I // 资源 (Table 或 Record ID)
effect v16 // allow 或 deny
creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
`
type Hooks struct {
OnCreatedTable func(table *TableSchema)
OnRemovedTable func(table *TableSchema)
OnUpdatedField func(table *TableSchema, field *FieldSchema)
OnRemovedField func(table *TableSchema, fieldName string)
OnUpdatingRow func(row map[string]any, table *TableSchema, fields []FieldSchema) error
OnUpdatedRows func(rows []map[string]any, table *TableSchema, fields []FieldSchema)
OnRemovedRows func(ids []string, table *TableSchema, fields []FieldSchema)
}
type TableDBUnauthorized struct {
base *db.DB
Hooks *Hooks
OnCreatedTable func(tableName string, record map[string]any)
OnRemovedTable func(tableName string)
OnUpdatedField func(tableId, fieldName string, record map[string]any)
OnRemovedField func(tableId, fieldName string)
OnUpdatingRow func(tableName string, row map[string]any) error
OnUpdatedRows func(tableName string, count int)
OnRemovedRows func(tableName string, ids []string)
}
// TableDB wraps the base go/db implementation to provide high-level abstractions.
type TableDB struct {
base *db.DB
userID string
hooks *Hooks
Hooks *Hooks
}
// GetDB retrieves a configured database instance. Must call Auth() before use.
func GetDB(name string, logger *log.Logger, redis string) *TableDBUnauthorized {
type App = TableDB
// GetDB retrieves a configured database instance.
func GetDB(name string, logger *log.Logger) *TableDB {
baseDB := db.GetDB(name, logger)
if redis != "" {
baseDB.Config.Redis = redis
}
return &TableDBUnauthorized{
base: baseDB,
Hooks: &Hooks{},
return &TableDB{
base: baseDB,
userID: SystemUserID,
Hooks: &Hooks{},
}
}
// Bootstrap initializes the system tables metadata required for tableDB to function.
func (d *TableDBUnauthorized) Bootstrap() error {
err := d.base.Sync(SystemSchema)
// Auth creates a new instance with the specified userID context.
func (d *TableDB) Auth(userID string) *App {
return &TableDB{
base: d.base,
userID: userID,
Hooks: d.Hooks,
}
}
// SyncSchema automatically applies the DSL schema to the underlying database.
func (d *TableDB) SyncSchema(schemaDSL string) error {
// 1. Sync to actual DB
// Underground rules (autoIndex, id normalization) are now handled internally by d.base.Sync
finalDSL := schemaDSL
if !strings.Contains(schemaDSL, "_Table") {
finalDSL = SystemSchema + "\n" + schemaDSL
}
err := d.base.Sync(finalDSL)
if err != nil {
return err
}
sys := d.Auth(SystemUserID)
groups := db.ParseSchema(SystemSchema)
for _, group := range groups {
for _, table := range group.Tables {
tRecord := map[string]any{
"name": table.Name,
"memo": table.Comment,
}
res := sys.base.Query("SELECT id FROM `_Table` WHERE name = ?", table.Name)
var tid string
rows := res.MapResults()
if len(rows) > 0 {
tid = cast.String(rows[0]["id"])
tRecord["id"] = tid
}
_ = sys.Table("_Table").Set(tRecord)
if tid == "" {
res = sys.base.Query("SELECT id FROM `_Table` WHERE name = ?", table.Name)
rows = res.MapResults()
if len(rows) > 0 {
tid = cast.String(rows[0]["id"])
// 2. Update _Table and _Field metadata
res := d.base.Query("SELECT name FROM sqlite_master WHERE type='table' AND name='_Table'")
if d.base.Config.Type == "mysql" {
res = d.base.Query("SELECT TABLE_NAME name FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND TABLE_NAME='_Table'", d.base.Config.DB)
}
if res.Error == nil && res.MapOnR1()["name"] != nil {
groups := db.ParseSchema(finalDSL)
for _, group := range groups {
for _, table := range group.Tables {
// Upsert _Table
tRecord := map[string]any{
"name": table.Name,
"memo": table.Comment,
"createTime": time.Now().UnixMilli(),
}
existingTable, _ := d.Table("_Table").List(map[string]any{"name": table.Name})
var tid string
if len(existingTable) > 0 {
tid = cast.String(existingTable[0]["id"])
tRecord["id"] = tid
}
_ = d.Table("_Table").Set(tRecord)
if tid == "" {
newTable, _ := d.Table("_Table").List(map[string]any{"name": table.Name})
if len(newTable) > 0 {
tid = cast.String(newTable[0]["id"])
}
}
}
if tid != "" {
for _, field := range table.Fields {
fName := field.Name
if fName == "id" || fName == "createTime" || fName == "creator" || fName == "updateTime" || fName == "updater" {
continue
if tid != "" {
// Update _Field
for _, field := range table.Fields {
fRecord := map[string]any{
"tableId": tid,
"name": field.Name,
"type": field.Type,
"isIndex": cast.If(field.Index != "", 1, 0),
"memo": field.Comment,
"createTime": time.Now().UnixMilli(),
}
existingField, _ := d.Table("_Field").List(map[string]any{"tableId": tid, "name": field.Name})
if len(existingField) > 0 {
fRecord["id"] = existingField[0]["id"]
}
_ = d.Table("_Field").Set(fRecord)
}
fRecord := map[string]any{
"tableId": tid,
"name": fName,
"type": field.Type,
"isIndex": cast.If(field.Index != "", 1, 0),
"memo": field.Comment,
}
fRes := sys.base.Query("SELECT id FROM `_Field` WHERE tableId = ? AND name = ?", tid, fName)
fRows := fRes.MapResults()
if len(fRows) > 0 {
fRecord["id"] = fRows[0]["id"]
}
_ = sys.Table("_Field").Set(fRecord)
}
}
}
}
return GlobalCache.Load(sys)
}
// Auth creates a new instance with the specified userID context.
func (d *TableDBUnauthorized) Auth(userID string) *TableDB {
return &TableDB{
base: d.base,
userID: userID,
hooks: d.Hooks,
}
}
// Auth creates a new instance with the specified userID context from an existing authorized instance.
func (d *TableDB) Auth(userID string) *TableDB {
return &TableDB{
base: d.base,
userID: userID,
hooks: d.hooks,
}
}
// Tables returns a list of tables that the current user has access to.
func (d *TableDB) Tables() ([]TableSchema, error) {
res, err := d.Table("_Table").List(nil)
if err != nil {
return nil, err
}
var tables []TableSchema
cast.Convert(&tables, res)
return tables, nil
}
// SetTable updates or creates the table metadata.
func (d *TableDB) SetTable(schema TableSchema) error {
record := map[string]any{
"name": schema.Name,
"memo": schema.Memo,
"enableRLS": schema.EnableRLS,
"settings": schema.Settings,
}
if schema.ID != "" {
record["id"] = schema.ID
}
return d.Table("_Table").Set(record)
}
// RemoveTable deletes a table.
func (d *TableDB) RemoveTable(name string) error {
tableRec := GlobalCache.GetTable(name)
if tableRec == nil {
return fmt.Errorf("table %s not found", name)
}
return d.Table("_Table").Remove(cast.String(tableRec["id"]))
}
// SetPolicy updates or creates an access policy.
func (d *TableDB) SetPolicy(policy PolicySchema) error {
if d.userID != SystemUserID {
if policy.Type == "inherit" {
return fmt.Errorf("only system user can set inherit policy")
}
if policy.Type == "table" {
// Only users with permission to the table can set type=table policy
for _, targetTable := range policy.Targets {
// Check if current user has 'full' access to the target table
pols := GlobalCache.GetFlatPolicies(d.userID, targetTable, "full")
hasFullAccess := false
for _, p := range pols {
if p.Condition == "" {
hasFullAccess = true
break
}
}
if !hasFullAccess {
return fmt.Errorf("permission denied to set policy for table %s", targetTable)
}
}
}
}
record := map[string]any{
"userID": policy.UserID,
"type": policy.Type,
"targets": policy.Targets,
"action": policy.Action,
"condition": policy.Condition,
"conditionArgs": policy.ConditionArgs,
"creator": d.userID,
}
if policy.ID != "" {
record["id"] = policy.ID
}
return d.Auth(SystemUserID).Table("_Policy").Set(record)
}
// ListPolicy retrieves policies based on filter.
func (d *TableDB) ListPolicy(filter map[string]any) ([]PolicySchema, error) {
res, err := d.Table("_Policy").List(filter)
if err != nil {
return nil, err
}
var policies []PolicySchema
cast.Convert(&policies, res)
return policies, nil
// 3. Reload cache
return GlobalCache.Load(d)
}
// Table returns an AI-friendly interface for multi-dimensional operations on a specific table.
@ -254,37 +152,50 @@ func (d *TableDB) Table(name string) *Table {
return NewTable(name, d)
}
// GetRawDB returns the underlying apigo.cc/go/db.DB. Only allowed for SystemUserID.
func (d *TableDB) GetRawDB() (*db.DB, error) {
if d.userID != SystemUserID {
return nil, fmt.Errorf("permission denied for GetRawDB")
}
return d.base, nil
// Base returns the underlying apigo.cc/go/db.DB for raw queries if needed.
func (d *TableDB) Base() *db.DB {
return d.base
}
// buildQuery constructs a SQL query from a QueryRequest with strict identifier validation and auth filtering.
func (d *TableDB) buildQuery(t *Table, req QueryRequest) (string, []any, error) {
tableName := t.Name
if GlobalCache.GetTable(tableName) == nil {
return "", nil, fmt.Errorf("invalid table: %s", tableName)
// Query performs a structured query.
func (d *TableDB) Query(req QueryRequest) ([]map[string]any, error) {
sql, args, err := d.BuildQuery(req)
if err != nil {
return nil, err
}
res := d.base.Query(sql, args...)
if res.Error != nil {
return nil, res.Error
}
return res.MapResults(), nil
}
// BuildQuery constructs a SQL query from a QueryRequest with strict identifier validation.
func (d *TableDB) BuildQuery(req QueryRequest) (string, []any, error) {
if GlobalCache.GetTable(req.Table) == nil {
return "", nil, fmt.Errorf("invalid table: %s", req.Table)
}
fieldList := req.Select
if len(fieldList) == 0 {
fieldList = GlobalCache.GetValidFields(tableName)
}
var validatedSelect []string
for _, s := range fieldList {
if !GlobalCache.IsValidField(tableName, s) {
return "", nil, fmt.Errorf("invalid field %s in table %s", s, tableName)
fields := "*"
if len(req.Select) > 0 {
validFields := GlobalCache.GetValidFields(req.Table)
fieldMap := make(map[string]bool)
for _, f := range validFields {
fieldMap[f] = true
}
validatedSelect = append(validatedSelect, "`"+s+"`")
var validatedSelect []string
for _, s := range req.Select {
if !fieldMap[s] {
return "", nil, fmt.Errorf("invalid field %s in table %s", s, req.Table)
}
validatedSelect = append(validatedSelect, "`"+s+"`")
}
fields = strings.Join(validatedSelect, ", ")
}
fieldsStr := strings.Join(validatedSelect, ", ")
var sql strings.Builder
fmt.Fprintf(&sql, "SELECT %s FROM `%s` ", fieldsStr, tableName)
fmt.Fprintf(&sql, "SELECT %s FROM `%s` ", fields, req.Table)
for _, join := range req.Joins {
if GlobalCache.GetTable(join.Table) == nil {
@ -302,42 +213,38 @@ func (d *TableDB) buildQuery(t *Table, req QueryRequest) (string, []any, error)
}
args := req.Args
whereStr := req.Where
// Apply auth filtering for the main table
var err error
whereStr, args, err = t.appendAuthAndConstraint(whereStr, args)
if err != nil {
return "", nil, err
}
if whereStr != "" {
if req.Where != "" {
sql.WriteString(" WHERE ")
sql.WriteString(whereStr)
sql.WriteString(req.Where)
}
orderBy := req.OrderBy
if orderBy == "" {
orderBy = "createTime DESC"
}
parts := strings.Fields(orderBy)
if len(parts) > 0 {
fieldName := parts[0]
if !GlobalCache.IsValidField(tableName, fieldName) {
return "", nil, fmt.Errorf("invalid order by field: %s", fieldName)
}
direction := ""
if len(parts) > 1 {
dir := strings.ToUpper(parts[1])
if dir == "ASC" || dir == "DESC" {
direction = " " + dir
} else {
return "", nil, fmt.Errorf("invalid order by direction: %s", parts[1])
if req.OrderBy != "" {
parts := strings.Fields(req.OrderBy)
if len(parts) > 0 {
fieldName := parts[0]
validFields := GlobalCache.GetValidFields(req.Table)
found := false
for _, f := range validFields {
if f == fieldName {
found = true
break
}
}
if !found {
return "", nil, fmt.Errorf("invalid order by field: %s", fieldName)
}
direction := ""
if len(parts) > 1 {
dir := strings.ToUpper(parts[1])
if dir == "ASC" || dir == "DESC" {
direction = " " + dir
} else {
return "", nil, fmt.Errorf("invalid order by direction: %s", parts[1])
}
}
fmt.Fprintf(&sql, " ORDER BY `%s` %s", fieldName, direction)
}
fmt.Fprintf(&sql, " ORDER BY `%s` %s", fieldName, direction)
}
if req.Limit > 0 {
@ -350,7 +257,7 @@ func (d *TableDB) buildQuery(t *Table, req QueryRequest) (string, []any, error)
return sql.String(), args, nil
}
// buildWhere is a helper to convert a map of conditions into a SQL WHERE clause and args.
// BuildWhere is a helper to convert a map of conditions into a SQL WHERE clause and args.
func buildWhere(filter map[string]any) (string, []any) {
if len(filter) == 0 {
return "", nil

24
go.mod
View File

@ -3,21 +3,21 @@ module apigo.cc/go/tableDB
go 1.25.0
require (
apigo.cc/go/cast v1.3.3
apigo.cc/go/db v1.3.4
apigo.cc/go/log v1.3.4
apigo.cc/go/cast v1.3.2
apigo.cc/go/db v1.3.1
apigo.cc/go/id v1.3.0
apigo.cc/go/log v1.3.2
)
require (
apigo.cc/go/config v1.3.1 // indirect
apigo.cc/go/crypto v1.3.1 // indirect
apigo.cc/go/encoding v1.3.1 // indirect
apigo.cc/go/file v1.3.2 // indirect
apigo.cc/go/id v1.3.1 // indirect
apigo.cc/go/rand v1.3.1 // indirect
apigo.cc/go/redis v1.3.2 // indirect
apigo.cc/go/safe v1.3.1 // indirect
apigo.cc/go/shell v1.3.1 // indirect
apigo.cc/go/config v1.3.0 // indirect
apigo.cc/go/crypto v1.3.0 // indirect
apigo.cc/go/encoding v1.3.0 // indirect
apigo.cc/go/file v1.3.0 // indirect
apigo.cc/go/rand v1.3.0 // indirect
apigo.cc/go/redis v1.3.0 // indirect
apigo.cc/go/safe v1.3.0 // indirect
apigo.cc/go/shell v1.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gomodule/redigo v2.0.0+incompatible // indirect
github.com/google/uuid v1.6.0 // indirect

48
go.sum
View File

@ -1,27 +1,27 @@
apigo.cc/go/cast v1.3.3 h1:aln5eDR5DZVWVzZ/y5SJh1gQNgWv2sT82I25NaO9g34=
apigo.cc/go/cast v1.3.3/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk=
apigo.cc/go/config v1.3.1 h1:wZzUh4oL+fGD6SayVgX6prLPMsniM25etWFcEH8XzIE=
apigo.cc/go/config v1.3.1/go.mod h1:7KHz/1WmtBLM762Lln/TaXh2dmlMvJTLhnlk33zbS3U=
apigo.cc/go/crypto v1.3.1 h1:ulQ2zX9bUWirk0sEacx1Srsjs2Jow7HlZq7ED7msNcg=
apigo.cc/go/crypto v1.3.1/go.mod h1:SwHlBFDPddttWgFFtzsEMla8CM/rcFy9nvdsJjW4CIs=
apigo.cc/go/db v1.3.4 h1:SDTK9+anU3vc30i5JT4y5NCkUByFxg+tX5Ju+Ye/jq4=
apigo.cc/go/db v1.3.4/go.mod h1:/uELxvpf9i6cE/2fS/LprJl5JoIX3w/K9qKMy7Hdr98=
apigo.cc/go/encoding v1.3.1 h1:y8O58KYAyulkThg1O2ji2BqjnFoSvk42sit9I3z+K7Y=
apigo.cc/go/encoding v1.3.1/go.mod h1:xAJk5b83VZ31mXMTnyp0dfMoBKfT/AHDn0u+cQfojgY=
apigo.cc/go/file v1.3.2 h1:pu4oiDyiqgj3/eykfnJf+/6+A9v/Z0b3ClP5XK+lwG4=
apigo.cc/go/file v1.3.2/go.mod h1:vci4h0Pz94mV6dkniQkuyBYERVYeq7/LX4jJVuCg9hs=
apigo.cc/go/id v1.3.1 h1:pkqi6VeWyQoHuIu0Zbx/RRxIAdM61Js0j6cY1M9XVCk=
apigo.cc/go/id v1.3.1/go.mod h1:P2/vl3tyW3US+ayOFSMoPIOCulNLBngNYPhXJC/Z7J4=
apigo.cc/go/log v1.3.4 h1:UT8Neb9r4QjjbCFbTzw+ZeTxd+DmdmR5gNExeR4Cj+g=
apigo.cc/go/log v1.3.4/go.mod h1:/Q/2r51xWSsrS4QN5U9jLiTw8n6qNC8kG9nuVHweY20=
apigo.cc/go/rand v1.3.1 h1:7FvsI6PtQ5XrWER0dTiLVo0p7GIxRidT/TBKhVy93j8=
apigo.cc/go/rand v1.3.1/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk=
apigo.cc/go/redis v1.3.2 h1:iUWL/CHHnfonz0dJq6/V4IG3QuXBoHA2L1xnoGEbNEQ=
apigo.cc/go/redis v1.3.2/go.mod h1:/k5wcfAzB9jrfd9otabio9CPUxEsLPgEs4oggBG5sbs=
apigo.cc/go/safe v1.3.1 h1:irTCqPAC97gGsX/Lw5AzLelDt1xXLEZIAaVhLELWe9Q=
apigo.cc/go/safe v1.3.1/go.mod h1:XdOpBhN2vkImalaykYXXmEpczqWa1y3ah6/Q72cdRqE=
apigo.cc/go/shell v1.3.1 h1:M8oD0b2HcJuCC6frQFx11b3UTcTx3lATX8XK+YXSVm8=
apigo.cc/go/shell v1.3.1/go.mod h1:ZMdJjpCpWdvsHKUXlelh/AxsV/nWdkH/k3lISfzMdUw=
apigo.cc/go/cast v1.3.2 h1:hh9MWDSwh3T/kQdCHjFpjDwHrh2A05Q4wt1AAWs8NBI=
apigo.cc/go/cast v1.3.2/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk=
apigo.cc/go/config v1.3.0 h1:TwI3bv3D+BJrAnFx+o62HQo3FarY2Ge3SCGsKchFYGg=
apigo.cc/go/config v1.3.0/go.mod h1:88lqKEBXlIExFKt1geLONVLYyM+QhRVpBe0ok3OEvjI=
apigo.cc/go/crypto v1.3.0 h1:rGRrrb5O+4M50X5hVUmJQbXx3l87zzlcgzGtUvZrZL8=
apigo.cc/go/crypto v1.3.0/go.mod h1:uSCcmbcFoiltUPMQTSuqmU9nfKEH/lRs7nQ7aa3Z4Mc=
apigo.cc/go/db v1.3.1 h1:X+iHMZ71OdWW8R91TFtE+aZr/CZPOtL7gIyt3tEkfyc=
apigo.cc/go/db v1.3.1/go.mod h1:Gc6HEmncoYRaaRy5yPjTegwc2pSwjNOTnINBmbomhcs=
apigo.cc/go/encoding v1.3.0 h1:8jqNHoZBR8vOU/BGsLFebfp1Txa1UxDRpd7YwzIFLJs=
apigo.cc/go/encoding v1.3.0/go.mod h1:kT/uUJiuAOkZ4LzUWrUtk/I0iL1D8aatvD+59bDnHBo=
apigo.cc/go/file v1.3.0 h1:xG9FcY3Rv6Br83r9pq9QsIXFrplx4g8ITOkHSzfzXRg=
apigo.cc/go/file v1.3.0/go.mod h1:pYHBlB/XwsrnWpEh7GIFpbiqobrExfiB+rEN8V2d2kY=
apigo.cc/go/id v1.3.0 h1:Tr2Yj0Rl19lfwW5wBTJ407o/zgo2oVRLE20WWEgJzdE=
apigo.cc/go/id v1.3.0/go.mod h1:AFH3kMFwENfXNyijnAFWEhSF1o3y++UBPem1IUlrcxA=
apigo.cc/go/log v1.3.2 h1:/m3V4MnlYnCG4XPHpWDsa4cw5suMaDVY1SgaVyjnBSo=
apigo.cc/go/log v1.3.2/go.mod h1:dz4bSz9BnOgutkUJJZfX3uDDwsMpUxt7WF50mLK9hgE=
apigo.cc/go/rand v1.3.0 h1:k+UFAhMySwXf+dq8Om9TniZV6fm6gAE0evbrqMEdwQU=
apigo.cc/go/rand v1.3.0/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk=
apigo.cc/go/redis v1.3.0 h1:3NJE3xPXzhCwL+Mh1iyphFrsKWEuPlY26LHJfMVFSeU=
apigo.cc/go/redis v1.3.0/go.mod h1:KPDPwMOER7WJX3Qev24LTeAOSmCl8OApe8iagPDxOUQ=
apigo.cc/go/safe v1.3.0 h1:uctdAUsphT9p60Tk4oS5xPCe0NoIdOHfsYv4PNS0Rok=
apigo.cc/go/safe v1.3.0/go.mod h1:tC9X14V+qh0BqIrVg4UkXbl+2pEN+lj2ZNI8IjDB6Fs=
apigo.cc/go/shell v1.3.0 h1:hdxuYPN/7T2BuM/Ja8AjVUhbRqU/wpi8OjcJVziJ0nw=
apigo.cc/go/shell v1.3.0/go.mod h1:aNJiRWibxlA485yX3t+07IVAbrALKmxzv4oGEUC+hK4=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=

591
table.go
View File

@ -3,10 +3,10 @@ package tableDB
import (
"fmt"
"strings"
"time"
"apigo.cc/go/cast"
"apigo.cc/go/db"
"apigo.cc/go/id"
)
// Table provides an AI-friendly interface for interacting with structured data or schema.
@ -15,15 +15,20 @@ type Table struct {
userID string
db *db.DB
app *TableDB
isMeta bool // True if Name ends with ":Schema"
constraint map[string]any
}
// NewTable creates a new Table instance. Handles ":Field" suffix.
// NewTable creates a new Table instance. Handles ":Schema" and ":Field" suffix.
func NewTable(name string, app *TableDB) *Table {
isMeta := false
actualName := name
var constraint map[string]any
if strings.HasSuffix(name, ":Field") {
if strings.HasSuffix(name, ":Schema") {
isMeta = true
actualName = strings.TrimSuffix(name, ":Schema")
} else if strings.HasSuffix(name, ":Field") {
tableName := strings.TrimSuffix(name, ":Field")
actualName = "_Field"
// lookup table_id
@ -41,74 +46,52 @@ func NewTable(name string, app *TableDB) *Table {
userID: app.userID,
db: app.base,
app: app,
isMeta: isMeta,
constraint: constraint,
}
}
func (t *Table) checkAuth(id string, action string) error {
if t.userID == SystemUserID {
func (t *Table) checkAuth(id string) error {
if t.userID == "_system" {
return nil
}
if action == "write" && (t.Name == "_Policy" || t.Name == "_Backup") {
// Check if table has 'creator' field
hasCreator := false
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
tid := cast.String(tableRec["id"])
fields := GlobalCache.GetFields(tid)
for _, f := range fields {
if f.Name == "creator" {
hasCreator = true
break
}
}
}
if !hasCreator && !strings.HasPrefix(t.Name, "_") {
return nil
}
if t.Name == "_Policy" || t.Name == "_Backup" {
return fmt.Errorf("permission denied for %s", t.Name)
}
tableRec := GlobalCache.GetTable(t.Name)
enableRLS := false
if tableRec != nil {
enableRLS = cast.Bool(tableRec["enableRLS"])
}
if !enableRLS && !strings.HasPrefix(t.Name, "_") {
return nil
}
policies := GlobalCache.GetFlatPolicies(t.userID, t.Name, action)
hasFullAccess := false
for _, p := range policies {
if p.Condition == "" {
hasFullAccess = true
break
// Check existing record
res := t.db.Query(fmt.Sprintf("SELECT creator FROM `%s` WHERE id = ?", t.Name), id)
rec := res.MapOnR1()
if len(rec) > 0 {
if cast.String(rec["creator"]) == t.userID {
return nil
}
}
if hasFullAccess {
return nil
}
// 构建合并查询：一次性判断 Creator 和 所有 Conditions
var authConditions []string
var authArgs []any
// 1. 追加 Creator 检查
hasCreator := true
if hasCreator {
authConditions = append(authConditions, "`creator` = ?")
authArgs = append(authArgs, t.userID)
}
// 2. 追加 Policy Conditions
for _, p := range policies {
if p.Condition != "" {
authConditions = append(authConditions, "("+p.Condition+")")
authArgs = append(authArgs, p.ConditionArgs...)
// Check _Policy
pol := t.db.Query("SELECT 1 FROM _Policy WHERE subject = ? AND resource = ? AND effect = 'allow' LIMIT 1", t.userID, id)
if len(pol.MapOnR1()) > 0 {
return nil
}
return fmt.Errorf("permission denied for %s record %s", t.Name, id)
}
// 3. 组装并执行终极 1-RTT 查询
if len(authConditions) > 0 {
authPart := "(" + strings.Join(authConditions, " OR ") + ")"
checkSQL := fmt.Sprintf("SELECT 1 FROM `%s` WHERE id = ? AND %s LIMIT 1", t.Name, authPart)
finalArgs := append([]any{id}, authArgs...)
checkRes := t.db.Query(checkSQL, finalArgs...)
if len(checkRes.MapOnR1()) > 0 {
return nil // 验证通过！
}
}
return fmt.Errorf("permission denied for %s record %s", t.Name, id)
return nil
}
func (t *Table) appendAuthAndConstraint(whereStr string, args []any) (string, []any, error) {
@ -123,61 +106,34 @@ func (t *Table) appendAuthAndConstraint(whereStr string, args []any) (string, []
}
}
if t.userID == SystemUserID {
if t.userID == "_system" {
return whereStr, args, nil
}
// Check if table has 'creator' field
hasCreator := false
tableRec := GlobalCache.GetTable(t.Name)
enableRLS := false
if tableRec != nil {
enableRLS = cast.Bool(tableRec["enableRLS"])
}
if !enableRLS && !strings.HasPrefix(t.Name, "_") {
return whereStr, args, nil
}
policies := GlobalCache.GetFlatPolicies(t.userID, t.Name, "read")
hasFullAccess := false
for _, p := range policies {
if p.Condition == "" {
hasFullAccess = true
break
}
}
if hasFullAccess {
return whereStr, args, nil
}
// Build dynamic SQL
var authConditions []string
var authArgs []any
// Check creator field
hasCreator := true
if hasCreator {
authConditions = append(authConditions, "`creator` = ?")
authArgs = append(authArgs, t.userID)
}
for _, p := range policies {
if p.Condition != "" {
authConditions = append(authConditions, "("+p.Condition+")")
authArgs = append(authArgs, p.ConditionArgs...)
tid := cast.String(tableRec["id"])
fields := GlobalCache.GetFields(tid)
for _, f := range fields {
if f.Name == "creator" {
hasCreator = true
break
}
}
}
authPart := "0"
if len(authConditions) > 0 {
authPart = "(" + strings.Join(authConditions, " OR ") + ")"
args = append(args, authArgs...)
}
if whereStr != "" {
whereStr = "(" + whereStr + ") AND " + authPart
} else {
whereStr = authPart
if hasCreator || strings.HasPrefix(t.Name, "_") {
if t.Name == "_Policy" || t.Name == "_Backup" {
return "", nil, fmt.Errorf("permission denied for %s", t.Name)
}
authWhere := fmt.Sprintf("(creator = '%s' OR id IN (SELECT resource FROM _Policy WHERE subject = '%s' AND effect = 'allow'))", t.userID, t.userID)
if whereStr != "" {
whereStr = "(" + whereStr + ") AND " + authWhere
} else {
whereStr = authWhere
}
}
return whereStr, args, nil
@ -196,9 +152,7 @@ func (t *Table) reconstructAndSyncSchema() error {
var sb strings.Builder
for _, tbl := range tables {
name := cast.String(tbl["name"])
if name == "" {
continue
}
if name == "" { continue }
tid := cast.String(tbl["id"])
tblFields := fieldMap[tid]
@ -213,23 +167,22 @@ func (t *Table) reconstructAndSyncSchema() error {
}
sb.WriteString("\n")
// Always ensure standard fields are present with appropriate indices
sb.WriteString(" id c10 PK\n")
sb.WriteString(" createTime bi I\n")
sb.WriteString(" creator v64\n")
sb.WriteString(" updateTime bi I\n")
sb.WriteString(" updater v64\n")
hasID := false
for _, f := range tblFields {
if cast.String(f["name"]) == "id" {
hasID = true
break
}
}
if !hasID {
sb.WriteString(" id c10 PK\n")
}
for _, f := range tblFields {
fname := cast.String(f["name"])
if fname == "id" || fname == "createTime" || fname == "creator" || fname == "updateTime" || fname == "updater" {
continue
}
ftype := cast.String(f["type"])
if ftype == "" {
ftype = "v255"
}
isIndex := cast.Bool(f["isIndex"]) || cast.Int(f["isIndex"]) == 1
if ftype == "" { ftype = "v255" }
isIndex := cast.Int(f["isIndex"]) == 1
fmemo := cast.String(f["memo"])
sb.WriteString(" " + fname + " " + ftype)
@ -244,224 +197,115 @@ func (t *Table) reconstructAndSyncSchema() error {
sb.WriteString("\n")
}
err := t.db.Sync(sb.String())
if err != nil {
return err
}
return GlobalCache.Load(t.app)
return t.db.Sync(sb.String())
}
// Set performs an upsert of one or more records.
func (t *Table) Set(data ...any) error {
// Set performs an upsert.
func (t *Table) Set(data any) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
return fmt.Errorf("no permission")
}
metaTouched := false
var updatedRows []map[string]any
record := make(map[string]any)
j, _ := cast.ToJSON(data)
_ = cast.UnmarshalJSON([]byte(j), &record)
// Pre-fetch schema context for business tables
var tableSchema *TableSchema
var fields []FieldSchema
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
cast.Convert(&tableSchema, tableRec)
fields = GlobalCache.GetFields(tableSchema.ID)
}
for _, d := range data {
record := make(map[string]any)
cast.Convert(&record, d)
if t.constraint != nil {
for k, v := range t.constraint {
record[k] = v
}
if t.isMeta {
dsl, ok := record["dsl"].(string)
if !ok {
return fmt.Errorf("schema dsl is required")
}
return t.app.SyncSchema(dsl)
}
if t.app.hooks.OnUpdatingRow != nil && t.Name != "_Table" && t.Name != "_Field" {
if err := t.app.hooks.OnUpdatingRow(record, tableSchema, fields); err != nil {
if t.constraint != nil {
for k, v := range t.constraint {
record[k] = v
}
}
if t.app.Hooks.OnUpdatingRow != nil && t.Name != "_Table" && t.Name != "_Field" {
if err := t.app.Hooks.OnUpdatingRow(t.Name, record); err != nil {
return err
}
}
idVal := record["id"]
var isInsert bool
if idVal == nil || cast.String(idVal) == "" {
record["id"] = id.MakeID(10)
isInsert = true
} else {
idStr := cast.String(idVal)
res := t.db.Query(fmt.Sprintf("SELECT id FROM `%s` WHERE id = ? LIMIT 1", t.Name), idStr)
rec := res.MapOnR1()
if rec != nil && len(rec) > 0 {
if err := t.checkAuth(idStr); err != nil {
return err
}
}
idVal := record["id"]
var isInsert bool
if idVal == nil || cast.String(idVal) == "" {
isInsert = true
isInsert = false
} else {
idStr := cast.String(idVal)
res := t.db.Query(fmt.Sprintf("SELECT id FROM `%s` WHERE id = ? LIMIT 1", t.Name), idStr)
rec := res.MapOnR1()
if rec != nil && len(rec) > 0 {
if err := t.checkAuth(idStr, "write"); err != nil {
return err
}
isInsert = false
} else {
isInsert = true
}
isInsert = true
}
}
// Always update updateTime and updater
record["updateTime"] = time.Now().UnixMilli()
record["updater"] = t.userID
var err error
if isInsert {
if t.userID != "_system" {
// Check if table has 'creator' field
hasCreator := false
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
tid := cast.String(tableRec["id"])
fields := GlobalCache.GetFields(tid)
for _, f := range fields {
if f.Name == "creator" {
hasCreator = true
break
}
}
}
var err error
if isInsert {
record["createTime"] = record["updateTime"]
if t.userID != SystemUserID {
if hasCreator || strings.HasPrefix(t.Name, "_") {
if t.Name == "_Policy" || t.Name == "_Backup" {
return fmt.Errorf("permission denied for %s", t.Name)
}
record["creator"] = t.userID
} else {
if record["creator"] == nil {
record["creator"] = t.userID
}
}
err = t.db.Insert(t.Name, record).Error
} else {
// Prevent overwriting CreateTime and Creator on update
delete(record, "createTime")
delete(record, "creator")
err = t.db.Update(t.Name, record, "id = ?", cast.String(record["id"])).Error
}
if err != nil {
return err
}
if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
metaTouched = true
if t.Name == "_Table" {
if isInsert && t.app.hooks.OnCreatedTable != nil {
var ts TableSchema
cast.Convert(&ts, record)
t.app.hooks.OnCreatedTable(&ts)
}
} else if t.Name == "_Field" {
if t.app.hooks.OnUpdatedField != nil {
var fs FieldSchema
cast.Convert(&fs, record)
// Look up table schema
var fts *TableSchema
GlobalCache.lock.RLock()
tableName := GlobalCache.TableIDMap[fs.TableID]
fTableRec := GlobalCache.Tables[tableName]
GlobalCache.lock.RUnlock()
if fTableRec != nil {
cast.Convert(&fts, fTableRec)
}
t.app.hooks.OnUpdatedField(fts, &fs)
}
}
} else {
updatedRows = append(updatedRows, record)
}
err = t.db.Insert(t.Name, record).Error
} else {
err = t.db.Update(t.Name, record, "id = ?", cast.String(record["id"])).Error
}
if metaTouched {
if t.Name != "_Policy" {
if err == nil {
if t.Name == "_Table" {
_ = t.reconstructAndSyncSchema()
}
_ = GlobalCache.Load(t.app)
}
if len(updatedRows) > 0 && t.app.hooks.OnUpdatedRows != nil {
t.app.hooks.OnUpdatedRows(updatedRows, tableSchema, fields)
}
return nil
}
// SetField adds or updates one or more fields. If table doesn't exist, it will be created.
func (t *Table) SetField(fields ...FieldSchema) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
}
// 1. Ensure table exists in _Table
tableRec := GlobalCache.GetTable(t.Name)
if tableRec == nil {
// Create table entry
err := t.app.SetTable(TableSchema{Name: t.Name})
if err != nil {
return err
}
tableRec = GlobalCache.GetTable(t.Name)
if tableRec == nil {
return fmt.Errorf("failed to create table entry for %s", t.Name)
}
// Automatically add default ID field to metadata
_ = t.app.Table(t.Name + ":Field").Set(map[string]any{
"name": "id",
"type": "c10",
"isIndex": 1,
"memo": "Primary Key",
})
}
// 2. Prepare field records
fieldTable := t.app.Table(t.Name + ":Field")
var fieldRecords []any
for _, f := range fields {
fRecord := map[string]any{
"name": f.Name,
"type": f.Type,
"isIndex": f.IsIndex,
"memo": f.Memo,
"settings": f.Settings,
}
if f.ID != "" {
fRecord["id"] = f.ID
if isInsert && t.app.Hooks.OnCreatedTable != nil {
t.app.Hooks.OnCreatedTable(cast.String(record["name"]), record)
}
} else if t.Name == "_Field" {
_ = t.reconstructAndSyncSchema()
if t.app.Hooks.OnUpdatedField != nil {
t.app.Hooks.OnUpdatedField(cast.String(record["tableId"]), cast.String(record["name"]), record)
}
} else {
// If ID is missing, try to find existing field to update
existing, _ := fieldTable.List(map[string]any{"name": f.Name})
if len(existing) > 0 {
fRecord["id"] = existing[0]["id"]
}
}
fieldRecords = append(fieldRecords, fRecord)
}
// 3. Batch Set fields
return fieldTable.Set(fieldRecords...)
}
// RemoveField deletes one or more fields by name.
func (t *Table) RemoveField(names ...string) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
}
tableRec := GlobalCache.GetTable(t.Name)
if tableRec == nil {
return fmt.Errorf("table %s not found", t.Name)
}
fieldTable := t.app.Table(t.Name + ":Field")
for _, name := range names {
existing, _ := fieldTable.List(map[string]any{"name": name})
if len(existing) > 0 {
err := fieldTable.Remove(cast.String(existing[0]["id"]))
if err != nil {
return err
if t.app.Hooks.OnUpdatedRows != nil {
t.app.Hooks.OnUpdatedRows(t.Name, 1)
}
}
}
return nil
return err
}
// Get retrieves a single record.
func (t *Table) Get(id string) (map[string]any, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
if t.isMeta {
return map[string]any{"name": t.Name, "type": "table"}, nil
}
if err := t.checkAuth(id, "read"); err != nil {
if err := t.checkAuth(id); err != nil {
return nil, err
}
@ -478,119 +322,80 @@ func (t *Table) Get(id string) (map[string]any, error) {
return record, nil
}
// Remove deletes one or more records by ID.
func (t *Table) Remove(ids ...string) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
// Remove deletes a record.
func (t *Table) Remove(id string) error {
if t.isMeta {
return fmt.Errorf("schema removal not allowed via Table API")
}
var removedIDs []string
// Pre-fetch schema context for business tables
var tableSchema *TableSchema
var fields []FieldSchema
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
cast.Convert(&tableSchema, tableRec)
fields = GlobalCache.GetFields(tableSchema.ID)
if err := t.checkAuth(id); err != nil {
return err
}
for _, id := range ids {
if err := t.checkAuth(id, "write"); err != nil {
return err
}
var record map[string]any
if t.Name == "_Table" || t.Name == "_Field" {
res := t.db.Query(fmt.Sprintf("SELECT * FROM `%s` WHERE id = ?", t.Name), id)
record = res.MapOnR1()
}
var record map[string]any
if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
res := t.db.Query(fmt.Sprintf("SELECT * FROM `%s` WHERE id = ?", t.Name), id)
record = res.MapOnR1()
}
res := t.db.Delete(t.Name, "id = ?", id)
if res.Error == nil {
if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
if t.Name != "_Policy" {
_ = t.reconstructAndSyncSchema()
}
_ = GlobalCache.Load(t.app)
if t.Name == "_Table" && record != nil && t.app.hooks.OnRemovedTable != nil {
var ts TableSchema
cast.Convert(&ts, record)
t.app.hooks.OnRemovedTable(&ts)
} else if t.Name == "_Field" && record != nil && t.app.hooks.OnRemovedField != nil {
var fts *TableSchema
GlobalCache.lock.RLock()
tableName := GlobalCache.TableIDMap[cast.String(record["tableId"])]
fTableRec := GlobalCache.Tables[tableName]
GlobalCache.lock.RUnlock()
if fTableRec != nil {
cast.Convert(&fts, fTableRec)
}
t.app.hooks.OnRemovedField(fts, cast.String(record["name"]))
}
} else {
removedIDs = append(removedIDs, id)
res := t.db.Delete(t.Name, "id = ?", id)
if res.Error == nil {
if t.Name == "_Table" {
_ = t.reconstructAndSyncSchema()
if record != nil && t.app.Hooks.OnRemovedTable != nil {
t.app.Hooks.OnRemovedTable(cast.String(record["name"]))
}
} else if t.Name == "_Field" {
_ = t.reconstructAndSyncSchema()
if record != nil && t.app.Hooks.OnRemovedField != nil {
t.app.Hooks.OnRemovedField(cast.String(record["tableId"]), cast.String(record["name"]))
}
} else {
return res.Error
if t.app.Hooks.OnRemovedRows != nil {
t.app.Hooks.OnRemovedRows(t.Name, []string{id})
}
}
}
if len(removedIDs) > 0 && t.app.hooks.OnRemovedRows != nil {
t.app.hooks.OnRemovedRows(removedIDs, tableSchema, fields)
}
return nil
return res.Error
}
// List retrieves multiple records.
func (t *Table) List(where any, args ...any) ([]map[string]any, error) {
req := QueryRequest{}
if t.isMeta {
return nil, fmt.Errorf("list not supported on schema tables")
}
query := fmt.Sprintf("SELECT * FROM `%s` ", t.Name)
whereStr := ""
if where != nil {
switch v := where.(type) {
case string:
req.Where = v
req.Args = args
whereStr = v
case map[string]any:
req.Where, req.Args = buildWhere(v)
whereStr, args = buildWhere(v)
}
}
return t.Query(req)
}
// Query performs a structured query on the current table.
func (t *Table) Query(req QueryRequest) ([]map[string]any, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
// Apply constraint to where clause
if t.constraint != nil {
if req.Where != "" {
// If it's a string, this is complex to merge safely without parsing SQL.
// Let's rely on appendAuthAndConstraint passing the constraint.
}
}
sql, args, err := t.app.buildQuery(t, req)
var err error
whereStr, args, err = t.appendAuthAndConstraint(whereStr, args)
if err != nil {
return nil, err
}
res := t.db.Query(sql, args...)
if whereStr != "" {
query += " WHERE " + whereStr
}
res := t.db.Query(query, args...)
if res.Error != nil {
return nil, res.Error
}
return res.MapResults(), nil
}
// Count returns the number of records.
func (t *Table) Count(where any, args ...any) (int64, error) {
if t.userID == "" {
return 0, fmt.Errorf("authentication required")
}
query := fmt.Sprintf("SELECT COUNT(*) FROM `%s` ", t.Name)
whereStr := ""
if where != nil {
@ -623,10 +428,6 @@ func (t *Table) Count(where any, args ...any) (int64, error) {
// CountBy returns counts grouped by a field.
func (t *Table) CountBy(field string) (map[any]int64, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
query := fmt.Sprintf("SELECT `%s`, COUNT(*) as cnt FROM `%s` ", field, t.Name)
whereStr, args, err := t.appendAuthAndConstraint("", nil)
@ -654,10 +455,6 @@ func (t *Table) CountBy(field string) (map[any]int64, error) {
// Fields returns field metadata.
func (t *Table) Fields() ([]FieldSchema, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
tid := ""
tableRecord := GlobalCache.GetTable(t.Name)
if tableRecord != nil {

305
tableDB_test.go
View File

@ -14,27 +14,20 @@ func setupDB(t *testing.T, dbFile string) *TableDB {
_ = os.Remove(dbFile)
logger := log.DefaultLogger
logger.SetLevel(log.ERROR)
dbInst := GetDB("sqlite://"+dbFile, logger, "")
// Bootstrap system tables
err := dbInst.Bootstrap()
if err != nil {
panic(fmt.Sprintf("Bootstrap failed in setupDB: %v", err))
}
return dbInst.Auth(SystemUserID)
dbInst := GetDB("sqlite://"+dbFile, logger)
return dbInst
}
func TestBootstrapAndSync(t *testing.T) {
dbFile := "test_bootstrap.db"
_ = os.Remove(dbFile)
unauth := GetDB("sqlite://"+dbFile, log.DefaultLogger, "")
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
// 1. Bootstrap: sync system DSL
err := unauth.Bootstrap()
// 1. Bootstrap: Sync empty DSL, should still create system tables
err := dbInst.SyncSchema("")
if err != nil {
t.Fatalf("Bootstrap failed: %v", err)
}
dbInst := unauth.Auth(SystemUserID)
// Verify system tables in cache
if GlobalCache.GetTable("_Table") == nil {
@ -44,21 +37,25 @@ func TestBootstrapAndSync(t *testing.T) {
t.Errorf("_Field not found in cache after bootstrap")
}
// 2. Create business table via API
err = dbInst.Table("users").SetField(
FieldSchema{Name: "name", Type: "v64", IsIndex: true},
FieldSchema{Name: "age", Type: "i"},
)
// 2. Sync business schema
businessDSL := `
== Business ==
users SD
id c10 PK
name v64 U
age i
`
err = dbInst.SyncSchema(businessDSL)
if err != nil {
t.Fatalf("Create table failed: %v", err)
t.Fatalf("Business sync failed: %v", err)
}
// Verify business table
if GlobalCache.GetTable("users") == nil {
t.Errorf("users table not found in cache after sync")
}
fields, _ := dbInst.Table("users").Fields()
if len(fields) < 3 { // id, name, age
fields := GlobalCache.GetValidFields("users")
if len(fields) < 3 { // id, name, age (and maybe others from SD)
t.Errorf("Expected at least 3 fields for users, got %v", fields)
}
}
@ -68,18 +65,41 @@ func TestAPIDrivenSchema(t *testing.T) {
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.SyncSchema("")
app := dbInst.Auth("admin")
// 1. Create table and add fields via SetField API
err := app.Table("tasks").SetField(
FieldSchema{Name: "title", Type: "v100", Memo: "Task title"},
FieldSchema{Name: "done", Type: "b", IsIndex: true},
)
// 1. Create table via _Table API
err := app.Table("_Table").Set(map[string]any{
"name": "tasks",
"memo": "Task list",
})
if err != nil {
t.Fatalf("Failed to create table/fields via API: %v", err)
t.Fatalf("Failed to create table via API: %v", err)
}
// 2. Verify table works
// Get tid
tbl, _ := app.Table("_Table").List(map[string]any{"name": "tasks"})
if len(tbl) == 0 {
t.Fatalf("Table tasks not found in _Table")
}
tid := cast.String(tbl[0]["id"])
// 2. Add fields via _Field API
_ = app.Table("_Field").Set(map[string]any{
"tableId": tid,
"name": "title",
"type": "v100",
"memo": "Task title",
})
_ = app.Table("_Field").Set(map[string]any{
"tableId": tid,
"name": "done",
"type": "b",
"isIndex": 1,
})
// 3. Verify table works
taskTable := app.Table("tasks")
err = taskTable.Set(map[string]any{
"title": "Fix tests",
@ -93,13 +113,6 @@ func TestAPIDrivenSchema(t *testing.T) {
if len(res) != 1 || res[0]["title"] != "Fix tests" {
t.Errorf("Query from tasks failed: %v", res)
}
// Verify ID is automatically generated
if idVal := cast.String(res[0]["id"]); len(idVal) == 0 {
t.Errorf("ID was not automatically generated")
} else {
fmt.Printf("Generated ID: %s\n", idVal)
}
}
func TestDataOperations(t *testing.T) {
@ -107,28 +120,20 @@ func TestDataOperations(t *testing.T) {
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.Table("products").SetField(
FieldSchema{Name: "name", Type: "v64", IsIndex: true},
FieldSchema{Name: "price", Type: "i"},
)
_ = dbInst.SyncSchema(`
== Test ==
products SD
id c10 PK
name v64 U
price i
`)
app := dbInst.Auth("admin")
table := app.Table("products")
// Create (Batch)
err := table.Set(
map[string]any{"name": "Laptop", "price": 1000},
map[string]any{"id": "p2", "name": "Mouse", "price": 50},
)
if err != nil {
t.Fatalf("Batch set failed: %v", err)
}
laptop, _ := table.List(map[string]any{"name": "Laptop"})
if len(laptop) == 0 {
t.Fatalf("Laptop not created")
}
laptopID := cast.String(laptop[0]["id"])
// Create
_ = table.Set(map[string]any{"name": "Laptop", "price": 1000})
_ = table.Set(map[string]any{"id": "p2", "name": "Mouse", "price": 50})
// Read
p1, _ := table.Get("p2")
@ -155,20 +160,15 @@ func TestDataOperations(t *testing.T) {
t.Errorf("Count failed: %d", cnt)
}
// Delete (Batch)
err = table.Remove("p2", laptopID)
if err != nil {
t.Fatalf("Batch remove failed: %v", err)
}
// Delete (Shadow Delete)
_ = table.Remove("p2")
p1_removed, _ := table.Get("p2")
if p1_removed != nil {
t.Errorf("Expected p2 to be removed")
}
// Verify shadow delete (actual table has _deleted suffix)
raw, _ := dbInst.GetRawDB()
res := raw.Query("SELECT name FROM sqlite_master WHERE name='products_deleted'")
res := dbInst.Base().Query("SELECT name FROM sqlite_master WHERE name='products_deleted'")
if res.MapOnR1()["name"] == nil {
t.Errorf("Shadow delete table products_deleted not found")
}
@ -179,11 +179,13 @@ func TestPermissionsAndAuth(t *testing.T) {
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.Table("secrets").SetField(
FieldSchema{Name: "content", Type: "t"},
FieldSchema{Name: "creator", Type: "v64"},
)
_ = dbInst.SetTable(TableSchema{Name: "secrets", EnableRLS: true})
_ = dbInst.SyncSchema(`
== Secret ==
secrets SD
id c10 PK
content t
creator v64
`)
user1 := dbInst.Auth("user1")
user2 := dbInst.Auth("user2")
@ -191,27 +193,28 @@ func TestPermissionsAndAuth(t *testing.T) {
// user1 creates a secret
_ = user1.Table("secrets").Set(map[string]any{"id": "s1", "content": "user1-secret"})
// user2 tries to get it
// user2 tries to get it -> should fail or return nil depending on implementation
// Current implementation: checkAuth returns error if no permission
_, err := user2.Table("secrets").Get("s1")
if err == nil {
t.Errorf("user2 should not have permission to get user1 secret")
}
// system gets it
sys := dbInst.Auth(SystemUserID)
// admin gets it -> _system or admin should have permission?
// Implementation says t.userID == "_system" bypasses.
sys := dbInst.Auth("_system")
s, err := sys.Table("secrets").Get("s1")
if err != nil || s == nil {
t.Errorf("system should have permission: %v", err)
}
// Test _Policy: Grant access to specific ID via condition
// Test _Policy
// allow user2 to see s1
err = sys.Table("_Policy").Set(map[string]any{
"userID": "user2",
"type": "table",
"targets": []string{"secrets"},
"action": "read",
"condition": "id = ?",
"conditionArgs": []any{"s1"},
"subject": "user2",
"action": "read",
"resource": "s1",
"effect": "allow",
})
if err != nil {
t.Fatalf("Failed to set policy: %v", err)
@ -224,100 +227,6 @@ func TestPermissionsAndAuth(t *testing.T) {
}
}
func TestInheritance(t *testing.T) {
dbFile := "test_inherit.db"
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.Table("docs").SetField(
FieldSchema{Name: "title", Type: "v64"},
FieldSchema{Name: "creator", Type: "v64"},
)
_ = dbInst.SetTable(TableSchema{Name: "docs", EnableRLS: true})
sys := dbInst.Auth(SystemUserID)
_ = sys.Table("docs").Set(map[string]any{"id": "d1", "title": "secret doc", "creator": "boss"})
// manager inherits boss
_ = sys.Table("_Policy").Set(map[string]any{
"userID": "manager",
"type": "inherit",
"targets": []string{"boss"},
})
// boss can read docs
_ = sys.Table("_Policy").Set(map[string]any{
"userID": "boss",
"type": "table",
"targets": []string{"docs"},
"action": "read",
})
manager := dbInst.Auth("manager")
d, err := manager.Table("docs").Get("d1")
if err != nil || d == nil {
t.Errorf("manager should inherit boss's permission to read docs: %v", err)
}
}
func TestPolicyInterfaces(t *testing.T) {
dbFile := "test_policy_api.db"
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
sys := dbInst.Auth(SystemUserID)
_ = sys.Table("data").SetField(FieldSchema{Name: "val", Type: "v10"})
_ = sys.SetTable(TableSchema{Name: "data", EnableRLS: true})
// 1. Normal user tries to set inherit policy (should fail)
user1 := dbInst.Auth("user1")
err := user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "inherit",
Targets: []string{"user1"},
})
if err == nil {
t.Errorf("Normal user should not be able to set inherit policy")
}
// 2. Normal user tries to set table policy for table they don't have full access to (should fail)
err = user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "table",
Targets: []string{"data"},
Action: "read",
})
if err == nil {
t.Errorf("User without full access to table should not be able to set policy for it")
}
// 3. System grants user1 full access to 'data'
_ = sys.SetPolicy(PolicySchema{
UserID: "user1",
Type: "table",
Targets: []string{"data"},
Action: "full",
})
// 4. Now user1 sets policy for user2 on 'data' (should succeed)
err = user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "table",
Targets: []string{"data"},
Action: "read",
})
if err != nil {
t.Errorf("User with full access should be able to set policy: %v", err)
}
// 5. List policies
pols, err := user1.ListPolicy(map[string]any{"userID": "user2"})
if err != nil || len(pols) == 0 || pols[0].UserID != "user2" {
t.Errorf("ListPolicy failed: %v, %v", err, pols)
}
}
func TestHooks(t *testing.T) {
dbFile := "test_hooks.db"
dbInst := setupDB(t, dbFile)
@ -326,21 +235,27 @@ func TestHooks(t *testing.T) {
var tableCreated string
var rowsUpdated int
dbInst.hooks.OnCreatedTable = func(table *TableSchema) {
tableCreated = table.Name
dbInst.Hooks.OnCreatedTable = func(tableName string, record map[string]any) {
tableCreated = tableName
}
dbInst.hooks.OnUpdatedRows = func(rows []map[string]any, table *TableSchema, fields []FieldSchema) {
rowsUpdated += len(rows)
dbInst.Hooks.OnUpdatedRows = func(tableName string, count int) {
rowsUpdated += count
}
_ = dbInst.SyncSchema("")
app := dbInst.Auth("admin")
// Trigger OnCreatedTable
_ = app.Table("hook_test").SetField(FieldSchema{Name: "val", Type: "v10"})
_ = app.Table("_Table").Set(map[string]any{"name": "hook_test"})
if tableCreated != "hook_test" {
t.Errorf("OnCreatedTable hook failed, got %s", tableCreated)
}
// Add a field to ensure table exists
tbl, _ := app.Table("_Table").List(map[string]any{"name": "hook_test"})
tid := cast.String(tbl[0]["id"])
_ = app.Table("_Field").Set(map[string]any{"tableId": tid, "name": "val", "type": "v10"})
// Trigger OnUpdatedRows
_ = app.Table("hook_test").Set(map[string]any{"id": "1", "val": "a"})
if rowsUpdated != 1 {
@ -353,26 +268,46 @@ func TestQueryValidationAndInjection(t *testing.T) {
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.Table("safe_table").SetField(FieldSchema{Name: "name", Type: "v64"})
app := dbInst.Auth("user1")
_ = dbInst.SyncSchema(`
== Test ==
safe_table SD
id c10 PK
name v64
`)
// Valid query
req := QueryRequest{
Select: []string{"id", "name"},
Table: "safe_table",
Select: []string{"id", "name"},
OrderBy: "name DESC",
}
_, err := app.Table("safe_table").Query(req)
_, _, err := dbInst.BuildQuery(req)
if err != nil {
t.Errorf("Valid query failed: %v", err)
}
// Invalid table
req.Table = "invalid_table"
_, _, err = dbInst.BuildQuery(req)
if err == nil {
t.Errorf("Should fail for invalid table")
}
// SQL Injection in Select
req.Table = "safe_table"
req.Select = []string{"name` FROM safe_table; --"}
_, err = app.Table("safe_table").Query(req)
_, _, err = dbInst.BuildQuery(req)
if err == nil {
t.Errorf("Should fail for injection in Select")
}
// SQL Injection in OrderBy
req.Select = []string{"name"}
req.OrderBy = "name; DROP TABLE safe_table"
_, _, err = dbInst.BuildQuery(req)
if err == nil {
t.Errorf("Should fail for injection in OrderBy")
}
}
func BenchmarkTableSet(b *testing.B) {
@ -381,14 +316,10 @@ func BenchmarkTableSet(b *testing.B) {
dbFile := fmt.Sprintf("bench_ops_%d.db", time.Now().UnixNano())
defer os.Remove(dbFile)
unauth := GetDB("sqlite://"+dbFile, logger, "")
_ = unauth.Bootstrap()
app := unauth.Auth(SystemUserID)
_ = app.Table("bench_ops").SetField(
FieldSchema{Name: "name", Type: "v50", IsIndex: true},
FieldSchema{Name: "val", Type: "i"},
)
dbInst := GetDB("sqlite://"+dbFile, logger)
_ = dbInst.SyncSchema("bench_ops SD\n id c10 PK\n name v50 U\n val i")
app := dbInst.Auth("admin")
table := app.Table("bench_ops")
b.ResetTimer()

45
types.go
View File

@ -1,48 +1,15 @@
package tableDB
type TableSchema struct {
ID string
Name string
Memo string
EnableRLS bool
Settings map[string]any
IsSecret bool // 是否敏感表（不被索引）
CreateTime int64
Creator string
UpdateTime int64
Updater string
}
type FlatPolicy struct {
Condition string
ConditionArgs []any
}
type PolicySchema struct {
ID string
UserID string
Type string // inherit, table
Targets []string
Action string // read, write, full
Condition string
ConditionArgs []any
CreateTime int64
Creator string
}
type FieldSchema struct {
ID string
TableID string
Name string
Type string
IsIndex bool
Memo string
Settings map[string]any
CreateTime int64
Creator string
TableID string
Name string
Type string
Index string
Comment string
}
type QueryRequest struct {
Table string
Select []string
Joins []JoinConfig
Where string