Compare commits

..

3 Commits
v1.1.0 ... main

10 changed files with 1315 additions and 607 deletions

View File

@ -1,5 +1,82 @@
# CHANGELOG # CHANGELOG
## v1.1.8 (2026-05-16)
- **文档增强**:
- **Hook 事件指南**: 在 `README.md` 中新增了 Hook 事件的详细声明与用法说明,涵盖从表结构变更到数据 CRUD 的全生命周期钩子。
## v1.1.7 (2026-05-15)
- **底层架构重构与标准字段**:
- **强制标准字段**: 所有数据表自动携带并维护 5 个标准字段(`id`, `creator`, `createTime`, `updater`, `updateTime`),极大地规范了业务记录生命周期。
- **默认倒序排列**: `List``Query` 现默认按 `createTime DESC` 返回数据,以符合现代业务开发直觉。
- **严格的 SELECT 安全**: `buildQuery` 彻底摒弃了 `SELECT *`,基于内存缓存中的合法字段列表动态拼接 SQL彻底杜绝已删除残余字段的泄露。
- **Hook 系统重构**:
- **增强上下文注入**: 全面重构了 `Hooks` 接口签名,现在 `OnUpdatedRows``OnRemovedRows` 等事件会完整返回 `rows/ids` 以及关联的 `TableSchema``FieldSchema`
- **RAG 友好型设计**: 增强后的 Hook 系统能够让外部模块(如 RAG 索引器)在零额外查询的情况下,直接获取所需的所有元数据与数据正文。
- **基础设施对齐与 ID 自动化**:
- **移除手动 ID 生成**: 去除了 `tableDB` 内部的 `record["id"] = id.MakeID(10)` 逻辑,完全下推至 `go/db` 基础设施处理。
- **自动随机 ID 识别**: 配合底层 `db` 升级,现在只要字段名为 `id`、类型为 `char(8~16)` 且在插入时为空,系统将自动产生全局随机 ID。
- **分布式 ID 支持**: `GetDB` 增加 `redis` 参数,用于配置底层 `db` 的 Redis 实例,确保在分布式环境下生成的 ID 具备全局唯一性。
- **架构重构稳定性修复**:
- **修复 Schema 重建 bug**: 修正了 `reconstructAndSyncSchema` 在生成 DSL 时丢失 `PK` (Primary Key) 标记的问题,确保动态创建的表始终具备正确的索引结构。
- **对齐 GetDB 签名**: 同步更新所有测试用例与示例文档,对齐最新的多参数初始化模型。
## v1.1.6 (2026-05-15)
- **零开销 ABAC 鉴权引擎**:
- **内存展平技术**: 引入 `SchemaCache` 策略展平算法,支持无限深度的用户权限继承(如 `Manager -> Boss`),并将复杂权限图谱在内存中转换为 O(1) 的平滑映射,彻底消除运行时递归与 N+1 数据库查询。
- **1-RTT 动态 SQL 注入**: 重构 `checkAuth``appendAuthAndConstraint`将行级过滤条件RLS直接下推至 SQL WHERE 子句中。无论是 `Get`, `List` 还是 `Remove`,所有权限校验均与业务查询合并执行,确保极致性能。
- **Opt-in 行级安全**: `_Table` 增加 `enableRLS` 开关。仅在显式开启的表上执行权限拦截,避免普通表(如字典表、日志表)产生不必要的计算开销。
- **全新策略管理 API**:
- 新增 `SetPolicy``ListPolicy` 接口,支持标准化的 ABAC 策略配置SQL 条件、参数绑定)。
- **权限分级管理**: 实现了自主授权逻辑。普通用户仅在拥有目标表 `full` 权限时方可为他人分配该表权限;`inherit` 类高阶策略仅限 `SystemUserID` 操作。
- **极致性能与校验**:
- **O(1) 字段验证**: 增加 `ValidFieldsMap` 缓存,消除 `buildQuery` 过程中高频的 Map 构建开销。
- **防死锁与容错**: 权限继承逻辑内置环状检测,自动忽略 `A -> B -> A` 式的逻辑死循环。
- **文档与工程化**:
- 全面重写 `README.md`,以 AI 驱动开发为中心,提供了清晰的 Schema 与 Policy 使用示例。
- 同步更新 `.profile/README.md`,对齐基础设施版本索引。
## v1.1.5 (2026-05-14)
- **强制身份校验模型**: 重构 `GetDB` 返回类型为 `*TableDBUnauthorized`,在编译期强制要求必须先调用 `Auth` 后才能执行后续数据表操作。
- **自定义元数据扩展**: 在系统表 `_Table``_Field` 中增加了 `settings` (JSON) 字段,业务方可通过 `TableSchema``FieldSchema` 存储自定义的表级或字段级配置。
- **Schema API 正规化**: 引入了完整的 `TableSchema` 结构体,并将 `SetTable``RemoveTable` API 上提至 `TableDB` 层级。彻底废弃并移除了遗留的 `Table("xxx:Schema")` 魔法后缀调用方式。
- **底层行为一致性**: `SetTable`, `RemoveTable`, `SetField`, `RemoveField` 现已彻底改写为底层封装调用 `Table("_Table")``Table("_Field")``Set/Remove` 方法,确保了系统自动更新的时间戳(`createTime`)、操作人(`creator`)及各种 Hook 生命周期事件能够被 100% 一致地触发。
## v1.1.4 (2026-05-14)
- **API 语义优化**:
- `Query` 方法从 `TableDB` 移动到 `Table` 对象,更符合面向对象调用习惯;`QueryRequest` 移除 `Table` 字段。
- 移除 `App` 类型别名,统一使用 `TableDB`
- `SyncSchema` 改为私有方法 `syncSchema`,外部调用统一通过 `SetTable``SetField` 进行结构维护。
- **结构维护增强**:
- 新增 `SetTable(memo)` 用于设置表元数据。
- `SetField` 参数从 `any` 改为强类型的 `FieldSchema` 切片,并自动忽略 `CreateTime` 等系统字段,增强类型安全性。
- `SetField` 在创建新表时会自动补全 `id` 字段的元数据,确保查询验证一致性。
- 新增 `Tables()` 方法,支持列出当前用户有权访问的所有数据表。
- **安全性完善**:
- 强化了所有公共 API 的 Auth 检查,确保未授权实例无法执行任何操作。
- 内部 `buildQuery` 现在需要明确传入表名,增强了多表操作时的上下文隔离。
- **测试回归**: 完全重构了测试用例,所有业务表创建均通过 `SetField` 接口完成,验证了公共 API 的闭环能力。
## v1.1.3 (2026-05-14)
- **API 统一与简化**: 引入 `SetField``RemoveField` 接口,支持通过统一的 Table API 动态管理表结构,实现了“操作表结构像操作数据一样简单”。
- **批量操作支持**: `Set`, `Remove`, `SetField`, `RemoveField` 全面支持变长参数Variadic arguments显著提升了批量数据处理效率。
- **安全性增强**:
- `GetDB` 返回实例默认未授权,强制调用 `Auth` 进行身份绑定。
- `GetRawDB` (原 `Base`) 现已增加权限校验,仅限 `_system` 身份访问。
- `Query` (App 级) 现已自动集成行级权限过滤。
- **性能与稳定性**: 优化了元数据同步流程,合并批量操作后的 Schema 重建;修复了 `Cache.Load` 在并发调用时的死锁隐患。
- **文档完善**: 重新编写了 README.md完全抹除了实现细节侧重于纯粹的 API 指南与最佳实践。
## v1.1.2 (2026-05-14)
- **文档重构**: 重新编写 README.md以 AI 为中心,侧重于 API 功能说明与使用指南,移除了无关的实现细节。
- **命名规范对齐**: 消灭了所有 Struct 中的 `json` tag全面对齐驼峰命名规范利用 `cast` 库自动处理大小写转换。
- **API 完整性**: 基于 `getExports` 工具补全了 API 文档,涵盖了常量、全局变量及完整的 Table 操作接口。
## v1.1.1 (2026-05-14)
- **基础设施对齐**: 深度对接 `@go/db` 的自动 ID 生成器 (`NextID`) 与版本管理特性。
- **元数据模型对齐**: 重构 `FieldSchema` 结构体,对齐 `_Field` 表物理结构,并统一使用 `cast.Convert` 进行 JSON 兼容的类型转换。
- **查询安全性强化**: 增强 `BuildQuery` 的标识符验证与自动转义Backticks防止潜在的 SQL 注入风险。
- **Schema 同步健壮性**: 优化 `reconstructAndSyncSchema``SyncSchema` 的元数据采集逻辑,支持更复杂的 DSL 结构。
## v1.1.0 ## v1.1.0
- **内置系统架构 (SystemSchema)**: `_Table`, `_Field`, `_Policy` 等核心元数据表现已内置,`SyncSchema` 会在启动时自动补齐,无需在业务 DSL 中显式定义。 - **内置系统架构 (SystemSchema)**: `_Table`, `_Field`, `_Policy` 等核心元数据表现已内置,`SyncSchema` 会在启动时自动补齐,无需在业务 DSL 中显式定义。
- **自动化 Schema 修复**: 修复了通过 API 创建表时缺少 `id` 字段导致的同步失败问题;`reconstructAndSyncSchema` 现在会自动为所有表补齐 `id c10 PK` 主键。 - **自动化 Schema 修复**: 修复了通过 API 创建表时缺少 `id` 字段导致的同步失败问题;`reconstructAndSyncSchema` 现在会自动为所有表补齐 `id c10 PK` 主键。

174
README.md
View File

@ -1,68 +1,144 @@
# apigo.cc/go/tableDB # go/tableDB v1.1.8
`tableDB`建立在 `apigo.cc/go/db` 基础上的更高层抽象工具库。提供对于动态表、数据权限隔离和 Hook 的生命周期追踪管理 `tableDB`基于 `@go/db` 的高级数据库抽象引擎,支持动态 Schema 管理、身份驱动的行级隔离 (RLS) 以及 ABAC 权限策略
## 使用指南 ## 🎯 核心设计理念
- **动态 Schema**: 像操作数据一样操作表和字段,无需手动 SQL 迁移。
- **内置标准字段**: 所有数据表强制且自动携带 5 个标准字段:`id` (全局唯一)、`creator``createTime``updater``updateTime`。系统会自动在插入和更新时维护它们。
- **身份驱动**: 所有操作强制绑定 `userID`,系统基于此自动拦截并填充创建者、更新者。
- **ABAC 权限模型**: 支持 SQL 条件过滤、属性权限及关系继承,替代传统的 Role 体系。
- **极速性能**: 通过内存展平策略实现零 RTT 鉴权,支持 O(1) 字段验证。
### 1. 初始化与 Auth ## 📦 安装
```bash
go get apigo.cc/go/tableDB
```
## 🛠 API 指南
### 1. 初始化与授权
```go ```go
package main // 获取未授权实例 (无法直接操作数据),可传入 redis 地址用于分布式 ID 生成
unauth := tableDB.GetDB("sqlite://data.db", logger, "")
import ( // 必须绑定身份后才能使用
"apigo.cc/go/tableDB" app := unauth.Auth("user_123")
"apigo.cc/go/log" sys := unauth.Auth(tableDB.SystemUserID) // 超管权限,绕过所有 RLS
```
### 2. 动态架构管理 (Schema)
```go
// 1. 设置表元数据
sys.SetTable(tableDB.TableSchema{
Name: "orders",
EnableRLS: true, // 开启行级安全拦截
Memo: "业务订单表",
})
// 2. 批量维护字段 (支持动态迁移)
app.Table("orders").SetField(
tableDB.FieldSchema{Name: "title", Type: "v128", IsIndex: true},
tableDB.FieldSchema{Name: "amount", Type: "i"},
tableDB.FieldSchema{Name: "status", Type: "i", IsIndex: true},
) )
func main() { // 3. 查看有权访问的表
app := tableDB.GetDB("sqlite://local.db", log.DefaultLogger) tables, _ := app.Tables()
```
// 在需要忽略权限的系统级后台执行时: ### 3. 数据操作 (CRUD)
systemApp := app.Auth(tableDB.SystemUserID) ```go
table := app.Table("orders")
// 在特定的业务上下文执行时,将强制携带 creator 约束 // 保存数据 (自动填充 creator)
userApp := app.Auth("user_12345") table.Set(map[string]any{"title": "测试订单", "amount": 100})
// 获取单条 (自动进行权限校验)
order, err := table.Get("id_xxx")
// 列表查询 (自动注入过滤条件: creator = userID OR policy_conditions)
list, _ := table.List(map[string]any{"amount >": 50})
// 结构化查询 (支持 Join 和 Select 验证)
res, _ := table.Query(tableDB.QueryRequest{
Select: []string{"id", "title"},
Where: "status = ?",
Args: []any{1},
})
```
### 4. 策略与权限管理 (ABAC)
策略允许用户访问非自己创建的数据。
```go
// 1. 授权策略: 允许 user_A 读取订单,但仅限金额小于 1000 的
sys.SetPolicy(tableDB.PolicySchema{
UserID: "user_A",
Type: "table",
Targets: []string{"orders"},
Action: "read",
Condition: "amount < ?",
ConditionArgs: []any{1000},
})
// 2. 继承策略: manager 继承 clerk 的所有权限
sys.SetPolicy(tableDB.PolicySchema{
UserID: "manager",
Type: "inherit",
Targets: []string{"clerk"},
})
// 3. 自主授权: 对表拥有 'full' 权限的用户,可以给其他人授权该表
manager.SetPolicy(tableDB.PolicySchema{
UserID: "clerk",
Type: "table",
Targets: []string{"orders"},
Action: "read",
})
```
## 🛡 安全契约
1. **隐式隔离**: 只要表开启了 `EnableRLS` 且包含 `creator` 字段,查询将自动限制在用户创建的数据范围内。
2. **零开销拦截**: 鉴权逻辑已下推至 SQL WHERE 层,消除 N+1 查询隐患。
3. **特权保护**: 系统表 `_Table`, `_Field`, `_Policy` 禁止普通用户通过 Table API 直接修改,必须通过专属 API 进行管理。
4. **命名规范**: 全局强制驼峰命名,禁止使用 `json` tag。
## 🔗 Hook 事件 (Hooks)
`tableDB` 提供了一系列 Hook允许开发者在表结构变更或数据更新时注入自定义逻辑缓存同步、消息推送、业务校验等
### 1. 注册 Hook
Hook 需要在 `Auth` 之前,在 `TableDBUnauthorized` 实例上注册。
```go
unauth := tableDB.GetDB("sqlite://data.db", logger, "")
// 监听数据更新 (用于缓存同步)
unauth.Hooks.OnUpdatedRows = func(rows []map[string]any, table *tableDB.TableSchema, fields []tableDB.FieldSchema) {
for _, row := range rows {
fmt.Printf("表 %s 的数据已更新: %v\n", table.Name, row["id"])
}
} }
```
### 2. DSL 动态表声明与同步 // 数据写入前校验 (返回 error 将中断操作)
unauth.Hooks.OnUpdatingRow = func(row map[string]any, table *tableDB.TableSchema, fields []tableDB.FieldSchema) error {
`TableDB` 提供底层隐藏表机制自动补全如 `id``autoIndex`,可用于初始化表结构。 if table.Name == "orders" && cast.Int(row["amount"]) > 1000000 {
return errors.New("金额过大,需要人工审核")
```go
schema := `
== UserGroup ==
users SD
name v50 U
age i
`
_ = app.SyncSchema(schema)
```
### 3. 生命周期 Hooks 订阅
通过注册 Hooks我们可以在发生元数据变更或行更新时进行回调拦截
```go
app.Hooks.OnUpdatingRow = func(tableName string, row map[string]any) error {
if tableName == "users" {
row["modifiedAt"] = time.Now().Unix()
} }
return nil return nil
} }
app := unauth.Auth("user_123")
``` ```
### 4. 动态表增删改查 ### 2. 支持的事件列表
底层调用与 `go/db` 使用习惯对齐,不同点是操作会被权限或 Hook 钩子影响: | 事件名 | 触发时机 | 参数说明 |
| :--- | :--- | :--- |
| `OnCreatedTable` | 新表创建后 | `table`: 表架构信息 |
| `OnRemovedTable` | 表被删除后 | `table`: 表架构信息 |
| `OnUpdatedField` | 字段新增或修改后 | `table`: 表架构,`field`: 字段架构 |
| `OnRemovedField` | 字段删除后 | `table`: 表架构,`fieldName`: 字段名 |
| `OnUpdatingRow` | **数据写入前** | `row`: 待写入数据,`table`: 表架构。**返回 error 可拦截写入**。 |
| `OnUpdatedRows` | 数据写入/更新后 | `rows`: 已更新的数据列表(包含完整字段) |
| `OnRemovedRows` | 数据删除后 | `ids`: 已删除的主键列表 |
```go
// 设定数据,如 ID 缺省会自动生成 c10 ID
_ = userApp.Table("users").Set(map[string]any{"name": "Alice"})
// 读取
record, _ := userApp.Table("users").Get("u1xxxx")
// 统计
count, _ := userApp.Table("users").Count(map[string]any{"age >": 20})
```

27
TEST.md
View File

@ -1,20 +1,33 @@
# 测试报告 # 测试报告
## 性能测试
| 测试项 | 性能指标 (v1.1.6) | 备注 |
| :--- | :--- | :--- |
| **BenchmarkTableSet** | ~760,000 ns/op | 包含 Schema 验证、身份注入、1-RTT 写入及自动 ID 生成 |
| **鉴权开销 (RTT)** | 1-RTT | 核心鉴权逻辑已合并至主查询/写入 SQL无额外网络往返 |
| **内存展平查询** | $O(1)$ | 字段有效性验证与策略检索均通过内存缓存实现,无递归开销 |
## 覆盖场景 ## 覆盖场景
1. **结构化数据操作与隔离** 1. **零开销 ABAC 鉴权引擎**
- 验证了在开启 `enableRLS` 时,系统自动注入 `creator = ? OR (conditions)`
- 验证了 `checkAuth` 在单条操作Get/Remove时的 1-RTT 性能闭环。
2. **多级权限继承与环状检测**
- 测试通过了 `Manager -> Clerk` 的权限继承Manager 自动获得 Clerk 的所有表级访问权。
- 验证了循环继承A -> B -> A场景下系统依然能保持稳定运行不发生内存溢出或死锁。
3. **自主授权体系**
- 验证了普通用户(非 `_system`)只有在对目标表拥有 `full` 权限时,才能通过 `SetPolicy` 为第三方授权。
- 验证了 `inherit` 类型的策略被严格限制在 `SystemUserID` 权限下操作。
4. **结构化数据操作与隔离**
- 测试通过了包含和排除 system userID 时的增删改查。 - 测试通过了包含和排除 system userID 时的增删改查。
- `Auth` 方法通过携带不同 userID 后实现了 `creator` 的数据隔离限制。 - `Auth` 方法通过携带不同 userID 后实现了 `creator` 的数据隔离限制。
2. **生命周期 Hook 触发机制** 5. **生命周期 Hook 触发机制**
- `OnUpdatingRow` 触发并允许修改将要入库的行数据。 - `OnUpdatingRow` 触发并允许修改将要入库的行数据。
- `OnUpdatedRows` 在更新结束后被触发。 - `OnUpdatedRows` 在更新结束后被触发。
- `OnRemovedRows` 在成功移除后被触发返回对应的 ID 集合。 - `OnRemovedRows` 在成功移除后被触发返回对应的 ID 集合。
3. **SQL注入黑盒测试** 6. **SQL注入黑盒测试**
- Table 名称注入被验证并阻止。 - Table 名称注入被验证并阻止。
- Field 查询字段注入被验证并阻止。 - Field 查询字段注入被验证并阻止。
- Join On 条件验证被拦截。 - Join On 条件验证被拦截。
- OrderBy 查询的语句结构注入被拦截。 - OrderBy 查询的语句结构注入被拦截。
## 性能测试
* 待核心 DB 层修复后回归测试性能。

192
cache.go
View File

@ -9,19 +9,24 @@ import (
type SchemaCache struct { type SchemaCache struct {
Tables map[string]map[string]any // name -> table record Tables map[string]map[string]any // name -> table record
TableIDMap map[string]string // id -> name
Fields map[string][]FieldSchema // tableId -> fields Fields map[string][]FieldSchema // tableId -> fields
Policies []map[string]any ValidFieldsMap map[string]map[string]bool // tableName -> fieldName -> true
FlatPolicies map[string]map[string]map[string][]FlatPolicy // userID -> tableName -> action -> []FlatPolicy
lock sync.RWMutex lock sync.RWMutex
lastLoad time.Time lastLoad time.Time
} }
var GlobalCache = &SchemaCache{ var GlobalCache = &SchemaCache{
Tables: make(map[string]map[string]any), Tables: make(map[string]map[string]any),
TableIDMap: make(map[string]string),
Fields: make(map[string][]FieldSchema), Fields: make(map[string][]FieldSchema),
ValidFieldsMap: make(map[string]map[string]bool),
FlatPolicies: make(map[string]map[string]map[string][]FlatPolicy),
} }
func (c *SchemaCache) Load(tDB *TableDB) error { func (c *SchemaCache) Load(tDB *TableDB) error {
dbInst := tDB.Base() dbInst := tDB.base
// Check if _Table exists first // Check if _Table exists first
res := dbInst.Query("SELECT name FROM sqlite_master WHERE type='table' AND name='_Table'") res := dbInst.Query("SELECT name FROM sqlite_master WHERE type='table' AND name='_Table'")
if dbInst.Config.Type == "mysql" { if dbInst.Config.Type == "mysql" {
@ -31,45 +36,173 @@ func (c *SchemaCache) Load(tDB *TableDB) error {
return nil // System tables not yet created return nil // System tables not yet created
} }
c.lock.Lock() tablesRes := dbInst.Query("SELECT * FROM `_Table`")
defer c.lock.Unlock() if tablesRes.Error != nil {
return tablesRes.Error
// Load Tables
tables, err := tDB.Table("_Table").List(nil)
if err != nil {
return err
} }
tables := tablesRes.MapResults()
fieldsRes := dbInst.Query("SELECT * FROM `_Field`")
if fieldsRes.Error != nil {
return fieldsRes.Error
}
fields := fieldsRes.MapResults()
policiesRes := dbInst.Query("SELECT * FROM `_Policy`")
if policiesRes.Error != nil {
return policiesRes.Error
}
policies := policiesRes.MapResults()
newTables := make(map[string]map[string]any) newTables := make(map[string]map[string]any)
newTableIDMap := make(map[string]string)
for _, t := range tables { for _, t := range tables {
newTables[cast.String(t["name"])] = t name := cast.String(t["name"])
newTables[name] = t
newTableIDMap[cast.String(t["id"])] = name
} }
c.Tables = newTables
// Load Fields
fields, err := tDB.Table("_Field").List(nil)
if err != nil {
return err
}
newFields := make(map[string][]FieldSchema) newFields := make(map[string][]FieldSchema)
newValidFieldsMap := make(map[string]map[string]bool)
for _, f := range fields { for _, f := range fields {
var fs FieldSchema var fs FieldSchema
cast.Convert(&fs, f) cast.Convert(&fs, f)
// If fs.TableID is empty, try to manually get it just in case
if fs.TableID == "" {
fs.TableID = cast.String(f["tableId"])
}
tid := fs.TableID tid := fs.TableID
newFields[tid] = append(newFields[tid], fs) newFields[tid] = append(newFields[tid], fs)
tableName := newTableIDMap[tid]
if tableName != "" {
if newValidFieldsMap[tableName] == nil {
newValidFieldsMap[tableName] = make(map[string]bool)
} }
newValidFieldsMap[tableName][fs.Name] = true
}
}
// Flatten Policies
newFlatPolicies := make(map[string]map[string]map[string][]FlatPolicy)
type rawPolicy struct {
UserID string
Type string
Targets []string
Action string
Condition string
ConditionArgs []any
}
var rawPolicies []rawPolicy
cast.Convert(&rawPolicies, policies)
userToRaw := make(map[string][]rawPolicy)
for _, p := range rawPolicies {
userToRaw[p.UserID] = append(userToRaw[p.UserID], p)
}
var flatten func(userID string, visited map[string]bool) map[string]map[string][]FlatPolicy
flatten = func(userID string, visited map[string]bool) map[string]map[string][]FlatPolicy {
if visited[userID] {
return nil
}
visited[userID] = true
defer delete(visited, userID)
result := make(map[string]map[string][]FlatPolicy) // table -> action -> []FlatPolicy
addPolicy := func(table, action string, fp FlatPolicy) {
if result[table] == nil {
result[table] = make(map[string][]FlatPolicy)
}
result[table][action] = append(result[table][action], fp)
if action == "full" {
result[table]["read"] = append(result[table]["read"], fp)
result[table]["write"] = append(result[table]["write"], fp)
}
}
for _, p := range userToRaw[userID] {
if p.Type == "table" {
cond := p.Condition
if cond == "null" {
cond = ""
}
var condArgs []any
for _, arg := range p.ConditionArgs {
if arg != nil {
condArgs = append(condArgs, arg)
}
}
for _, targetTable := range p.Targets {
addPolicy(targetTable, p.Action, FlatPolicy{
Condition: cond,
ConditionArgs: condArgs,
})
}
} else if p.Type == "inherit" {
for _, parentID := range p.Targets {
parentFlat := flatten(parentID, visited)
for table, actions := range parentFlat {
for action, fps := range actions {
if result[table] == nil {
result[table] = make(map[string][]FlatPolicy)
}
result[table][action] = append(result[table][action], fps...)
}
}
}
}
}
return result
}
for userID := range userToRaw {
newFlatPolicies[userID] = flatten(userID, make(map[string]bool))
}
c.lock.Lock()
defer c.lock.Unlock()
c.Tables = newTables
c.TableIDMap = newTableIDMap
c.Fields = newFields c.Fields = newFields
c.ValidFieldsMap = newValidFieldsMap
// Load Policies c.FlatPolicies = newFlatPolicies
policies, err := tDB.Table("_Policy").List(nil)
if err != nil {
return err
}
c.Policies = policies
c.lastLoad = time.Now() c.lastLoad = time.Now()
return nil return nil
} }
func (c *SchemaCache) IsValidField(tableName, fieldName string) bool {
c.lock.RLock()
defer c.lock.RUnlock()
if fieldName == "id" || fieldName == "createTime" || fieldName == "creator" || fieldName == "updateTime" || fieldName == "updater" {
return true
}
if c.ValidFieldsMap[tableName] == nil {
return false
}
return c.ValidFieldsMap[tableName][fieldName]
}
func (c *SchemaCache) GetFlatPolicies(userID, tableName, action string) []FlatPolicy {
c.lock.RLock()
defer c.lock.RUnlock()
userPol := c.FlatPolicies[userID]
if userPol == nil {
return nil
}
tablePol := userPol[tableName]
if tablePol == nil {
return nil
}
return tablePol[action]
}
func (c *SchemaCache) GetTable(name string) map[string]any { func (c *SchemaCache) GetTable(name string) map[string]any {
c.lock.RLock() c.lock.RLock()
defer c.lock.RUnlock() defer c.lock.RUnlock()
@ -94,8 +227,19 @@ func (c *SchemaCache) GetValidFields(tableName string) []string {
tid := cast.String(table["id"]) tid := cast.String(table["id"])
fields := c.Fields[tid] fields := c.Fields[tid]
var names []string var names []string
standardFields := map[string]bool{
"id": true,
"createTime": true,
"creator": true,
"updateTime": true,
"updater": true,
}
for _, f := range fields { for _, f := range fields {
names = append(names, f.Name) names = append(names, f.Name)
delete(standardFields, f.Name)
}
for f := range standardFields {
names = append(names, f)
} }
return names return names
} }

323
db.go
View File

@ -3,7 +3,6 @@ package tableDB
import ( import (
"fmt" "fmt"
"strings" "strings"
"time"
"apigo.cc/go/cast" "apigo.cc/go/cast"
"apigo.cc/go/db" "apigo.cc/go/db"
@ -18,8 +17,13 @@ _Table SD // 核心表:存储所有表的元数据
id c10 PK id c10 PK
name v64 U // 表名 name v64 U // 表名
memo t // 备注 memo t // 备注
createTime bi // 创建时间 enableRLS b // 是否开启行级安全
settings o // 设置
isSecret b // 是否敏感表(不被索引)
createTime bi I // 创建时间
creator v64 // 创建者 creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
_Field SD // 核心表:存储所有字段的元数据 _Field SD // 核心表:存储所有字段的元数据
id c10 PK id c10 PK
@ -28,123 +32,221 @@ _Field SD // 核心表:存储所有字段的元数据
type v32 // 字段类型 type v32 // 字段类型
isIndex b // 是否索引 isIndex b // 是否索引
memo t // 备注 memo t // 备注
createTime bi // 创建时间 settings o // 设置
createTime bi I // 创建时间
creator v64 // 创建者 creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
_Policy SD // 核心表:访问策略 _Policy SD // 核心表:访问策略
subject v64 I // 主体 (UserID 或 Role) id c10 PK
action v32 I // 动作 userID c10 I // 策略拥有者
resource v128 I // 资源 (Table 或 Record ID) type v32 // 策略类型 (inherit, table)
effect v16 // allow 或 deny targets o // 作用目标数组 (inherit: userIDs, table: tableNames)
action v16 // 动作 (read, write, full)
condition v1024 // SQL WHERE 条件片段
conditionArgs o // 条件对应的参数数组
createTime bi I // 创建时间
creator v64 // 创建者 creator v64 // 创建者
updateTime bi I // 更新时间
updater v64 // 更新者
` `
type Hooks struct { type Hooks struct {
OnCreatedTable func(tableName string, record map[string]any) OnCreatedTable func(table *TableSchema)
OnRemovedTable func(tableName string) OnRemovedTable func(table *TableSchema)
OnUpdatedField func(tableId, fieldName string, record map[string]any) OnUpdatedField func(table *TableSchema, field *FieldSchema)
OnRemovedField func(tableId, fieldName string) OnRemovedField func(table *TableSchema, fieldName string)
OnUpdatingRow func(tableName string, row map[string]any) error OnUpdatingRow func(row map[string]any, table *TableSchema, fields []FieldSchema) error
OnUpdatedRows func(tableName string, count int) OnUpdatedRows func(rows []map[string]any, table *TableSchema, fields []FieldSchema)
OnRemovedRows func(tableName string, ids []string) OnRemovedRows func(ids []string, table *TableSchema, fields []FieldSchema)
}
type TableDBUnauthorized struct {
base *db.DB
Hooks *Hooks
} }
// TableDB wraps the base go/db implementation to provide high-level abstractions. // TableDB wraps the base go/db implementation to provide high-level abstractions.
type TableDB struct { type TableDB struct {
base *db.DB base *db.DB
userID string userID string
Hooks *Hooks hooks *Hooks
} }
type App = TableDB // GetDB retrieves a configured database instance. Must call Auth() before use.
func GetDB(name string, logger *log.Logger, redis string) *TableDBUnauthorized {
// GetDB retrieves a configured database instance.
func GetDB(name string, logger *log.Logger) *TableDB {
baseDB := db.GetDB(name, logger) baseDB := db.GetDB(name, logger)
return &TableDB{ if redis != "" {
baseDB.Config.Redis = redis
}
return &TableDBUnauthorized{
base: baseDB, base: baseDB,
userID: SystemUserID,
Hooks: &Hooks{}, Hooks: &Hooks{},
} }
} }
// Auth creates a new instance with the specified userID context. // Bootstrap initializes the system tables metadata required for tableDB to function.
func (d *TableDB) Auth(userID string) *App { func (d *TableDBUnauthorized) Bootstrap() error {
return &TableDB{ err := d.base.Sync(SystemSchema)
base: d.base,
userID: userID,
Hooks: d.Hooks,
}
}
// SyncSchema automatically applies the DSL schema to the underlying database.
func (d *TableDB) SyncSchema(schemaDSL string) error {
// 1. Sync to actual DB
// Underground rules (autoIndex, id normalization) are now handled internally by d.base.Sync
finalDSL := schemaDSL
if !strings.Contains(schemaDSL, "_Table") {
finalDSL = SystemSchema + "\n" + schemaDSL
}
err := d.base.Sync(finalDSL)
if err != nil { if err != nil {
return err return err
} }
// 2. Update _Table and _Field metadata sys := d.Auth(SystemUserID)
res := d.base.Query("SELECT name FROM sqlite_master WHERE type='table' AND name='_Table'") groups := db.ParseSchema(SystemSchema)
if d.base.Config.Type == "mysql" {
res = d.base.Query("SELECT TABLE_NAME name FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND TABLE_NAME='_Table'", d.base.Config.DB)
}
if res.Error == nil && res.MapOnR1()["name"] != nil {
groups := db.ParseSchema(finalDSL)
for _, group := range groups { for _, group := range groups {
for _, table := range group.Tables { for _, table := range group.Tables {
// Upsert _Table
tRecord := map[string]any{ tRecord := map[string]any{
"name": table.Name, "name": table.Name,
"memo": table.Comment, "memo": table.Comment,
"createTime": time.Now().UnixMilli(),
} }
existingTable, _ := d.Table("_Table").List(map[string]any{"name": table.Name}) res := sys.base.Query("SELECT id FROM `_Table` WHERE name = ?", table.Name)
var tid string var tid string
if len(existingTable) > 0 { rows := res.MapResults()
tid = cast.String(existingTable[0]["id"]) if len(rows) > 0 {
tid = cast.String(rows[0]["id"])
tRecord["id"] = tid tRecord["id"] = tid
} }
_ = d.Table("_Table").Set(tRecord) _ = sys.Table("_Table").Set(tRecord)
if tid == "" { if tid == "" {
newTable, _ := d.Table("_Table").List(map[string]any{"name": table.Name}) res = sys.base.Query("SELECT id FROM `_Table` WHERE name = ?", table.Name)
if len(newTable) > 0 { rows = res.MapResults()
tid = cast.String(newTable[0]["id"]) if len(rows) > 0 {
tid = cast.String(rows[0]["id"])
} }
} }
if tid != "" { if tid != "" {
// Update _Field
for _, field := range table.Fields { for _, field := range table.Fields {
fName := field.Name
if fName == "id" || fName == "createTime" || fName == "creator" || fName == "updateTime" || fName == "updater" {
continue
}
fRecord := map[string]any{ fRecord := map[string]any{
"tableId": tid, "tableId": tid,
"name": field.Name, "name": fName,
"type": field.Type, "type": field.Type,
"isIndex": cast.If(field.Index != "", 1, 0), "isIndex": cast.If(field.Index != "", 1, 0),
"memo": field.Comment, "memo": field.Comment,
"createTime": time.Now().UnixMilli(),
} }
existingField, _ := d.Table("_Field").List(map[string]any{"tableId": tid, "name": field.Name}) fRes := sys.base.Query("SELECT id FROM `_Field` WHERE tableId = ? AND name = ?", tid, fName)
if len(existingField) > 0 { fRows := fRes.MapResults()
fRecord["id"] = existingField[0]["id"] if len(fRows) > 0 {
} fRecord["id"] = fRows[0]["id"]
_ = d.Table("_Field").Set(fRecord)
} }
_ = sys.Table("_Field").Set(fRecord)
} }
} }
} }
} }
// 3. Reload cache return GlobalCache.Load(sys)
return GlobalCache.Load(d) }
// Auth creates a new instance with the specified userID context.
func (d *TableDBUnauthorized) Auth(userID string) *TableDB {
return &TableDB{
base: d.base,
userID: userID,
hooks: d.Hooks,
}
}
// Auth creates a new instance with the specified userID context from an existing authorized instance.
func (d *TableDB) Auth(userID string) *TableDB {
return &TableDB{
base: d.base,
userID: userID,
hooks: d.hooks,
}
}
// Tables returns a list of tables that the current user has access to.
func (d *TableDB) Tables() ([]TableSchema, error) {
res, err := d.Table("_Table").List(nil)
if err != nil {
return nil, err
}
var tables []TableSchema
cast.Convert(&tables, res)
return tables, nil
}
// SetTable updates or creates the table metadata.
func (d *TableDB) SetTable(schema TableSchema) error {
record := map[string]any{
"name": schema.Name,
"memo": schema.Memo,
"enableRLS": schema.EnableRLS,
"settings": schema.Settings,
}
if schema.ID != "" {
record["id"] = schema.ID
}
return d.Table("_Table").Set(record)
}
// RemoveTable deletes a table.
func (d *TableDB) RemoveTable(name string) error {
tableRec := GlobalCache.GetTable(name)
if tableRec == nil {
return fmt.Errorf("table %s not found", name)
}
return d.Table("_Table").Remove(cast.String(tableRec["id"]))
}
// SetPolicy updates or creates an access policy.
func (d *TableDB) SetPolicy(policy PolicySchema) error {
if d.userID != SystemUserID {
if policy.Type == "inherit" {
return fmt.Errorf("only system user can set inherit policy")
}
if policy.Type == "table" {
// Only users with permission to the table can set type=table policy
for _, targetTable := range policy.Targets {
// Check if current user has 'full' access to the target table
pols := GlobalCache.GetFlatPolicies(d.userID, targetTable, "full")
hasFullAccess := false
for _, p := range pols {
if p.Condition == "" {
hasFullAccess = true
break
}
}
if !hasFullAccess {
return fmt.Errorf("permission denied to set policy for table %s", targetTable)
}
}
}
}
record := map[string]any{
"userID": policy.UserID,
"type": policy.Type,
"targets": policy.Targets,
"action": policy.Action,
"condition": policy.Condition,
"conditionArgs": policy.ConditionArgs,
"creator": d.userID,
}
if policy.ID != "" {
record["id"] = policy.ID
}
return d.Auth(SystemUserID).Table("_Policy").Set(record)
}
// ListPolicy retrieves policies based on filter.
func (d *TableDB) ListPolicy(filter map[string]any) ([]PolicySchema, error) {
res, err := d.Table("_Policy").List(filter)
if err != nil {
return nil, err
}
var policies []PolicySchema
cast.Convert(&policies, res)
return policies, nil
} }
// Table returns an AI-friendly interface for multi-dimensional operations on a specific table. // Table returns an AI-friendly interface for multi-dimensional operations on a specific table.
@ -152,50 +254,37 @@ func (d *TableDB) Table(name string) *Table {
return NewTable(name, d) return NewTable(name, d)
} }
// Base returns the underlying apigo.cc/go/db.DB for raw queries if needed. // GetRawDB returns the underlying apigo.cc/go/db.DB. Only allowed for SystemUserID.
func (d *TableDB) Base() *db.DB { func (d *TableDB) GetRawDB() (*db.DB, error) {
return d.base if d.userID != SystemUserID {
return nil, fmt.Errorf("permission denied for GetRawDB")
}
return d.base, nil
} }
// Query performs a structured query. // buildQuery constructs a SQL query from a QueryRequest with strict identifier validation and auth filtering.
func (d *TableDB) Query(req QueryRequest) ([]map[string]any, error) { func (d *TableDB) buildQuery(t *Table, req QueryRequest) (string, []any, error) {
sql, args, err := d.BuildQuery(req) tableName := t.Name
if err != nil { if GlobalCache.GetTable(tableName) == nil {
return nil, err return "", nil, fmt.Errorf("invalid table: %s", tableName)
}
res := d.base.Query(sql, args...)
if res.Error != nil {
return nil, res.Error
}
return res.MapResults(), nil
} }
// BuildQuery constructs a SQL query from a QueryRequest with strict identifier validation. fieldList := req.Select
func (d *TableDB) BuildQuery(req QueryRequest) (string, []any, error) { if len(fieldList) == 0 {
if GlobalCache.GetTable(req.Table) == nil { fieldList = GlobalCache.GetValidFields(tableName)
return "", nil, fmt.Errorf("invalid table: %s", req.Table)
}
fields := "*"
if len(req.Select) > 0 {
validFields := GlobalCache.GetValidFields(req.Table)
fieldMap := make(map[string]bool)
for _, f := range validFields {
fieldMap[f] = true
} }
var validatedSelect []string var validatedSelect []string
for _, s := range req.Select { for _, s := range fieldList {
if !fieldMap[s] { if !GlobalCache.IsValidField(tableName, s) {
return "", nil, fmt.Errorf("invalid field %s in table %s", s, req.Table) return "", nil, fmt.Errorf("invalid field %s in table %s", s, tableName)
} }
validatedSelect = append(validatedSelect, "`"+s+"`") validatedSelect = append(validatedSelect, "`"+s+"`")
} }
fields = strings.Join(validatedSelect, ", ") fieldsStr := strings.Join(validatedSelect, ", ")
}
var sql strings.Builder var sql strings.Builder
fmt.Fprintf(&sql, "SELECT %s FROM `%s` ", fields, req.Table) fmt.Fprintf(&sql, "SELECT %s FROM `%s` ", fieldsStr, tableName)
for _, join := range req.Joins { for _, join := range req.Joins {
if GlobalCache.GetTable(join.Table) == nil { if GlobalCache.GetTable(join.Table) == nil {
@ -213,24 +302,29 @@ func (d *TableDB) BuildQuery(req QueryRequest) (string, []any, error) {
} }
args := req.Args args := req.Args
if req.Where != "" { whereStr := req.Where
sql.WriteString(" WHERE ")
sql.WriteString(req.Where) // Apply auth filtering for the main table
var err error
whereStr, args, err = t.appendAuthAndConstraint(whereStr, args)
if err != nil {
return "", nil, err
} }
if req.OrderBy != "" { if whereStr != "" {
parts := strings.Fields(req.OrderBy) sql.WriteString(" WHERE ")
sql.WriteString(whereStr)
}
orderBy := req.OrderBy
if orderBy == "" {
orderBy = "createTime DESC"
}
parts := strings.Fields(orderBy)
if len(parts) > 0 { if len(parts) > 0 {
fieldName := parts[0] fieldName := parts[0]
validFields := GlobalCache.GetValidFields(req.Table) if !GlobalCache.IsValidField(tableName, fieldName) {
found := false
for _, f := range validFields {
if f == fieldName {
found = true
break
}
}
if !found {
return "", nil, fmt.Errorf("invalid order by field: %s", fieldName) return "", nil, fmt.Errorf("invalid order by field: %s", fieldName)
} }
@ -245,7 +339,6 @@ func (d *TableDB) BuildQuery(req QueryRequest) (string, []any, error) {
} }
fmt.Fprintf(&sql, " ORDER BY `%s` %s", fieldName, direction) fmt.Fprintf(&sql, " ORDER BY `%s` %s", fieldName, direction)
} }
}
if req.Limit > 0 { if req.Limit > 0 {
fmt.Fprintf(&sql, " LIMIT %d", req.Limit) fmt.Fprintf(&sql, " LIMIT %d", req.Limit)
@ -257,7 +350,7 @@ func (d *TableDB) BuildQuery(req QueryRequest) (string, []any, error) {
return sql.String(), args, nil return sql.String(), args, nil
} }
// BuildWhere is a helper to convert a map of conditions into a SQL WHERE clause and args. // buildWhere is a helper to convert a map of conditions into a SQL WHERE clause and args.
func buildWhere(filter map[string]any) (string, []any) { func buildWhere(filter map[string]any) (string, []any) {
if len(filter) == 0 { if len(filter) == 0 {
return "", nil return "", nil

24
go.mod
View File

@ -3,21 +3,21 @@ module apigo.cc/go/tableDB
go 1.25.0 go 1.25.0
require ( require (
apigo.cc/go/cast v1.3.2 apigo.cc/go/cast v1.3.3
apigo.cc/go/db v1.3.1 apigo.cc/go/db v1.3.4
apigo.cc/go/id v1.3.0 apigo.cc/go/log v1.3.4
apigo.cc/go/log v1.3.2
) )
require ( require (
apigo.cc/go/config v1.3.0 // indirect apigo.cc/go/config v1.3.1 // indirect
apigo.cc/go/crypto v1.3.0 // indirect apigo.cc/go/crypto v1.3.1 // indirect
apigo.cc/go/encoding v1.3.0 // indirect apigo.cc/go/encoding v1.3.1 // indirect
apigo.cc/go/file v1.3.0 // indirect apigo.cc/go/file v1.3.2 // indirect
apigo.cc/go/rand v1.3.0 // indirect apigo.cc/go/id v1.3.1 // indirect
apigo.cc/go/redis v1.3.0 // indirect apigo.cc/go/rand v1.3.1 // indirect
apigo.cc/go/safe v1.3.0 // indirect apigo.cc/go/redis v1.3.2 // indirect
apigo.cc/go/shell v1.3.0 // indirect apigo.cc/go/safe v1.3.1 // indirect
apigo.cc/go/shell v1.3.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gomodule/redigo v2.0.0+incompatible // indirect github.com/gomodule/redigo v2.0.0+incompatible // indirect
github.com/google/uuid v1.6.0 // indirect github.com/google/uuid v1.6.0 // indirect

48
go.sum
View File

@ -1,27 +1,27 @@
apigo.cc/go/cast v1.3.2 h1:hh9MWDSwh3T/kQdCHjFpjDwHrh2A05Q4wt1AAWs8NBI= apigo.cc/go/cast v1.3.3 h1:aln5eDR5DZVWVzZ/y5SJh1gQNgWv2sT82I25NaO9g34=
apigo.cc/go/cast v1.3.2/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk= apigo.cc/go/cast v1.3.3/go.mod h1:lGlwImiOvHxG7buyMWhFzcdvQzmSaoKbmr7bcDfUpHk=
apigo.cc/go/config v1.3.0 h1:TwI3bv3D+BJrAnFx+o62HQo3FarY2Ge3SCGsKchFYGg= apigo.cc/go/config v1.3.1 h1:wZzUh4oL+fGD6SayVgX6prLPMsniM25etWFcEH8XzIE=
apigo.cc/go/config v1.3.0/go.mod h1:88lqKEBXlIExFKt1geLONVLYyM+QhRVpBe0ok3OEvjI= apigo.cc/go/config v1.3.1/go.mod h1:7KHz/1WmtBLM762Lln/TaXh2dmlMvJTLhnlk33zbS3U=
apigo.cc/go/crypto v1.3.0 h1:rGRrrb5O+4M50X5hVUmJQbXx3l87zzlcgzGtUvZrZL8= apigo.cc/go/crypto v1.3.1 h1:ulQ2zX9bUWirk0sEacx1Srsjs2Jow7HlZq7ED7msNcg=
apigo.cc/go/crypto v1.3.0/go.mod h1:uSCcmbcFoiltUPMQTSuqmU9nfKEH/lRs7nQ7aa3Z4Mc= apigo.cc/go/crypto v1.3.1/go.mod h1:SwHlBFDPddttWgFFtzsEMla8CM/rcFy9nvdsJjW4CIs=
apigo.cc/go/db v1.3.1 h1:X+iHMZ71OdWW8R91TFtE+aZr/CZPOtL7gIyt3tEkfyc= apigo.cc/go/db v1.3.4 h1:SDTK9+anU3vc30i5JT4y5NCkUByFxg+tX5Ju+Ye/jq4=
apigo.cc/go/db v1.3.1/go.mod h1:Gc6HEmncoYRaaRy5yPjTegwc2pSwjNOTnINBmbomhcs= apigo.cc/go/db v1.3.4/go.mod h1:/uELxvpf9i6cE/2fS/LprJl5JoIX3w/K9qKMy7Hdr98=
apigo.cc/go/encoding v1.3.0 h1:8jqNHoZBR8vOU/BGsLFebfp1Txa1UxDRpd7YwzIFLJs= apigo.cc/go/encoding v1.3.1 h1:y8O58KYAyulkThg1O2ji2BqjnFoSvk42sit9I3z+K7Y=
apigo.cc/go/encoding v1.3.0/go.mod h1:kT/uUJiuAOkZ4LzUWrUtk/I0iL1D8aatvD+59bDnHBo= apigo.cc/go/encoding v1.3.1/go.mod h1:xAJk5b83VZ31mXMTnyp0dfMoBKfT/AHDn0u+cQfojgY=
apigo.cc/go/file v1.3.0 h1:xG9FcY3Rv6Br83r9pq9QsIXFrplx4g8ITOkHSzfzXRg= apigo.cc/go/file v1.3.2 h1:pu4oiDyiqgj3/eykfnJf+/6+A9v/Z0b3ClP5XK+lwG4=
apigo.cc/go/file v1.3.0/go.mod h1:pYHBlB/XwsrnWpEh7GIFpbiqobrExfiB+rEN8V2d2kY= apigo.cc/go/file v1.3.2/go.mod h1:vci4h0Pz94mV6dkniQkuyBYERVYeq7/LX4jJVuCg9hs=
apigo.cc/go/id v1.3.0 h1:Tr2Yj0Rl19lfwW5wBTJ407o/zgo2oVRLE20WWEgJzdE= apigo.cc/go/id v1.3.1 h1:pkqi6VeWyQoHuIu0Zbx/RRxIAdM61Js0j6cY1M9XVCk=
apigo.cc/go/id v1.3.0/go.mod h1:AFH3kMFwENfXNyijnAFWEhSF1o3y++UBPem1IUlrcxA= apigo.cc/go/id v1.3.1/go.mod h1:P2/vl3tyW3US+ayOFSMoPIOCulNLBngNYPhXJC/Z7J4=
apigo.cc/go/log v1.3.2 h1:/m3V4MnlYnCG4XPHpWDsa4cw5suMaDVY1SgaVyjnBSo= apigo.cc/go/log v1.3.4 h1:UT8Neb9r4QjjbCFbTzw+ZeTxd+DmdmR5gNExeR4Cj+g=
apigo.cc/go/log v1.3.2/go.mod h1:dz4bSz9BnOgutkUJJZfX3uDDwsMpUxt7WF50mLK9hgE= apigo.cc/go/log v1.3.4/go.mod h1:/Q/2r51xWSsrS4QN5U9jLiTw8n6qNC8kG9nuVHweY20=
apigo.cc/go/rand v1.3.0 h1:k+UFAhMySwXf+dq8Om9TniZV6fm6gAE0evbrqMEdwQU= apigo.cc/go/rand v1.3.1 h1:7FvsI6PtQ5XrWER0dTiLVo0p7GIxRidT/TBKhVy93j8=
apigo.cc/go/rand v1.3.0/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk= apigo.cc/go/rand v1.3.1/go.mod h1:mZ/4Soa3bk+XvDaqPWJuUe1bfEi4eThBj1XmEAuYxsk=
apigo.cc/go/redis v1.3.0 h1:3NJE3xPXzhCwL+Mh1iyphFrsKWEuPlY26LHJfMVFSeU= apigo.cc/go/redis v1.3.2 h1:iUWL/CHHnfonz0dJq6/V4IG3QuXBoHA2L1xnoGEbNEQ=
apigo.cc/go/redis v1.3.0/go.mod h1:KPDPwMOER7WJX3Qev24LTeAOSmCl8OApe8iagPDxOUQ= apigo.cc/go/redis v1.3.2/go.mod h1:/k5wcfAzB9jrfd9otabio9CPUxEsLPgEs4oggBG5sbs=
apigo.cc/go/safe v1.3.0 h1:uctdAUsphT9p60Tk4oS5xPCe0NoIdOHfsYv4PNS0Rok= apigo.cc/go/safe v1.3.1 h1:irTCqPAC97gGsX/Lw5AzLelDt1xXLEZIAaVhLELWe9Q=
apigo.cc/go/safe v1.3.0/go.mod h1:tC9X14V+qh0BqIrVg4UkXbl+2pEN+lj2ZNI8IjDB6Fs= apigo.cc/go/safe v1.3.1/go.mod h1:XdOpBhN2vkImalaykYXXmEpczqWa1y3ah6/Q72cdRqE=
apigo.cc/go/shell v1.3.0 h1:hdxuYPN/7T2BuM/Ja8AjVUhbRqU/wpi8OjcJVziJ0nw= apigo.cc/go/shell v1.3.1 h1:M8oD0b2HcJuCC6frQFx11b3UTcTx3lATX8XK+YXSVm8=
apigo.cc/go/shell v1.3.0/go.mod h1:aNJiRWibxlA485yX3t+07IVAbrALKmxzv4oGEUC+hK4= apigo.cc/go/shell v1.3.1/go.mod h1:ZMdJjpCpWdvsHKUXlelh/AxsV/nWdkH/k3lISfzMdUw=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo= filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc= filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=

489
table.go
View File

@ -3,10 +3,10 @@ package tableDB
import ( import (
"fmt" "fmt"
"strings" "strings"
"time"
"apigo.cc/go/cast" "apigo.cc/go/cast"
"apigo.cc/go/db" "apigo.cc/go/db"
"apigo.cc/go/id"
) )
// Table provides an AI-friendly interface for interacting with structured data or schema. // Table provides an AI-friendly interface for interacting with structured data or schema.
@ -15,20 +15,15 @@ type Table struct {
userID string userID string
db *db.DB db *db.DB
app *TableDB app *TableDB
isMeta bool // True if Name ends with ":Schema"
constraint map[string]any constraint map[string]any
} }
// NewTable creates a new Table instance. Handles ":Schema" and ":Field" suffix. // NewTable creates a new Table instance. Handles ":Field" suffix.
func NewTable(name string, app *TableDB) *Table { func NewTable(name string, app *TableDB) *Table {
isMeta := false
actualName := name actualName := name
var constraint map[string]any var constraint map[string]any
if strings.HasSuffix(name, ":Schema") { if strings.HasSuffix(name, ":Field") {
isMeta = true
actualName = strings.TrimSuffix(name, ":Schema")
} else if strings.HasSuffix(name, ":Field") {
tableName := strings.TrimSuffix(name, ":Field") tableName := strings.TrimSuffix(name, ":Field")
actualName = "_Field" actualName = "_Field"
// lookup table_id // lookup table_id
@ -46,53 +41,75 @@ func NewTable(name string, app *TableDB) *Table {
userID: app.userID, userID: app.userID,
db: app.base, db: app.base,
app: app, app: app,
isMeta: isMeta,
constraint: constraint, constraint: constraint,
} }
} }
func (t *Table) checkAuth(id string) error { func (t *Table) checkAuth(id string, action string) error {
if t.userID == "_system" { if t.userID == SystemUserID {
return nil return nil
} }
// Check if table has 'creator' field if action == "write" && (t.Name == "_Policy" || t.Name == "_Backup") {
hasCreator := false return fmt.Errorf("permission denied for %s", t.Name)
}
tableRec := GlobalCache.GetTable(t.Name) tableRec := GlobalCache.GetTable(t.Name)
enableRLS := false
if tableRec != nil { if tableRec != nil {
tid := cast.String(tableRec["id"]) enableRLS = cast.Bool(tableRec["enableRLS"])
fields := GlobalCache.GetFields(tid) }
for _, f := range fields {
if f.Name == "creator" { if !enableRLS && !strings.HasPrefix(t.Name, "_") {
hasCreator = true return nil
}
policies := GlobalCache.GetFlatPolicies(t.userID, t.Name, action)
hasFullAccess := false
for _, p := range policies {
if p.Condition == "" {
hasFullAccess = true
break break
} }
} }
} if hasFullAccess {
if !hasCreator && !strings.HasPrefix(t.Name, "_") {
return nil return nil
} }
if t.Name == "_Policy" || t.Name == "_Backup" { // 构建合并查询:一次性判断 Creator 和 所有 Conditions
return fmt.Errorf("permission denied for %s", t.Name) var authConditions []string
var authArgs []any
// 1. 追加 Creator 检查
hasCreator := true
if hasCreator {
authConditions = append(authConditions, "`creator` = ?")
authArgs = append(authArgs, t.userID)
} }
// Check existing record
res := t.db.Query(fmt.Sprintf("SELECT creator FROM `%s` WHERE id = ?", t.Name), id) // 2. 追加 Policy Conditions
rec := res.MapOnR1() for _, p := range policies {
if len(rec) > 0 { if p.Condition != "" {
if cast.String(rec["creator"]) == t.userID { authConditions = append(authConditions, "("+p.Condition+")")
return nil authArgs = append(authArgs, p.ConditionArgs...)
} }
// Check _Policy
pol := t.db.Query("SELECT 1 FROM _Policy WHERE subject = ? AND resource = ? AND effect = 'allow' LIMIT 1", t.userID, id)
if len(pol.MapOnR1()) > 0 {
return nil
} }
// 3. 组装并执行终极 1-RTT 查询
if len(authConditions) > 0 {
authPart := "(" + strings.Join(authConditions, " OR ") + ")"
checkSQL := fmt.Sprintf("SELECT 1 FROM `%s` WHERE id = ? AND %s LIMIT 1", t.Name, authPart)
finalArgs := append([]any{id}, authArgs...)
checkRes := t.db.Query(checkSQL, finalArgs...)
if len(checkRes.MapOnR1()) > 0 {
return nil // 验证通过!
}
}
return fmt.Errorf("permission denied for %s record %s", t.Name, id) return fmt.Errorf("permission denied for %s record %s", t.Name, id)
} }
return nil
}
func (t *Table) appendAuthAndConstraint(whereStr string, args []any) (string, []any, error) { func (t *Table) appendAuthAndConstraint(whereStr string, args []any) (string, []any, error) {
if t.constraint != nil { if t.constraint != nil {
@ -106,34 +123,61 @@ func (t *Table) appendAuthAndConstraint(whereStr string, args []any) (string, []
} }
} }
if t.userID == "_system" { if t.userID == SystemUserID {
return whereStr, args, nil return whereStr, args, nil
} }
// Check if table has 'creator' field
hasCreator := false
tableRec := GlobalCache.GetTable(t.Name) tableRec := GlobalCache.GetTable(t.Name)
enableRLS := false
if tableRec != nil { if tableRec != nil {
tid := cast.String(tableRec["id"]) enableRLS = cast.Bool(tableRec["enableRLS"])
fields := GlobalCache.GetFields(tid) }
for _, f := range fields {
if f.Name == "creator" { if !enableRLS && !strings.HasPrefix(t.Name, "_") {
hasCreator = true return whereStr, args, nil
}
policies := GlobalCache.GetFlatPolicies(t.userID, t.Name, "read")
hasFullAccess := false
for _, p := range policies {
if p.Condition == "" {
hasFullAccess = true
break break
} }
} }
if hasFullAccess {
return whereStr, args, nil
} }
if hasCreator || strings.HasPrefix(t.Name, "_") { // Build dynamic SQL
if t.Name == "_Policy" || t.Name == "_Backup" { var authConditions []string
return "", nil, fmt.Errorf("permission denied for %s", t.Name) var authArgs []any
// Check creator field
hasCreator := true
if hasCreator {
authConditions = append(authConditions, "`creator` = ?")
authArgs = append(authArgs, t.userID)
} }
authWhere := fmt.Sprintf("(creator = '%s' OR id IN (SELECT resource FROM _Policy WHERE subject = '%s' AND effect = 'allow'))", t.userID, t.userID)
for _, p := range policies {
if p.Condition != "" {
authConditions = append(authConditions, "("+p.Condition+")")
authArgs = append(authArgs, p.ConditionArgs...)
}
}
authPart := "0"
if len(authConditions) > 0 {
authPart = "(" + strings.Join(authConditions, " OR ") + ")"
args = append(args, authArgs...)
}
if whereStr != "" { if whereStr != "" {
whereStr = "(" + whereStr + ") AND " + authWhere whereStr = "(" + whereStr + ") AND " + authPart
} else { } else {
whereStr = authWhere whereStr = authPart
}
} }
return whereStr, args, nil return whereStr, args, nil
@ -152,7 +196,9 @@ func (t *Table) reconstructAndSyncSchema() error {
var sb strings.Builder var sb strings.Builder
for _, tbl := range tables { for _, tbl := range tables {
name := cast.String(tbl["name"]) name := cast.String(tbl["name"])
if name == "" { continue } if name == "" {
continue
}
tid := cast.String(tbl["id"]) tid := cast.String(tbl["id"])
tblFields := fieldMap[tid] tblFields := fieldMap[tid]
@ -167,22 +213,23 @@ func (t *Table) reconstructAndSyncSchema() error {
} }
sb.WriteString("\n") sb.WriteString("\n")
hasID := false // Always ensure standard fields are present with appropriate indices
for _, f := range tblFields {
if cast.String(f["name"]) == "id" {
hasID = true
break
}
}
if !hasID {
sb.WriteString(" id c10 PK\n") sb.WriteString(" id c10 PK\n")
} sb.WriteString(" createTime bi I\n")
sb.WriteString(" creator v64\n")
sb.WriteString(" updateTime bi I\n")
sb.WriteString(" updater v64\n")
for _, f := range tblFields { for _, f := range tblFields {
fname := cast.String(f["name"]) fname := cast.String(f["name"])
if fname == "id" || fname == "createTime" || fname == "creator" || fname == "updateTime" || fname == "updater" {
continue
}
ftype := cast.String(f["type"]) ftype := cast.String(f["type"])
if ftype == "" { ftype = "v255" } if ftype == "" {
isIndex := cast.Int(f["isIndex"]) == 1 ftype = "v255"
}
isIndex := cast.Bool(f["isIndex"]) || cast.Int(f["isIndex"]) == 1
fmemo := cast.String(f["memo"]) fmemo := cast.String(f["memo"])
sb.WriteString(" " + fname + " " + ftype) sb.WriteString(" " + fname + " " + ftype)
@ -197,26 +244,34 @@ func (t *Table) reconstructAndSyncSchema() error {
sb.WriteString("\n") sb.WriteString("\n")
} }
return t.db.Sync(sb.String()) err := t.db.Sync(sb.String())
if err != nil {
return err
}
return GlobalCache.Load(t.app)
} }
// Set performs an upsert. // Set performs an upsert of one or more records.
func (t *Table) Set(data any) error { func (t *Table) Set(data ...any) error {
if t.userID == "" { if t.userID == "" {
return fmt.Errorf("no permission") return fmt.Errorf("authentication required")
} }
metaTouched := false
var updatedRows []map[string]any
// Pre-fetch schema context for business tables
var tableSchema *TableSchema
var fields []FieldSchema
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
cast.Convert(&tableSchema, tableRec)
fields = GlobalCache.GetFields(tableSchema.ID)
}
for _, d := range data {
record := make(map[string]any) record := make(map[string]any)
j, _ := cast.ToJSON(data) cast.Convert(&record, d)
_ = cast.UnmarshalJSON([]byte(j), &record)
if t.isMeta {
dsl, ok := record["dsl"].(string)
if !ok {
return fmt.Errorf("schema dsl is required")
}
return t.app.SyncSchema(dsl)
}
if t.constraint != nil { if t.constraint != nil {
for k, v := range t.constraint { for k, v := range t.constraint {
@ -224,8 +279,8 @@ func (t *Table) Set(data any) error {
} }
} }
if t.app.Hooks.OnUpdatingRow != nil && t.Name != "_Table" && t.Name != "_Field" { if t.app.hooks.OnUpdatingRow != nil && t.Name != "_Table" && t.Name != "_Field" {
if err := t.app.Hooks.OnUpdatingRow(t.Name, record); err != nil { if err := t.app.hooks.OnUpdatingRow(record, tableSchema, fields); err != nil {
return err return err
} }
} }
@ -233,14 +288,13 @@ func (t *Table) Set(data any) error {
idVal := record["id"] idVal := record["id"]
var isInsert bool var isInsert bool
if idVal == nil || cast.String(idVal) == "" { if idVal == nil || cast.String(idVal) == "" {
record["id"] = id.MakeID(10)
isInsert = true isInsert = true
} else { } else {
idStr := cast.String(idVal) idStr := cast.String(idVal)
res := t.db.Query(fmt.Sprintf("SELECT id FROM `%s` WHERE id = ? LIMIT 1", t.Name), idStr) res := t.db.Query(fmt.Sprintf("SELECT id FROM `%s` WHERE id = ? LIMIT 1", t.Name), idStr)
rec := res.MapOnR1() rec := res.MapOnR1()
if rec != nil && len(rec) > 0 { if rec != nil && len(rec) > 0 {
if err := t.checkAuth(idStr); err != nil { if err := t.checkAuth(idStr, "write"); err != nil {
return err return err
} }
isInsert = false isInsert = false
@ -249,63 +303,165 @@ func (t *Table) Set(data any) error {
} }
} }
// Always update updateTime and updater
record["updateTime"] = time.Now().UnixMilli()
record["updater"] = t.userID
var err error var err error
if isInsert { if isInsert {
if t.userID != "_system" { record["createTime"] = record["updateTime"]
// Check if table has 'creator' field if t.userID != SystemUserID {
hasCreator := false
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
tid := cast.String(tableRec["id"])
fields := GlobalCache.GetFields(tid)
for _, f := range fields {
if f.Name == "creator" {
hasCreator = true
break
}
}
}
if hasCreator || strings.HasPrefix(t.Name, "_") {
if t.Name == "_Policy" || t.Name == "_Backup" { if t.Name == "_Policy" || t.Name == "_Backup" {
return fmt.Errorf("permission denied for %s", t.Name) return fmt.Errorf("permission denied for %s", t.Name)
} }
record["creator"] = t.userID record["creator"] = t.userID
} else {
if record["creator"] == nil {
record["creator"] = t.userID
} }
} }
err = t.db.Insert(t.Name, record).Error err = t.db.Insert(t.Name, record).Error
} else { } else {
// Prevent overwriting CreateTime and Creator on update
delete(record, "createTime")
delete(record, "creator")
err = t.db.Update(t.Name, record, "id = ?", cast.String(record["id"])).Error err = t.db.Update(t.Name, record, "id = ?", cast.String(record["id"])).Error
} }
if err == nil { if err != nil {
return err
}
if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
metaTouched = true
if t.Name == "_Table" { if t.Name == "_Table" {
_ = t.reconstructAndSyncSchema() if isInsert && t.app.hooks.OnCreatedTable != nil {
if isInsert && t.app.Hooks.OnCreatedTable != nil { var ts TableSchema
t.app.Hooks.OnCreatedTable(cast.String(record["name"]), record) cast.Convert(&ts, record)
t.app.hooks.OnCreatedTable(&ts)
} }
} else if t.Name == "_Field" { } else if t.Name == "_Field" {
_ = t.reconstructAndSyncSchema() if t.app.hooks.OnUpdatedField != nil {
if t.app.Hooks.OnUpdatedField != nil { var fs FieldSchema
t.app.Hooks.OnUpdatedField(cast.String(record["tableId"]), cast.String(record["name"]), record) cast.Convert(&fs, record)
// Look up table schema
var fts *TableSchema
GlobalCache.lock.RLock()
tableName := GlobalCache.TableIDMap[fs.TableID]
fTableRec := GlobalCache.Tables[tableName]
GlobalCache.lock.RUnlock()
if fTableRec != nil {
cast.Convert(&fts, fTableRec)
}
t.app.hooks.OnUpdatedField(fts, &fs)
}
} }
} else { } else {
if t.app.Hooks.OnUpdatedRows != nil { updatedRows = append(updatedRows, record)
t.app.Hooks.OnUpdatedRows(t.Name, 1)
}
} }
} }
if metaTouched {
if t.Name != "_Policy" {
_ = t.reconstructAndSyncSchema()
}
_ = GlobalCache.Load(t.app)
}
if len(updatedRows) > 0 && t.app.hooks.OnUpdatedRows != nil {
t.app.hooks.OnUpdatedRows(updatedRows, tableSchema, fields)
}
return nil
}
// SetField adds or updates one or more fields. If table doesn't exist, it will be created.
func (t *Table) SetField(fields ...FieldSchema) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
}
// 1. Ensure table exists in _Table
tableRec := GlobalCache.GetTable(t.Name)
if tableRec == nil {
// Create table entry
err := t.app.SetTable(TableSchema{Name: t.Name})
if err != nil {
return err return err
} }
tableRec = GlobalCache.GetTable(t.Name)
if tableRec == nil {
return fmt.Errorf("failed to create table entry for %s", t.Name)
}
// Automatically add default ID field to metadata
_ = t.app.Table(t.Name + ":Field").Set(map[string]any{
"name": "id",
"type": "c10",
"isIndex": 1,
"memo": "Primary Key",
})
}
// 2. Prepare field records
fieldTable := t.app.Table(t.Name + ":Field")
var fieldRecords []any
for _, f := range fields {
fRecord := map[string]any{
"name": f.Name,
"type": f.Type,
"isIndex": f.IsIndex,
"memo": f.Memo,
"settings": f.Settings,
}
if f.ID != "" {
fRecord["id"] = f.ID
} else {
// If ID is missing, try to find existing field to update
existing, _ := fieldTable.List(map[string]any{"name": f.Name})
if len(existing) > 0 {
fRecord["id"] = existing[0]["id"]
}
}
fieldRecords = append(fieldRecords, fRecord)
}
// 3. Batch Set fields
return fieldTable.Set(fieldRecords...)
}
// RemoveField deletes one or more fields by name.
func (t *Table) RemoveField(names ...string) error {
if t.userID == "" {
return fmt.Errorf("authentication required")
}
tableRec := GlobalCache.GetTable(t.Name)
if tableRec == nil {
return fmt.Errorf("table %s not found", t.Name)
}
fieldTable := t.app.Table(t.Name + ":Field")
for _, name := range names {
existing, _ := fieldTable.List(map[string]any{"name": name})
if len(existing) > 0 {
err := fieldTable.Remove(cast.String(existing[0]["id"]))
if err != nil {
return err
}
}
}
return nil
}
// Get retrieves a single record. // Get retrieves a single record.
func (t *Table) Get(id string) (map[string]any, error) { func (t *Table) Get(id string) (map[string]any, error) {
if t.isMeta { if t.userID == "" {
return map[string]any{"name": t.Name, "type": "table"}, nil return nil, fmt.Errorf("authentication required")
} }
if err := t.checkAuth(id); err != nil { if err := t.checkAuth(id, "read"); err != nil {
return nil, err return nil, err
} }
@ -322,80 +478,119 @@ func (t *Table) Get(id string) (map[string]any, error) {
return record, nil return record, nil
} }
// Remove deletes a record. // Remove deletes one or more records by ID.
func (t *Table) Remove(id string) error { func (t *Table) Remove(ids ...string) error {
if t.isMeta { if t.userID == "" {
return fmt.Errorf("schema removal not allowed via Table API") return fmt.Errorf("authentication required")
} }
if err := t.checkAuth(id); err != nil { var removedIDs []string
// Pre-fetch schema context for business tables
var tableSchema *TableSchema
var fields []FieldSchema
tableRec := GlobalCache.GetTable(t.Name)
if tableRec != nil {
cast.Convert(&tableSchema, tableRec)
fields = GlobalCache.GetFields(tableSchema.ID)
}
for _, id := range ids {
if err := t.checkAuth(id, "write"); err != nil {
return err return err
} }
var record map[string]any var record map[string]any
if t.Name == "_Table" || t.Name == "_Field" { if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
res := t.db.Query(fmt.Sprintf("SELECT * FROM `%s` WHERE id = ?", t.Name), id) res := t.db.Query(fmt.Sprintf("SELECT * FROM `%s` WHERE id = ?", t.Name), id)
record = res.MapOnR1() record = res.MapOnR1()
} }
res := t.db.Delete(t.Name, "id = ?", id) res := t.db.Delete(t.Name, "id = ?", id)
if res.Error == nil { if res.Error == nil {
if t.Name == "_Table" { if t.Name == "_Table" || t.Name == "_Field" || t.Name == "_Policy" {
if t.Name != "_Policy" {
_ = t.reconstructAndSyncSchema() _ = t.reconstructAndSyncSchema()
if record != nil && t.app.Hooks.OnRemovedTable != nil {
t.app.Hooks.OnRemovedTable(cast.String(record["name"]))
} }
} else if t.Name == "_Field" { _ = GlobalCache.Load(t.app)
_ = t.reconstructAndSyncSchema()
if record != nil && t.app.Hooks.OnRemovedField != nil { if t.Name == "_Table" && record != nil && t.app.hooks.OnRemovedTable != nil {
t.app.Hooks.OnRemovedField(cast.String(record["tableId"]), cast.String(record["name"])) var ts TableSchema
cast.Convert(&ts, record)
t.app.hooks.OnRemovedTable(&ts)
} else if t.Name == "_Field" && record != nil && t.app.hooks.OnRemovedField != nil {
var fts *TableSchema
GlobalCache.lock.RLock()
tableName := GlobalCache.TableIDMap[cast.String(record["tableId"])]
fTableRec := GlobalCache.Tables[tableName]
GlobalCache.lock.RUnlock()
if fTableRec != nil {
cast.Convert(&fts, fTableRec)
}
t.app.hooks.OnRemovedField(fts, cast.String(record["name"]))
} }
} else { } else {
if t.app.Hooks.OnRemovedRows != nil { removedIDs = append(removedIDs, id)
t.app.Hooks.OnRemovedRows(t.Name, []string{id})
}
}
} }
} else {
return res.Error return res.Error
} }
}
if len(removedIDs) > 0 && t.app.hooks.OnRemovedRows != nil {
t.app.hooks.OnRemovedRows(removedIDs, tableSchema, fields)
}
return nil
}
// List retrieves multiple records. // List retrieves multiple records.
func (t *Table) List(where any, args ...any) ([]map[string]any, error) { func (t *Table) List(where any, args ...any) ([]map[string]any, error) {
if t.isMeta { req := QueryRequest{}
return nil, fmt.Errorf("list not supported on schema tables")
}
query := fmt.Sprintf("SELECT * FROM `%s` ", t.Name)
whereStr := ""
if where != nil { if where != nil {
switch v := where.(type) { switch v := where.(type) {
case string: case string:
whereStr = v req.Where = v
req.Args = args
case map[string]any: case map[string]any:
whereStr, args = buildWhere(v) req.Where, req.Args = buildWhere(v)
} }
} }
var err error return t.Query(req)
whereStr, args, err = t.appendAuthAndConstraint(whereStr, args) }
// Query performs a structured query on the current table.
func (t *Table) Query(req QueryRequest) ([]map[string]any, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
// Apply constraint to where clause
if t.constraint != nil {
if req.Where != "" {
// If it's a string, this is complex to merge safely without parsing SQL.
// Let's rely on appendAuthAndConstraint passing the constraint.
}
}
sql, args, err := t.app.buildQuery(t, req)
if err != nil { if err != nil {
return nil, err return nil, err
} }
res := t.db.Query(sql, args...)
if whereStr != "" {
query += " WHERE " + whereStr
}
res := t.db.Query(query, args...)
if res.Error != nil { if res.Error != nil {
return nil, res.Error return nil, res.Error
} }
return res.MapResults(), nil return res.MapResults(), nil
} }
// Count returns the number of records. // Count returns the number of records.
func (t *Table) Count(where any, args ...any) (int64, error) { func (t *Table) Count(where any, args ...any) (int64, error) {
if t.userID == "" {
return 0, fmt.Errorf("authentication required")
}
query := fmt.Sprintf("SELECT COUNT(*) FROM `%s` ", t.Name) query := fmt.Sprintf("SELECT COUNT(*) FROM `%s` ", t.Name)
whereStr := "" whereStr := ""
if where != nil { if where != nil {
@ -428,6 +623,10 @@ func (t *Table) Count(where any, args ...any) (int64, error) {
// CountBy returns counts grouped by a field. // CountBy returns counts grouped by a field.
func (t *Table) CountBy(field string) (map[any]int64, error) { func (t *Table) CountBy(field string) (map[any]int64, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
query := fmt.Sprintf("SELECT `%s`, COUNT(*) as cnt FROM `%s` ", field, t.Name) query := fmt.Sprintf("SELECT `%s`, COUNT(*) as cnt FROM `%s` ", field, t.Name)
whereStr, args, err := t.appendAuthAndConstraint("", nil) whereStr, args, err := t.appendAuthAndConstraint("", nil)
@ -455,6 +654,10 @@ func (t *Table) CountBy(field string) (map[any]int64, error) {
// Fields returns field metadata. // Fields returns field metadata.
func (t *Table) Fields() ([]FieldSchema, error) { func (t *Table) Fields() ([]FieldSchema, error) {
if t.userID == "" {
return nil, fmt.Errorf("authentication required")
}
tid := "" tid := ""
tableRecord := GlobalCache.GetTable(t.Name) tableRecord := GlobalCache.GetTable(t.Name)
if tableRecord != nil { if tableRecord != nil {

View File

@ -14,20 +14,27 @@ func setupDB(t *testing.T, dbFile string) *TableDB {
_ = os.Remove(dbFile) _ = os.Remove(dbFile)
logger := log.DefaultLogger logger := log.DefaultLogger
logger.SetLevel(log.ERROR) logger.SetLevel(log.ERROR)
dbInst := GetDB("sqlite://"+dbFile, logger) dbInst := GetDB("sqlite://"+dbFile, logger, "")
return dbInst // Bootstrap system tables
err := dbInst.Bootstrap()
if err != nil {
panic(fmt.Sprintf("Bootstrap failed in setupDB: %v", err))
}
return dbInst.Auth(SystemUserID)
} }
func TestBootstrapAndSync(t *testing.T) { func TestBootstrapAndSync(t *testing.T) {
dbFile := "test_bootstrap.db" dbFile := "test_bootstrap.db"
dbInst := setupDB(t, dbFile) _ = os.Remove(dbFile)
unauth := GetDB("sqlite://"+dbFile, log.DefaultLogger, "")
defer os.Remove(dbFile) defer os.Remove(dbFile)
// 1. Bootstrap: Sync empty DSL, should still create system tables // 1. Bootstrap: sync system DSL
err := dbInst.SyncSchema("") err := unauth.Bootstrap()
if err != nil { if err != nil {
t.Fatalf("Bootstrap failed: %v", err) t.Fatalf("Bootstrap failed: %v", err)
} }
dbInst := unauth.Auth(SystemUserID)
// Verify system tables in cache // Verify system tables in cache
if GlobalCache.GetTable("_Table") == nil { if GlobalCache.GetTable("_Table") == nil {
@ -37,25 +44,21 @@ func TestBootstrapAndSync(t *testing.T) {
t.Errorf("_Field not found in cache after bootstrap") t.Errorf("_Field not found in cache after bootstrap")
} }
// 2. Sync business schema // 2. Create business table via API
businessDSL := ` err = dbInst.Table("users").SetField(
== Business == FieldSchema{Name: "name", Type: "v64", IsIndex: true},
users SD FieldSchema{Name: "age", Type: "i"},
id c10 PK )
name v64 U
age i
`
err = dbInst.SyncSchema(businessDSL)
if err != nil { if err != nil {
t.Fatalf("Business sync failed: %v", err) t.Fatalf("Create table failed: %v", err)
} }
// Verify business table // Verify business table
if GlobalCache.GetTable("users") == nil { if GlobalCache.GetTable("users") == nil {
t.Errorf("users table not found in cache after sync") t.Errorf("users table not found in cache after sync")
} }
fields := GlobalCache.GetValidFields("users") fields, _ := dbInst.Table("users").Fields()
if len(fields) < 3 { // id, name, age (and maybe others from SD) if len(fields) < 3 { // id, name, age
t.Errorf("Expected at least 3 fields for users, got %v", fields) t.Errorf("Expected at least 3 fields for users, got %v", fields)
} }
} }
@ -65,41 +68,18 @@ func TestAPIDrivenSchema(t *testing.T) {
dbInst := setupDB(t, dbFile) dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile) defer os.Remove(dbFile)
_ = dbInst.SyncSchema("")
app := dbInst.Auth("admin") app := dbInst.Auth("admin")
// 1. Create table via _Table API // 1. Create table and add fields via SetField API
err := app.Table("_Table").Set(map[string]any{ err := app.Table("tasks").SetField(
"name": "tasks", FieldSchema{Name: "title", Type: "v100", Memo: "Task title"},
"memo": "Task list", FieldSchema{Name: "done", Type: "b", IsIndex: true},
}) )
if err != nil { if err != nil {
t.Fatalf("Failed to create table via API: %v", err) t.Fatalf("Failed to create table/fields via API: %v", err)
} }
// Get tid // 2. Verify table works
tbl, _ := app.Table("_Table").List(map[string]any{"name": "tasks"})
if len(tbl) == 0 {
t.Fatalf("Table tasks not found in _Table")
}
tid := cast.String(tbl[0]["id"])
// 2. Add fields via _Field API
_ = app.Table("_Field").Set(map[string]any{
"tableId": tid,
"name": "title",
"type": "v100",
"memo": "Task title",
})
_ = app.Table("_Field").Set(map[string]any{
"tableId": tid,
"name": "done",
"type": "b",
"isIndex": 1,
})
// 3. Verify table works
taskTable := app.Table("tasks") taskTable := app.Table("tasks")
err = taskTable.Set(map[string]any{ err = taskTable.Set(map[string]any{
"title": "Fix tests", "title": "Fix tests",
@ -113,6 +93,13 @@ func TestAPIDrivenSchema(t *testing.T) {
if len(res) != 1 || res[0]["title"] != "Fix tests" { if len(res) != 1 || res[0]["title"] != "Fix tests" {
t.Errorf("Query from tasks failed: %v", res) t.Errorf("Query from tasks failed: %v", res)
} }
// Verify ID is automatically generated
if idVal := cast.String(res[0]["id"]); len(idVal) == 0 {
t.Errorf("ID was not automatically generated")
} else {
fmt.Printf("Generated ID: %s\n", idVal)
}
} }
func TestDataOperations(t *testing.T) { func TestDataOperations(t *testing.T) {
@ -120,20 +107,28 @@ func TestDataOperations(t *testing.T) {
dbInst := setupDB(t, dbFile) dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile) defer os.Remove(dbFile)
_ = dbInst.SyncSchema(` _ = dbInst.Table("products").SetField(
== Test == FieldSchema{Name: "name", Type: "v64", IsIndex: true},
products SD FieldSchema{Name: "price", Type: "i"},
id c10 PK )
name v64 U
price i
`)
app := dbInst.Auth("admin") app := dbInst.Auth("admin")
table := app.Table("products") table := app.Table("products")
// Create // Create (Batch)
_ = table.Set(map[string]any{"name": "Laptop", "price": 1000}) err := table.Set(
_ = table.Set(map[string]any{"id": "p2", "name": "Mouse", "price": 50}) map[string]any{"name": "Laptop", "price": 1000},
map[string]any{"id": "p2", "name": "Mouse", "price": 50},
)
if err != nil {
t.Fatalf("Batch set failed: %v", err)
}
laptop, _ := table.List(map[string]any{"name": "Laptop"})
if len(laptop) == 0 {
t.Fatalf("Laptop not created")
}
laptopID := cast.String(laptop[0]["id"])
// Read // Read
p1, _ := table.Get("p2") p1, _ := table.Get("p2")
@ -160,15 +155,20 @@ products SD
t.Errorf("Count failed: %d", cnt) t.Errorf("Count failed: %d", cnt)
} }
// Delete (Shadow Delete) // Delete (Batch)
_ = table.Remove("p2") err = table.Remove("p2", laptopID)
if err != nil {
t.Fatalf("Batch remove failed: %v", err)
}
p1_removed, _ := table.Get("p2") p1_removed, _ := table.Get("p2")
if p1_removed != nil { if p1_removed != nil {
t.Errorf("Expected p2 to be removed") t.Errorf("Expected p2 to be removed")
} }
// Verify shadow delete (actual table has _deleted suffix) // Verify shadow delete (actual table has _deleted suffix)
res := dbInst.Base().Query("SELECT name FROM sqlite_master WHERE name='products_deleted'") raw, _ := dbInst.GetRawDB()
res := raw.Query("SELECT name FROM sqlite_master WHERE name='products_deleted'")
if res.MapOnR1()["name"] == nil { if res.MapOnR1()["name"] == nil {
t.Errorf("Shadow delete table products_deleted not found") t.Errorf("Shadow delete table products_deleted not found")
} }
@ -179,13 +179,11 @@ func TestPermissionsAndAuth(t *testing.T) {
dbInst := setupDB(t, dbFile) dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile) defer os.Remove(dbFile)
_ = dbInst.SyncSchema(` _ = dbInst.Table("secrets").SetField(
== Secret == FieldSchema{Name: "content", Type: "t"},
secrets SD FieldSchema{Name: "creator", Type: "v64"},
id c10 PK )
content t _ = dbInst.SetTable(TableSchema{Name: "secrets", EnableRLS: true})
creator v64
`)
user1 := dbInst.Auth("user1") user1 := dbInst.Auth("user1")
user2 := dbInst.Auth("user2") user2 := dbInst.Auth("user2")
@ -193,28 +191,27 @@ secrets SD
// user1 creates a secret // user1 creates a secret
_ = user1.Table("secrets").Set(map[string]any{"id": "s1", "content": "user1-secret"}) _ = user1.Table("secrets").Set(map[string]any{"id": "s1", "content": "user1-secret"})
// user2 tries to get it -> should fail or return nil depending on implementation // user2 tries to get it
// Current implementation: checkAuth returns error if no permission
_, err := user2.Table("secrets").Get("s1") _, err := user2.Table("secrets").Get("s1")
if err == nil { if err == nil {
t.Errorf("user2 should not have permission to get user1 secret") t.Errorf("user2 should not have permission to get user1 secret")
} }
// admin gets it -> _system or admin should have permission? // system gets it
// Implementation says t.userID == "_system" bypasses. sys := dbInst.Auth(SystemUserID)
sys := dbInst.Auth("_system")
s, err := sys.Table("secrets").Get("s1") s, err := sys.Table("secrets").Get("s1")
if err != nil || s == nil { if err != nil || s == nil {
t.Errorf("system should have permission: %v", err) t.Errorf("system should have permission: %v", err)
} }
// Test _Policy // Test _Policy: Grant access to specific ID via condition
// allow user2 to see s1
err = sys.Table("_Policy").Set(map[string]any{ err = sys.Table("_Policy").Set(map[string]any{
"subject": "user2", "userID": "user2",
"type": "table",
"targets": []string{"secrets"},
"action": "read", "action": "read",
"resource": "s1", "condition": "id = ?",
"effect": "allow", "conditionArgs": []any{"s1"},
}) })
if err != nil { if err != nil {
t.Fatalf("Failed to set policy: %v", err) t.Fatalf("Failed to set policy: %v", err)
@ -227,6 +224,100 @@ secrets SD
} }
} }
func TestInheritance(t *testing.T) {
dbFile := "test_inherit.db"
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
_ = dbInst.Table("docs").SetField(
FieldSchema{Name: "title", Type: "v64"},
FieldSchema{Name: "creator", Type: "v64"},
)
_ = dbInst.SetTable(TableSchema{Name: "docs", EnableRLS: true})
sys := dbInst.Auth(SystemUserID)
_ = sys.Table("docs").Set(map[string]any{"id": "d1", "title": "secret doc", "creator": "boss"})
// manager inherits boss
_ = sys.Table("_Policy").Set(map[string]any{
"userID": "manager",
"type": "inherit",
"targets": []string{"boss"},
})
// boss can read docs
_ = sys.Table("_Policy").Set(map[string]any{
"userID": "boss",
"type": "table",
"targets": []string{"docs"},
"action": "read",
})
manager := dbInst.Auth("manager")
d, err := manager.Table("docs").Get("d1")
if err != nil || d == nil {
t.Errorf("manager should inherit boss's permission to read docs: %v", err)
}
}
func TestPolicyInterfaces(t *testing.T) {
dbFile := "test_policy_api.db"
dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile)
sys := dbInst.Auth(SystemUserID)
_ = sys.Table("data").SetField(FieldSchema{Name: "val", Type: "v10"})
_ = sys.SetTable(TableSchema{Name: "data", EnableRLS: true})
// 1. Normal user tries to set inherit policy (should fail)
user1 := dbInst.Auth("user1")
err := user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "inherit",
Targets: []string{"user1"},
})
if err == nil {
t.Errorf("Normal user should not be able to set inherit policy")
}
// 2. Normal user tries to set table policy for table they don't have full access to (should fail)
err = user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "table",
Targets: []string{"data"},
Action: "read",
})
if err == nil {
t.Errorf("User without full access to table should not be able to set policy for it")
}
// 3. System grants user1 full access to 'data'
_ = sys.SetPolicy(PolicySchema{
UserID: "user1",
Type: "table",
Targets: []string{"data"},
Action: "full",
})
// 4. Now user1 sets policy for user2 on 'data' (should succeed)
err = user1.SetPolicy(PolicySchema{
UserID: "user2",
Type: "table",
Targets: []string{"data"},
Action: "read",
})
if err != nil {
t.Errorf("User with full access should be able to set policy: %v", err)
}
// 5. List policies
pols, err := user1.ListPolicy(map[string]any{"userID": "user2"})
if err != nil || len(pols) == 0 || pols[0].UserID != "user2" {
t.Errorf("ListPolicy failed: %v, %v", err, pols)
}
}
func TestHooks(t *testing.T) { func TestHooks(t *testing.T) {
dbFile := "test_hooks.db" dbFile := "test_hooks.db"
dbInst := setupDB(t, dbFile) dbInst := setupDB(t, dbFile)
@ -235,27 +326,21 @@ func TestHooks(t *testing.T) {
var tableCreated string var tableCreated string
var rowsUpdated int var rowsUpdated int
dbInst.Hooks.OnCreatedTable = func(tableName string, record map[string]any) { dbInst.hooks.OnCreatedTable = func(table *TableSchema) {
tableCreated = tableName tableCreated = table.Name
} }
dbInst.Hooks.OnUpdatedRows = func(tableName string, count int) { dbInst.hooks.OnUpdatedRows = func(rows []map[string]any, table *TableSchema, fields []FieldSchema) {
rowsUpdated += count rowsUpdated += len(rows)
} }
_ = dbInst.SyncSchema("")
app := dbInst.Auth("admin") app := dbInst.Auth("admin")
// Trigger OnCreatedTable // Trigger OnCreatedTable
_ = app.Table("_Table").Set(map[string]any{"name": "hook_test"}) _ = app.Table("hook_test").SetField(FieldSchema{Name: "val", Type: "v10"})
if tableCreated != "hook_test" { if tableCreated != "hook_test" {
t.Errorf("OnCreatedTable hook failed, got %s", tableCreated) t.Errorf("OnCreatedTable hook failed, got %s", tableCreated)
} }
// Add a field to ensure table exists
tbl, _ := app.Table("_Table").List(map[string]any{"name": "hook_test"})
tid := cast.String(tbl[0]["id"])
_ = app.Table("_Field").Set(map[string]any{"tableId": tid, "name": "val", "type": "v10"})
// Trigger OnUpdatedRows // Trigger OnUpdatedRows
_ = app.Table("hook_test").Set(map[string]any{"id": "1", "val": "a"}) _ = app.Table("hook_test").Set(map[string]any{"id": "1", "val": "a"})
if rowsUpdated != 1 { if rowsUpdated != 1 {
@ -268,46 +353,26 @@ func TestQueryValidationAndInjection(t *testing.T) {
dbInst := setupDB(t, dbFile) dbInst := setupDB(t, dbFile)
defer os.Remove(dbFile) defer os.Remove(dbFile)
_ = dbInst.SyncSchema(` _ = dbInst.Table("safe_table").SetField(FieldSchema{Name: "name", Type: "v64"})
== Test ==
safe_table SD app := dbInst.Auth("user1")
id c10 PK
name v64
`)
// Valid query // Valid query
req := QueryRequest{ req := QueryRequest{
Table: "safe_table",
Select: []string{"id", "name"}, Select: []string{"id", "name"},
OrderBy: "name DESC", OrderBy: "name DESC",
} }
_, _, err := dbInst.BuildQuery(req) _, err := app.Table("safe_table").Query(req)
if err != nil { if err != nil {
t.Errorf("Valid query failed: %v", err) t.Errorf("Valid query failed: %v", err)
} }
// Invalid table
req.Table = "invalid_table"
_, _, err = dbInst.BuildQuery(req)
if err == nil {
t.Errorf("Should fail for invalid table")
}
// SQL Injection in Select // SQL Injection in Select
req.Table = "safe_table"
req.Select = []string{"name` FROM safe_table; --"} req.Select = []string{"name` FROM safe_table; --"}
_, _, err = dbInst.BuildQuery(req) _, err = app.Table("safe_table").Query(req)
if err == nil { if err == nil {
t.Errorf("Should fail for injection in Select") t.Errorf("Should fail for injection in Select")
} }
// SQL Injection in OrderBy
req.Select = []string{"name"}
req.OrderBy = "name; DROP TABLE safe_table"
_, _, err = dbInst.BuildQuery(req)
if err == nil {
t.Errorf("Should fail for injection in OrderBy")
}
} }
func BenchmarkTableSet(b *testing.B) { func BenchmarkTableSet(b *testing.B) {
@ -316,10 +381,14 @@ func BenchmarkTableSet(b *testing.B) {
dbFile := fmt.Sprintf("bench_ops_%d.db", time.Now().UnixNano()) dbFile := fmt.Sprintf("bench_ops_%d.db", time.Now().UnixNano())
defer os.Remove(dbFile) defer os.Remove(dbFile)
dbInst := GetDB("sqlite://"+dbFile, logger) unauth := GetDB("sqlite://"+dbFile, logger, "")
_ = dbInst.SyncSchema("bench_ops SD\n id c10 PK\n name v50 U\n val i") _ = unauth.Bootstrap()
app := unauth.Auth(SystemUserID)
_ = app.Table("bench_ops").SetField(
FieldSchema{Name: "name", Type: "v50", IsIndex: true},
FieldSchema{Name: "val", Type: "i"},
)
app := dbInst.Auth("admin")
table := app.Table("bench_ops") table := app.Table("bench_ops")
b.ResetTimer() b.ResetTimer()

View File

@ -1,15 +1,48 @@
package tableDB package tableDB
type TableSchema struct {
ID string
Name string
Memo string
EnableRLS bool
Settings map[string]any
IsSecret bool // 是否敏感表(不被索引)
CreateTime int64
Creator string
UpdateTime int64
Updater string
}
type FlatPolicy struct {
Condition string
ConditionArgs []any
}
type PolicySchema struct {
ID string
UserID string
Type string // inherit, table
Targets []string
Action string // read, write, full
Condition string
ConditionArgs []any
CreateTime int64
Creator string
}
type FieldSchema struct { type FieldSchema struct {
ID string
TableID string TableID string
Name string Name string
Type string Type string
Index string IsIndex bool
Comment string Memo string
Settings map[string]any
CreateTime int64
Creator string
} }
type QueryRequest struct { type QueryRequest struct {
Table string
Select []string Select []string
Joins []JoinConfig Joins []JoinConfig
Where string Where string