727 lines
12 KiB
Markdown
727 lines
12 KiB
Markdown
# bcap API 手册
|
||
|
||
开发者参考文档。快速接入说明见 [`../README.md`](../README.md)。
|
||
|
||
## 1. 主包职责
|
||
|
||
主包负责：
|
||
|
||
- 统一的报文事实层
|
||
- 统一的 flow 表达
|
||
- 统一的轻量 hint 层
|
||
- 以 TCP 为重点的轻状态跟踪
|
||
|
||
主包不负责：
|
||
|
||
- 报告生成
|
||
- 业务规则判断
|
||
- 用户侧展示文案
|
||
- 动作编排
|
||
- 深度流重组
|
||
- 应用层协议解析框架
|
||
|
||
核心对象：
|
||
|
||
- `Decoder`
|
||
- 单包事实解码
|
||
- `Tracker`
|
||
- flow 状态与轻量提示
|
||
- `Analyzer`
|
||
- `Decoder + Tracker` 组合入口
|
||
|
||
## 2. 公开入口
|
||
|
||
### 2.1 `Decoder`
|
||
|
||
构造：
|
||
|
||
```go
|
||
decoder := bcap.NewDecoder()
|
||
```
|
||
|
||
主要方法：
|
||
|
||
```go
|
||
Decode(packet gopacket.Packet) (Packet, error)
|
||
DecodeWithOptions(packet gopacket.Packet, opts DecodeOptions) (Packet, error)
|
||
```
|
||
|
||
使用时机：
|
||
|
||
- 只想获得单包事实
|
||
- 自己维护状态机
|
||
- 想把 `bcap` 当作标准化事实提取层
|
||
|
||
### 2.2 `Tracker`
|
||
|
||
构造：
|
||
|
||
```go
|
||
tracker := bcap.NewTracker()
|
||
tracker := bcap.NewTrackerWithConfig(cfg)
|
||
```
|
||
|
||
主要方法：
|
||
|
||
```go
|
||
Observe(packet Packet) (Observation, error)
|
||
CleanupExpiredFlows() int
|
||
ActiveFlowCount() int
|
||
Stop()
|
||
```
|
||
|
||
使用时机：
|
||
|
||
- 已经在别处完成解码
|
||
- 只想复用 `bcap` 的 flow / hint 跟踪能力
|
||
|
||
### 2.3 `Analyzer`
|
||
|
||
构造：
|
||
|
||
```go
|
||
analyzer := bcap.NewAnalyzer()
|
||
analyzer := bcap.NewAnalyzerWithConfig(cfg)
|
||
```
|
||
|
||
主要方法：
|
||
|
||
```go
|
||
ObservePacket(packet gopacket.Packet) (Observation, error)
|
||
ObservePacketWithOptions(packet gopacket.Packet, opts DecodeOptions) (Observation, error)
|
||
Decoder() *Decoder
|
||
Tracker() *Tracker
|
||
Stop()
|
||
```
|
||
|
||
使用时机：
|
||
|
||
- 在线抓包
|
||
- 离线遍历
|
||
- 绝大多数直接接入型工具
|
||
|
||
## 3. `DecodeOptions`
|
||
|
||
`DecodeOptions` 目前包含两个字段：
|
||
|
||
```go
|
||
type DecodeOptions struct {
|
||
BaseTime time.Time
|
||
SrcMACOverride net.HardwareAddr
|
||
}
|
||
```
|
||
|
||
字段：
|
||
|
||
- `BaseTime`
|
||
- 用于计算 `Packet.Meta.RelativeTime`
|
||
- `Analyzer.ObservePacket(...)` 未显式传入时，会自动以首包时间作为基准
|
||
- `SrcMACOverride`
|
||
- 用于覆盖源 MAC
|
||
|
||
## 4. 数据模型
|
||
|
||
### 4.1 `Packet`
|
||
|
||
`Packet` 表示单包事实。
|
||
|
||
```go
|
||
type Packet struct {
|
||
Meta Meta
|
||
Link LinkFacts
|
||
Network NetworkFacts
|
||
Transport TransportFacts
|
||
Raw RawFacts
|
||
}
|
||
```
|
||
|
||
不包含跨包推断结果。
|
||
|
||
### 4.2 `Meta`
|
||
|
||
```go
|
||
type Meta struct {
|
||
Timestamp time.Time
|
||
TimestampMicros int64
|
||
RelativeTime time.Duration
|
||
CaptureLength int
|
||
Length int
|
||
}
|
||
```
|
||
|
||
字段：
|
||
|
||
- `Timestamp` / `TimestampMicros`
|
||
- 捕获时间
|
||
- `RelativeTime`
|
||
- 相对 `BaseTime` 的时间差
|
||
- `CaptureLength`
|
||
- 实际捕获长度
|
||
- `Length`
|
||
- 原始报文长度
|
||
|
||
### 4.3 `LinkFacts`
|
||
|
||
```go
|
||
type LinkFacts struct {
|
||
Kind LinkKind
|
||
SrcMAC net.HardwareAddr
|
||
DstMAC net.HardwareAddr
|
||
}
|
||
```
|
||
|
||
当前支持的链路类型：
|
||
|
||
- `LinkKindEthernet`
|
||
- `LinkKindLinuxSLL`
|
||
- `LinkKindLinuxSLL2`
|
||
- `LinkKindUnknown`
|
||
|
||
### 4.4 `NetworkFacts`
|
||
|
||
```go
|
||
type NetworkFacts struct {
|
||
Family NetworkFamily
|
||
SrcIP string
|
||
DstIP string
|
||
TTL uint8
|
||
HopLimit uint8
|
||
ProtocolNumber uint16
|
||
ARP *ARPFacts
|
||
}
|
||
```
|
||
|
||
当前支持的网络族：
|
||
|
||
- `NetworkFamilyIPv4`
|
||
- `NetworkFamilyIPv6`
|
||
- `NetworkFamilyARP`
|
||
- `NetworkFamilyUnknown`
|
||
|
||
备注：
|
||
|
||
- IPv4 报文主要填 `TTL`
|
||
- IPv6 报文主要填 `HopLimit`
|
||
- ARP 报文会同时填充 `ARP`
|
||
|
||
### 4.5 `TransportFacts`
|
||
|
||
```go
|
||
type TransportFacts struct {
|
||
Kind ProtocolKind
|
||
Payload int
|
||
TCP *TCPFacts
|
||
UDP *UDPFacts
|
||
ICMP *ICMPFacts
|
||
Unknown *UnknownTransportFacts
|
||
}
|
||
```
|
||
|
||
当前支持协议：
|
||
|
||
- `ProtocolTCP`
|
||
- `ProtocolUDP`
|
||
- `ProtocolICMPv4`
|
||
- `ProtocolICMPv6`
|
||
- `ProtocolARP`
|
||
- `ProtocolUnknown`
|
||
|
||
### 4.6 协议事实结构
|
||
|
||
#### `TCPFacts`
|
||
|
||
```go
|
||
type TCPFacts struct {
|
||
SrcPort string
|
||
DstPort string
|
||
Seq uint32
|
||
Ack uint32
|
||
Window uint16
|
||
SYN bool
|
||
ACK bool
|
||
FIN bool
|
||
RST bool
|
||
ECE bool
|
||
CWR bool
|
||
PSH bool
|
||
Checksum uint16
|
||
Payload int
|
||
}
|
||
```
|
||
|
||
#### `UDPFacts`
|
||
|
||
```go
|
||
type UDPFacts struct {
|
||
SrcPort string
|
||
DstPort string
|
||
Length uint16
|
||
Payload int
|
||
}
|
||
```
|
||
|
||
#### `ICMPFacts`
|
||
|
||
```go
|
||
type ICMPFacts struct {
|
||
Version int
|
||
Type uint8
|
||
Code uint8
|
||
Checksum uint16
|
||
ID uint16
|
||
Seq uint16
|
||
Payload int
|
||
}
|
||
```
|
||
|
||
#### `ARPFacts`
|
||
|
||
```go
|
||
type ARPFacts struct {
|
||
Operation uint16
|
||
SenderMAC net.HardwareAddr
|
||
TargetMAC net.HardwareAddr
|
||
SenderIP string
|
||
TargetIP string
|
||
}
|
||
```
|
||
|
||
### 4.7 `FlowKey` / `FlowRef`
|
||
|
||
推荐使用结构化 flow，不把字符串 key 当成唯一公共接口。
|
||
|
||
```go
|
||
type Endpoint struct {
|
||
IP string
|
||
Port string
|
||
}
|
||
|
||
type FlowKey struct {
|
||
Family NetworkFamily
|
||
Protocol ProtocolKind
|
||
Src Endpoint
|
||
Dst Endpoint
|
||
}
|
||
|
||
type FlowRef struct {
|
||
Forward FlowKey
|
||
Reverse FlowKey
|
||
Stable string
|
||
}
|
||
```
|
||
|
||
方法：
|
||
|
||
```go
|
||
func (f FlowKey) StableString() string
|
||
```
|
||
|
||
字段：
|
||
|
||
- `Forward`
|
||
- 当前方向
|
||
- `Reverse`
|
||
- 反向方向
|
||
- `Stable`
|
||
- 稳定字符串表达，可用于日志和 map key
|
||
|
||
### 4.8 `Observation`
|
||
|
||
```go
|
||
type Observation struct {
|
||
Packet Packet
|
||
Flow FlowRef
|
||
Hints HintSet
|
||
}
|
||
```
|
||
|
||
- `Packet`
|
||
- 报文事实
|
||
- `Flow`
|
||
- 流引用
|
||
- `Hints`
|
||
- 推断结果
|
||
|
||
## 5. Hint 模型
|
||
|
||
### 5.1 `HintSet`
|
||
|
||
```go
|
||
type HintSet struct {
|
||
Summary SummaryHint
|
||
Tags []Tag
|
||
|
||
TCP *TCPHint
|
||
UDP *UDPHint
|
||
ICMP *ICMPHint
|
||
ARP *ARPHint
|
||
}
|
||
```
|
||
|
||
- `Summary`
|
||
- 摘要代码
|
||
- `Tags`
|
||
- 标签集合
|
||
- 协议专属 hint
|
||
- 放在对应子结构中
|
||
|
||
### 5.2 TCP hints
|
||
|
||
```go
|
||
type TCPHint struct {
|
||
Phase TCPPhase
|
||
Event TCPEvent
|
||
LegacyState uint8
|
||
Seq uint32
|
||
Ack uint32
|
||
Window uint16
|
||
Payload int
|
||
Retransmission bool
|
||
Keepalive bool
|
||
KeepaliveResponse bool
|
||
RST bool
|
||
ECE bool
|
||
CWR bool
|
||
}
|
||
```
|
||
|
||
Phase：
|
||
|
||
- `TCPPhaseHandshake`
|
||
- `TCPPhaseEstablished`
|
||
- `TCPPhaseTeardown`
|
||
- `TCPPhaseSpecial`
|
||
- `TCPPhaseUnknown`
|
||
|
||
Event：
|
||
|
||
- `TCPEventSYN`
|
||
- `TCPEventSYNACK`
|
||
- `TCPEventHandshakeACK`
|
||
- `TCPEventACK`
|
||
- `TCPEventRetransmission`
|
||
- `TCPEventKeepalive`
|
||
- `TCPEventKeepaliveResp`
|
||
- `TCPEventFIN`
|
||
- `TCPEventFINACK`
|
||
- `TCPEventTeardownACK`
|
||
- `TCPEventRST`
|
||
- `TCPEventECE`
|
||
- `TCPEventCWR`
|
||
- `TCPEventUnknown`
|
||
|
||
常见 Tag：
|
||
|
||
- `TagTCPHandshakeSYN`
|
||
- `TagTCPHandshakeSYNACK`
|
||
- `TagTCPHandshakeACK`
|
||
- `TagTCPTeardownFIN`
|
||
- `TagTCPTeardownFINACK`
|
||
- `TagTCPTeardownACK`
|
||
- `TagTCPPacket`
|
||
- `TagTCPRetransmit`
|
||
- `TagTCPKeepalive`
|
||
- `TagTCPKeepaliveResp`
|
||
- `TagTCPRst`
|
||
- `TagTCPEce`
|
||
- `TagTCPCwr`
|
||
|
||
### 5.3 UDP / ICMP / ARP hints
|
||
|
||
#### `UDPHint`
|
||
|
||
```go
|
||
type UDPHint struct {
|
||
Payload int
|
||
}
|
||
```
|
||
|
||
对应 Tag：
|
||
|
||
- `TagUDPPacket`
|
||
|
||
#### `ICMPHint`
|
||
|
||
```go
|
||
type ICMPHint struct {
|
||
Version int
|
||
Type uint8
|
||
Code uint8
|
||
IsEcho bool
|
||
IsEchoReply bool
|
||
IsUnreachable bool
|
||
IsTimeExceeded bool
|
||
}
|
||
```
|
||
|
||
对应 Tag：
|
||
|
||
- `TagICMPPacket`
|
||
- `TagICMPEchoRequest`
|
||
- `TagICMPEchoReply`
|
||
- `TagICMPUnreachable`
|
||
- `TagICMPTimeExceeded`
|
||
|
||
#### `ARPHint`
|
||
|
||
```go
|
||
type ARPHint struct {
|
||
Operation uint16
|
||
Request bool
|
||
Reply bool
|
||
}
|
||
```
|
||
|
||
对应 Tag：
|
||
|
||
- `TagARPRequest`
|
||
- `TagARPReply`
|
||
|
||
## 6. TCP 跟踪边界
|
||
|
||
`Tracker` 对 TCP 只做轻量跟踪，不做完整 TCP 栈模拟。
|
||
|
||
当前覆盖：
|
||
|
||
- 握手识别
|
||
- 挥手识别
|
||
- 普通 ACK 识别
|
||
- 重传识别
|
||
- keepalive 识别
|
||
- keepalive response 识别
|
||
- RST / ECE / CWR 识别
|
||
|
||
实现说明：
|
||
|
||
- 基于 flow 跟踪
|
||
- 使用有限 segment 记忆辅助判断重传
|
||
- keepalive 使用启发式判断
|
||
- 支持超时清理
|
||
|
||
不覆盖：
|
||
|
||
- 严格 TCP 协议验证
|
||
- 深度重组
|
||
- 业务级根因结论
|
||
|
||
## 7. 配置
|
||
|
||
### 7.1 `PacketsConfig`
|
||
|
||
```go
|
||
type PacketsConfig struct {
|
||
ConnectionTimeout time.Duration
|
||
CleanupInterval time.Duration
|
||
}
|
||
```
|
||
|
||
字段：
|
||
|
||
- `ConnectionTimeout`
|
||
- flow 状态保留时长
|
||
- `CleanupInterval`
|
||
- 自动清理周期
|
||
|
||
默认值：
|
||
|
||
```go
|
||
const (
|
||
DefaultConnectionTimeout = 5 * time.Minute
|
||
DefaultCleanupInterval = 1 * time.Minute
|
||
)
|
||
```
|
||
|
||
默认配置：
|
||
|
||
```go
|
||
cfg := bcap.DefaultConfig()
|
||
```
|
||
|
||
### 7.2 生命周期方法
|
||
|
||
`Tracker`：
|
||
|
||
- `CleanupExpiredFlows() int`
|
||
- `ActiveFlowCount() int`
|
||
- `Stop()`
|
||
|
||
`Analyzer`：
|
||
|
||
- `Decoder() *Decoder`
|
||
- `Tracker() *Tracker`
|
||
- `Stop()`
|
||
|
||
备注：
|
||
|
||
- 长生命周期进程退出前调用 `Stop()`
|
||
- 如需自行控制清理节奏，可将 `CleanupInterval` 设为 `0`，再手动调用 `CleanupExpiredFlows()`
|
||
|
||
## 8. 错误模型
|
||
|
||
`bcap` 使用 `*ParseError` 表达解析问题：
|
||
|
||
```go
|
||
type ParseError struct {
|
||
Type ParseErrorType
|
||
Layer string
|
||
Message string
|
||
Err error
|
||
}
|
||
```
|
||
|
||
错误类型：
|
||
|
||
- `ErrTypeLinkLayer`
|
||
- `ErrTypeNetwork`
|
||
- `ErrTypeTransport`
|
||
- `ErrTypeUnsupported`
|
||
|
||
常见触发条件：
|
||
|
||
- 没有有效网络层
|
||
- 协议层对象类型不匹配
|
||
- 跟踪阶段缺少必须的协议事实
|
||
|
||
## 9. 子包
|
||
|
||
### 9.1 `libpcap`
|
||
|
||
`libpcap` 子包负责在线抓包输入。
|
||
|
||
入口：
|
||
|
||
- `FindAllDevs()`
|
||
- `NewCatch(host, filter)`
|
||
- `NewCatchEth(eth, filter)`
|
||
- `SetRecall(func(gopacket.Packet))`
|
||
- `Run()`
|
||
- `Stop()`
|
||
|
||
常见组合：
|
||
|
||
- 子包接收 `gopacket.Packet`
|
||
- 主包 `Analyzer` 做解析和提示
|
||
- 上层工具做展示和统计
|
||
|
||
### 9.2 `nfq`
|
||
|
||
`nfq` 子包负责 NFQUEUE 输入适配。
|
||
|
||
入口：
|
||
|
||
- `NewNfQueue(ctx, queid, maxqueue)`
|
||
- `SetRecall(func(id uint32, q *nfqueue.Nfqueue, p Packet))`
|
||
- `Run()`
|
||
- `Stop()`
|
||
|
||
备注：
|
||
|
||
- `nfq.Packet` 只是 NFQUEUE 输入包装
|
||
- 它不是主包的 `bcap.Packet`
|
||
- 一般仍然从其中取 `gopacket.Packet`，再交给 `Analyzer` 或 `Decoder`
|
||
|
||
## 10. 示例
|
||
|
||
### 10.1 直接用 `Analyzer`
|
||
|
||
```go
|
||
analyzer := bcap.NewAnalyzer()
|
||
defer analyzer.Stop()
|
||
|
||
obs, err := analyzer.ObservePacket(packet)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
|
||
fmt.Println(obs.Flow.Stable)
|
||
fmt.Println(obs.Packet.Transport.Kind)
|
||
fmt.Println(obs.Hints.Summary.Code)
|
||
```
|
||
|
||
### 10.2 自己管理解码和跟踪
|
||
|
||
```go
|
||
decoder := bcap.NewDecoder()
|
||
tracker := bcap.NewTracker()
|
||
defer tracker.Stop()
|
||
|
||
decoded, err := decoder.DecodeWithOptions(packet, bcap.DecodeOptions{
|
||
BaseTime: firstPacketTS,
|
||
})
|
||
if err != nil {
|
||
return err
|
||
}
|
||
|
||
obs, err := tracker.Observe(decoded)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
```
|
||
|
||
### 10.3 典型离线遍历
|
||
|
||
```go
|
||
f, err := os.Open("sample.pcap")
|
||
if err != nil {
|
||
return err
|
||
}
|
||
defer f.Close()
|
||
|
||
reader, err := pcapgo.NewReader(f)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
|
||
source := gopacket.NewPacketSource(reader, reader.LinkType())
|
||
analyzer := bcap.NewAnalyzer()
|
||
defer analyzer.Stop()
|
||
|
||
for packet := range source.Packets() {
|
||
obs, err := analyzer.ObservePacket(packet)
|
||
if err != nil {
|
||
continue
|
||
}
|
||
fmt.Println(obs.Packet.Network.SrcIP, obs.Hints.Summary.Code)
|
||
}
|
||
```
|
||
|
||
## 11. 辅助函数
|
||
|
||
主包还提供两个轻量格式化函数：
|
||
|
||
- `FormatDuration(time.Duration) string`
|
||
- `FormatBytes(uint64) string`
|
||
|
||
用于轻量日志与终端展示。
|
||
|
||
## 12. 迁移说明
|
||
|
||
主包旧接口已经被清理，删除项包括：
|
||
|
||
- `Packets`
|
||
- `PacketInfo`
|
||
- `ParsePacket`
|
||
- `NewPackets`
|
||
- `NewPacketsWithConfig`
|
||
- `LegacyPacketInfoFromObservation`
|
||
- `GetStateDescription`
|
||
- `PrintStats`
|
||
- `ExportConnectionsToJSON`
|
||
|
||
迁移方向：
|
||
|
||
- 旧的单包解析入口迁到 `Decoder`
|
||
- 旧的“包 + 状态”混合对象迁到 `Observation`
|
||
- 旧的连接跟踪职责迁到 `Tracker`
|
||
- 大部分调用场景直接迁到 `Analyzer`
|
||
|
||
旧代码如果大量依赖字符串 key：
|
||
|
||
- 新代码优先改为使用 `FlowRef.Forward` / `FlowRef.Reverse`
|
||
- 如果只是为了日志或 map key，可以继续使用 `FlowRef.Stable`
|
||
|
||
## 13. 相关文档
|
||
|
||
- 快速入口见 [`../README.md`](../README.md)
|
||
- 设计备忘见 [`dev.md`](./dev.md)
|