eBPF

Статьи в разделе

• Tetragon Filemon

  Tetragon File Monitoring

eBPF File monitoring

History

Early approach: periodic scanning of the file-system and comparing the expected state with the actual state. Limitations:

• can only be used to detect modifications and not reads to files;
• unreliable because a modification can go undetected if the file is modified, then returned back to its original state before the scanning occurs:
  • A quick attacker can read/modify target file, and clean up their tracks before the periodic scan.

Later approach: in-kernel inotify:

• addresses unreliability - executed inline with the operation;
• no way to associate or filter operations using the execution context (e.g., pid or cgroup) of the process doing the operation -> no way to filter events based on which Kubernetes workload performed the file access;
• lack of flexibility in the actions taken when a file is accessed:
  • When a monitored file is accessed,  it will send an event to user-space and it’s up to the user-space agent to do the rest:
    • Example: when monitoring the directory /private/data, the sequence of operations would be:

1. Agent adds /private into the directories to be watched
2. Application creates /private/data directory
3. inotify sends an event to the agent that a directory /private/data was created
4. Agent adds /private/data to the directories to be watched

If a file was created and/or accessed in /private/data/* between steps 2 and 4, there will be no inotify event for the access prior to it being added to the watch list.

• Possible to modify the kernel to add execution context to inotify events:
  • long process, might take years until a new kernel reaches production.

eBPF allows FIM implementation to correlate file access events with execution context such as process information (e.g., credentials) and its cloud native identity (e.g., k8s workload), perform inline updates to its internal state to avoid races, as well as implement inline enforcement on file operations.

Path-based FIM with eBPF

Install eBPF hooks (kprobes, specifically) to track file operations and implement File Integrity Monitoring (FIM).

• install these hooks into system calls: the open system call to determine when a file is opened and for what access (read or write):
  • Hooking into system calls, however, might lead to time-of-check to time-of-use (TOCTOU) issues: the memory location with the pathname to be accessed belongs to user-space, and user-space can change it after the hook runs, but before the pathname is used to perform the actual open in-kernel operation:

Hooking into a (kernel) function that happens after the path is copied from user-space to kernel-space avoids this problem since the hook operates on memory that the user-space application cannot change. Hence, instead of a system call we will install a hook into a security_function. Specifically, we will hook into the security_file_permission function which is called on every file access (there is also security_file_open which is executed whenever a file is opened). Information about the process and its parent such as binary, arguments, credentials, and others. In cloud-native environments, the events also contain information about the container and the pod that this process belongs to.

If no filtering is applied, we get a file-all policy: generate events for every file access:

• Many file accesses happen in a system at any point in time, and monitoring all of them is not a good practice because generating an event for each one incurs significant overhead;
• file-all policy does not inform users about what file was actually accessed;
•  We create a second version of the policy where sensitive ssh server private keys are monitored:

Tetragon policy with filtering of SSH keys:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "sensivite-files"
spec:
  kprobes:
  - call: "security_file_permission"
	syscall: false
	args:
	- index: 0
  	   type: "file" #(struct file *) получаем путь
	selectors:
	- matchArgs: 	 
  	- index: 0
    	operator: "Equal"
    	values:
    	- "/etc/shadow"
    	- "/etc/sudoers"
    	- "/etc/ssh/ssh_host_ecdsa_key"

• Filtering in-kernel & deciding at the eBPF hook whether the event is of interest to the user or not, means that no pointless events will be generated and processed by the agent;
• The alternative is to do filtering in user-space tends to induce significant overhead for events that happen very frequently in a system (such as file access). For more details, see Tetragon’s 1.0 release blog post.

 * security_file_permission - the eBPF hook is called on every file access in the system  * Use security_file_open and have the eBPF hook be executed whenever a file is opened. However, it means that if a file is already opened before the hook is installed, the hook will not be called and certain accesses may be missed  * Hooks into other functions such as security_file_truncate or security_file_ioctl for other operations;

eBPF lets you do observability and do inline enforcement by stopping an operation from happening by modifying the request.

Example of denying /usr/bin.cat ssh files access:

      matchBinaries:
      - operator: "In"
        values:
        - "/usr/bin/cat"
      matchActions:
      - action: Override
        argError: -1

It is impossible to do proper enforcement without in-kernel filtering, because by the time the event has reached user-space it is already too late if the operation has already executed.

Path FIM limitations

• Paths are taken from from struct file arguments of functions such as security_file_open;
• The same file can have multiple names in a Linux system:
  • If a policy monitors /etc/ssh/ssh_host_rsa_key but the same underlying file is accessed via a different name, the access will go unnoticed;
  • Same file can have multiple names are hard links, bind mounts, and chroot.
• Hard link to the file /etc/ssh/ssh_host_rsa_key named, for example, /mykey accesses via /mykey will not be caught by policies such as file-ssh-keys:
  • Creating hard links requires appropriate permissions (when  fs.protected_hardlinks  is set to 1, creating a link requires certain permissions on the target file);
  • bind mount requires CAP_SYS_ADMIN;
  • chroot requires CAP_CHROOT.
• We need the ability to monitor file accesses regardless of the name with which the file is accessed.

inode-based FIM with eBPF

An inode number uniquely identifies an underlying file in a single filesystem. Example:

# stat /etc/ssh/ssh_host_ecdsa_key | grep Inode
Device: 259,2   Inode: 36176340	Links: 1
# ln /etc/ssh/ssh_host_ecdsa_key /key
# stat /key | grep Inode
Device: 259,2   Inode: 36176340	Links: 2
# touch /key2
# mount --bind /etc/ssh/ssh_host_ecdsa_key /key2
# stat /key2 | grep Inode
Device: 259,2   Inode: 36176340	Links: 2

Диаграмма работы сканера

sequenceDiagram
autonumber
actor U as User
participant A as Агент
participant S as Сканнер
participant F as Файл
participant B as Программа eBPF
participant I as Карта inodes
U->>A: политика
activate A
activate U
A->>S: шаблон
S->>F: получить inode
activate F
activate S
F-->>S: inode
S->>I: обновить список inodes
deactivate S
activate I
F->>B: событие
deactivate F
activate B
loop Синк в ядре
B->>I: запрос списка inodes
I-->>B: список inodes
end
deactivate I
B-->>A: событие
deactivate B
A-->>U: уведомление
deactivate A
deactivate U