How to Configure Apple Unified Logs (os_log) Streaming to Splunk via Fluent-Bit

The macOS unified logging system (os_log) is a profound leap forward in system telemetry compared to traditional POSIX text logs (like /var/log/system.log). Introduced in macOS Sierra, it centralizes all kernel, daemon, and application logs into a highly compressed, binary format. While incredibly efficient for local storage, this binary architecture creates a massive problem for enterprise security teams: you cannot easily ingest os_log data into a SIEM (Security Information and Event Management) platform like Splunk using standard syslog forwarders.

Attempting to run log show --stream via a shell script and piping it to a file is unreliable, resource-intensive, and prone to formatting errors.

The modern, architecturally sound solution is to deploy a lightweight data shipper—specifically Fluent-Bit—configured to natively execute the macOS log binary, parse the JSON output in memory, and stream it securely to a Splunk HTTP Event Collector (HEC) endpoint.

This guide explains how to architect this pipeline and configure Fluent-Bit to ship macOS unified logs in real-time.

The Architecture of the macOS Log Pipeline

The pipeline consists of three stages:

1. Extraction: We utilize the built-in macOS log command-line utility. Crucially, we use the --style json flag to ensure the output is structured, rather than raw text.
2. Ingestion & Parsing: Fluent-Bit (running as a LaunchDaemon) executes the log command via its exec input plugin. Fluent-Bit parses the resulting JSON stream, allowing for filtering (e.g., dropping noisy subsystem logs).
3. Transmission: Fluent-Bit routes the structured JSON payloads to Splunk via HTTPS using the Splunk HEC output plugin.

Step 1: Installing Fluent-Bit on macOS

Unlike Linux, macOS does not have a native package manager integrated by Apple. While Homebrew is popular for developers, deploying software across an enterprise fleet requires native PKG installers deployed via MDM (e.g., Jamf Pro).

For this guide, you can compile Fluent-Bit or use pre-compiled binaries provided by the community (or Calyptia).

Assuming the fluent-bit binary is installed at /opt/fluent-bit/bin/fluent-bit, and the configuration file is at /opt/fluent-bit/etc/fluent-bit.conf.

Step 2: Configuring the Fluent-Bit Input (The exec Plugin)

The core of the solution is the exec input plugin. We will instruct Fluent-Bit to run a continuous log stream command.

Open fluent-bit.conf and define the input:

[INPUT]
    Name           exec
    Tag            macos.unified_log
    Command        log stream --style json --predicate 'messageType == info OR messageType == error'
    Parser         json
    Interval_Sec   1
    Interval_NSec  0

Critical Consideration: The Predicate Filter
The macOS unified log generates thousands of events per second. If you stream the entire raw firehose to Splunk, you will obliterate your indexing license and network bandwidth. You must use the --predicate flag to filter events at the source.

A better, security-focused predicate might target the Endpoint Security subsystem or authentication events:

Command log stream --style json --predicate 'subsystem == "com.apple.EndpointSecurity" OR subsystem == "com.apple.Authorization"'

Step 3: Configuring the Splunk HEC Output

Next, configure Fluent-Bit to send the tagged JSON data to Splunk.

You must have a Splunk HTTP Event Collector (HEC) token generated in your Splunk deployment.

Add the output block to fluent-bit.conf:

[OUTPUT]
    Name           splunk
    Match          macos.unified_log
    Host           splunk.internalcorp.com
    Port           8088
    TLS            On
    TLS.Verify     On
    Splunk_Token   xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    Splunk_Send_Raw Off
    Format         json

Setting Splunk_Send_Raw Off ensures that Fluent-Bit wraps the log event in the specific JSON structure ({ "event": { ... } }) required by the Splunk HEC API.

Step 4: Deploying as a macOS LaunchDaemon

Fluent-Bit must run continuously in the background as root (since reading the unified log requires elevated privileges). This requires a macOS LaunchDaemon.

Create a property list file (plist) at /Library/LaunchDaemons/com.fluentbit.unifiedlog.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.fluentbit.unifiedlog</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/fluent-bit/bin/fluent-bit</string>
        <string>-c</string>
        <string>/opt/fluent-bit/etc/fluent-bit.conf</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/var/log/fluent-bit.err</string>
    <key>StandardOutPath</key>
    <string>/var/log/fluent-bit.log</string>
</dict>
</plist>

Load the daemon into the system to start the streaming process:

sudo launchctl load -w /Library/LaunchDaemons/com.fluentbit.unifiedlog.plist

Conclusion

By leveraging Fluent-Bit’s exec plugin, security engineers can unlock the profound telemetry buried within Apple’s proprietary unified logging format. Translating the binary stream into structured JSON directly at the endpoint and streaming it via HEC allows Splunk to ingest, index, and alert on critical macOS security events in true real-time, bridging the gap between Apple’s ecosystem and enterprise SOC operations.