Skip to content
Provara

Monitoring & Audit

Audit Log

Section titled “Audit Log”

Every command execution is recorded in runtime/audit.log as newline-delimited JSON (JSON-lines format).

Log Entries

Section titled “Log Entries”

Command Executed:

{
  "event": "command_executed",
  "run_id": "20250115_143122_e5f6g7h8",
  "command": "Get-Process python",
  "exit_code": 0,
  "duration_ms": 142.5,
  "ts": "2025-01-15T14:31:22"
}

Command Blocked:

{
  "event": "command_blocked",
  "run_id": "20250115_143200_x1y2z3w4",
  "reason": "deny:\\b(Remove-Item|del)",
  "command": "Remove-Item C:\\Windows\\*",
  "ts": "2025-01-15T14:32:00"
}

Fields

Section titled “Fields”
FieldDescription
eventEvent type: command_executed or command_blocked
run_idUnique execution identifier
commandThe command (truncated to 200 chars)
exit_codeProcess exit code (only for executed commands)
duration_msExecution time (only for executed commands)
reasonBlock reason (only for blocked commands)
tsISO 8601 timestamp

Execution Records

Section titled “Execution Records”

Each completed execution is stored as a JSON file in runtime/runs/{run_id}/record.json:

{
  "run_id": "20250115_143122_e5f6g7h8",
  "cwd": "C:\\provara",
  "command": "Get-Process python",
  "exit_code": 0,
  "stdout": "Handles  NPM(K)    PM(K)...",
  "stderr": "",
  "ts": 1705329082.456,
  "duration_ms": 142.5
}

This includes full stdout/stderr output, which the audit log does not.

Health Endpoint

Section titled “Health Endpoint”

The /health endpoint provides real-time operational metrics:

Terminal window
Invoke-RestMethod http://127.0.0.1:8787/health
{
  "status": "healthy",
  "uptime_seconds": 3621.4,
  "memory_mb": 42.7,
  "pending_count": 3,
  "runs_count": 15,
  "timestamp": "2025-01-15T14:30:22"
}

Key metrics:

  • uptime_seconds — Time since server start
  • memory_mb — Server process memory usage (via psutil)
  • pending_count — Commands awaiting approval
  • runs_count — Total execution records

Real-Time Events (SSE)

Section titled “Real-Time Events (SSE)”

The /events endpoint provides Server-Sent Events for live monitoring:

const events = new EventSource('http://127.0.0.1:8787/events');
events.onmessage = (e) => console.log(JSON.parse(e.data));

Event types:

  • run_started — Command execution begins
  • run_finished — Command execution completes (includes exit code, output)

Monitoring Scripts

Section titled “Monitoring Scripts”

Check Pending Queue Depth

Section titled “Check Pending Queue Depth”
Terminal window
$token = Get-Content runtime\agent_hub_token.txt
$headers = @{ "X-Agent-Token" = $token }


$pending = Invoke-RestMethod http://127.0.0.1:8787/pending -Headers $headers
Write-Host "Pending commands: $($pending.Count)"


# Alert if queue is backing up
if ($pending.Count -gt 10) {
    Write-Warning "High pending queue: $($pending.Count) commands"
}

Audit Log Analysis

Section titled “Audit Log Analysis”
Terminal window
# Count executions in the last hour
$oneHourAgo = (Get-Date).AddHours(-1).ToString("yyyy-MM-ddTHH:mm")
Get-Content runtime\audit.log |
    ConvertFrom-Json |
    Where-Object { $_.ts -gt $oneHourAgo -and $_.event -eq "command_executed" } |
    Measure-Object


# Find all blocked commands
Get-Content runtime\audit.log |
    ConvertFrom-Json |
    Where-Object { $_.event -eq "command_blocked" } |
    Select-Object ts, reason, command


# Average execution time
Get-Content runtime\audit.log |
    ConvertFrom-Json |
    Where-Object { $_.event -eq "command_executed" } |
    Measure-Object -Property duration_ms -Average

Recent Runs Summary

Section titled “Recent Runs Summary”
Terminal window
Get-ChildItem runtime\runs -Directory |
    Sort-Object LastWriteTime -Descending |
    Select-Object -First 10 |
    ForEach-Object {
        $record = Get-Content "$($_.FullName)\record.json" | ConvertFrom-Json
        [PSCustomObject]@{
            RunId      = $record.run_id
            Command    = $record.command.Substring(0, [Math]::Min(50, $record.command.Length))
            ExitCode   = $record.exit_code
            DurationMs = [Math]::Round($record.duration_ms, 1)
        }
    } | Format-Table

Server Logging

Section titled “Server Logging”

The API server logs to stdout with structured format:

2025-01-15 14:30:22 | INFO     | src.server.main | Agent Hub API starting...
2025-01-15 14:30:22 | INFO     | src.server.main | Token required: True
2025-01-15 14:31:00 | INFO     | src.server.main | Queued command 20250115_143100_abc: whoami...
2025-01-15 14:31:15 | INFO     | src.server.exec.runner | [20250115_143100_abc] Executing: whoami
2025-01-15 14:31:15 | INFO     | src.server.exec.runner | [20250115_143100_abc] Completed in 52.3ms

Set AGENT_HUB_LOG_LEVEL=DEBUG for verbose output including policy pattern matches.

Denied Commands Archive

Section titled “Denied Commands Archive”

Denied commands are stored in runtime/denied/ with additional metadata:

{
  "command": "Remove-Item C:\\important",
  "note": "agent cleanup request",
  "pending_id": "20250115_143022_a1b2c3d4",
  "ts": 1705329022.123,
  "denied_ts": 1705329045.678,
  "denied_at": "2025-01-15T14:30:45"
}

Next Steps

Section titled “Next Steps”