Quickstart: Connect Kafka in 15 Minutes

This guide walks through connecting a Kafka broker, selecting topics to monitor, and triggering a test schema detection. By the end you'll have Schema Guard running on at least one topic.

Prerequisites

  • A Streamforge account (free to create at streamforgehq.com)
  • Kafka bootstrap server address and read-only credentials (SASL/PLAIN or mTLS)
  • At least one active Kafka topic with recent traffic

Step 1: Create your account

Sign up at streamforgehq.com. No payment info required. Your free Solo tier activates immediately with one pipeline slot.

Step 2: Add your Kafka broker

In your workspace, navigate to Pipelines and click Add pipeline. Select Kafka as the broker type.

Connection details 
Bootstrap servers:  kafka.internal.example.com:9092
Security protocol:  SASL_SSL
SASL mechanism:     PLAIN
Username:          streamforge-readonly
Password:          ••••••••••••

Streamforge uses read-only credentials. We never write to your broker, never modify your consumer groups or topic configurations.

Step 3: Select topics to monitor

After connecting, Streamforge lists all available topics. Select the topics you want Schema Guard to watch. For your first setup, start with one high-value topic (e.g., an orders or events topic).

Topic selection output 
CONNECTED  kafka.internal.example.com:9092
Found 24 topics

Selected for monitoring:
  orders-v2.checkout      # 2,847 msg/min
  payments.events         # 1,203 msg/min

SAMPLING  Schema baseline building... (24h to complete)

Step 4: Wait for baseline (24 hours)

Streamforge samples up to 10,000 events per topic to build the schema fingerprint. You can check progress in the Dashboard. After baseline completes, Schema Guard starts validating every incoming event.

Step 5: Set your alert channel

Go to Workspace settings > Alerts and add a destination. Here's an example Slack webhook configuration:

JSON 
POST /api/v1/workspace/alerts
{
  "type": "slack",
  "webhook_url": "https://hooks.slack.com/services/T00/B00/xxx",
  "dedup_window_minutes": 15,
  "test_fire": true
}

Setting test_fire: true triggers a test alert immediately so you can confirm Slack delivery before any real incident.

Step 6: Trigger a test detection

Once baseline is complete, you can trigger a test schema violation using our test producer tool. This sends one event with a deliberate null field violation so you can confirm the detection pipeline is working end-to-end.

CLI 
# Install the Streamforge CLI
npm install -g @streamforge/cli

# Run a test violation on a topic
sfhq test-violation \
  --topic orders-v2.checkout \
  --violation null_field \
  --field event_id

Expected output in your Slack channel within 90 seconds:

Slack alert 
[Streamforge] Schema drift detected
Topic:      orders-v2.checkout
Field:      event_id
Violation:  null (expected: string, required)
Action:     quarantined to orders-v2.checkout.dlq
View in DLQ Inspector: app.streamforgehq.com/dlq/...

Next steps

  • Add more topics to monitor from the Pipelines dashboard
  • Configure routing rules for each topic (quarantine vs warn vs block)
  • Review the API Reference to automate pipeline management
  • Upgrade to Team tier for multi-channel alerts and 30-day DLQ retention