Harbor Lighthouse: We Fixed What Everyone Hates About Telemetry Collection

Remember when we open-sourced Telemetry Harbor? We built a complete, production-ready platform for ingesting, storing, and visualizing telemetry data. people loved it. But we kept hearing the same question:

"This is incredible for the backend... but what about actually collecting and sending the data?"

And they were right. We solved the hard problem of building a scalable time-series database with beautiful visualizations. But we left people stuck with the same old pain points:

• Writing custom Python scripts that break in production
• Managing different agents for different data sources
• Dealing with network failures and lost data
• Manual configuration on every single device

Today, that ends.

Meet Harbor Lighthouse: The telemetry agent that actually makes sense. It's open source, written in Go for blazing performance, and it works seamlessly with both our cloud platform and your self-hosted OSS deployment.

The Problem With Existing Solutions

Let's be honest about the current state of telemetry collection:

Telegraf: Powerful But Painful

Telegraf is plugin-driven with over 300 available plugins, which sounds great until you realize you need to:

• Learn TOML configuration syntax
• Figure out which of those 300 plugins you actually need
• Manually configure each input, processor, aggregator, and output
• Update and maintain configs across hundreds of devices
• Hope your specific use case has a plugin (and if not, write your own)

Don't get us wrong, Telegraf is a solid tool. But it assumes you have time to become a configuration expert.

OpenTelemetry Collector: Comprehensive But Complex

Managing separate agents for metrics, logs, and traces creates operational complexity. The OpenTelemetry Collector tries to solve this with a unified approach, but introduces its own challenges:

• Complex multi-component architecture (receivers, processors, exporters, connectors)
• Significant learning curve for proper deployment
• Resource overhead that grows with scale
• Different deployment patterns trading off complexity, scalability, and isolation

Custom Scripts: "It'll Just Take 5 Minutes"

We've all been there. You think: "I'll just write a quick Python script to collect this data." Three weeks later:

• The script crashes when WiFi drops
• You're dealing with retry logic and exponential backoff
• Memory leaks appear under load
• You need to deploy it on 50 servers and realize you forgot logging
• The original developer left and nobody understands the code

There had to be a better way.

What We Built Instead

Harbor Lighthouse is the telemetry agent we wish existed when we started building Harbor Scale. Here's what makes it different:

One Command. Seriously.

Install Lighthouse on any system:

# Linux, Mac, Raspberry Pi
curl -sL get.harborscale.com | sudo bash

# Windows PowerShell (as Administrator)
iwr get.harborscale.com | iex

That's it. You now have a telemetry agent running as a system service.

Configuration That Makes Sense

No TOML files. No YAML nightmares. Just straightforward commands:

# Monitor a server
lighthouse --add \
  --name "production-server-01" \
  --harbor-id "123" \
  --key "your_api_key" \
  --source linux

# Monitor a website
lighthouse --add \
  --name "website-monitor" \
  --harbor-id "123" \
  --key "your_api_key" \
  --source uptime \
  --param target_url="https://example.com"

# Monitor Docker containers
lighthouse --add \
  --name "docker-host" \
  --harbor-id "123" \
  --key "your_api_key" \
  --source docker

Done. Three monitors configured in three commands. Each one is independently tracked, managed, and monitored.

Built-In Sources That Actually Work

Lighthouse comes with native collectors for the things people actually monitor:

• System monitors (linux, windows, macos): CPU, RAM, disk, network, uptime
• Docker Engine (docker): Container state, uptime, resource usage per container
• HTTP Uptime (uptime): Website availability, latency, response codes
• AI Agents (ollama): Track your local LLM farm's VRAM usage and model loading
• Starlink (starlink): Signal health, obstructions, packet loss
• Meshtastic LoRa (meshtastic): Battery, environmental data, signal stats for mesh networks

Unlike Telegraf's plugin sprawl, these are curated, tested, and work out of the box.

The Secret Weapon: exec Mode

Here's where Lighthouse gets really interesting. The exec collector lets you turn any script into a telemetry stream:

lighthouse --add \
  --name "custom-metrics" \
  --harbor-id "123" \
  --key "your_api_key" \
  --source exec \
  --param command="python3 /opt/your-script.py"

Your script just needs to output JSON to STDOUT. That's it. No SDK, no library, no integration hell.

Single device:

{
  "temperature": 24.5,
  "humidity": 60
}

Multiple devices (gateway mode):

[
  {
    "ship_id": "sensor_living_room",
    "temperature": 22.0
  },
  {
    "ship_id": "sensor_kitchen",
    "temperature": 25.5
  }
]

Python, Bash, Node, Rust, a compiled binary, doesn't matter. If it can print JSON, Lighthouse can collect it.

Network Resilience Built In

Unlike fragile custom scripts, Lighthouse handles the hard networking problems:

• Automatic retry with exponential backoff
• Local queuing when network is down
• Batch uploads to reduce overhead
• Graceful degradation under load

You don't write any of this. It just works.

Self-Hosted or Cloud, Your Choice

Lighthouse works with both:

Harbor Scale Cloud:

lighthouse --add \
  --name "server-01" \
  --harbor-id "123" \
  --key "hs_live_key_xxx" \
  --source linux

Self-Hosted OSS:

lighthouse --add \
  --name "local-server" \
  --endpoint "http://192.168.1.50:8000" \
  --key "your_oss_api_key" \
  --source linux

Same agent. Same features. Just change the endpoint.

Auto-Updates That Don't Break Things

Lighthouse checks for updates every 24 hours and upgrades itself automatically.

Don't want auto-updates? Turn them off:

lighthouse --autoupdate=false

Monitor Management Built In

Managing one monitor is easy. Managing 100 monitors is where most solutions fall apart. Lighthouse makes it simple:

# See all monitors and their health
lighthouse --list

# Check logs for a specific monitor
lighthouse --logs "production-server-01"

# Remove a monitor
lighthouse --remove "production-server-01"

Every monitor is independently manageable. No master configuration file to break.

Why We Built It In Go

We learned our lesson when we rewrote Harbor's ingest pipeline from Python to Go and got 10x performance improvement.

For Lighthouse:

• Single binary: No Python dependencies, no pip installs, no virtual environments
• Low memory footprint: Runs on a Raspberry Pi Zero without breaking a sweat
• Fast startup: Milliseconds, not seconds
• Cross-compilation: Build for any platform from any platform
• Concurrent by design: Go's goroutines make handling multiple monitors trivial

Real-World Use Cases

IoT Gateway with Meshtastic

You have a network of LoRa devices using Meshtastic. Each node reports battery, temperature, humidity, signal strength. How do you get all that data into a central monitoring system?

With Lighthouse:

# Install the Meshtastic engine
curl -sL get.harborscale.com/meshtastic | sudo bash

# Configure Lighthouse
lighthouse --add \
  --name "meshtastic-gateway" \
  --harbor-id "123" \
  --key "your_api_key" \
  --source exec \
  --param command="mesh_engine --ttl 3600"

Done. Your gateway is now forwarding telemetry from every node in your mesh network.

Multi-Cloud Infrastructure

You run services across AWS, Azure, and on-premises servers. Each environment has different monitoring needs:

# AWS EC2 instances
lighthouse --add --name "aws-web-1" --source linux

# Azure VMs
lighthouse --add --name "azure-db-1" --source linux

# On-prem Docker hosts
lighthouse --add --name "onprem-docker-1" --source docker

# On-prem websites
lighthouse --add --name "website-check" --source uptime \
  --param target_url="https://internal-app.company.com"

One agent, unified monitoring across your entire infrastructure.

Development Team Monitoring

Your team runs multiple projects, each with their own monitoring needs:

# Production database metrics (custom script)
lighthouse --add --name "prod-db" --source exec \
  --param command="/opt/scripts/postgres-metrics.py"

# Staging API health check
lighthouse --add --name "staging-api" --source uptime \
  --param target_url="https://staging-api.company.com/health"

# Development server system metrics
lighthouse --add --name "dev-server" --source linux

Each team member can add monitors independently without breaking anyone else's configuration.

How It Compares

How It Fits Together

Harbor Lighthouse completes the puzzle. It sits at the edge of your infrastructure, feeding data into the platform you already know.

• Harbor Lighthouse (The Collector) → Runs on your devices.
• Harbor Scale Cloud / Telemetry Harbor OSS  (The Backend) → Ingests, stores, and queries data.
• Grafana (The View) → Visualizes your metrics.

Whether you use our managed Harbor Scale Cloud or deploy the Open Source Stack yourself, Lighthouse works exactly the same way.

Get Started

Choose your deployment path. You can be up and running in under a minute.

Option A: The 60-Second Cloud Start

Install the Agent

curl -sL get.harborscale.com | sudo bash

Add new monitor

lighthouse --add \
  --name "my-first-monitor" \
  --harbor-id "YOUR_HARBOR_ID" \
  --key "YOUR_API_KEY" \
  --source linux

Option B: The Self-Hosted OSS Path

Install The OSS Version

git clone https://github.com/HarborScale/telemetry-harbor-oss.git
cd telemetry-harbor-oss && docker compose up -d

#Please change the default values!

Add new monitor

curl -sL get.harborscale.com | sudo bash

lighthouse --add \
  --name "local-monitor" \
  --endpoint "http://localhost:8000" \
  --key "your_oss_api_key" \
  --source linux

That’s it. Check your logs with lighthouse --logs "monitor-name" and watch the data flow.

It's All Open Source

• Harbor Lighthouse: github.com/HarborScale/harbor-lighthouse
• Harbor Lighthouse Installer: github.com/HarborScale/harbor-lighthouse-installer
• Telemetry Harbor OSS: github.com/HarborScale/telemetry-harbor-oss

Fork it. Modify it. Deploy it anywhere. Build commercial products with it. We want you to succeed.

Why We're Doing This

We didn't build Lighthouse to kill Telegraf or compete with OpenTelemetry. We built it because we saw people struggle with the same problems over and over:

• Senior engineers spending days debugging TOML files
• DevOps teams managing configuration drift across hundreds of servers
• Developers writing the same network retry logic for the hundredth time
• Companies abandoning observability projects because they're "too complex"

Monitoring your infrastructure shouldn't be harder than building your infrastructure.

The telemetry space needed a tool that:

• Just works out of the box
• Scales from 1 device to 10,000
• Handles the hard networking problems automatically
• Doesn't require a PhD to configure
• Supports both cloud and self-hosted deployments
• Is fast, reliable, and free

So we built it. And now it's yours.

What's Next

• More native sources: PostgreSQL, MySQL, Redis, MongoDB collectors
• Plugin system: Community-contributed collectors (but keeping the core simple)

But we're letting the community guide the roadmap. What do you need? Open an issue. Want to contribute? PRs welcome.