Connectors

Connectors are infrastructure integrations that ingest external events into Flowman. They are not workflows, and they do not contain workflow mappings. A connector monitors an external system, applies ingestion filters, normalizes matching records, stores them in Convex, and emits platform events for the trigger engine.

Control plane and execution plane

• Flowman web is the control plane: create, edit, enable, disable, test, and inspect connector state from /p/<slug>/connectors.
• connector-runtime is the execution plane: a continuously running Bun process in apps/connector-runtime.
• The runtime subscribes to Convex sync signals over a WebSocket, loads connectors that are both enabled and polling-enabled when configuration changes, starts one isolated async worker per connector, and stops workers when either flag is turned off.

Ingestion vs routing

Connector configuration answers how do we access and ingest this external source? Trigger rules answer what workflow should run because of this event? Keep those separate. The connector stays reusable and transport-focused; workflow decisions live under the project Triggers section.

Supported connectors

• IMAP: poll an email mailbox, apply ingestion filters, store normalized emails, and emit email.received events.
• GitHub: connect a GitHub App installation so a run can read live pull request status on demand and finish when the PR is merged. Unlike IMAP, this is an on-demand integration, not a polling worker.

Runtime configuration

Connector-runtime can run by itself. In a split deployment, run one connector-runtime host for polling connectors, then run coding-agent runtimes separately on the VPS hosts that have repository checkouts.

pnpm runtime:connector

For local all-in-one development, the supervisor still starts both connector-runtime and coding-agent-runtime, restarts them after unexpected exits, and watches local process heartbeat files for stuck processes.

pnpm runtime:supervisor

Direct runtime scripts, the supervisor, and child runtimes load .env.local. Connector hosts require CONNECTOR_RUNTIME_TOKEN and NEXT_PUBLIC_CONVEX_URL. Coding-agent hosts require CODING_AGENT_RUNTIME_TOKEN, FLOWMAN_ENVIRONMENT, and the same Convex URL. Create or rotate the token from Project settings before starting the host.

The supervisor starts child runtimes with compact logs by default. Set LOG_PRETTY=true for compact terminal logs in direct runtime processes, and LOG_LEVEL=info or another Pino level to control verbosity. Pretty logs are color-coded on TTY output; set LOG_COLOR=true to force colors or LOG_COLOR=falseto disable them. Without LOG_PRETTY, runtimes emit structured JSON logs.

Connector runtime options

• Config sync is event-driven: Flowman web enqueues connectorRuntimeSignals when connectors are created, updated, enabled, disabled, or reset. Connector-runtime watches those signals and reloads enabled connectors only when they change.
• CONNECTOR_WORKER_HEALTH_MS: local interval for restarting crashed connector workers without polling Convex (default 30s).
• IMAP pollingInterval still controls how often each worker polls the mailbox. That is external to Convex and unchanged by the event-driven config sync.

Supervisor options

• RUNTIME_SUPERVISOR_SERVICES: comma-separated services to run, such as connector,coding.
• RUNTIME_SUPERVISOR_HEARTBEAT_MS: how often each runtime writes .runtime/heartbeats/*.json.
• RUNTIME_SUPERVISOR_STALE_MS: heartbeat age before the supervisor restarts a runtime.
• RUNTIME_SUPERVISOR_RESTART_BACKOFF_MS: initial restart backoff after unexpected exits.
• RUNTIME_SUPERVISOR_SHUTDOWN_GRACE_MS: graceful shutdown window before force-killing a child process.

Core tables

• connectors: project-scoped connector configuration, authentication settings, polling controls, and ingestion filters. IMAP connector configuration is normalized and redacted by a single shared module before it reaches UI, workflow, or runtime callers.
• connectorWorkerState: checkpoint, worker status, heartbeat, and last error.
• connectorRuntimeSignals: event-driven wakeups for connector-runtime config sync.
• connectorRuns: polling run history, processed counts, matched counts, checkpoints, and errors.
• Source-specific records, such as emails, are documented on the connector type page.