1
Connect
Each connector brings its own fields, validation, connection test, and catalog discovery.
Data Pipelines You Configure, Not Code
CARBON™ is GSPANN's metadata-driven ingestion and ETL platform. One YAML file defines the source, target, extraction strategy, load pattern, mappings, and audit settings. Run it from the console, or hand it to the copilot and your own AI agents.
A new data source usually costs your team in one of two ways. Build it in Python or Spark, and your engineers maintain that code for as long as the pipeline runs. Buy a managed service, and your pipeline logic moves into its platform, billed by usage. CARBON™ is a custom-grade framework delivered as a product: each pipeline is a YAML file your team owns and can change.
Key Capabilities
What Changes for Your Data Team
No-Code Configuration
Declare source-to-target mappings in YAML. Guided screens generate and preview the YAML before you create a pipeline, or you can upload your own.
Extensible Connectors
Relational databases, SaaS APIs, cloud storage, and warehouses. Add a source by configuration. Forms are rendered from connector metadata.
Unified Audit and Monitoring
Every job logs and reports errors in the same format. Run history, row counts, throughput, and live logs sit in one console.
Portable Runtime
Open PySpark and SQL on any Spark-compliant runtime, so you can shift for cost or speed.
Duplicate-Run Protection
A running pipeline cannot be launched twice. Run state survives restarts, and orphaned runs are reconciled when the backend starts.
Built-In Catalog Discovery
Browse and search large catalogs of tables and objects. Columns and sample data load only when you ask for them.
Why CARBON
The Fourth Approach to Data Integration
Weigh time to first run against control over your pipeline logic.
| Approach | Examples | Trade-Off |
|---|---|---|
| Managed SaaS | Fivetran, Workato | Fast to start. Your logic lives in their platform, and the bill grows with usage. |
| Cloud-Native | Azure Data Factory | Flexible within one cloud, but you build inside their box. |
| Custom Build | Python, Spark | Total control, but slow to build and yours to maintain forever. |
| ✓CARBON™ | A framework, delivered | The flexibility of code with the speed of a platform. You can change any pipeline file and add your own connector modules. |
How It Works
From Source to Warehouse, Configured in YAML
Connectors, load logic, scheduling, and audit live in one configuration per pipeline. Standard ingestion patterns need no custom Spark code.
source: connection: salesforce_prod extract_type: web_service object: Account target: connection: snowflake_dw table: crm.customer ingestion_pattern: delta_watermark_sync mappings: - {from: Phone, to: phone_number} audit: {enabled: true}
successcrm_customer_20260827_184501triggered by: console
2
Extract
Three extraction types, set by the connector you pick.
3
Load
Five load patterns, from atomic replace to append-only history.
4
Deliver
Land data in the cloud warehouse you already run.
Governed runs. Every run is audited: source, target, row counts, status, and timing.
Scheduled or on demand. Run on a cron schedule or trigger a pipeline when you need it.
Usage
Built for People and AI Agents
All three interfaces reach the same capabilities. The copilot calls the same Model Context Protocol (MCP) tools that external agents use, so one set of permissions governs both.
Console
A full interface for teams who prefer screens.
- ✓ Executions: run pipelines, schedule them, watch live status, and cancel a run
- ✓ Integrations: create and edit YAML through a guided workflow
- ✓ Connections: test a connection before saving it and see where it is used
- ✓ Logs: search across pipelines
AI Copilot
Build and run pipelines by asking in plain language.
- ✓ Inspect runs, logs, schedules, and failures
- ✓ Explore real tables and suggest mappings
- ✓ Build and edit configurations
- ✓ Start and cancel runs, after you approve
- ✓ Build and Run work only when your deployment turns them on
MCP Server
Connect any MCP client, such as Claude Code or a Slack bot, to 12+ typed tools.
- ✓ Read-only by default
- ✓ Optional write tools to author pipelines
- ✓ Optional execution tools to run them
- ✓ Tokens scoped to pipeline patterns
Key Features
Run and Monitor Every Pipeline From One Console
Real screens from the CARBON app. Click any screen to see it full size.
Executions
Run any pipeline and watch its status and throughput live. The last five runs show as colored squares, and clicking one opens that run.
- ✓ Every process YAML is listed, including pipelines that have never run
- ✓ Schedule runs by the minute, hour, day, week, month, or quarter
- ✓ Cancel a running pipeline. Cancelling does not roll back rows already loaded, so check the target before you rely on it
Integrations
Search, filter, create, upload, and edit process YAML through a guided workflow.
- ✓ Source, target, mapping, then a reviewed YAML preview
- ✓ Uploaded YAML shows the same workflow preview before you create the pipeline
Connections
Test each connection before you save it, and see which integrations use it.
- ✓ Secrets stay masked
- ✓ Each connection shows whether it is a source, a target, both, or unused
Logs
Search execution logs across pipelines by time window and status.
- ✓ Active runs refresh while you watch
- ✓ Switch between the workflow view and the full raw log
Copilot
Ask in plain language from the spark button.
- ✓ Inspects, explores, builds, and runs through the same MCP tools
- ✓ Asks before it changes anything
Patterns
Extract and Load Patterns
Pick the pattern that fits the shape of the data. CARBON stages each load before merging it into the target, and writes an audit record for every run.
SaaS Platforms
web_serviceAPI-token extraction for customer and employee data, without manual exports.
Relational Databases
relational_dbSQL extraction of facts and dimensions from OLTP systems into one source of truth.
File Systems
file_systemStructured and unstructured files, including logs and event data, from cloud storage or on-premises systems.
| Load Pattern | Behavior | Best For |
|---|---|---|
| Atomic Full Refresh | overwrite | Small lookup and reference tables. Replaces the dataset in one transaction. |
| High-Volume Snapshot | snapshot | Loading a point-in-time snapshot of large sources. |
| Delta Watermark Sync | upsert / merge | The main incremental strategy. Syncs only new or changed records since the last run. |
| Comparative Delta Detect | upsert / merge | Sources without timestamps. Compares a full scan to find inserts and updates. |
| Sequential History Append | append only | Logs, events, telemetry, and audit trails that are added in order and never modified. |
Governance
Automation With the Guardrails in Place
Agents and the copilot get narrow permissions, and everything they do is on the record.
Everything Authenticated
Every API route except the health check needs a bearer token. Tokens are stored as hashes and expire within 90 days. Revoking one takes effect on the next request.
Scoped Tokens
A console token is rejected by the agent endpoint, and an agent token is rejected by the console API. Limit any token to pipelines such as crm_*.
You Approve Changes
The copilot shows a dry run before any edit, and says what a run will do to the target table before it starts. Approve it, or reject it with a note for a revised proposal.
Full Audit Trail
Agent and copilot actions, including refused ones, record who acted, which tool they called, whether it was allowed, and which run it started.
Rate Limits and Cooldowns
Per-token caps and a minimum gap between agent-triggered runs stop a retrying agent from flooding the engine.
Secure by Design
Code is bytecode-encoded and license-validated, and ships in a Docker runtime on Python 3.10 and Apache Spark 3.3.
What Agents Can Never Do
- Touch connections. An agent that could edit credentials could point a pipeline at any host, so someone on your team sets up every source
- Read row data. Schema discovery returns column names and types only
- See secrets or endpoints. Responses are filtered to an allowlist before they leave the server
What the Copilot Will Not Do
- Create or edit connections
- Create or delete schedules
- Read row data from any source or target
- Change anything without asking you first
GSPANN Accelerators
Works With Your GSPANN Accelerators
Cadmium™
AI-powered CDP accelerator that consumes CARBON data feeds to build unified customer profiles.
Learn more →BEAT™
Automated ETL testing that validates data quality across CARBON pipelines, from source to target.
Learn more →Nitrate™
AI-driven data observability that verifies data accuracy at every transformation layer of your CARBON pipelines.
Learn more →See CARBON Run on Your Sources
Book a walkthrough and watch a YAML file become an audited, scheduled pipeline.
Insights
