# Data Pipeline

> How events flow from your app to destinations - streams, functions, and real-time routing

Source: https://docs.graph8.com/developers/data-pipeline/

graph8 includes a built-in customer data platform (CDP) powered by ClickHouse and PostgreSQL. Every event you send is stored, processed, and available immediately - contacts, companies, enrichment, sequences, and campaigns all work out of the box. No warehouse setup required.

**Optionally**, you can route a copy of your events to your own warehouse (BigQuery, Snowflake, etc.) or other tools. This is for teams that want their graph8 data alongside their other data sources.

> **Tip: CDP included, warehouses optional**
> Unlike Segment where you MUST configure a destination to store data, graph8's CDP stores everything by default. Adding BigQuery or Snowflake is optional - for teams that also want data in their own warehouse.

## How Data Flows

```
Your App (SDK / API)
    │
    ▼
  Streams (collect events)
    │
    ▼
  Functions (transform, filter, enrich)
    │
    ├──→ graph8 CDP (automatic - contacts, enrichment, sequences, campaigns)
    │
    └──→ Your warehouse (optional - BigQuery, Snowflake, Postgres, ...)
         + analytics (GA4, Amplitude, Mixpanel)
         + CRM (HubSpot)
```

Every `g8.track()`, `g8.identify()`, and `g8.page()` call automatically enters the graph8 CDP. If you've configured additional destinations, events are also fanned out to those simultaneously.

## Streams

A stream is a data source - your website, mobile app, or server. Each stream has:

| Component | Purpose |
|-----------|---------|
| **Browser key** (write key) | Public key for the [JavaScript SDK](https://docs.graph8.com/developers/sdk/) |
| **Server key** | Private key for server-side tracking |
| **Domains** | Allowed source domains (CORS) |

When you set up graph8, a default "Website" stream is created automatically. Use the write key in your SDK init:

```typescript
g8.init({ writeKey: 'YOUR_BROWSER_KEY' });
```

Manage streams in **Connections > Sites** in your graph8 dashboard.

## Functions

Functions are JavaScript code that runs on every event before it reaches destinations. Use them to:

- **Filter** out test/bot traffic
- **Transform** event shapes to match destination schemas
- **Enrich** events with external data (API calls, warehouse lookups)
- **Route** events to specific tables or destinations

See the full [Functions guide](https://docs.graph8.com/developers/functions/) for API reference and code examples.

## Destinations

Events are routed to all connected destinations in real-time. No extra code needed - configure once in the dashboard, every event flows automatically.

**Supported destinations:**

**Data Warehouses**

    BigQuery, Snowflake, ClickHouse, Redshift, Postgres, MySQL, S3

**Analytics & Marketing**

    Google Analytics 4, Amplitude, Mixpanel, Facebook Conversion API

**CRM & Sales**

    HubSpot, plus any HTTP API via webhook destinations

**graph8 CDP**

    Contacts, companies, enrichment, sequences, campaigns (automatic)

See the full [Destinations guide](https://docs.graph8.com/developers/destinations/) for setup instructions.

## Live Events Debugger

See events arriving in real-time. After installing the SDK, open **Connections > Live Events** to verify data is flowing correctly.

- Filter by stream, destination, or event type
- Inspect full event payloads
- See function execution results
- Debug destination delivery

See the [Live Events guide](https://docs.graph8.com/developers/live-events/) for details.

## Schema Management

graph8 automatically manages database schemas for warehouse destinations:

- **Auto-create tables** when new event types arrive
- **Auto-add columns** when events contain new properties
- **Type inference** from JSON to SQL types (string, integer, timestamp, etc.)

No DDL required. Send events, graph8 handles the schema.

## Replay

When you add a new destination, you can replay historical events to backfill it. All events are stored in the data lake and can be replayed with date filtering.

## What graph8 Replaces

| Before | After |
|--------|-------|
| Segment (tracking + destinations) | `g8.track()` + Connections |
| Clearbit (enrichment) | `g8.enrich.person()` |
| Outreach (sequences) | `g8.sequences.add()` |
| Calendly (booking) | `g8.calendar.show()` |
| Intercom (chat + copilot) | `g8.chat.open()` + `g8.copilot.open()` |
| Typeform (forms) | `g8.forms.lookup()` |
| HubSpot CRM | Built-in contacts, companies, lists, deals |

One platform. One SDK. One bill.