# OpenSearch (Raw Data Access)

> Structured search, count, aggregate, and bulk ID resolution against the graph8 mashup OpenSearch indices

Source: https://docs.graph8.com/developers/opensearch/

> **Caution: RAW field names — not snake_case**
> Filter, group-by, and sort fields on every OpenSearch endpoint use the **RAW index field names** — all UPPERCASE, e.g. `COMPANY_INDUSTRY`, `COMPANY_NAICS`, `COMPANY_COUNTRY`. These are **not** the friendly snake_case names used by `/search/contacts`. A typo in a field name returns **0 hits silently** rather than an error — double-check names before filtering.

These endpoints provide generic, read-only, structured access to the graph8 mashup OpenSearch indices. They back the `g8_opensearch_*` MCP tool family. Use them when the higher-level CRM or prospecting endpoints (`/contacts`, `/companies`, `/enrichment/*`, `/search/contacts`) don't expose the field you need to filter on.

The `search`, `count`, and `aggregate` routes require an active subscription (same gate as `/search/contacts` and `/enrichment/enrich`). The `resolve` route does not.

> **Note: Index shape — contacts vs. companies**
> `mashup_contacts` and `mashup_companies` are the **same physical, contact-grained index** distinguished by your filters. A `count` or `search` over `mashup_companies` counts contact rows, not unique companies. For a unique-company total, use `aggregate` with `uniqExact` on `COMPANY_ID`.

> **Note: Observed value shapes**
> - `COMPANY_COUNTRY` is free-text — both `"US"` and `"United States"` occur; OR both forms in your filter.
> - `COMPANY_NAICS` is a multi-value array that may carry both `"3311"` and `"3311.00000"`.
> - `COMPANY_EMPLOYEE_COUNT` and `COMPANY_REVENUE` are band strings (not numbers), so `sum`/`avg` on them is not meaningful.

---

## Search

`POST /opensearch/search`

Structured search across a whitelisted set of mashup indices. The request body declares filters, fields, sort, and limit as typed inputs — the handler translates them into a safe `bool/term` OpenSearch query. Raw OpenSearch DSL is not accepted.

### Request Body

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `index` | string | **Yes** | — | Index to search. One of: `mashup_contacts`, `mashup_companies`, `hem2contact`, `ip2company` |
| `filters` | object | No | — | Dict of field name to value. A scalar value becomes a `term` filter; a list becomes a `terms` (OR) filter. Fields must be RAW UPPERCASE index names. |
| `fields` | array | No | — | List of RAW field names to project from `_source`. Omit to return all fields. |
| `sort` | array | No | — | List of `{"FIELD_NAME": "asc"}` or `{"FIELD_NAME": "desc"}` sort objects. |
| `limit` | integer | No | `50` | Number of hits to return (1–200). |
| `cursor` | array | No | `null` | `search_after` token from a prior response — pass to retrieve the next page. |
| `track_total_hits` | integer (0–1000000) | No | `null` | Exact-count ceiling for the `total` field (e.g. `100000`). By default, OpenSearch caps `total` at **10000** for any larger result set. Pass a value in the range `0–1000000` to get an exact count up to that ceiling in the same round trip. Does not lift pagination depth — keep paging with `cursor`. For a count with no row payload, use `/opensearch/count` instead (never capped). |

### Example

**cURL**

```bash
curl -X POST "https://be.graph8.com/api/v1/opensearch/search" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "index": "mashup_contacts",
    "filters": {
      "COMPANY_DOMAIN": "acme.com",
      "COMPANY_INDUSTRY": ["SaaS", "Fintech"]
    },
    "fields": ["COMPANY_DOMAIN", "CONTACT_ID"],
    "sort": [{ "COMPANY_REVENUE": "desc" }],
    "limit": 50,
    "cursor": null,
    "track_total_hits": null
  }'
```

### Response

```json
{
  "data": {
    "index": "mashup_contacts",
    "total": 1,
    "hits": [
      {
        "id": "abc",
        "score": 1.5,
        "source": { "COMPANY_DOMAIN": "acme.com" },
        "sort": [42]
      }
    ],
    "cursor": [42],
    "took_ms": 7
  }
}
```

| Field | Description |
|-------|-------------|
| `index` | The index that was searched |
| `total` | Hit count, capped at 10000 by default. Pass `track_total_hits` for a higher exact-count ceiling. |
| `hits` | Array of matching documents. Each has `id`, `score`, `source` (projected fields), and `sort` (search_after token for this row). |
| `cursor` | Pass as `cursor` on the next request to retrieve the following page. `null` when there are no more results. |
| `took_ms` | OpenSearch query duration in milliseconds |

---

## Count

`POST /opensearch/count`

Exact document count for a whitelisted index using the native OpenSearch `_count` API — **not** subject to the 10000 `track_total_hits` ceiling. Uses the same `index` and `filters` shape as `search`; paging fields are intentionally absent because a count returns no rows. This is the correct call for TAM cell sizing.

### Request Body

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `index` | string | **Yes** | Index to count. Same enum as `search`: `mashup_contacts`, `mashup_companies`, `hem2contact`, `ip2company` |
| `filters` | object | No | Same semantics as `search` — scalar value becomes `term`, list becomes `terms` (OR). Fields must be RAW UPPERCASE index names. |

### Example

**cURL**

```bash
curl -X POST "https://be.graph8.com/api/v1/opensearch/count" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "index": "mashup_companies",
    "filters": {
      "COMPANY_NAICS": ["3311", "3311.00000"],
      "COMPANY_COUNTRY": ["US", "United States"]
    }
  }'
```

### Response

```json
{ "data": { "index": "mashup_companies", "count": 17842 } }
```

> **Note:**
> This counts documents on the contact-grained index. For unique companies, use `aggregate` with `uniqExact` on `COMPANY_ID`.

---

## Aggregate

`POST /opensearch/aggregate`

Generic, allowlist-bounded GROUP BY — the TAM matrix-walk primitive. Translates `group_by` into nested OpenSearch `terms` aggregations with metric sub-aggregations, then applies `having`, `sort`, and `limit` to the flattened groups.

### Request Body

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `index` | string | **Yes** | — | Index to aggregate. Same enum as `search`. |
| `filters` | object | No | — | Pre-aggregation filters. Same semantics as `search`. Applied before grouping. |
| `group_by` | array | **Yes** | — | List of 1–3 RAW UPPERCASE keyword fields to group by (e.g. `["COMPANY_STATE", "COMPANY_EMPLOYEE_COUNT"]`). Capped at 3 dimensions. |
| `agg` | array | No | `[{"function": "count"}]` | List of metric aggregation objects. Defaults to a single document count. See functions table below. |
| `having` | object | No | — | Post-aggregation filter by metric alias. Operators: `eq`, `gte`, `lte`, `gt`, `lt`. Example: `{"count": {"gte": 50}}`. |
| `sort` | array | No | — | Sort groups by a metric alias. Example: `[{"count": "desc"}]`. |
| `limit` | integer | No | `100` | Maximum number of groups to return (1–1000). `total_groups` reflects the count of matching groups before this slice. |

#### `agg` object fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `function` | string | **Yes** | Aggregation function. One of: `count`, `uniqExact`, `sum`, `avg`, `min`, `max`. `count` uses the bucket document count and does not require a `field`. All others require a numeric `field`. |
| `field` | string | Conditional | RAW UPPERCASE field name to aggregate. Required for all functions except `count`. |
| `alias` | string | No | Name to use for this metric in `having`, `sort`, and response output. Defaults to the function name. |

> **Note: Accuracy**
> `terms` bucket counts and `uniqExact` (HLL) are **approximate** on a sharded index — treat them as well-calibrated estimates, not exact figures. Per-level bucket size and returned `total_groups` are bounded server-side so a single call cannot fan out into a full-cluster scan.

### Example

**cURL**

```bash
curl -X POST "https://be.graph8.com/api/v1/opensearch/aggregate" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "index": "mashup_companies",
    "filters": {
      "COMPANY_NAICS": ["3311", "3311.00000"]
    },
    "group_by": ["COMPANY_STATE", "COMPANY_EMPLOYEE_COUNT"],
    "agg": [
      { "function": "count" },
      { "function": "uniqExact", "field": "COMPANY_DOMAIN", "alias": "unique_domains" }
    ],
    "having": { "count": { "gte": 50 } },
    "sort": [{ "count": "desc" }],
    "limit": 500
  }'
```

### Response

```json
{
  "data": {
    "index": "mashup_companies",
    "groups": [
      {
        "key": { "COMPANY_STATE": "OH", "COMPANY_EMPLOYEE_COUNT": "11-50" },
        "agg": { "count": 1247, "unique_domains": 1184 }
      }
    ],
    "total_groups": 312,
    "took_ms": 84
  }
}
```

| Field | Description |
|-------|-------------|
| `index` | The index that was aggregated |
| `groups` | Array of group objects. Each has `key` (the group-by field values) and `agg` (metric values keyed by alias). |
| `total_groups` | Count of groups matching `having` before the `limit` slice. |
| `took_ms` | OpenSearch query duration in milliseconds |

---

## Resolve

`POST /opensearch/resolve`

Bulk HEM-to-contact-ID or IP-to-company-ID resolution, optimised for batches up to 500. The response includes **every** input as a key — `null` for misses — so callers can iterate results without re-querying.

This endpoint does **not** require an active subscription.

### Request Body

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `kind` | string | **Yes** | Resolution type. One of: `hem_to_contact`, `ip_to_company` |
| `ids` | array | **Yes** | List of 1–500 string identifiers to resolve. For `hem_to_contact`, pass SHA-256 hashed email values. For `ip_to_company`, pass IP address strings. |

### Example

**cURL**

```bash
curl -X POST "https://be.graph8.com/api/v1/opensearch/resolve" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "hem_to_contact",
    "ids": ["sha256hash1", "sha256hash2"]
  }'
```

### Response

```json
{
  "data": {
    "kind": "hem_to_contact",
    "results": {
      "sha256hash1": "C1",
      "sha256hash2": null
    },
    "matched": 1,
    "requested": 2
  }
}
```

| Field | Description |
|-------|-------------|
| `kind` | The resolution type that was requested |
| `results` | Object keyed by every input identifier. Value is the resolved ID string on a hit, or `null` on a miss. |
| `matched` | Number of inputs that resolved to an ID |
| `requested` | Total number of input identifiers submitted |