Application datastore

The application datastore provides custom relational tables (such as users, orders, documents, and events) stored within the same globally consistent backend as agent memory.

Unified storage allows SQL predicates and vector distance rankings to execute at a single snapshot without cross-database synchronization.

Datasets

A dataset is a container of your tables. It is a peer of an agent workspace, not nested under one, and (EnterpriseId, DatasetId) is the isolation boundary for everything in it, exactly as (EnterpriseId, AgentInstanceId) is for memory.

POST /v1/datasets
{
  "dataset_id": "acme.prod",
  "display_name": "Production",
  "location": "global"
}

Set create_api_key to get the dataset and a key scoped to it in one call:

POST /v1/datasets
{
  "dataset_id": "acme.prod",
  "location": "global",
  "create_api_key": true,
  "api_key_scopes": [
    "datastore.data:read",
    "datastore.data:write",
    "datastore.schema:manage"
  ]
}

This needs iam.apikeys:create as well as datastore.datasets:create, because minting a credential is a management-class act wherever it happens. Creating a dataset does not become a way around it. The built-in member role holds the second and not the first, so a member creates datasets and cannot mint keys for them.

The response carries api_key_secret once. The key is an ordinary one on the api-keys surface: its dataset selector is exactly this dataset, its scopes and selectors are immutable, and it is revocable independently. Omit api_key_scopes and it gets the member-equivalent default, which can read and write data but cannot declare tables.

A dataset id has the same shape as an agent id: at most 128 characters of lowercase letters, digits, ., _ or -, starting and ending with a letter or digit. The . character serves as the access-selector separator, so an id carrying a selector metacharacter could be crafted to widen access.

The same lifecycle from the CLI:

jnh datasets create acme.prod --name Production --location global \
    --with-key --key-scope datastore.data:read \
                --key-scope datastore.data:write \
                --key-scope datastore.schema:manage

jnh datasets list
jnh datasets get acme.prod
jnh datasets delete acme.prod       # prompts for the id; --yes to skip

list walks to exhaustion, so what it prints is every dataset your selectors reach and never a first page that looks complete. Add -o json to any of these for the raw API response, or -o name for the ids alone; see CLI scripting and automation.

Choosing a location

A dataset picks its own read/write geography at creation and keeps it for life. The available locations are set by your operator.

Location	Characteristics	Trade-offs
Single-region e.g. us-central	lowest write latency	remote reads from other regions
Multi-region e.g. global	globally consistent reads	higher write latency

A multi-region location is a deliberate trade rather than a default: globally consistent reads are bought with write latency. If you pick one for global reads and find writes slower than you want, bounded-staleness reads recover read latency without giving up the write geography.

Declaring tables

You declare logical tables and columns. The control plane validates them, mints a physical identifier for every object, and creates real tables.

POST /v1/datasets/acme.prod/schema:declare
{
  "tables": [
    {
      "name": "orders",
      "columns": [
        { "name": "order_id", "type": "COLUMN_TYPE_STRING" },
        {
          "name": "customer_id",
          "type": "COLUMN_TYPE_STRING",
          "nullable": true
        },
        {
          "name": "total_cents",
          "type": "COLUMN_TYPE_INT64",
          "nullable": true
        }
      ],
      "primary_key": ["order_id"],
      "indexes": [
        { "name": "by_customer", "columns": ["customer_id"] }
      ]
    }
  ]
}

You never see the physical name, and you never need to: every later request names orders. A name your dataset's catalog does not hold is not found, and it is never passed through into a query, which is what makes identifier injection structurally impossible rather than merely filtered.

Two things the platform adds and you cannot express:

• Every table's primary key is led by (EnterpriseId, DatasetId). Your declared key follows.
• Secondary index keys are led by the same columns, so a unique index is unique within your dataset, never across the shared database.

Schema changes are asynchronous

Schema changes are queued and applied in the background. A declaration returns with each table at SCHEMA_STATUS_PENDING; poll GET /v1/datasets/{id}/schema until every table is SCHEMA_STATUS_READY before committing data to it.

Evolution is additive

Change	Result
re-declaring an unchanged table	no-op
adding a nullable column	online ALTER TABLE
adding a NOT NULL column	refused (existing rows would have no value)
dropping a column, retyping one, changing the primary key	refused
tightening a column to NOT NULL	refused

Destructive changes are refused rather than gated behind a flag. They can rewrite or lose live data, so they deserve an explicit migration path, which this version does not offer. Plan your keys accordingly.

Declaring from the CLI

From the CLI a declaration is always a file. There is no flag form and no prompt-driven builder: the shape is nested enough that flags could not carry it, and a schema belongs in version control rather than in shell history.

Start from a dataset that is ACTIVE (a freshly created one in a location whose database must be provisioned first comes back PROVISIONING):

jnh datasets create acme.prod --name Production --location global
jnh datasets get acme.prod          # wait for ACTIVE

Write the declaration - the body of schema:declare above, as JSON or YAML, snake_case or camelCase, and column types may drop their COLUMN_TYPE_ prefix:

# orders.yaml
tables:
  - name: orders
    primary_key: [order_id]
    columns:
      - { name: order_id,    type: string }
      - { name: customer_id, type: string, nullable: true }
      - { name: total_cents, type: int64,  nullable: true }
      - { name: created_at,  type: timestamp, nullable: true }
    indexes:
      - { name: by_customer, columns: [customer_id] }

The types are string, int64, float64, bool, timestamp, date, bytes, json, and vector (which needs a vector block; see Vector columns). A dataset_id in the file is dropped: the dataset is the one on the command line, so a file cannot silently retarget another dataset.

Then preview, apply, and read the result back:

jnh datasets schema declare acme.prod --from-file orders.yaml --dry-run
jnh datasets schema declare acme.prod --from-file orders.yaml
jnh datasets schema get acme.prod

declare reads the current catalog first and prints what would change (new tables, new columns, new indexes) before issuing anything:

+ table   orders (4 column(s), 1 index(es))

--dry-run stops there, issuing nothing. A destructive change from the table above is refused there too, naming the column, without a request:

REFUSED   orders.total_cents would be retyped from int64 to string

Adding a NOT NULL column is a ! warning rather than a local refusal, because only the server knows whether the table already holds rows:

! orders.region is NOT NULL: adding it is refused if orders already holds rows

A declaration that would change nothing says so and issues no DDL, so you learn that immediately instead of waiting seconds per table to be told the same thing. Otherwise it applies and polls until every declared table is READY, because schema changes are asynchronous:

TABLE   STATUS   DETAIL
orders  pending
All 1 table(s) are ready.

--no-wait returns at PENDING instead, and --wait-timeout (default 5m) bounds the poll. Declaring needs datastore.schema:manage; without it the diff still prints and the request then fails, exiting 3.

Read the catalog back at any time with schema get, which renders each table's columns (marking the ones the platform maintains), its primary key, its indexes, its apply status, and your usage against the tier's ceilings:

TABLE orders (ready)
  COLUMN       TYPE       NULL  KEY  NOTES
  order_id     string     no    pk1
  customer_id  string     yes
  total_cents  int64      yes
  created_at   timestamp  yes

  INDEX        COLUMNS      UNIQUE
  by_customer  customer_id  no

USAGE              USED  CEILING
tables             1     100
indexes            1     50
columns per table  4     128

Add -o json to either command for the raw API response.

Reading and writing

Rows are written with data:commit and read with data:query. Both are covered on their own page: Reading and writing rows.

Vector columns

Any table may declare a fixed-width embedding column, searched by exact COSINE_DISTANCE:

{
  "name": "embedding",
  "type": "COLUMN_TYPE_VECTOR",
  "nullable": true,
  "vector": {
    "dimensions": 3072,
    "source_column": "body"
  }
}

The same column in a jnh datasets schema declare file:

# docs.yaml
tables:
  - name: docs
    primary_key: [doc_id]
    columns:
      - { name: doc_id, type: string }
      - { name: body,   type: string, nullable: true }
      - name: embedding
        type: vector
        nullable: true
        vector:
          dimensions: 3072      # the model's width
          source_column: body   # embedded server-side

There is no ANN index, deliberately: every query is already clamped to a dataset slice, which makes the scan small and an approximate index pointless.

The two embedding paths

source_column is the whole of the opt-in, and it decides which path a write takes:

You send	source_column set	Result
a vector	either	stored exactly as given; the model is not invoked
nothing	yes	generated server-side, in the same statement as the row
nothing	no	typed error, never a silent NULL
explicit null_value	either	stored as NULL; the model is not invoked

The last two rows are the important pair. Omitting a vector is almost always an oversight, and a row whose embedding silently held nothing would be invisible to the one operation the column exists for. Sending an explicit null is a different statement ("no vector yet") and is accepted, which is how you insert rows now and embed them in a later pass.

A managed column must declare the model's width

If you set source_column, dimensions must equal the width of the model registered for your dataset's location (3072 today). A narrower column is rejected by the backend rather than silently truncated, so this is caught at declare time with a clear message. A caller-supplied-only column is unconstrained: bring vectors of any width from your own model.

Truncation

A managed vector column brings two platform-maintained columns (jennah_<col>_truncated, jennah_<col>_token_count). They are returned by queries like any other column, cannot be written by you, and count against your per-table column ceiling, because the cost belongs to the table that opted in.

truncated=true means the stored content is complete but the stored embedding is not: the tail of your source column is unreachable by search. token_count is what the model counted in the content it was given, so it exceeds the model's limit rather than equalling it. Divide by the limit for a rough sense of how many pieces to split into.

Set reject_on_truncation on a commit to make truncation fatal instead of reported: no row from any table in that commit is written. It never fires for a vector you supplied, because the platform embedded nothing and so has nothing to report.

Searching an embedding column is part of the query envelope, so it is documented with the read path: see searching vector columns.

Access control

The datastore view of the model described in Roles and access control, which covers the full permission catalog, custom roles, and the selector grammar. Three independent checks, and none substitutes for another:

Layer	Answers	Carried by
Tenancy	which enterprise and dataset?	your credential (enterprise) + the route (dataset)
Permission	what kind of operation?	your role, or an API key's scopes
Dataset selector	which datasets?	your role, or an API key's selectors

EnterpriseId comes only from your verified credential, never from a body or a path. DatasetId comes from the route.

Permissions

Permission	Grants
datastore.datasets:create / :read / :delete	dataset lifecycle
datastore.schema:read	read the logical schema
datastore.schema:manage	declare and evolve tables
datastore.data:read / :write	data:query / data:commit
datastore.access:read / :manage	see / set dataset selectors

The built-in member role holds the data-plane set but not datastore.schema:manage: declaring a table issues DDL against a shared database, so schema authority is always a deliberate grant. It is grantable to an API key, a service building an app has to be able to declare its own tables.

datastore.access:manage is management-class, so it can never appear in an API key's scope: a key can never widen anyone's dataset reach.

Dataset access is default-deny

A member holds datastore.data:read and datastore.data:write and reaches no dataset, because their dataset selector set is empty. The permissions say what kind of operation; the selectors say where.

Dataset selectors

Same grammar as agent selectors, in a separate namespace: an exact acme.prod, a subtree acme.*, or * for every dataset. Matching is segment-anchored, so acme.* does not match acmecorp.prod.

The two namespaces never interact. Blanket agent reach grants no dataset reach and cannot be used to grant dataset reach to someone else, because containment is checked per namespace. That also means dataset-access administration can be delegated to someone who administers no agent access, and vice versa.

A per-project key is just an API key with one dataset selector:

POST /v1/apikeys
{
  "label": "acme-prod-app",
  "scopes": [
    "datastore.data:read",
    "datastore.data:write"
  ],
  "dataset_selectors": ["acme.prod"]
}

Its secret is shown once. Its scopes and selectors are immutable. There is no rotation RPC anywhere in Jennah, so "rotate" means revoke and create.

The application datastore design incorporates the following specifications:

• No raw SQL: Requests use structured JSON schemas. Raw SQL input is not accepted to avoid injection vulnerabilities.
• Atomic arithmetic: The set_expressions field provides atomic single-column arithmetic operations without requiring complex expression evaluation trees.
• Dataset isolation: The dataset is the unit of isolation. Principals granted access to a dataset have access across its tables. Segregate data into separate datasets to isolate access.
• Single-dataset query scope: Each query is scoped to a single dataset. Cross-dataset joins are not supported.
• No triggers or stored procedures: Business logic is handled at the application layer.
• Client-provided idempotency keys: Clients supply idempotency keys to ensure retry safety across network interruptions.

Limits

Per-dataset ceilings on tables, columns, and indexes, plus datasets per enterprise and a data:query result ceiling. Read your usage and the ceilings from GET /v1/datasets/{id}/schema.

A result count above your tier's ceiling is clamped, not refused. Asking for more rows than your plan allows is served short. A counted ceiling exceeded returns RESOURCE_EXHAUSTED with no DDL issued, never a raw backend error.

Ceilings are report-only today

Every datastore ceiling currently records an overage rather than refusing, while real usage is measured. Build against the published numbers regardless, because they become enforcing without notice.