Datasets

Datasets

Brane has a built-in concept of datasets — named data collections that live at specific domains. Datasets are central to Brane’s privacy model: they have a location, and policies control who can access them.

How Datasets Work

A dataset is a named reference to data stored at a specific worker node. When a workflow uses a dataset, Brane:

1. Identifies which worker node holds the data
2. Routes the computation to that node (or transfers the data if policy allows)
3. Makes the data available to the package container via a mounted file path

Using Datasets in Workflows

In BraneScript, create a dataset reference:

let data := new Data { name := "patient_records" };

Pass it to a function:

import my_analysis;
let result := analyze(data);
println(result);

The analyze function receives the dataset as a file path through an environment variable.

Creating Datasets

Datasets are registered on worker nodes by system administrators. They map a name to a location on disk:

# On the worker node
branectl data create patient_records /data/patients/

The data itself stays on the worker node’s filesystem and is never copied unless explicitly allowed by policy.

Intermediate Results

Functions can produce intermediate datasets that are stored temporarily and passed between workflow steps:

import preprocessor;
import analyzer;

let cleaned := preprocess(raw_data);
let result := analyze(cleaned);
println(result);

The cleaned intermediate result is stored at the worker that produced it and can be passed to subsequent functions. Brane manages the lifecycle of intermediate results automatically.

Access Control

Dataset access is governed by the domain’s policies. Common policy patterns:

• Allow local processing: Functions can read the data on the same node
• Deny export: Raw data cannot be sent to other nodes
• Allow aggregated results: Only computed summaries can leave the node

See the Policy Guide for details on writing these rules.

Next Steps

• Learn about BraneScript workflows
• Understand policy enforcement