Pipelines

A pipeline is an import or export feed of one collection, declared beside it in src/data/collection/<m>/+pipeline.ts. Its rows enter through the collection’s write pipeline as the caller, so the grant, the transform and approval all apply.

// src/data/collection/roster/roster_entry/+pipeline.ts (templates/hr-payroll, excerpt)
import { pipeline } from '@norbital-ai/bolt';

/**
 * The work-day sheet: one row per person-day — shift, clock, overtime, incentive hours, leave. …
 */
export default pipeline('roster_entry', {
  import: {
    description:
      'The work-day sheet: per employee, the days it covers become exactly what it states — rows created or updated, unsettled days left out or blanked removed, a leave code recorded as a day of time off. …',
    input: {
      rows: {
        kind: 'list',
        max: 10_000,
        of: {
          kind: 'object',
          fields: {
            row: { kind: 'number' },
            employee_number: { kind: 'text', label: SHEET_COLUMNS.employee_number },
            work_date: { kind: 'date', label: SHEET_COLUMNS.work_date },
            shift_code: { kind: 'text', label: SHEET_COLUMNS.shift_code, optional: true },
            clock_in: { kind: 'time', label: SHEET_COLUMNS.clock_in, optional: true },
            clock_out: { kind: 'time', label: SHEET_COLUMNS.clock_out, optional: true }
            // …
          }
        }
      }
    },
    records: (input) =>
      input.rows.map((row): SheetRecord => ({
        row: row.row,
        employee_number: row.employee_number.trim(),
        work_date: String(row.work_date)
        // …
      })),
    known: function* (ctx, records, { context }) { … },
    map: (record, { known }) => sheetRow(record, known),
    onConflict: 'update'
    // …
  }
});

Import

import declares a typed input; records, which turns the input into remote records; an optional known, one read of the rows already on file; map, which turns one record into one insert or null to skip it; and onConflict (update or keep), required on a collection whose model has a key.

Export

export declares select, an optional where and a format (csv, xlsx or json). The file is the run’s output, readable by the member who started it.

Runs

From the browser each feed is a run of the platform automation <collection>.pipeline, started with bolt.start by a member whose grants admit it: a create grant to import, a read grant to export. The view toolbar offers Import… and Export for the feeds the viewer may run, and the run shows its progress and its file.

When a pipeline is not enough

A pipeline covers a feed of one collection. An import that resolves or creates rows of other collections, or refuses with named rows, is a collection action; an export that assembles related rows, documents or several files is an automation that returns files on its run.