// STRYKE-ARROW — ENGINEERING REPORT

>_EXECUTIVE SUMMARY

stryke-arrow is one of the opt-in connector packages in the stryke ecosystem. Apache Arrow + Parquet + Arrow IPC + Feather + arrow-CSV + arrow-JSON for stryke. Opt-in package, kept out of the stryke core binary so the daily-driver install stays slim. The heavy arrow-rs / parquet code ships as a Rust cdylib that stryke dlopens in-process on first use Arrow.

Core stays small on purpose — most one-liner / awk-replacement work doesn't need 200 transitive crates of columnar data infrastructure. arrow-rs + parquet hit a different scale, so they ship as an opt-in package.

~ARCHITECTURE

In-process cdylib design: the stryke side is a thin .stk wrapper that calls FFI symbols on the dlopen'd library. No subprocess, no pipe — calls return on the same thread that made them.

$WHY OPT-IN (NOT BUILTIN)

Core stays small on purpose — most one-liner / awk-replacement work doesn't need 200 transitive crates of columnar data infrastructure. arrow-rs + parquet hit a different scale, so they ship as an opt-in package.

The trade-off is intentional. The core stryke binary stays under ~40 MB precisely because each connector ships separately. Daily-driver work (one-liners, awk replacement, data scripting) doesn't need MongoDB drivers, AWS SDKs, or Spark Connect bindings linked in.

&FFI ENVELOPE

JSON in / JSON out across the FFI boundary. Each Arrow::* wrapper serializes its args to a JSON dict and calls the matching arrow__* symbol resolved out of libstryke_arrow.{dylib,so}; the cdylib returns a JSON CString the wrapper parses. The cdylib exposes 52 arrow__* entry points (the [ffi].exports list in stryke.toml) across four roles — read (version, read, read_columnar, schema, stats), write (write), conversion (convert), and compute / aggregation / numeric transforms (filter, filter_in, filter_not_in, filter_str, select, drop, distinct, drop_nulls, keep_nulls, fill_null, sort, reverse, gather, top_k, value_counts, slice, head, tail, with_row_index, count, null_counts, shape, concat, hstack, rename, cast, sum, mean, min_max, std, median, quantile, corr, describe, aggregate, unique, clip, scale, add_const, abs, round, add_column, sample, fold_case, metadata). Each is an extern "C" fn arrow__<verb>(*const c_char) -> *mut c_char; the returned CString is freed via the cdylib-exported stryke_free_cstring, wired automatically by rust_ffi::load_cdylib.

# request shape (built by the .stk wrapper)
{"path": "<file>", "format": ..., "...": ...}

# success — per-verb shape:
#   read            → {"columns": [...], "rows": [{col: val, ...}, ...]}
#   read_columnar   → {"columns": [...], "num_rows": N, "data": {col: [...]}}
#   schema          → {"fields": [{name, type, nullable}, ...]}
#   stats (parquet) → {"num_rows", "num_row_groups", "columns": [{name, null_count, min, max}, ...]}
#   write / convert → {"path": ..., "rows": N}
#   compute (filter/sort/select/…) → {"dst": ..., "rows": N}
#   count           → {"src": ..., "rows": N}

# failure
{"error": "<msg>"}        # wrapper die()s with the message

/SCOPE

See the README's "Why this is a package" + "API reference" sections for the authoritative scope. The package is intentionally narrower than its underlying SDK / driver — the goal is "useful from a shell pipeline", not "complete API coverage". The 58-function Arrow::* surface breaks down as follows:

Read the full per-signature reference in the Docs page or the README. Every function above is either an arrow__* cdylib export or a pure-stryke helper layered over one; none round-trips bulk data through the stryke process except the row-returning readers.

#PROJECT METADATA