All notable changes to ScrivaDB (formerly FileDB v2) are documented here. The format is based on Keep a Changelog, and the project follows Semantic Versioning.
As of v1.0.0 the public surface is frozen. ScrivaDB follows semver, and the
1.0 release establishes the guarantee that had been deferred through the 0.x
line: no breaking changes without a major (v2) bump. This covers all three
public surfaces:
proto/scriva.proto — RPCs, message fields, enum
values, and the REST routes generated from them);scriva and scriva-cli commands, flags, and config-file keys);engine, scriva, store, query — import paths
and exported types/signatures).Concretely, within the v1 line:
v2.0.0) are the only place breaking changes may land — a
removed/renamed RPC, message field, flag, exported type, or signature. A future
breaking Go revision ships under a separate import path (.../engine/v2),
leaving v1 importers untouched.v1.(y+1).0) are additive only — new RPCs, message fields,
flags, methods, or options that don't break code or clients built against
v1.0. Every addition is noted here.v1.y.(z+1)) are bug-fix only and never change a documented
API or the on-disk format.omitempty, so a
newer engine reads older segments without migration.Depend on a tagged version (go get …@v1), and read this changelog before
upgrading — but within v1 no upgrade requires a code or client migration. See
docs/embedding.md for the
embedding-specific contract.
Note on v1.2.0.
v1.2.0was the initial ScrivaDB tag and is installable as a Go module (go get github.com/srjn45/scriva@v1.2.0), but its release pipeline failed at the multi-arch Docker publish step, so no binaries, image, or package-manager artifacts were published for it. v1.2.1 is the first release with the complete distribution set and supersedes v1.2.0; the only code difference is the release-pipeline fix. New users should adopt v1.2.1.
The project formerly known as FileDB v2 is now ScrivaDB. This is a rename —
the same storage engine, wire semantics, and on-disk format as FileDB v1.1.0,
re-baselined as v1.2.1 — continuing the tag line after FileDB's v1.1.0
(the v1.0.0/v1.1.0 tags are already published under the old name) — under a
new name (scriva), a new Go module path, and new distribution channels. There is no data migration: an existing data
directory opens unchanged. The break is entirely in identifiers — import paths,
the proto package, binary/env/metric names — so anything that references those
must be updated. See the migration note below.
github.com/srjn45/filedbv2 → github.com/srjn45/scriva.
The embedded façade is the root package scriva (scriva.Open); the public
engine, store, and query packages move with the module. Update every
import path.filedb.v1 → scriva.v1
and the service FileDB → Scriva. Because the fully-qualified service
name changed, every gRPC method path changed (e.g.
/filedb.v1.FileDB/Insert → /scriva.v1.Scriva/Insert). Regenerate any
hand-built stubs; the official SDKs and the checked-in OpenAPI spec
(docs/openapi/scriva.swagger.json) are already regenerated. The REST routes
(/v1/...) are unchanged.filedb/filedb-cli → scriva/scriva-cli.FILEDB_API_KEY → SCRIVA_API_KEY. The legacy
FILEDB_API_KEY name is still honoured as a deprecated fallback for one
release cycle, printing a one-time deprecation warning to stderr when used;
it will be removed in a later release. Switch to SCRIVA_API_KEY.filedb_* prefix is renamed outright to
scriva_* (scriva_collection_records_total, scriva_collection_segments_total,
scriva_collection_bytes, scriva_compaction_runs_total,
scriva_compaction_duration_seconds, scriva_grpc_request_duration_seconds,
scriva_scan_rows_scanned, scriva_quota_rejected_total). There is no dual
emission — update your dashboards and alerts to the new names./tmp/filedb.sock → /tmp/scriva.sock,
the example config file filedb.yaml → scriva.yaml, and the CLI REPL
history file ~/.filedb_history → ~/.scriva_history. The Docker image is
now ghcr.io/srjn45/scriva.Binaries — Homebrew (cask via the srjn45/scriva tap), Scoop, apt/rpm
packages, and an AUR package, in addition to the tagged tarball releases.
Docker — multi-arch (amd64 + arm64), cosign-signed images published to
both ghcr.io/srjn45/scriva and Docker Hub.
Client SDKs — the seven official clients publish under their new registry names:
| Language | Install |
|---|---|
| Python | pip install scriva |
| JavaScript / TypeScript | npm i scriva |
| PHP | composer require srjn45/scriva |
| Java | com.srjn45:scriva-client (Maven Central) |
| Ruby | gem install scriva |
| Rust | cargo add scriva |
| C# / .NET | dotnet add package Scriva.Client |
ScrivaDB is a continuation of FileDB v2, not a new project — the FileDB v1.1.0
codebase renamed and re-tagged under the ScrivaDB name (first full release
v1.2.1). The two do not share a
module path or a version line, so upgrading is a deliberate migration, not a
go get -u:
github.com/srjn45/filedbv2 (and its /engine, /store, /query
sub-packages) keeps building against the existing FileDB tags (v1.1.0 and
earlier). Those tags are frozen and remain resolvable.github.com/srjn45/scriva
(façade) or github.com/srjn45/scriva/engine at v1.2.1, install the new
binaries/SDKs above, and update the renamed identifiers listed under Breaking
changes.The FileDB v1.1.0 and earlier entries below are preserved verbatim as the historical record under the former name.
The first release on the frozen v1 surface: two correctness fixes to the
segment read/write paths plus one additive engine symbol. Fully backward
compatible — no wire or on-disk format change and no migration from v1.0.0.
ReadAt no longer fails with "token too long" on records over 64 KiB.
Segment.ReadAt scanned with a default 64 KiB line buffer while the full-scan
paths (ScanAll/ScanFrom) used 16 MiB, so a record between 64 KiB and 16 MiB
was writable and visible to scans but unreadable by offset — every point
lookup (Get/FindById) that resolved through the index failed. All three
read paths now share a single 16 MiB scanner via a common helper so their
buffer sizes can't drift apart again. (#78, #79)Segment.Append now rejects a record whose
encoded form (including its trailing newline) exceeds the 16 MiB scan-buffer
limit with a new typed engine.ErrRecordTooLarge, instead of writing a
record no read path could ever scan back. The server maps it to gRPC
InvalidArgument. The append stays atomic — a rejected write persists
nothing. The gRPC transport's 4 MiB max-message size already sits below this
ceiling, so the guard matters most for the embedded façade and internal
re-append paths (compaction, replication). (#80, #81)docs/architecture.md. (#77)FileDB's 1.0 release and public-API freeze. It rolls up the v0.9.0 Security
wave — mutual TLS (S1), audit logging (S2), per-collection ACLs (S3), and
per-tenant quotas (S4) — and establishes the stability guarantee: the gRPC/REST
surface, the CLI, and the engine/filedb/store/query embedding API will
not break without a major (v2) bump. No migration from v0.8.0 — the wire and
on-disk formats are unchanged.
proto/filedb.proto), the
CLI, and the Go embedding API (engine/filedb/store/query).
engine.CollectionConfig.Quotas (the DB-wide
map[string]Quota overlaid onto a collection's MaxRecords/MaxBytes at
open time) was deliberately kept as-is. It is consistent with the existing
DB-wide-setting-on-CollectionConfig precedent (ReplicationRingSize,
Follower), and the alternative — a separate DB-level config type and a
changed engine.Open signature — would be a larger breaking change than the
mixed-scope wart it removes. The embedded façade continues to set
MaxRecords/MaxBytes directly and leave Quotas nil.docs/embedding.md and this file now state
the frozen-surface guarantee instead of the former pre-1.0 "minor bumps may
break" framing.quotas: section mapping a collection name to a
max_records and/or max_bytes budget (either may be 0/omitted for
unlimited). It is config-file only — a per-collection map does not fit a
flat flag. The embedded façade gets matching filedb.WithMaxRecords /
filedb.WithMaxBytes collection options for parity.engine.ErrResourceExhausted, which the server maps to gRPC
ResourceExhausted. max_bytes is measured against the collection's
total on-disk segment size (the same figure CollectionStats.SizeBytes
reports).Insert, InsertMany,
keyed insert, an inserting Upsert, and transaction inserts. An in-place
Update/UpdateByKey, a compare-and-swap, an Upsert that replaces, and
a Delete are never refused, so a tenant sitting at its limit can still
edit or delete to recover. InsertMany and CommitTx are checked as a
whole batch atomically: a batch that would breach the budget writes
nothing. (InsertMany is now a genuinely atomic engine operation.)filedb_quota_rejected_total{collection} counter
tracks refused writes, and the per-collection gauges gain
filedb_collection_bytes{collection} alongside the existing record/segment
gauges, so consumption and the rejections it triggers are both visible. The
metric is wired via a server-side observer hook — the engine still imports no
metrics package. No proto/API change.docs/getting-started.md
for a YAML example and docs/architecture.md
for how write-path enforcement works.collections: list on a keys: entry: when present, the key
may only act on the listed collections; an RPC targeting any other collection
is rejected with PermissionDenied. Omitted/empty means all
collections — the historical behaviour.GetCollection() accessor: unary RPCs are checked before the handler runs;
collection-scoped streams (Watch, Find, Aggregate) are checked on
the first client message. RPCs with no collection field (e.g.
ListCollections) are not collection-scoped and remain callable by a
restricted key.docs/getting-started.md
for a YAML example and docs/architecture.md
for how enforcement works.audit_log): --audit-log <path> writes an
append-only NDJSON stream — one self-contained JSON record per line —
carrying the principal (API-key name, mTLS cert subject, anonymous when
auth is off, or unauthenticated for a rejected call), the RPC method,
the target (collection/key/id where applicable), the outcome
(ok or the gRPC status code), and auth_failure: true on a rejected call.InsertMany,
Upsert), schema changes (CreateCollection/DropCollection,
EnsureIndex/DropIndex), transaction control, the admin Compact and
Promote, and any RPC — read or write — rejected by the auth layer.
Successful reads are not audited. Exactly one record is emitted per RPC.docs/getting-started.md for the
format and docs/operations.md for the
retention/rotation runbook.x-api-key header. It is off by default and composes with API keys.
tls_client_ca, tls_client_auth):
--tls-client-ca <bundle.pem> sets the CA that signs trusted client certs,
and --tls-client-auth off|require|verify-if-given chooses the policy —
require mandates a valid client cert on every connection, verify-if-given
accepts a cert when presented but does not require one. Both require server
TLS (--tls-cert/--tls-key) to also be set.x-api-key always wins and its scope is enforced;
a request with no API key but a verified client cert is authenticated
as the certificate's principal (subject CN, else the first SAN) with
read-write scope. Per-certificate scoping / ACLs are deferred to S3.--tls-client-auth require, the local REST gateway dials
gRPC over the Unix socket so REST keeps working.docs/getting-started.md
for a full CA/cert setup walkthrough.Replication & high availability — a node can follow a leader, serve reads from its applied state, and be promoted on failover — plus a compaction/shutdown crash-consistency fix.
R1 — leader→follower replication (log shipping). A follower node tails a
new server-streaming Replicate RPC that ships every committed segment entry
(post-fsync) tagged with a monotonic global LSN, and applies it through the
engine's normal write path so its primary and secondary indexes match the
leader's exactly. Replication is asynchronous (bounded lag) with a
per-follower shipped-LSN channel that lays the groundwork for R2.
ReplicationStatus RPC (GET /v1/replication/status) reports the leader
LSN and, per connected follower, its shipped LSN, lag, and connect time.--replicate-from <leader-addr> (config:
replicate_from). A fresh follower bootstraps from a Snapshot and then
catches up from the stream; a restarted follower resumes from its persisted
applied-LSN with no gaps and no duplication (apply is idempotent by record
revision). Tune the leader's in-memory resume buffer with
--replication-ring-size (config: replication_ring_size, default 8192).engine package stays dependency-free: it exposes the LSN
feed and apply as plain Go types (engine.ReplicationEntry,
DB.SubscribeReplication, DB.ApplyReplication, DB.ReplicationStatus); the
server maps them to proto. Leader-side sequencing is opt-in via
CollectionConfig.ReplicationRingSize, so the default/embedded write path is
unchanged.replication.json at the data-dir root holds the leader LSN
and follower applied-LSN watermarks (additive; absent on non-replicated DBs).R2 — read replicas & follower reads. A node started with
--replicate-from now serves read RPCs (Find, FindById, FindByKey,
Aggregate, plus CollectionStats/ListCollections/ListIndexes/Watch)
from its applied state, so reads scale horizontally across followers.
FAILED_PRECONDITION and
the message read-only replica; write to the leader. The guard covers every
mutating RPC — Insert/InsertMany/Update/Delete, the keyed and
compare-and-swap writes (Upsert/UpdateByKey/DeleteByKey/UpdateIfRev),
CreateCollection/DropCollection, EnsureIndex/DropIndex, the
transaction RPCs, and Compact. It is a single pair of gRPC interceptors
installed only in follower mode, keyed on the generated method names, so the
read-only contract lives in one auditable place.ReplicationStatusResponse gains an additive
applied_lsn field: query a follower's ReplicationStatus
(GET /v1/replication/status) for its applied LSN and diff it against the
leader's leader_lsn to bound how stale a follower read may be. applied_lsn
is 0 on a leader.engine package is unchanged — role/routing lives entirely in
the server layer; the engine only exposes the existing DB.AppliedLSN().R3 — manual failover & role management. A new admin Promote RPC
(POST /v1/replication/promote) and filedb-cli promote command flip a
caught-up follower into a leader: it stops replicating from its upstream, lifts
the read-only guard, and begins accepting writes — the documented path for
recovering from a leader loss without an external coordinator.
FAILED_PRECONDITION when the follower's replication lag (last-known leader
LSN minus applied LSN) exceeds a configurable threshold — --promote-max-lag
(config: promote_max_lag, default 0 = must be fully caught up). Pass
--force (proto force) to override the guard when the leader is
unrecoverable and some divergence is acceptable. Promoting a node that is not
a follower is refused (nothing to promote). The response reports the new
role, the LSN the new leader continues from, and the lag at promotion.Promote lifts them live — no restart. A promoted
leader reseeds its LSN counter above the replicated tail, so it never reuses
an LSN the old leader assigned.Promote requires a read-write API key. Finer-grained admin ACLs (an
admin scope) are deferred to S3; until then a read-write key is the admin
boundary.engine package stays proto-free: the role-flip and lag guard
live in engine as plain Go (DB.Promote, DB.IsFollower,
DB.NoteLeaderLSN, typed engine.ErrReplicaLagExceeded/ErrNotFollower);
the server maps them to proto and enforces auth. Promotion is one-way —
automatic leader election (consensus) remains out of scope. See
docs/operations.md for the manual-failover runbook.Close() now serializes with an in-flight compaction pass (via
compactMu) instead of persisting the final index while the pass is
mid-swap; a pass that was still blocked on the lock when Close finished
aborts instead of mutating the layout afterwards.index.json whose
entries reference segment files that no longer exist — or offsets past a
segment's end — is treated as stale and rebuilt from the segments (the
documented source of truth), along with the secondary indexes. Directories
already corrupted by earlier builds heal on first open.compact.manifest) before touching any segment file,
renames temps over their final names first, deletes only old segments whose
names were not reused, and retires the manifest after the post-swap index
persist. A leftover manifest at open rolls the swap forward idempotently
and forces an index rebuild; the old remove-then-rename order could strand
the only copy of the sealed data in .compact_* temp files an open never
discovers. Swap manifests are excluded from snapshots for the same reason
index.json is.This release rolls up the operability/observability and network-API-parity work accumulated since v0.3.0, plus the follow-on sweep that brought all seven client SDKs up to the new wire surface. It collapses the interim internal milestone labels (v0.6.0 operability, v0.7.0 API parity) into a single tagged release. No breaking changes to existing behaviour; every addition is opt-in or additive.
Operability & observability. The server is no longer a black box:
it emits structured request logs (PR-A), exposes health/readiness probes for
load balancers and orchestrators (PR-A), can shed load under pressure with
opt-in concurrency, in-flight, and per-key rate limits (PR-B), can emit
OpenTelemetry traces to an OTLP collector (PR-C), and surfaces per-query cost
through a slow-query log and a rows-scanned metric (PR-D). No breaking changes —
logging defaults to human-readable text at info, the probes are additive, and
every limit, tracing, and the slow-query log are off by default.
log/slog (zero new dependencies). A unary and stream
gRPC interceptor emits exactly one structured record per RPC — method,
principal (the authenticated API-key name), duration, and status code —
at info for success and error for failure. Two flags configure output:
--log-level (debug|info|warn|error, default info) and --log-format
(json|text, default text), also settable via log_level / log_format
in the config file. The embeddable engine/store/query packages remain
free of any logging dependency (enforced by make deps-check).grpc.health.v1.Health service is
registered on the TCP and Unix gRPC servers; it reports SERVING once
listeners are up and flips to NOT_SERVING at the start of graceful shutdown
so in-flight RPCs drain cleanly. The REST gateway gains GET /healthz
(liveness) and GET /readyz (readiness — 200 when the DB is open and the
data directory is writable, 503 otherwise).RESOURCE_EXHAUSTED instead of
growing goroutines/FDs without bound. --max-concurrent-streams caps the
HTTP/2 streams per gRPC connection (0 = library default). --max-inflight
installs a server-wide in-flight semaphore: once the ceiling is saturated,
further calls are rejected immediately rather than queued (0 = unlimited).
--rate-limit throttles each API-key principal to a requests/sec budget via
an independent token bucket, so one principal being throttled never affects
another (0 = disabled). All three are also settable via
max_concurrent_streams / max_inflight / rate_limit in the config file.
The limiter interceptors chain after auth (to read the resolved principal) and
before logging (so a shed request is still logged); the embeddable engine
keeps its zero transport dependencies (rate limiting lives entirely in the
server layer via golang.org/x/time/rate).--otlp-endpoint is set. When enabled, a tracing interceptor (unary and
stream) starts one span per RPC named after the method and tagged with
rpc.method and the returned rpc.grpc.status_code; it is chained outermost
so the span wraps the whole handler and its context flows into the engine.
--otlp-sample-ratio (default 1.0) sets the root sampling fraction. Spans
export to an OTLP/gRPC collector. Engine scan and compaction cost surface as
child engine.scan / engine.compaction spans through the existing
engine.CollectionConfig hook pattern (new OnScan hook), so a slow Find
shows which segment scan dominated — the embeddable engine gains no OTel
dependency (the server owns the SDK; enforced by make deps-check). Both flags
are also settable via otlp_endpoint / otlp_sample_ratio in the config file.Collection.ScanStream now returns a
ScanStats value alongside its error — RowsScanned (live records examined),
RowsReturned (records emitted), and IndexUsed (whether a secondary index
produced the candidate set). It is plain data, so the embeddable engine gains
no dependency; the planner already knew index-vs-scan, and this simply surfaces
it. The server turns it into two operator signals: a new --slow-query-ms
flag (also slow_query_ms in the config file, 0 = disabled) logs any Find
reaching that duration at WARN with the filter shape (fields/ops, never
values), rows_scanned vs rows_returned, index_used, and duration; and
a filedb_scan_rows_scanned Prometheus histogram records rows examined per
query, labelled by collection. Together they let an operator find unindexed hot
queries from logs and metrics. The scan cost reaches metrics via the same
server-layer hook pattern as compaction — the engine never imports metrics
(enforced by make deps-check).Network/engine API parity. The wire API is catching up to the embedded engine: the richest engine operations — natural string keys, upsert, and optimistic-concurrency updates — are now reachable from any gRPC/REST client, not just an in-process Go program. Additive only: new field numbers and new RPCs, so pre-existing clients are unaffected.
Upsert (insert-or-replace by key), FindByKey,
UpdateByKey, DeleteByKey, and UpdateIfRev (compare-and-swap
on a record's revision). A keyed insert is expressed by setting the new
optional key field on the existing Insert request (POST /v1/{collection}/records with a key, .../records:upsert,
.../keys/{key}, and .../keys/{key}:cas on REST) — empty key preserves
the unchanged server-assigned-id behaviour.key and rev: Record
gained fields 5/6 and InsertResponse/UpdateResponse the same pair (all
additive field numbers).engine.ErrDuplicateKey →
AlreadyExists, engine.ErrKeyNotFound → NotFound, and setting the
reserved _key field via data → InvalidArgument. UpdateIfRev treats a
stale revision or missing key as a clean swapped=false no-op, not an error.upsert, find-by-key, update-by-key, delete-by-key,
update-if-rev, plus an insert --key flag for keyed create. The 7-language
SDK parity sweep for these operations is a separate follow-on wave.Find, FindById, and FindByKey gained
an optional repeated fields projection: when non-empty, only those
top-level fields are returned in each record's data, so wide documents transmit
only what the caller asked for. id, key, and rev are always included;
an empty fields returns the full record (backward compatible); an unknown or
absent field is silently omitted (not an error). The engine applies the
projection in ScanStream (via a new exported engine.ProjectData helper)
after filtering and ordering, so an order_by field need not be projected
and the embeddable engine keeps its zero transport dependencies. New CLI flag
--fields (comma-separated) on find, get, and find-by-key. Additive
field numbers only; the SDK parity sweep is a separate follow-on wave.Find can now sort
by several fields and paginate deep result sets in O(page) instead of O(offset):
repeated OrderBy { field, desc } order_by_fields on FindRequest applies sort keys lexicographically (first
dominant), each with its own direction, reusing the same type-aware
query.Compare as the filter operators. The record id is always the implicit
final tiebreaker, so the ordering is total and pages are stable.string page_token on both FindRequest and
FindResponse. A response carries the next page_token on its final streamed
message when more rows remain (empty = last page); feed it back to seek strictly
past the rows already returned rather than counting past them. Concatenated pages
cover every matching row exactly once — no duplicates, no gaps — even under
concurrent inserts. The cursor is a base64 of compact JSON encoding the last
(sort-key tuple, id); its codec lives entirely in the engine
(engine.ScanOptions.PageToken / ScanStats.NextPageToken, new
engine.SortField), so the embeddable engine keeps zero transport
dependencies. A malformed token → engine.ErrInvalidPageToken → gRPC
InvalidArgument. offset is retained for backward compatibility.--order-by field[:asc|:desc] (multi-field) and a
--page-token passthrough on find; a full page prints its
next-page-token: for the next fetch. The 7-language SDK parity sweep is a
separate follow-on wave.FindRequest.order_by and
descending fields are now marked [deprecated = true]. They still work — and
are honoured only when order_by_fields is empty — for one release, then
will be removed (a minor-bump break, per the versioning policy above). Migrate
order_by:"f", descending:d → order_by_fields:[{field:"f", desc:d}]. The CLI
--descending flag likewise still applies to a single bare --order-by; prefer
the field:desc form.Aggregate RPC (POST /v1/{collection}/aggregate) computes a count and
the numeric aggregations sum/avg/min/max over the live records matching
the same Filter as Find, optionally grouped by a field — so clients no
longer pull a whole collection just to count or total it.
group_by empty) streams a single result over the whole
filtered set; grouped streams one result per distinct group_by value, in
ascending group order. Each result carries the type-preserved group_value
(number/string/bool, or null for the whole-set group), the count, and — when
a numeric field is named — sum/avg/min/max with a numeric flag.
Only records whose field is numeric contribute (per the same query.AsNumber
rules the filter/sort use); avg divides by that numeric count (SQL AVG
semantics, ignoring absent/non-numeric values). sum/avg/min/max require
a field — requesting one without it is InvalidArgument.engine.Aggregate (in engine/aggregate.go) folds each matching record into
its group's accumulator, so memory is bounded by the number of distinct
groups, not the collection size. A whole-set count reuses the existing
Collection.Count fast path (answered from the primary/secondary index without
reading segments where possible); grouped/filtered aggregations reuse the same
index-aware forEachMatch scan as Count/Find. The engine returns plain Go
structs (AggregateSpec / GroupResult) the server maps to proto, so the
embeddable engine keeps zero transport dependencies (enforced by
make deps-check). A new exported query.AsNumber gives the numeric reduction
the exact type rules of query.Compare.aggregate <collection> [filter-json] with --group-by, numeric
--field, and comma-separated --aggs count,sum,avg,min,max (default
count), reusing the existing filter-JSON style. The 7-language SDK parity
sweep is a separate follow-on wave.All seven client SDKs brought to v0.7.0 wire parity. The follow-on sweep
promised by N1–N4 landed: every SDK now covers the full new surface —
keyed CRUD / upsert / find-by-key / update-by-key / delete-by-key and
update-if-rev CAS with key/rev surfaced on records and typed
not-found / already-exists errors (N1); the fields read projection (N2);
multi-field order_by with opaque keyset page_token pagination (N3); and the
streaming aggregate / count / group_by RPC (N4). Each client refreshed its
vendored filedb.proto and regenerated stubs against the current API; examples
and READMEs demonstrate the new methods. Python (#56), JavaScript/TypeScript
(#58), PHP (#57), Rust (#59), Java (#60), Ruby (#62), and C#/.NET (#61). The Go
server CI is unchanged (it does not build the clients); each client was verified
against a locally built v0.7.0 server where a toolchain was available.
Client parity release. Per-record TTL is now exposed over the wire, and all seven client SDKs reach the full server API surface. No breaking changes — every addition is optional and backward compatible.
ttl_seconds on Insert,
InsertMany, and Update, and default_ttl_seconds on CreateCollection.
On insert, 0 inherits the collection default and a value > 0 overrides it;
on update, 0 is sticky (keeps the record's existing deadline) and > 0
resets it; negative values are rejected. TTL was previously usable only via the
embedded engine and the server-wide --default-ttl; it is now a first-class
wire parameter. (#38)compact, snapshot / snapshot-to-file, and the per-record TTL
parameters, and surface the OVERFLOW watch op.proto/filedb.proto that had drifted behind
canonical — missing the Compact/Snapshot RPCs, the TTL fields, and the
OVERFLOW enum value — so the generated stubs could not reach the new surface.Maintenance release.
roadmap.md to
docs/warden-roadmap.md so it no longer ships in the module root. No API,
on-disk-format, or behavior change.This release makes FileDB embeddable as a Go library for the first time, in
addition to the standalone server. The storage engine can now run entirely
in-process — no gRPC, no network, no daemon — via the new filedb façade and the
promoted public engine / store / query packages. It also folds in the
durability-hardening and query-at-scale work that landed since v0.1.0.
internal/engine, internal/store, and
internal/query are promoted to public engine/, store/, and query/. You
can now go get github.com/srjn45/filedbv2/engine and open a database
in-process. A CI gate (make deps-check) keeps the engine free of
gRPC/protobuf/Prometheus/cobra dependencies, and an embeddemo/ module proves
the engine builds standalone. (EMB-1, EMB-3)scriva façade — root package. scriva.Open(dir, opts…) roots a
store at a directory and lazily opens-or-creates named collections with
per-collection options (WithUniqueIndex, WithCollectionSyncMode, …).
Existing collections on disk are discovered automatically; db.Engine() drops
to the underlying *engine.DB. (EMB-2)scriva.Open defaults every
collection to SyncModeInterval at a 1s cadence (bounded crash-loss window
without a per-write fsync); opt back into SyncModeAlways per collection. The
raw engine default (SyncModeNone) is unchanged. (OPS-1)InsertWithKey, FindByKey,
UpdateByKey, DeleteByKey on Collection. Keys live in a reserved _key
field enforced unique by a secondary index; O(1) lookup. New typed errors
engine.ErrKeyNotFound and engine.ErrReservedField. (KEY-1)EnsureUniqueIndex(field) and the typed
engine.ErrDuplicateKey, enforced on insert, update, and CommitTx; the
unique flag survives persist/reopen/rebuild. (KEY-2)Rev (1 on insert, +1 per update). New engine.Record{ID,Key,Rev,Ts,Data}
and Get/GetByKey expose it without a body read; UpdateIfRev (optimistic
concurrency) and UpdateIfMatch (predicate) apply a write only if a condition
on the current record still holds, in a single critical section. A stale
revision, false predicate, or missing key is a clean (false, nil) no-op.
store.Entry and ScanResult gain Rev. (KEY-3)Upsert(key, data) inserts-if-absent / replaces-if-present in one
critical section, returning the resulting Record and emitting the matching
Watch event. (KEY-4)Count(filter) returns the number of matching
live records without materializing them (O(1) for match-all and indexed
equality); Exists(key) is an O(1) key-presence check. (QRY-3)Collection.LoadJSONL(r, keyField) streams NDJSON through the
normal write path (indexes, uniqueness, durability, Watch all apply) as an
atomic, all-or-nothing batch; errors name the 1-based line number and leave
the collection untouched. (OPS-3)docs/embedding.md — full API reference for the embedding surface:
façade, keyed ops, CAS, upsert, querying, the in-process Watch contract
(including the OpOverflow resync sentinel), durability modes, store.Entry,
the versioning/stability policy, and a JSON-store migration guide with a worked
example. A runnable examples/watch program and an Example_watch test
demonstrate in-process subscriptions. (EMB-4, OPS-2)engine.DB.CollectionWithConfig(name, cfg) — open-or-create a collection
with an explicit per-collection config (lets the façade give each collection
its own durability/compaction settings).meta.json writes, and propagated rotation errors.--tx-timeout GCs abandoned BeginTx sessions.OpOverflow
sentinel (resync cue) instead of silently dropping events.Find — limit/offset/order_by are pushed into
the engine and results stream as they are read, with context cancellation; a
limited query is bounded by page size, not collection size.order_by — ordering uses the same type-aware
comparison as the filter operators (numbers order numerically).gt/gte/lt/lte lookups.--default-ttl); expired records are hidden from
reads immediately and reclaimed by compaction.internal/… → public) affects only code that imported the previously
internal packages — external consumers could not, so this is additive in
practice.Initial release: the feature-complete core.
BeginTx/CommitTx/RollbackTx), and a
server-streaming Watch change feed.--sync none|always|interval), optional TLS, YAML
config file, and Prometheus metrics..fql batch
scripting.v* tags.Can you improve this documentation?Edit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |