mneme — Lilush API

←index

Overview

Single-file embedded key-value database with multiple keyspaces, typed values, TTL/expiry, sorted sets, lists, vector similarity search, BM25 full-text search, and optional encryption at rest. All operations are synchronous. Concurrency: single writer, multiple readers (file-level locking, mmap snapshots). Database files use a copy-on-write B+ tree with double-buffered meta pages for atomic commits.

Submodules

ModuleDescription
mneme.cryptoEncryption key management for MNEME databases.

Functions

NameSignature
ks:getks:get(key) -> value, err
ks:setks:set(key, value, opts) -> ok, err
ks:set_bytesks:set_bytes(key, value, opts) -> ok, err
ks:set_integerks:set_integer(key, value, opts) -> ok, err
ks:set_floatks:set_float(key, value, opts) -> ok, err
ks:delks:del(key) -> ok, err
ks:existsks:exists(key) -> exists
ks:incrks:incr(key, delta) -> new_value, err
ks:decrks:decr(key, delta) -> new_value, err
ks:ttlks:ttl(key) -> seconds
ks:persistks:persist(key) -> ok, err
ks:purge_expiredks:purge_expired() -> count, err
ks:rangeks:range(start_key, end_key) -> iterator
ks:countks:count(prefix) -> n
ks:statsks:stats() -> stats, err
ks:batchks:batch(fn) -> ok, err
ks:vsetks:vset(key, vec) -> ok, err
ks:vgetks:vget(key) -> vec, err
ks:vsearchks:vsearch(query, opts) -> results
ks:vdimks:vdim() -> dim
zs:addzs:add(score, member, ...) -> ok, err
zs:remzs:rem(member) -> ok, err
zs:scorezs:score(member) -> score
zs:cardzs:card() -> count
zs:rankzs:rank(member) -> rank
zs:range_by_scorezs:range_by_score(min, max, opts) -> results
zs:range_by_rankzs:range_by_rank(start, stop) -> results
zs:iterzs:iter(opts) -> iterator
zs:batch_incrzs:batch_incr(members, delta) -> ok, err
ks:sorted_setks:sorted_set(name) -> zset
lst:rpushlst:rpush(value) -> length, err
lst:lpushlst:lpush(value) -> length, err
lst:rpoplst:rpop() -> value
lst:lpoplst:lpop() -> value
lst:lenlst:len() -> length
lst:indexlst:index(idx) -> value
lst:rangelst:range(start, stop) -> items
ks:listks:list(name) -> list
ft:putft:put(doc_id, text) -> ok, err
ft:getft:get(doc_id) -> text, err
ft:delft:del(doc_id) -> ok, err
ft:existsft:exists(doc_id) -> exists
ft:searchft:search(query, opts) -> results
ft:scanft:scan(prefix) -> iterator
ft:countft:count(prefix) -> n
ft:statsft:stats() -> stats, err
ft:pause_indexingft:pause_indexing()
ft:rebuild_indexft:rebuild_index() -> ok, err
db:ft_keyspacedb:ft_keyspace(name, opts) -> ft, err
db:closedb:close() -> ok, err
db:infodb:info() -> info, err
db:keyspacedb:keyspace(name, opts) -> ks, err
db:keyspacesdb:keyspaces() -> names, err
db:drop_keyspacedb:drop_keyspace(name) -> ok, err
db:add_encryption_keydb:add_encryption_key(pubkey) -> ok, err
db:remove_encryption_keydb:remove_encryption_key(fingerprint) -> ok, err
db:list_encryption_keysdb:list_encryption_keys() -> keys, err
db:statsdb:stats() -> stats, err
db:compactdb:compact(output_path) -> ok, err
openopen(path, opts) -> db, err

ks:get(key) -> value, err

Get the value for a key

Returns the value associated with the key, or nil if not found or expired. Type is preserved: integers return as numbers, floats as numbers, bytes as strings. Encrypted keyspaces decrypt transparently.

ks:set(key, value, opts) -> ok, err

Set a key-value pair with optional TTL

Stores the value under the key. Type is auto-detected: strings become bytes, integers that fit in int64 become integers, other numbers become floats. Pass opts.ttl (seconds) to set an expiration time.

ks:set_bytes(key, value, opts) -> ok, err

Set a key-value pair with explicit bytes type

ks:set_integer(key, value, opts) -> ok, err

Set a key-value pair with explicit integer type

ks:set_float(key, value, opts) -> ok, err

Set a key-value pair with explicit float type

ks:del(key) -> ok, err

Delete a key

ks:exists(key) -> exists

Check if a key exists

ks:incr(key, delta) -> new_value, err

Increment an integer key

Atomically increments the integer stored at key by delta (default 1). If the key does not exist, it is initialized to 0 before incrementing. Returns the new value. Fails with "type mismatch" if the key holds a non-integer type.

ks:decr(key, delta) -> new_value, err

Decrement an integer key

ks:ttl(key) -> seconds

Get remaining TTL for a key

Returns the remaining time-to-live in seconds, or nil if the key has no TTL set, does not exist, or has already expired.

ks:persist(key) -> ok, err

Remove the TTL from a key

ks:purge_expired() -> count, err

Remove all expired keys from the keyspace

ks:range(start_key, end_key) -> iterator

Iterate keys in a lexicographic range

Returns a stateful iterator that yields (key, value) pairs for keys in the half-open range [start_key, end_key). Both bounds are optional: nil start begins from the first key, nil end continues to the last.

ks:count(prefix) -> n

Count keys matching a prefix

ks:stats() -> stats, err

Get B-tree statistics for the keyspace

Returns a table with keys, depth, leaf_pages, branch_pages, and overflow_pages fields.

ks:batch(fn) -> ok, err

Execute multiple operations in a single transaction

Calls fn(batch) where the batch object supports set, set_integer, set_float, del, and incr methods. All operations share a single write transaction and are committed with one fsync on success. If fn raises an error, the transaction is aborted.

ks:vset(key, vec) -> ok, err

Store a vector

Stores a float32 vector (a Lua table of numbers). The dimension is inferred from the table length and must be consistent across all vectors in the keyspace. Not supported with encryption.

ks:vget(key) -> vec, err

Retrieve a vector

ks:vsearch(query, opts) -> results

Search for similar vectors

Returns an array of {key, score} tables sorted by similarity. Options: metric ("cosine", "dot", or "l2"; default "cosine"), top_k (max results; default 10), filter (key prefix filter). Cosine: 0=identical, 2=opposite. Dot: higher=more similar. L2: 0=identical.

ks:vdim() -> dim

Get the vector dimension of the keyspace

zs:add(score, member, ...) -> ok, err

Add or update members with scores

Accepts one or more (score, member) pairs as varargs. If a member already exists, its score is updated. New members increment the cardinality.

zs:rem(member) -> ok, err

Remove a member from the sorted set

zs:score(member) -> score

Get the score of a member

zs:card() -> count

Get the cardinality of the sorted set

zs:rank(member) -> rank

Get the rank of a member in ascending score order

zs:range_by_score(min, max, opts) -> results

Get members within a score range

Returns an array of {score, member} tables for members with scores in [min, max]. Options: limit (max results), reverse (descending order; when true, pass (high, low) as min/max).

zs:range_by_rank(start, stop) -> results

Get members by rank range

Returns an array of {score, member} tables for members at ranks in [start, stop] (0-based, inclusive).

zs:iter(opts) -> iterator

Iterate all members in score order

Returns a stateful iterator that yields (score, member) pairs in ascending score order. Pass opts.reverse = true for descending order.

zs:batch_incr(members, delta) -> ok, err

Increment scores for multiple members in one transaction

Atomically increments the score of each member in the array by delta (default 1). New members are created with score equal to delta. Uses a single transaction with one fdatasync.

ks:sorted_set(name) -> zset

Get a sorted set handle within this keyspace

lst:rpush(value) -> length, err

Push a value to the right (tail) of the list

lst:lpush(value) -> length, err

Push a value to the left (head) of the list

lst:rpop() -> value

Pop a value from the right (tail) of the list

lst:lpop() -> value

Pop a value from the left (head) of the list

lst:len() -> length

Get the length of the list

lst:index(idx) -> value

Get element at index

Returns the element at the given 0-based index. Negative indices count from the end: -1 is the last element.

lst:range(start, stop) -> items

Get a range of elements

Returns an array of values for elements at indices [start, stop] (0-based, inclusive). Negative indices wrap from the end. Defaults: start=0, stop=-1 (entire list).

ks:list(name) -> list

Get a list handle within this keyspace

ft:put(doc_id, text) -> ok, err

Index a document for full-text search

Stores the document text and indexes it for BM25 search. If a document with the same ID already exists, it is updated (old index entries are removed first). When indexing is paused, the text is stored but not indexed until rebuild_index() is called.

ft:get(doc_id) -> text, err

Retrieve the original text of a document

ft:del(doc_id) -> ok, err

Delete a document and its index entries

ft:exists(doc_id) -> exists

Check if a document exists

ft:search(query, opts) -> results

Search for documents matching a query

Returns an array of {key, score} tables ranked by BM25 score (descending). Options: top_k (max results; default 10), k1 (BM25 k1; default 1.2), b (BM25 b; default 0.75). The query is analyzed with the same analyzer used to index the documents.

ft:scan(prefix) -> iterator

Iterate documents by ID prefix

Returns a stateful iterator that yields (doc_id, text) pairs for all documents whose ID matches the prefix (or all documents if omitted).

ft:count(prefix) -> n

Count documents matching a prefix

ft:stats() -> stats, err

Get corpus statistics

Returns a table with doc_count, unique_terms, avg_doc_len, and total_doc_len fields.

ft:pause_indexing()

Pause automatic indexing for bulk loading

After calling this, ft:put() stores document text but skips index updates. Call ft:rebuild_index() after bulk loading to build the index in a single batch.

ft:rebuild_index() -> ok, err

Rebuild the full-text index from stored documents

Deletes all existing index entries and rebuilds from stored document text. Commits in batches of 1000 documents for large corpora. Resumes automatic indexing when complete.

db:ft_keyspace(name, opts) -> ft, err

Get or create a full-text search keyspace

Returns a full-text keyspace handle with BM25 ranked search. Options: analyzer (custom tokenizer function), analyzer_name (string label), k1 (BM25 k1; default 1.2), b (BM25 b; default 0.75). Encryption is not supported for FT keyspaces.

db:close() -> ok, err

Close the database

db:info() -> info, err

Get low-level database info

db:keyspace(name, opts) -> ks, err

Get or create a keyspace

Returns a keyspace handle. The keyspace is created on first write if it does not exist. Pass opts.encrypted = true to enable per-keyspace encryption (requires the database to be opened with an encryption key).

db:keyspaces() -> names, err

List all keyspace names

db:drop_keyspace(name) -> ok, err

Drop a keyspace and its data

db:add_encryption_key(pubkey) -> ok, err

Authorize an additional encryption key

Adds a new Ed25519 public key (32 bytes) to the database's authorized key list. The current key (from encryption.key_file) is used to unwrap the DEK, which is then re-wrapped for the new key.

db:remove_encryption_key(fingerprint) -> ok, err

Revoke an authorized encryption key

Removes the key slot matching the given 32-byte SHA-256 fingerprint. Cannot remove the last remaining key slot.

db:list_encryption_keys() -> keys, err

List authorized encryption keys

Returns an array of {fingerprint} tables, one per authorized key slot.

db:stats() -> stats, err

Get database-level statistics

Returns a table with page_count, retired_pages, live_pages, reclaimable_pages, bloat_ratio, keyspaces, and file_size fields.

db:compact(output_path) -> ok, err

Compact the database to a new file

Copies all live (non-expired) data to a new database file, reclaiming free pages and removing fragmentation. The current database is not modified. Commits in batches of 10000 keys per keyspace.

open(path, opts) -> db, err

Open or create a MNEME database

Opens the database file at path. Creates it if it does not exist (unless opts.create is false). Options: readonly (open read-only), create (set to false to fail if file does not exist), sync ("none" to disable fsync, useful for bulk loading), encryption (table with key_file path or seed+pubkey for Ed25519-based encryption at rest).