Single-file embedded key-value database with multiple keyspaces, typed values, TTL/expiry, sorted sets, lists, vector similarity search, BM25 full-text search, and optional encryption at rest. All operations are synchronous. Concurrency: single writer, multiple readers (file-level locking, mmap snapshots). Database files use a copy-on-write B+ tree with double-buffered meta pages for atomic commits.
| Module | Description |
|---|---|
| mneme.crypto | Encryption key management for MNEME databases. |
| Name | Signature |
|---|---|
ks:get | ks:get(key) -> value, err |
ks:set | ks:set(key, value, opts) -> ok, err |
ks:set_bytes | ks:set_bytes(key, value, opts) -> ok, err |
ks:set_integer | ks:set_integer(key, value, opts) -> ok, err |
ks:set_float | ks:set_float(key, value, opts) -> ok, err |
ks:del | ks:del(key) -> ok, err |
ks:exists | ks:exists(key) -> exists |
ks:incr | ks:incr(key, delta) -> new_value, err |
ks:decr | ks:decr(key, delta) -> new_value, err |
ks:ttl | ks:ttl(key) -> seconds |
ks:persist | ks:persist(key) -> ok, err |
ks:purge_expired | ks:purge_expired() -> count, err |
ks:range | ks:range(start_key, end_key) -> iterator |
ks:count | ks:count(prefix) -> n |
ks:stats | ks:stats() -> stats, err |
ks:batch | ks:batch(fn) -> ok, err |
ks:vset | ks:vset(key, vec) -> ok, err |
ks:vget | ks:vget(key) -> vec, err |
ks:vsearch | ks:vsearch(query, opts) -> results |
ks:vdim | ks:vdim() -> dim |
zs:add | zs:add(score, member, ...) -> ok, err |
zs:rem | zs:rem(member) -> ok, err |
zs:score | zs:score(member) -> score |
zs:card | zs:card() -> count |
zs:rank | zs:rank(member) -> rank |
zs:range_by_score | zs:range_by_score(min, max, opts) -> results |
zs:range_by_rank | zs:range_by_rank(start, stop) -> results |
zs:iter | zs:iter(opts) -> iterator |
zs:batch_incr | zs:batch_incr(members, delta) -> ok, err |
ks:sorted_set | ks:sorted_set(name) -> zset |
lst:rpush | lst:rpush(value) -> length, err |
lst:lpush | lst:lpush(value) -> length, err |
lst:rpop | lst:rpop() -> value |
lst:lpop | lst:lpop() -> value |
lst:len | lst:len() -> length |
lst:index | lst:index(idx) -> value |
lst:range | lst:range(start, stop) -> items |
ks:list | ks:list(name) -> list |
ft:put | ft:put(doc_id, text) -> ok, err |
ft:get | ft:get(doc_id) -> text, err |
ft:del | ft:del(doc_id) -> ok, err |
ft:exists | ft:exists(doc_id) -> exists |
ft:search | ft:search(query, opts) -> results |
ft:scan | ft:scan(prefix) -> iterator |
ft:count | ft:count(prefix) -> n |
ft:stats | ft:stats() -> stats, err |
ft:pause_indexing | ft:pause_indexing() |
ft:rebuild_index | ft:rebuild_index() -> ok, err |
db:ft_keyspace | db:ft_keyspace(name, opts) -> ft, err |
db:close | db:close() -> ok, err |
db:info | db:info() -> info, err |
db:keyspace | db:keyspace(name, opts) -> ks, err |
db:keyspaces | db:keyspaces() -> names, err |
db:drop_keyspace | db:drop_keyspace(name) -> ok, err |
db:add_encryption_key | db:add_encryption_key(pubkey) -> ok, err |
db:remove_encryption_key | db:remove_encryption_key(fingerprint) -> ok, err |
db:list_encryption_keys | db:list_encryption_keys() -> keys, err |
db:stats | db:stats() -> stats, err |
db:compact | db:compact(output_path) -> ok, err |
open | open(path, opts) -> db, err |
ks:get(
key) ->value,err
Get the value for a key
Returns the value associated with the key, or nil if not found or expired. Type is preserved: integers return as numbers, floats as numbers, bytes as strings. Encrypted keyspaces decrypt transparently.
ks:set(
key,value,opts) ->ok,err
Set a key-value pair with optional TTL
Stores the value under the key. Type is auto-detected: strings become bytes,
integers that fit in int64 become integers, other numbers become floats.
Pass opts.ttl (seconds) to set an expiration time.
ks:set_bytes(
key,value,opts) ->ok,err
Set a key-value pair with explicit bytes type
ks:set_integer(
key,value,opts) ->ok,err
Set a key-value pair with explicit integer type
ks:set_float(
key,value,opts) ->ok,err
Set a key-value pair with explicit float type
ks:del(
key) ->ok,err
Delete a key
ks:exists(
key) ->exists
Check if a key exists
ks:incr(
key,delta) ->new_value,err
Increment an integer key
Atomically increments the integer stored at key by delta (default 1). If the key does not exist, it is initialized to 0 before incrementing. Returns the new value. Fails with "type mismatch" if the key holds a non-integer type.
ks:decr(
key,delta) ->new_value,err
Decrement an integer key
ks:ttl(
key) ->seconds
Get remaining TTL for a key
Returns the remaining time-to-live in seconds, or nil if the key has no TTL set, does not exist, or has already expired.
ks:persist(
key) ->ok,err
Remove the TTL from a key
ks:purge_expired() ->
count,err
Remove all expired keys from the keyspace
ks:range(
start_key,end_key) ->iterator
Iterate keys in a lexicographic range
Returns a stateful iterator that yields (key, value) pairs for keys
in the half-open range [start_key, end_key). Both bounds are optional:
nil start begins from the first key, nil end continues to the last.
ks:count(
prefix) ->n
Count keys matching a prefix
ks:stats() ->
stats,err
Get B-tree statistics for the keyspace
Returns a table with keys, depth, leaf_pages, branch_pages, and
overflow_pages fields.
ks:batch(
fn) ->ok,err
Execute multiple operations in a single transaction
Calls fn(batch) where the batch object supports set, set_integer,
set_float, del, and incr methods. All operations share a single
write transaction and are committed with one fsync on success. If fn
raises an error, the transaction is aborted.
ks:vset(
key,vec) ->ok,err
Store a vector
Stores a float32 vector (a Lua table of numbers). The dimension is inferred from the table length and must be consistent across all vectors in the keyspace. Not supported with encryption.
ks:vget(
key) ->vec,err
Retrieve a vector
ks:vsearch(
query,opts) ->results
Search for similar vectors
Returns an array of {key, score} tables sorted by similarity.
Options: metric ("cosine", "dot", or "l2"; default "cosine"),
top_k (max results; default 10), filter (key prefix filter).
Cosine: 0=identical, 2=opposite. Dot: higher=more similar. L2: 0=identical.
ks:vdim() ->
dim
Get the vector dimension of the keyspace
zs:add(
score,member,...) ->ok,err
Add or update members with scores
Accepts one or more (score, member) pairs as varargs. If a member already exists, its score is updated. New members increment the cardinality.
zs:rem(
member) ->ok,err
Remove a member from the sorted set
zs:score(
member) ->score
Get the score of a member
zs:card() ->
count
Get the cardinality of the sorted set
zs:rank(
member) ->rank
Get the rank of a member in ascending score order
zs:range_by_score(
min,max,opts) ->results
Get members within a score range
Returns an array of {score, member} tables for members with scores in
[min, max]. Options: limit (max results), reverse (descending order;
when true, pass (high, low) as min/max).
zs:range_by_rank(
start,stop) ->results
Get members by rank range
Returns an array of {score, member} tables for members at ranks in
[start, stop] (0-based, inclusive).
zs:iter(
opts) ->iterator
Iterate all members in score order
Returns a stateful iterator that yields (score, member) pairs in
ascending score order. Pass opts.reverse = true for descending order.
zs:batch_incr(
members,delta) ->ok,err
Increment scores for multiple members in one transaction
Atomically increments the score of each member in the array by delta (default 1). New members are created with score equal to delta. Uses a single transaction with one fdatasync.
ks:sorted_set(
name) ->zset
Get a sorted set handle within this keyspace
lst:rpush(
value) ->length,err
Push a value to the right (tail) of the list
lst:lpush(
value) ->length,err
Push a value to the left (head) of the list
lst:rpop() ->
value
Pop a value from the right (tail) of the list
lst:lpop() ->
value
Pop a value from the left (head) of the list
lst:len() ->
length
Get the length of the list
lst:index(
idx) ->value
Get element at index
Returns the element at the given 0-based index. Negative indices count from the end: -1 is the last element.
lst:range(
start,stop) ->items
Get a range of elements
Returns an array of values for elements at indices [start, stop]
(0-based, inclusive). Negative indices wrap from the end. Defaults:
start=0, stop=-1 (entire list).
ks:list(
name) ->list
Get a list handle within this keyspace
ft:put(
doc_id,text) ->ok,err
Index a document for full-text search
Stores the document text and indexes it for BM25 search. If a document
with the same ID already exists, it is updated (old index entries are
removed first). When indexing is paused, the text is stored but not
indexed until rebuild_index() is called.
ft:get(
doc_id) ->text,err
Retrieve the original text of a document
ft:del(
doc_id) ->ok,err
Delete a document and its index entries
ft:exists(
doc_id) ->exists
Check if a document exists
ft:search(
query,opts) ->results
Search for documents matching a query
Returns an array of {key, score} tables ranked by BM25 score (descending).
Options: top_k (max results; default 10), k1 (BM25 k1; default 1.2),
b (BM25 b; default 0.75). The query is analyzed with the same analyzer
used to index the documents.
ft:scan(
prefix) ->iterator
Iterate documents by ID prefix
Returns a stateful iterator that yields (doc_id, text) pairs for all
documents whose ID matches the prefix (or all documents if omitted).
ft:count(
prefix) ->n
Count documents matching a prefix
ft:stats() ->
stats,err
Get corpus statistics
Returns a table with doc_count, unique_terms, avg_doc_len, and
total_doc_len fields.
ft:pause_indexing()
Pause automatic indexing for bulk loading
After calling this, ft:put() stores document text but skips index
updates. Call ft:rebuild_index() after bulk loading to build the
index in a single batch.
ft:rebuild_index() ->
ok,err
Rebuild the full-text index from stored documents
Deletes all existing index entries and rebuilds from stored document text. Commits in batches of 1000 documents for large corpora. Resumes automatic indexing when complete.
db:ft_keyspace(
name,opts) ->ft,err
Get or create a full-text search keyspace
Returns a full-text keyspace handle with BM25 ranked search. Options:
analyzer (custom tokenizer function), analyzer_name (string label),
k1 (BM25 k1; default 1.2), b (BM25 b; default 0.75). Encryption
is not supported for FT keyspaces.
db:close() ->
ok,err
Close the database
db:info() ->
info,err
Get low-level database info
db:keyspace(
name,opts) ->ks,err
Get or create a keyspace
Returns a keyspace handle. The keyspace is created on first write if it
does not exist. Pass opts.encrypted = true to enable per-keyspace
encryption (requires the database to be opened with an encryption key).
db:keyspaces() ->
names,err
List all keyspace names
db:drop_keyspace(
name) ->ok,err
Drop a keyspace and its data
db:add_encryption_key(
pubkey) ->ok,err
Authorize an additional encryption key
Adds a new Ed25519 public key (32 bytes) to the database's authorized key
list. The current key (from encryption.key_file) is used to unwrap the
DEK, which is then re-wrapped for the new key.
db:remove_encryption_key(
fingerprint) ->ok,err
Revoke an authorized encryption key
Removes the key slot matching the given 32-byte SHA-256 fingerprint. Cannot remove the last remaining key slot.
db:list_encryption_keys() ->
keys,err
List authorized encryption keys
Returns an array of {fingerprint} tables, one per authorized key slot.
db:stats() ->
stats,err
Get database-level statistics
Returns a table with page_count, retired_pages, live_pages,
reclaimable_pages, bloat_ratio, keyspaces, and file_size fields.
db:compact(
output_path) ->ok,err
Compact the database to a new file
Copies all live (non-expired) data to a new database file, reclaiming free pages and removing fragmentation. The current database is not modified. Commits in batches of 10000 keys per keyspace.
open(
path,opts) ->db,err
Open or create a MNEME database
Opens the database file at path. Creates it if it does not exist (unless
opts.create is false). Options: readonly (open read-only),
create (set to false to fail if file does not exist),
sync ("none" to disable fsync, useful for bulk loading),
encryption (table with key_file path or seed+pubkey for
Ed25519-based encryption at rest).