Hitchy Odem Manual

Appearance

Adapters ​

Models are relying on adapters for accessing its data in a connected data storage. When defining a model an adapter may be provided to use for all instances of resulting model instead of some default adapter.

This document is about those adapters, how to use them and how to create one yourself.

Available adapters ​

There are two adapters distributed as part of @hitchy/plugin-odem and exposed as service components:

In addition, separate plugins are capable of providing additional adapters, just like:

Adapters expose an API very similar to commonly available key-value stores such as Redis, Etcd and similar making it rather easy to use those instead of a local filesystem for persistently storing managed data.

Storing data in volatile memory ​

The service component OdemAdapterMemory of Hitchy's document-oriented database is an adapter most useful for developing and testing applications as it does not actually save any data but manages records in volatile memory, only.

Creating instances does not take any options. Different instances manage different sets of records, so it's even possible to have different sets of data separately managed in volatile memory.

javascript
const adapter = new api.services.OdemAdapterMemory();

This adapter may be provided on calling Model.define() explicitly. In a default setup it is also used whenever omitting provision of adapter on calling that function.

Storing data in local filesystem ​

This adapter is meant to implement a very basic opportunity to persistently save records without relying on any additional software. It is thus suitable for smaller, single-node applications e.g. desktop software based on Electron.

When using it all records are saved in a folder of your local filesystem. That's why you should provide the path name of folder to contain all those files as option on creating instances of OdemAdapterFile:

javascript
const adapter = new api.services.OdemAdapterFile( {
   dataSource: "/path/name/of/a/folder"
} );

Storing data in etcd cluster ​

Separate plugin @hitchy/plugin-odem-etcd enables data being persisted in an etcd cluster, too. This option is intended for use with production setups running in a cluster enabling horizontal scaling of your Hitchy application.

See the plugin's documentation for up-to-date information on how to integrate it with your application. The following excerpt is meant to demonstrate the ease of integrating Odem with etcd cluster instead of some local filesystem, only:

javascript
import FS from "node:fs";

const adapter = new api.services.OdemAdapterEtcd( {
    hosts: [
        "https://etcd1:2379",
        "https://etcd2:2379",
        "https://etcd3:2379",
    ],
    // optional: share single etcd cluster with multiple apps
    prefix: "apps/my-app",
    auth: {
        username: "etcd-login-user",
        password: "secret-pw",
    },
    // optional: authenticate server and client on encrypted connections
    credentials: {
        rootCertificate: FS.readFileSync( "path/to/ca.pem" ),
        certChain: FS.readFileSync( "path/to/cert.pem" ),
        privateKey: FS.readFileSync( "path/to/key.pem" ),
    }
} );

Configuring default adapter ​

In compliance with Hitchy's conventions you may create a file config/database.js in your Hitchy-based project's folder with content like this:

javascript
import File from "node:fs";

export default function() {
    return {
        database: {
            default : new this.services.OdemAdapterFile( {
                dataSource: "/path/name/of/a/folder"
            } ),
        },
    };
};

This example is replacing the default adapter storing all data in volatile memory with another adapter storing all data in a folder of your local filesystem.

Custom adapter implementation ​

Custom adapters can be implemented as services of your runtime, either as part of your application or in a plugin.

They always must inherit from api.services.OdemAdapterAbstract. Inheriting one service from another one requires your code to comply with Hitchy's common module pattern as it provides access on all existing components.

javascript
export default function() {
	return class OdemAdapterCustom extends this.service.OdemAdapter {
		// TODO implement your adapter class here
	}
};

For the class to be discovered and exposed as a service component with the same name, you need to put this example in a file api/service/odem/adapter/custom.js of your application or some plugin you are implementing.

You can then use this service when configuring your database context in file config/database.js:

javascript
import File from "node:fs";

export default function() {
    return {
        database: {
            default : new this.services.OdemAdapterCustom(),
        },
    };
};

Every adapter has to implement the following set of instance methods.

Context & Glossary

Understanding the API described below might require to understand every adapter's context as well as certain terms.

First of all, adapters are expected to operate on key-value-stores, only. There is no support for connecting to an RDBMS other than using it to flatly store pairs of keys and values in a single table.

The key is a string and has to comply with a certain format managed by Odem. It is representing a model and the UUID of one of its instances. Odem's model implementation includes methods for comnbining a model's name and the UUID of one of its instances and convert them into such a key to be used by the backend and vice versa.

On the other hand, the according value is assumed to be a serialized record of that instance's properties.

Using this pattern, there is exactly one key-value-pair per item of a model in a datasource some adapter is connecting with.

can v0.12.0 ​

This property provides a collection of boolean values each indicating whether current adapter instances is featuring a certain capability. This is used by Odem's model handling to decide whether certain actions are available or not. In some situations, the model handling is adjusting its behavior if possible when a particular feature of current adapter is marked as not available here.

Adapters come with different capabilities

There are adapters available that lack certain features in selected situations such as etcd client lacking support for streaming entries while in a transaction. This collection of capabilities has been introduced to account for that and e.g. stick to an existing cache if consuming code is still trying to fetch a list of entries while in a transaction.

Be aware of the fact that an adapter's capabilities may change based on its current status. For example, this applies to clones of adapters provided in context of transactions.

read ​

If true, the adapter is capable of reading a selected entry's record from the connected data source.

remove ​

If true, the adapter is capable of deleting a selected entry from the connected data source.

stream ​

If true, the adapter is capable of providing a stream of entries e.g. to match against some filter criteria. It applies to streaming entries' keys as well as their records.

test ​

If true, the adapter is capable of testing whether some particular entry is available in connected data source or not.

transact ​

If true, the adapter is capable of starting a transaction for applying complex adjustments to the connected data source.

watch ​

If true, the adapter is capable of monitoring entries of connected data source for modifications made by current runtime or any other runtime connected to the same data source.

write ​

If true, the adapter is capable of writing a selected entry's record to the connected data source. This includes creating new entries there.

create( keyTemplate, properties ) ​

This method creates a record for storing provided properties in a new KV pair at its connected datasource. It returns a promise resolved with the unique key that has been assigned to it.

The adapter or its connected datasource is responsible for safely picking a unique key for the new KV pair. A key template is provided to derive that unique key from a UUID assigned to the new KV pair.

javascript
const newKey = await MyModel.adapter.create( "odems/person/%u", { 
    firstName: "John",
    lastName: "Doe",
    born: "1983-09-03T00:00:00+1:00"
} );

Keys and key templates are opaque

Keys are basically path-like strings containing the name of a model and the UUID of one of the named model's items. However, the particular format of keys is managed by Odem. Thus, do not rely on a certain pattern but stick to the idea of keys being like POSIX-style path names at most.

The same applies to key templates. They are strings. They are like POSIX-style path names. Apart from that, the adapter is expected to replace any contained occurrence of %u with the UUID it has picked for the created KV-pair, only.

has( key ) ​

The method detects if connected datasource has a KV pair with the given key or not. It returns a promise resolved with true if a matching KV pair has been found for the given key and with false if there is no such KV pair.

javascript
const recordExists = await MyModel.adapter.has( keyOfJohnDoe );

purge() ​

This method is removing all existing KV pairs from its connected datasource, thus eventually is removing all data this adapter is managing for Odem.

The return value is a promise resolved once all data has been purged.

javascript
await MyModel.adapter.purge();

read( key ) ​

The method reads the record of serialized properties that has been stored as value of a KV pair addressed by the given key in the connected datasource. It returns a promise resolved with that record if given key is addresing some KV pair. It is rejected otherwise.

javascript
const properties = await MyModel.adapter.read( keyOfJohnDoe );

remove( key ) ​

The method removes a KV pair addresses by provided key from the connected datasource. It returns a promise resolved as soon as the record has been removed.

javascript
await MyModel.adapter.remove( key );

stream( options ) ​

The method creates an object-mode readable stream delivering keys, values or pairs of the two for every KV pair in the connected datasource.

It accepts a set of options to customize the created stream:

  • prefix adjusts the stream to consider KV pairs with a key matching this prefix, only. It defaults to empty string to have the stream process all KV pairs of the datasource.

  • maxDepth is a non-negative integer selecting the maximum number of levels to descend into the hierarchy of keys as they resemble path names of a filesystem. It is starting from optionally given prefix so that e.g. 0 is considering a sole KV pair with its key matching that prefix, only.

    By combining prefix and maxDepth, enumeration can be limited from ascending too far out of hierarchy and from descending too deeply into it.

    When omitted, the descending into a hierarchy of keys isn't limited at all.

  • separator customizes the character separating segments of the path-like keys from each other.

    It defaults to a forward slash to have POSIX-style path names as keys.

  • target is a string and must be one out of these options:

    • "key" requests resulting stream to produce the keys of every considered KV pair
    • "value" requests resulting stream to produce the values a.k.a. the records of properties of every considered KV pair
    • "entry" requests resulting stream to produce the key and value as a two-item array for every processed KV pair. This is the default.

  • invalidPolicy is a string and must be one out of these options:

    • "skip"" requests to ignore KV pairs e.g. with invalid keys or missing a value just as if they haven't existed in the first place. This is the default.
    • "fail" requests to emit error event on resulting stream and destroy it e.g. on processing a KV pair with an invalid key or without a proper value.
javascript
const keys = MyModel.adapter.stream( {
    target: "key"
} );

keys.on( "data", console.log );
keys.on( "error", console.error );

transaction( callback ) ​

This method requests to execute provided callback in a transaction. It usually prevents other code from adjusting the connected datasource while the transaction is active. In addition, it ensures to either have all adjustments made by the callback applied to the datasource or none of them in case the callback is failing by throwing an exception.

The callback is invoked with a custom copy of current adapter. The callback is required to use only that copy for interacting with the datasource. The callback is expected to return a promise which is either resolved to commit the transaction or rejected to roll it back.

The method transaction() is returning a promise itself which is either resolved as soon as changes made by a completing callback have been applied to the connected datasource or rejected as soon as temporary changes made to connected datasource by a failing callback have been rolled back.

Keep in mind that long-running callbacks lock the connected datasource at a particular focus level and thus may prevent other code from proceeding.

javascript
await MyModel.adapter.transaction( async tx => {
	// use provided copy of adapter, only
	const properties = await tx.read( someKey );
	properties.someValue++;
	await tx.write( someKey, properties );
} );
javascript
await MyModel.adapter.transaction( async tx => {
	// DO NOT use original adapter in a transaction's code
	const properties = await MyModel.adapter.read( someKey );
	properties.someValue++;
	await MyModel.adapter.write( someKey, properties );
} );

write( key, properties ) ​

The method writes provided record of properties to the KV pair addressed by given key in the connected datasource. It returns a promise resolved as soon as the record has been written.

javascript
await MyModel.adapter.write( key, properties );

Adapter.benefitsFromCaching ​

This static boolean property indicates whether a model using this adapter would benefit from locally caching fetched data or not. It is true by default.

When implementing your own adapter, it might come with its own caching. Exposing false in this property may prevent Odem from wasting memory in those cases.