SSI-Service Database Considerations

The SSI-Services currently uses BoltDB as its database, which is well-suited for its current needs. However, as the SSI-Services expands and is deployed in more production environments, it may be necessary to support a wider range of database plug-ins. This is because different production environments may have different requirements and constraints, such as different performance, scalability, and security needs.

The data in ssi-service:

  • needs to live for a long time. There will be many objects like revocation vcs that will need to exist indefinitely
  • has moderate document size. A complex signed JWT object credential application can take up a few 100 of kb
  • lends itself to document-based storage

To support a wider range of database plug-ins, the SSI-Services will need to be designed in a modular and flexible way. This will allow different database plug-ins to be easily integrated into the system, and will enable the SSI-Services to take advantage of the unique features and capabilities of each database.

We currently have a modular design where we can ‘plug in’ any database without much coding or refactoring. It would only take the time to implement the database functions for that chosen DB.

Alternatively we can use a DAL to allow us to easily plug into any number of databases which me and other believe is the correct approach.

DAL Options in golang:
upper/db - GitHub - upper/db: Data access layer for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.
mgo - GitHub - go-mgo/mgo: The MongoDB driver for Go. UNMAINTAINED - SEE BELOW
gorm - GORM Guides | GORM - The fantastic ORM library for Golang, aims to be developer friendly.
dalgo - GitHub - strongo/dalgo: Database Abstraction Layer (DAL) in Go language
other - ??

Overall, the ability to support a wide range of database plug-ins is important for the future expansion and success of the SSI-Services. By designing the system in a modular and flexible way, and by thoroughly testing each database plug-in, the SSI-Services will be able to adapt to the needs of different production environments and provide high-quality services to its users.

I Would love the community’s thoughts on this and if anyone has had any gotchas implementing any of these into golang!

1 Like

Document based should make abstracting this easier than say a relational database.

As mentioned elsewhere GitHub - philippgille/gokv: Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more) as a data access layer is quite interesting with its set of implementations.

Would also be interesting to try things out with a diverse set of implementation, when a DAL is picked, ie if say an s3 backend provides satisfactory performance that is interesting as it gives people an option to scale without hassle on AWS for example (blob style document stores I expect are slowest latency wise but maybe not).

1 Like

I wish i was part of the team so I could make a difference