This chapter provides an overview from DAMA-DMBOK2 on data storage and operations (excerpted from pages 169-180), and then covers the additional data storage and operations responsibilities needed for blockchain to work well within our organizations.

Overview from DAMA-DMBOK2

This domain includes the design, implementation, and support of stored data, to maximize its value throughout its lifecycle, from creation/acquisition to disposal. Data storage and operations comprises two sub-activities:

• Database support focuses on activities related to the data lifecycle, from initial implementation of a database environment, through obtaining, backing up, and purging data. Monitoring and tuning are critical to ensuring good database performance.
• Database technology support involves defining technical requirements that will meet organizational needs, defining technical architecture, installing and administering technology, and resolving issues related to technology.

Database administrators (DBAs) play key roles in both aspects of data storage and operations. The role of DBA is the most established and most widely-adopted data professional role; database administration practices are perhaps the most mature of all data management practices. DBAs also play dominant roles in data operations and data security.

Companies rely on their information systems to operate. Data Storage and Operations activities are crucial to organizations that rely on data. Business continuity is the primary driver of these activities. If a system becomes unavailable, company operations may be impaired or stopped completely. A reliable data storage infrastructure for IT operations minimizes the risk of disruption.

The CAP Theorem (or Brewer’s Theorem) was developed in response to a shift toward more distributed systems (Brewer, 2000). The theorem asserts that a distributed system must make trade-offs between properties, which include:

• Consistency. The system must operate as designed and expected at all times.
• Availability. The system must be available when requested and must respond to each request.
• Partition Tolerance. The system must be able to continue operations during occasions of data loss or partial system failure.

The CAP Theorem states that at most two of the three properties can exist in any shared-data system.

Additional responsibilities due to blockchain

The ledger represents the database in blockchain. The DBA will need to align more closely with data architects and data modelers to achieve success with building and supporting blockchain ledgers.

Achieving performance

Recall the CAP Theorem, where you can pick at most two out of three of Consistency, Availability, or Partition Tolerance. With blockchain, however, you can pick all three. We can build a blockchain application that provides a high amount of integrity through multiple recordkeepers, which gives us consistency. The system can always be available and partition tolerant because it is distributed; if a few recordkeepers malfunction, the others will continue to operate.

The cost for achieving all three, however, is performance. Blockchain applications can be very slow because of all of the checking required by recordkeepers. It takes time (and resources) to make sure transactions are accurate, and to make sure that no one is trying to cheat the system.

Performance means the number of transactions that can be processed in a certain amount of time. Bitcoin, for example, currently has a maximum limit of seven transactions per second. Compare this with the VISA credit card processing network, which routinely handles 2,000 transactions per second, and can accommodate peak volumes of 10,000 transactions per second.31

The DBA must balance performance with other limiting factors, such as the number of recordkeepers required to confirm a transaction. The more recordkeepers that are used to confirm a transaction, the worse the performance, but the greater the chance that the transaction is valid. The fewer recordkeepers used, the better the performance, but the lower the chance the transaction is valid.

Defining permanence

Blockchain can store the data for “forever.” But what does “forever” mean, practically speaking? In some scenarios is “forever” too long?

For example, someone creates their will using blockchain. Fifty years later, someone will have the unfortunate task of invoking that will. Will the smart contract still be there to initiate the necessary transactions?

On the other side, what if an organization needs to delete data after seven years? Can that be programmed into a blockchain smart contract?

The DBA must get involved to make sure smart contracts remain active, and that there is a deactivation or archival process in place for the ledger.

Accepting majority rule

It is very important for mission-critical applications that either the entire transaction completes, or it rolls back. There can be no intermediary state. I can’t give $10 to my brother and still have that $10 in my account. It either got transferred to my brother or rolled back into my account.

With blockchain, however, it is not as simple as a “yes” or a “no.” “Yes” the transaction completed or “no” it rolled back. Some recordkeepers will say “yes,” some will say “no” or not respond, and whether a transaction completes or not depends on a majority vote by the recordkeepers.

The DBA will need to be comfortable in certain scenarios where a majority and not 100% of the recordkeepers confirm a transaction.

Supporting many recordkeepers

The DBA traditionally is responsible for keeping a database server up and running. With blockchain, however, there could be hundreds or thousands of servers that need support. Additional maintenance tools must be created for administration, or additional DBA resources will be needed.