org.apache.pulsar.broker.transaction.buffer

Interface TransactionBuffer

All Known Implementing Classes:

TopicTransactionBuffer, TransactionBufferDisable

@Beta
public interface TransactionBuffer

A class represent a transaction buffer. The transaction buffer is per partition. All the messages published within transactions are appended to a transaction buffer. They are not visible to consumers or readers before the transaction is committed.

When committing transaction starts, the broker will append a `COMMITTED` marker to the data partition first to mark the transaction is committed. The broker knows the data ledger of the commit marker and calls #commitTxn(TxnID, long, long) to commit and seal the buffer.

When the marker is appended to the data partition, all the entries are visible to the consumers. So a transaction reader TransactionBufferReader will be opened to read the entries when the broker sees the commit marker. There is a chance broker crashes after writing the marker to data partition but before committing the transaction in transaction buffer. That is fine. Because the transaction buffer will fail opening the transaction buffer reader since the transaction is still marked as open. The broker can keep retry until the TC (transaction coordinator) eventually commits the buffer again.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Void>	abortTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark) Abort the transaction and all the entries of this transaction will be discarded.
CompletableFuture<Position>	appendBufferToTxn(org.apache.pulsar.client.api.transaction.TxnID txnId, long sequenceId, io.netty.buffer.ByteBuf buffer) Append the buffer to the transaction buffer.
CompletableFuture<Void>	checkIfTBRecoverCompletely(boolean isTxn) Wait TransactionBuffer Recovers completely.
CompletableFuture<Void>	clearSnapshot() Clear up the snapshot of the TransactionBuffer.
CompletableFuture<Void>	closeAsync() Close the buffer asynchronously.
CompletableFuture<Void>	commitTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark) Commit the transaction and seal the buffer for this transaction.
PositionImpl	getMaxReadPosition() Get the can read max position.
org.apache.pulsar.common.policies.data.TransactionBufferStats	getStats() Get transaction stats in buffer.
org.apache.pulsar.common.policies.data.TransactionInBufferStats	getTransactionInBufferStats(org.apache.pulsar.client.api.transaction.TxnID txnID) Get transaction in buffer stats.
CompletableFuture<TransactionMeta>	getTransactionMeta(org.apache.pulsar.client.api.transaction.TxnID txnID) Return the metadata of a transaction in the buffer.
boolean	isTxnAborted(org.apache.pulsar.client.api.transaction.TxnID txnID) Close the buffer asynchronously.
CompletableFuture<TransactionBufferReader>	openTransactionBufferReader(org.apache.pulsar.client.api.transaction.TxnID txnID, long startSequenceId) Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.
CompletableFuture<Void>	purgeTxns(List<Long> dataLedgers) Purge all the data of the transactions who are committed and stored in the provided data ledgers.
void	syncMaxReadPositionForNormalPublish(PositionImpl position) Sync max read position for normal publish.

Method Detail

getTransactionMeta

CompletableFuture<TransactionMeta> getTransactionMeta(org.apache.pulsar.client.api.transaction.TxnID txnID)

Return the metadata of a transaction in the buffer.

Parameters:

txnID - the transaction id

Returns:

a future represents the result of the operation

Throws:

org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

appendBufferToTxn

CompletableFuture<Position> appendBufferToTxn(org.apache.pulsar.client.api.transaction.TxnID txnId,
                                              long sequenceId,
                                              io.netty.buffer.ByteBuf buffer)

Append the buffer to the transaction buffer.

The entry will be indexed by txnId and sequenceId.

Parameters:

txnId - the transaction id

sequenceId - the sequence id of the entry in this transaction buffer.

buffer - the entry buffer

Returns:

a future represents the result of the operation.

Throws:

org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionSealedException - if the transaction has been sealed.

openTransactionBufferReader

CompletableFuture<TransactionBufferReader> openTransactionBufferReader(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                                                       long startSequenceId)

Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.

Parameters:

txnID - transaction id

startSequenceId - the sequence id to start read

Returns:

a future represents the result of open operation.

Throws:

org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

commitTxn

CompletableFuture<Void> commitTxn(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                  long lowWaterMark)

Commit the transaction and seal the buffer for this transaction.

If a transaction is sealed, no more entries can be appendBufferToTxn(TxnID, long, ByteBuf).

Parameters:

txnID - the transaction id

lowWaterMark - the low water mark of this transaction

Returns:

a future represents the result of commit operation.

Throws:

org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

abortTxn

CompletableFuture<Void> abortTxn(org.apache.pulsar.client.api.transaction.TxnID txnID,
                                 long lowWaterMark)

Abort the transaction and all the entries of this transaction will be discarded.

Parameters:

txnID - the transaction id

lowWaterMark - the low water mark of this transaction

Returns:

a future represents the result of abort operation.

Throws:

org.apache.pulsar.broker.transaction.buffer.exceptions.TransactionNotFoundException - if the transaction is not in the buffer.

purgeTxns

CompletableFuture<Void> purgeTxns(List<Long> dataLedgers)

Purge all the data of the transactions who are committed and stored in the provided data ledgers.

This method will be called by the broker before they delete the ledgers. It ensures that all the transactions committed in those ledgers can be purged.

Parameters:

dataLedgers - the list of data ledgers.

Returns:

a future represents the result of purge operations.

clearSnapshot

CompletableFuture<Void> clearSnapshot()

Clear up the snapshot of the TransactionBuffer.

Returns:

Clear up operation result.

closeAsync

CompletableFuture<Void> closeAsync()

Close the buffer asynchronously.

Returns:

isTxnAborted

boolean isTxnAborted(org.apache.pulsar.client.api.transaction.TxnID txnID)

Close the buffer asynchronously.

Parameters:

txnID - TxnID txnId.

Returns:

the txnId is aborted.

syncMaxReadPositionForNormalPublish

void syncMaxReadPositionForNormalPublish(PositionImpl position)

Sync max read position for normal publish.

Parameters:

position - PositionImpl the position to sync.

getMaxReadPosition

PositionImpl getMaxReadPosition()

Get the can read max position.

Returns:

the stable position.

getTransactionInBufferStats

org.apache.pulsar.common.policies.data.TransactionInBufferStats getTransactionInBufferStats(org.apache.pulsar.client.api.transaction.TxnID txnID)

Get transaction in buffer stats.

Returns:

the transaction in buffer stats.

getStats

org.apache.pulsar.common.policies.data.TransactionBufferStats getStats()

Get transaction stats in buffer.

Returns:

the transaction stats in buffer.

checkIfTBRecoverCompletely

CompletableFuture<Void> checkIfTBRecoverCompletely(boolean isTxn)

Wait TransactionBuffer Recovers completely. Take snapshot after TB Recovers completely.

Parameters:

isTxn -

Returns:

a future which has completely if isTxn = false. Or a future return by takeSnapshot.