org.apache.pulsar.compaction

Class TwoPhaseCompactor

java.lang.Object

org.apache.pulsar.compaction.Compactor

org.apache.pulsar.compaction.TwoPhaseCompactor

public class TwoPhaseCompactor
extends Compactor

Compaction will go through the topic in two passes. The first pass selects latest offset for each key in the topic. Then the second pass writes these values to a ledger.

The two passes are required to avoid holding the payloads of each of the latest values in memory, as the payload can be many orders of magnitude larger than a message id.

Field Summary

Fields

Modifier and Type	Field and Description
static Duration	PHASE_ONE_LOOP_READ_TIMEOUT

Fields inherited from class org.apache.pulsar.compaction.Compactor

COMPACTION_SUBSCRIPTION, conf, scheduler

Constructor Summary

Constructors

Constructor and Description
TwoPhaseCompactor(ServiceConfiguration conf, org.apache.pulsar.client.api.PulsarClient pulsar, BookKeeper bk, ScheduledExecutorService scheduler)

Method Summary

Modifier and Type	Method and Description
protected CompletableFuture<Long>	doCompaction(RawReader reader, BookKeeper bk)

Methods inherited from class org.apache.pulsar.compaction.Compactor

compact

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PHASE_ONE_LOOP_READ_TIMEOUT

public static final Duration PHASE_ONE_LOOP_READ_TIMEOUT

Constructor Detail

TwoPhaseCompactor

public TwoPhaseCompactor(ServiceConfiguration conf,
                         org.apache.pulsar.client.api.PulsarClient pulsar,
                         BookKeeper bk,
                         ScheduledExecutorService scheduler)

Method Detail

doCompaction

protected CompletableFuture<Long> doCompaction(RawReader reader,
                                               BookKeeper bk)

Specified by:

doCompaction in class Compactor