org.apache.pulsar.broker.service.persistent

Class PersistentDispatcherMultipleConsumers

java.lang.Object

org.apache.pulsar.broker.service.AbstractBaseDispatcher

org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers

All Implemented Interfaces:

AsyncCallbacks.ReadEntriesCallback, Dispatcher

Direct Known Subclasses:

PersistentStickyKeyDispatcherMultipleConsumers, PersistentStreamingDispatcherMultipleConsumers

public class PersistentDispatcherMultipleConsumers
extends AbstractDispatcherMultipleConsumers
implements Dispatcher, AsyncCallbacks.ReadEntriesCallback

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	PersistentDispatcherMultipleConsumers.ReadType

Field Summary

Fields

Modifier and Type	Field and Description
protected static AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumers>	BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER
protected ManagedCursor	cursor
protected Optional<DispatchRateLimiter>	dispatchRateLimiter
protected boolean	havePendingRead
protected boolean	havePendingReplayRead
protected com.google.common.collect.Range<PositionImpl>	lastIndividualDeletedRangeFromCursorRecovery
protected PositionImpl	minReplayedPosition
protected String	name
protected int	readBatchSize
protected Backoff	readFailureBackoff
protected MessageRedeliveryController	redeliveryMessages
protected RedeliveryTracker	redeliveryTracker
protected boolean	shouldRewindBeforeReadingOrReplaying
protected PersistentTopic	topic
protected static AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumers>	TOTAL_AVAILABLE_PERMITS_UPDATER
protected int	totalAvailablePermits
protected int	totalUnackedMessages

Fields inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

consumerList, consumerSet, currentConsumerRoundRobinIndex, FALSE, IS_CLOSED_UPDATER, TRUE

Fields inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

serviceConfig, subscription

Constructor Summary

Constructors

Constructor and Description
PersistentDispatcherMultipleConsumers(PersistentTopic topic, ManagedCursor cursor, Subscription subscription)
PersistentDispatcherMultipleConsumers(PersistentTopic topic, ManagedCursor cursor, Subscription subscription, boolean allowOutOfOrderDelivery)

Method Summary

Modifier and Type	Method and Description
void	addConsumer(Consumer consumer)
protected boolean	addMessageToReplay(long ledgerId, long entryId)
protected boolean	addMessageToReplay(long ledgerId, long entryId, long stickyKeyHash)
void	addUnAckedMessages(int numberOfMessages)
protected Set<? extends Position>	asyncReplayEntries(Set<? extends Position> positions)
protected Set<? extends Position>	asyncReplayEntriesInOrder(Set<? extends Position> positions)
void	blockDispatcherOnUnackedMsgs()
protected org.apache.commons.lang3.tuple.Pair<Integer,Long>	calculateToRead(int currentTotalAvailablePermits)
protected void	cancelPendingRead()
boolean	canUnsubscribe(Consumer consumer)
boolean	checkAndUnblockIfStuck() Checks if dispatcher is stuck and unblocks the dispatch if needed.
void	clearDelayedMessages()
CompletableFuture<Void>	close() mark dispatcher closed to stop new incoming requests and disconnect all consumers.
void	consumerFlow(Consumer consumer, int additionalNumberOfMessages) Indicates that this consumer is now ready to receive more messages.
void	cursorIsReset()
CompletableFuture<Void>	disconnectActiveConsumers(boolean isResetCursor) Disconnect active consumers.
CompletableFuture<Void>	disconnectAllConsumers(boolean isResetCursor) disconnect all consumers.
CopyOnWriteArrayList<Consumer>	getConsumers()
protected int	getFirstAvailableConsumerPermits()
protected Set<PositionImpl>	getMessagesToReplayNow(int maxMessagesToRead)
String	getName()
long	getNumberOfDelayedMessages()
Optional<DispatchRateLimiter>	getRateLimiter()
RedeliveryTracker	getRedeliveryTracker()
protected int	getStickyKeyHash(Entry entry)
PersistentTopic	getTopic()
int	getTotalUnackedMessages()
org.apache.pulsar.common.api.proto.CommandSubscribe.SubType	getType()
void	initializeDispatchRateLimiterIfNeeded(Optional<org.apache.pulsar.common.policies.data.Policies> policies)
protected boolean	isAtleastOneConsumerAvailable() returns true only if AbstractDispatcherMultipleConsumers.consumerList has atleast one unblocked consumer and have available permits.
boolean	isBlockedDispatcherOnUnackedMsgs()
boolean	isConsumerAvailable(Consumer consumer)
boolean	isConsumerConnected()
protected boolean	isConsumersExceededOnSubscription() Determine whether the number of consumers on the subscription reaches the threshold.
void	readEntriesComplete(List<Entry> entries, Object ctx)
void	readEntriesFailed(ManagedLedgerException exception, Object ctx)
void	readMoreEntries()
void	redeliverUnacknowledgedMessages(Consumer consumer)
void	redeliverUnacknowledgedMessages(Consumer consumer, List<PositionImpl> positions)
void	removeConsumer(Consumer consumer)
void	reset() mark dispatcher open to serve new incoming requests.
void	resetCloseFuture()
protected void	sendMessagesToConsumers(PersistentDispatcherMultipleConsumers.ReadType readType, List<Entry> entries)
protected boolean	shouldPauseDeliveryForDelayTracker()
boolean	trackDelayedDelivery(long ledgerId, long entryId, org.apache.pulsar.common.api.proto.MessageMetadata msgMetadata) Check with dispatcher if the message should be added to the delayed delivery tracker.
void	unBlockDispatcherOnUnackedMsgs()
void	updateRateLimiter(org.apache.pulsar.common.policies.data.DispatchRate dispatchRate)

Methods inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

getNextConsumer, getRandomConsumer, isClosed

Methods inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

computeReadLimits, filterEntriesForConsumer, filterEntriesForConsumer, isConsumersExceededOnSubscription, peekStickyKey, updateEntryWrapperWithMetadata

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.broker.service.Dispatcher

disconnectAllConsumers, isClosed, markDeletePositionMoveForward

Field Detail

topic

protected final PersistentTopic topic

cursor

protected final ManagedCursor cursor

lastIndividualDeletedRangeFromCursorRecovery

protected volatile com.google.common.collect.Range<PositionImpl> lastIndividualDeletedRangeFromCursorRecovery

redeliveryMessages

protected final MessageRedeliveryController redeliveryMessages

redeliveryTracker

protected final RedeliveryTracker redeliveryTracker

havePendingRead

protected volatile boolean havePendingRead

havePendingReplayRead

protected volatile boolean havePendingReplayRead

minReplayedPosition

protected volatile PositionImpl minReplayedPosition

shouldRewindBeforeReadingOrReplaying

protected boolean shouldRewindBeforeReadingOrReplaying

name

protected final String name

TOTAL_AVAILABLE_PERMITS_UPDATER

protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumers> TOTAL_AVAILABLE_PERMITS_UPDATER

totalAvailablePermits

protected volatile int totalAvailablePermits

readBatchSize

protected volatile int readBatchSize

readFailureBackoff

protected final Backoff readFailureBackoff

totalUnackedMessages

protected volatile int totalUnackedMessages

BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER

protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumers> BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER

dispatchRateLimiter

protected Optional<DispatchRateLimiter> dispatchRateLimiter

Constructor Detail

PersistentDispatcherMultipleConsumers

public PersistentDispatcherMultipleConsumers(PersistentTopic topic,
                                             ManagedCursor cursor,
                                             Subscription subscription)

PersistentDispatcherMultipleConsumers

public PersistentDispatcherMultipleConsumers(PersistentTopic topic,
                                             ManagedCursor cursor,
                                             Subscription subscription,
                                             boolean allowOutOfOrderDelivery)

Method Detail

addConsumer

public void addConsumer(Consumer consumer)
                 throws BrokerServiceException

Specified by:

addConsumer in interface Dispatcher

Throws:

BrokerServiceException

isConsumersExceededOnSubscription

protected boolean isConsumersExceededOnSubscription()

Description copied from class: AbstractBaseDispatcher

Determine whether the number of consumers on the subscription reaches the threshold.

Specified by:

isConsumersExceededOnSubscription in class AbstractBaseDispatcher

Returns:

removeConsumer

public void removeConsumer(Consumer consumer)
                    throws BrokerServiceException

Specified by:

removeConsumer in interface Dispatcher

Throws:

BrokerServiceException

consumerFlow

public void consumerFlow(Consumer consumer,
                         int additionalNumberOfMessages)

Description copied from interface: Dispatcher

Indicates that this consumer is now ready to receive more messages.

Specified by:

consumerFlow in interface Dispatcher

readMoreEntries

public void readMoreEntries()

calculateToRead

protected org.apache.commons.lang3.tuple.Pair<Integer,Long> calculateToRead(int currentTotalAvailablePermits)

asyncReplayEntries

protected Set<? extends Position> asyncReplayEntries(Set<? extends Position> positions)

asyncReplayEntriesInOrder

protected Set<? extends Position> asyncReplayEntriesInOrder(Set<? extends Position> positions)

isConsumerConnected

public boolean isConsumerConnected()

Specified by:

isConsumerConnected in interface Dispatcher

Overrides:

isConsumerConnected in class AbstractDispatcherMultipleConsumers

getConsumers

public CopyOnWriteArrayList<Consumer> getConsumers()

Specified by:

getConsumers in interface Dispatcher

Overrides:

getConsumers in class AbstractDispatcherMultipleConsumers

canUnsubscribe

public boolean canUnsubscribe(Consumer consumer)

Specified by:

canUnsubscribe in interface Dispatcher

Overrides:

canUnsubscribe in class AbstractDispatcherMultipleConsumers

close

public CompletableFuture<Void> close()

Description copied from interface: Dispatcher

mark dispatcher closed to stop new incoming requests and disconnect all consumers.

Specified by:

close in interface Dispatcher

Returns:

disconnectAllConsumers

public CompletableFuture<Void> disconnectAllConsumers(boolean isResetCursor)

Description copied from interface: Dispatcher

disconnect all consumers.

Specified by:

disconnectAllConsumers in interface Dispatcher

Returns:

cancelPendingRead

protected void cancelPendingRead()

Overrides:

cancelPendingRead in class AbstractDispatcherMultipleConsumers

disconnectActiveConsumers

public CompletableFuture<Void> disconnectActiveConsumers(boolean isResetCursor)

Description copied from interface: Dispatcher

Disconnect active consumers.

Specified by:

disconnectActiveConsumers in interface Dispatcher

resetCloseFuture

public void resetCloseFuture()

Specified by:

resetCloseFuture in interface Dispatcher

Overrides:

resetCloseFuture in class AbstractBaseDispatcher

reset

public void reset()

Description copied from interface: Dispatcher

mark dispatcher open to serve new incoming requests.

Specified by:

reset in interface Dispatcher

getType

public org.apache.pulsar.common.api.proto.CommandSubscribe.SubType getType()

Specified by:

getType in interface Dispatcher

Overrides:

getType in class AbstractDispatcherMultipleConsumers

readEntriesComplete

public void readEntriesComplete(List<Entry> entries,
                                Object ctx)

Specified by:

readEntriesComplete in interface AsyncCallbacks.ReadEntriesCallback

sendMessagesToConsumers

protected void sendMessagesToConsumers(PersistentDispatcherMultipleConsumers.ReadType readType,
                                       List<Entry> entries)

readEntriesFailed

public void readEntriesFailed(ManagedLedgerException exception,
                              Object ctx)

Specified by:

readEntriesFailed in interface AsyncCallbacks.ReadEntriesCallback

isAtleastOneConsumerAvailable

protected boolean isAtleastOneConsumerAvailable()

returns true only if AbstractDispatcherMultipleConsumers.consumerList has atleast one unblocked consumer and have available permits.

Returns:

getFirstAvailableConsumerPermits

protected int getFirstAvailableConsumerPermits()

isConsumerAvailable

public boolean isConsumerAvailable(Consumer consumer)

Specified by:

isConsumerAvailable in class AbstractDispatcherMultipleConsumers

redeliverUnacknowledgedMessages

public void redeliverUnacknowledgedMessages(Consumer consumer)

Specified by:

redeliverUnacknowledgedMessages in interface Dispatcher

redeliverUnacknowledgedMessages

public void redeliverUnacknowledgedMessages(Consumer consumer,
                                            List<PositionImpl> positions)

Specified by:

redeliverUnacknowledgedMessages in interface Dispatcher

addUnAckedMessages

public void addUnAckedMessages(int numberOfMessages)

Specified by:

addUnAckedMessages in interface Dispatcher

isBlockedDispatcherOnUnackedMsgs

public boolean isBlockedDispatcherOnUnackedMsgs()

blockDispatcherOnUnackedMsgs

public void blockDispatcherOnUnackedMsgs()

unBlockDispatcherOnUnackedMsgs

public void unBlockDispatcherOnUnackedMsgs()

getTotalUnackedMessages

public int getTotalUnackedMessages()

getName

public String getName()

getRedeliveryTracker

public RedeliveryTracker getRedeliveryTracker()

Specified by:

getRedeliveryTracker in interface Dispatcher

getRateLimiter

public Optional<DispatchRateLimiter> getRateLimiter()

Specified by:

getRateLimiter in interface Dispatcher

updateRateLimiter

public void updateRateLimiter(org.apache.pulsar.common.policies.data.DispatchRate dispatchRate)

Specified by:

updateRateLimiter in interface Dispatcher

initializeDispatchRateLimiterIfNeeded

public void initializeDispatchRateLimiterIfNeeded(Optional<org.apache.pulsar.common.policies.data.Policies> policies)

Specified by:

initializeDispatchRateLimiterIfNeeded in interface Dispatcher

trackDelayedDelivery

public boolean trackDelayedDelivery(long ledgerId,
                                    long entryId,
                                    org.apache.pulsar.common.api.proto.MessageMetadata msgMetadata)

Description copied from interface: Dispatcher

Check with dispatcher if the message should be added to the delayed delivery tracker. Return true if the message should be delayed and ignored at this point.

Specified by:

trackDelayedDelivery in interface Dispatcher

getMessagesToReplayNow

protected Set<PositionImpl> getMessagesToReplayNow(int maxMessagesToRead)

shouldPauseDeliveryForDelayTracker

protected boolean shouldPauseDeliveryForDelayTracker()

getNumberOfDelayedMessages

public long getNumberOfDelayedMessages()

Specified by:

getNumberOfDelayedMessages in interface Dispatcher

clearDelayedMessages

public void clearDelayedMessages()

Specified by:

clearDelayedMessages in interface Dispatcher

cursorIsReset

public void cursorIsReset()

Specified by:

cursorIsReset in interface Dispatcher

addMessageToReplay

protected boolean addMessageToReplay(long ledgerId,
                                     long entryId,
                                     long stickyKeyHash)

addMessageToReplay

protected boolean addMessageToReplay(long ledgerId,
                                     long entryId)

checkAndUnblockIfStuck

public boolean checkAndUnblockIfStuck()

Description copied from interface: Dispatcher

Checks if dispatcher is stuck and unblocks the dispatch if needed.

Specified by:

checkAndUnblockIfStuck in interface Dispatcher

getTopic

public PersistentTopic getTopic()

getStickyKeyHash

protected int getStickyKeyHash(Entry entry)