org.apache.pulsar.broker.resourcegroup

Class ResourceGroupService

java.lang.Object

org.apache.pulsar.broker.resourcegroup.ResourceGroupService

public class ResourceGroupService
extends Object

The ResourceGroupService contains APIs to manipulate resource groups. It is assumed that there is a single instance of the ResourceGroupService on each broker. The control plane operations are somewhat stringent, and throw exceptions if (for instance) the op refers to a resource group which does not exist. The data plane operations (e.g., increment stats) throw exceptions as a last resort; if (e.g.) increment stats refers to a non-existent resource group, the stats are quietly not incremented.

See Also:

PulsarService

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ResourceGroupService.ResourceGroupOpStatus
protected static class	ResourceGroupService.ResourceGroupUsageStatsType

Field Summary

Fields

Modifier and Type	Field and Description
protected static long	maxIntervalForSuppressingReportsMSecs
protected static int	MaxUsageReportSuppressRounds
protected ResourceQuotaCalculator	quotaCalculator
protected static float	UsageReportSuppressionTolerancePercentage

Constructor Summary

Constructors

Constructor and Description
ResourceGroupService(PulsarService pulsar) Default constructor.
ResourceGroupService(PulsarService pulsar, TimeUnit timescale, ResourceUsageTopicTransportManager transportMgr, ResourceQuotaCalculator quotaCalc)

Method Summary

Modifier and Type	Method and Description
protected void	aggregateResourceGroupLocalUsages()
protected void	calculateQuotaForAllResourceGroups()
ResourceGroup	getNamespaceResourceGroup(org.apache.pulsar.common.naming.NamespaceName namespaceName) Return the resource group associated with a namespace.
protected long	getNumResourceGroups() Get the current number of RGs.
protected ResourceGroup.BytesAndMessagesCount	getPublishRateLimiters(String rgName)
protected static double	getRgLocalUsageByteCount(String rgName, String monClassName)
protected static double	getRgLocalUsageMessageCount(String rgName, String monClassName)
protected static double	getRgNamespaceRegistersCount(String rgName)
protected static double	getRgNamespaceUnRegistersCount(String rgName)
protected static double	getRgQuotaByteCount(String rgName, String monClassName)
protected static io.prometheus.client.Summary.Child.Value	getRgQuotaCalculationTime()
protected static double	getRgQuotaMessageCount(String rgName, String monClassName)
protected static double	getRgTenantRegistersCount(String rgName)
protected static double	getRgTenantUnRegistersCount(String rgName)
protected static double	getRgUpdatesCount(String rgName)
protected ResourceGroup.BytesAndMessagesCount	getRGUsage(String rgName, ResourceGroup.ResourceGroupMonitoringClass monClass, ResourceGroupService.ResourceGroupUsageStatsType statsType)
protected static io.prometheus.client.Summary.Child.Value	getRgUsageAggregationLatency()
protected boolean	incrementUsage(String tenantName, String nsName, ResourceGroup.ResourceGroupMonitoringClass monClass, ResourceGroup.BytesAndMessagesCount incStats) Increments usage stats for the resource groups associated with the given namespace and tenant.
void	registerNameSpace(String resourceGroupName, org.apache.pulsar.common.naming.NamespaceName fqNamespaceName) Registers a namespace as a user of a resource group.
void	registerTenant(String resourceGroupName, String tenantName) Registers a tenant as a user of a resource group.
void	resourceGroupCreate(String rgName, org.apache.pulsar.common.policies.data.ResourceGroup rgConfig) Create RG.
void	resourceGroupCreate(String rgName, org.apache.pulsar.common.policies.data.ResourceGroup rgConfig, ResourceUsagePublisher rgPublisher, ResourceUsageConsumer rgConsumer) Create RG, with non-default functions for resource-usage transport-manager.
void	resourceGroupDelete(String name) Delete RG.
ResourceGroup	resourceGroupGet(String resourceGroupName) Get a copy of the RG with the given name.
Set<String>	resourceGroupGetAll()
void	resourceGroupUpdate(String rgName, org.apache.pulsar.common.policies.data.ResourceGroup rgConfig) Update RG.
void	unRegisterNameSpace(String resourceGroupName, org.apache.pulsar.common.naming.NamespaceName fqNamespaceName) UnRegisters a namespace from a resource group.
void	unRegisterTenant(String resourceGroupName, String tenantName) UnRegisters a tenant from a resource group.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

quotaCalculator

protected final ResourceQuotaCalculator quotaCalculator

MaxUsageReportSuppressRounds

protected static final int MaxUsageReportSuppressRounds

See Also:

Constant Field Values

maxIntervalForSuppressingReportsMSecs

protected static long maxIntervalForSuppressingReportsMSecs

UsageReportSuppressionTolerancePercentage

protected static final float UsageReportSuppressionTolerancePercentage

See Also:

Constant Field Values

Constructor Detail

ResourceGroupService

public ResourceGroupService(PulsarService pulsar)

Default constructor.

ResourceGroupService

public ResourceGroupService(PulsarService pulsar,
                            TimeUnit timescale,
                            ResourceUsageTopicTransportManager transportMgr,
                            ResourceQuotaCalculator quotaCalc)

Method Detail

resourceGroupCreate

public void resourceGroupCreate(String rgName,
                                org.apache.pulsar.common.policies.data.ResourceGroup rgConfig)
                         throws PulsarAdminException

Create RG.

Throws:

if - RG with that name already exists.

PulsarAdminException

resourceGroupCreate

public void resourceGroupCreate(String rgName,
                                org.apache.pulsar.common.policies.data.ResourceGroup rgConfig,
                                ResourceUsagePublisher rgPublisher,
                                ResourceUsageConsumer rgConsumer)
                         throws PulsarAdminException

Create RG, with non-default functions for resource-usage transport-manager.

Throws:

if - RG with that name already exists (even if the resource usage handlers are different).

PulsarAdminException

resourceGroupGet

public ResourceGroup resourceGroupGet(String resourceGroupName)

Get a copy of the RG with the given name.

resourceGroupUpdate

public void resourceGroupUpdate(String rgName,
                                org.apache.pulsar.common.policies.data.ResourceGroup rgConfig)
                         throws PulsarAdminException

Update RG.

Throws:

if - RG with that name does not exist.

PulsarAdminException

resourceGroupGetAll

public Set<String> resourceGroupGetAll()

resourceGroupDelete

public void resourceGroupDelete(String name)
                         throws PulsarAdminException

Delete RG.

Throws:

if - RG with that name does not exist, or if the RG exists but is still in use.

PulsarAdminException

getNumResourceGroups

protected long getNumResourceGroups()

Get the current number of RGs. For testing.

registerTenant

public void registerTenant(String resourceGroupName,
                           String tenantName)
                    throws PulsarAdminException

Registers a tenant as a user of a resource group.

Parameters:

resourceGroupName -

tenantName -

Throws:

if - the RG does not exist, or if the NS already references the RG.

PulsarAdminException

unRegisterTenant

public void unRegisterTenant(String resourceGroupName,
                             String tenantName)
                      throws PulsarAdminException

UnRegisters a tenant from a resource group.

Parameters:

resourceGroupName -

tenantName -

Throws:

if - the RG does not exist, or if the tenant does not references the RG yet.

PulsarAdminException

registerNameSpace

public void registerNameSpace(String resourceGroupName,
                              org.apache.pulsar.common.naming.NamespaceName fqNamespaceName)
                       throws PulsarAdminException

Registers a namespace as a user of a resource group.

Parameters:

resourceGroupName -

fqNamespaceName - (i.e., in "tenant/Namespace" format)

Throws:

if - the RG does not exist, or if the NS already references the RG.

PulsarAdminException

unRegisterNameSpace

public void unRegisterNameSpace(String resourceGroupName,
                                org.apache.pulsar.common.naming.NamespaceName fqNamespaceName)
                         throws PulsarAdminException

UnRegisters a namespace from a resource group.

Parameters:

resourceGroupName -

fqNamespaceName - i.e., in "tenant/Namespace" format)

Throws:

if - the RG does not exist, or if the NS does not references the RG yet.

PulsarAdminException

getNamespaceResourceGroup

public ResourceGroup getNamespaceResourceGroup(org.apache.pulsar.common.naming.NamespaceName namespaceName)

Return the resource group associated with a namespace.

Parameters:

namespaceName -

Throws:

if - the RG does not exist, or if the NS already references the RG.

incrementUsage

protected boolean incrementUsage(String tenantName,
                                 String nsName,
                                 ResourceGroup.ResourceGroupMonitoringClass monClass,
                                 ResourceGroup.BytesAndMessagesCount incStats)
                          throws PulsarAdminException

Increments usage stats for the resource groups associated with the given namespace and tenant. Expected to be called when a message is produced or consumed on a topic, or when we calculate usage periodically in the background by going through broker-service stats. [Not yet decided which model we will follow.] Broker-service stats will be cumulative, while calls from the topic produce/consume code will be per-produce/consume. If the tenant and NS are associated with different RGs, the statistics on both RGs are updated. If the tenant and NS are associated with the same RG, the stats on the RG are updated only once (to avoid a direct double-counting). ToDo: will this distinction result in "expected semantics", or shock from users? For now, the only caller is internal to this class.

Parameters:

tenantName -

nsName -

monClass -

incStats -

Throws:

PulsarAdminException

getRGUsage

protected ResourceGroup.BytesAndMessagesCount getRGUsage(String rgName,
                                                         ResourceGroup.ResourceGroupMonitoringClass monClass,
                                                         ResourceGroupService.ResourceGroupUsageStatsType statsType)
                                                  throws PulsarAdminException

Throws:

PulsarAdminException

getPublishRateLimiters

protected ResourceGroup.BytesAndMessagesCount getPublishRateLimiters(String rgName)
                                                              throws PulsarAdminException

Throws:

PulsarAdminException

getRgQuotaByteCount

protected static double getRgQuotaByteCount(String rgName,
                                            String monClassName)

getRgQuotaMessageCount

protected static double getRgQuotaMessageCount(String rgName,
                                               String monClassName)

getRgLocalUsageByteCount

protected static double getRgLocalUsageByteCount(String rgName,
                                                 String monClassName)

getRgLocalUsageMessageCount

protected static double getRgLocalUsageMessageCount(String rgName,
                                                    String monClassName)

getRgUpdatesCount

protected static double getRgUpdatesCount(String rgName)

getRgTenantRegistersCount

protected static double getRgTenantRegistersCount(String rgName)

getRgTenantUnRegistersCount

protected static double getRgTenantUnRegistersCount(String rgName)

getRgNamespaceRegistersCount

protected static double getRgNamespaceRegistersCount(String rgName)

getRgNamespaceUnRegistersCount

protected static double getRgNamespaceUnRegistersCount(String rgName)

getRgUsageAggregationLatency

protected static io.prometheus.client.Summary.Child.Value getRgUsageAggregationLatency()

getRgQuotaCalculationTime

protected static io.prometheus.client.Summary.Child.Value getRgQuotaCalculationTime()

aggregateResourceGroupLocalUsages

protected void aggregateResourceGroupLocalUsages()

calculateQuotaForAllResourceGroups

protected void calculateQuotaForAllResourceGroups()