Package org.apache.kafka.clients.consumer.internals

Class AbstractFetch

java.lang.Object
org.apache.kafka.clients.consumer.internals.AbstractFetch
All Implemented Interfaces:
Closeable, AutoCloseable
Direct Known Subclasses:
Fetcher, FetchRequestManager
public abstract class AbstractFetch extends Object implements Closeable
AbstractFetch represents the basic state and logic for record fetching processing.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected static interface 
    AbstractFetch.ResponseHandler<T>
    Defines the contract for handling fetch responses from brokers.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final org.apache.kafka.common.utils.BufferSupplier
    decompressionBufferSupplier
     
    protected final FetchBuffer
    fetchBuffer
     
    protected final FetchConfig
    fetchConfig
     
    protected final org.apache.kafka.common.utils.LogContext
    logContext
     
    protected final ConsumerMetadata
    metadata
     
    protected final FetchMetricsManager
    metricsManager
     
    protected final Set<Integer>
    nodesWithPendingFetchRequests
     
    protected final SubscriptionState
    subscriptions
     
    protected final org.apache.kafka.common.utils.Time
    time
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    AbstractFetch(org.apache.kafka.common.utils.LogContext logContext, ConsumerMetadata metadata, SubscriptionState subscriptions, FetchConfig fetchConfig, FetchBuffer fetchBuffer, FetchMetricsManager metricsManager, org.apache.kafka.common.utils.Time time, org.apache.kafka.clients.ApiVersions apiVersions)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    void
    close(org.apache.kafka.common.utils.Timer timer)
     
    protected void
    closeInternal(org.apache.kafka.common.utils.Timer timer)
    This method is called by close(Timer) which is guarded by the IdempotentCloser) such as to only be executed once the first time that any of the close() methods are called.
    protected org.apache.kafka.common.requests.FetchRequest.Builder
    createFetchRequest(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData requestData)
    Creates a new fetch request in preparation for sending to the Kafka cluster.
    void
    handleCloseFetchSessionFailure(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, Throwable t)
     
    protected void
    handleCloseFetchSessionSuccess(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, org.apache.kafka.clients.ClientResponse ignored)
     
    protected void
    handleFetchFailure(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, Throwable t)
    Implements the core logic for a failed fetch response.
    protected void
    handleFetchSuccess(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, org.apache.kafka.clients.ClientResponse resp)
    Implements the core logic for a successful fetch response.
    boolean
    hasAvailableFetches()
    Return whether we have any completed fetches that are fetchable.
    protected abstract boolean
    isUnavailable(org.apache.kafka.common.Node node)
    Check if the node is disconnected and unavailable for immediate reconnection (i.e.
    protected abstract void
    maybeThrowAuthFailure(org.apache.kafka.common.Node node)
    Checks for an authentication error on a given node and throws the exception if it exists.
    protected Map<org.apache.kafka.common.Node,org.apache.kafka.clients.FetchSessionHandler.FetchRequestData>
    prepareCloseFetchSessionRequests()
     
    protected Map<org.apache.kafka.common.Node,org.apache.kafka.clients.FetchSessionHandler.FetchRequestData>
    prepareFetchRequests()
    Create fetch requests for all nodes for which we have assigned partitions that have no existing requests in flight.
    protected org.apache.kafka.clients.FetchSessionHandler
    sessionHandler(int node)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • logContext

      protected final org.apache.kafka.common.utils.LogContext logContext

    • metadata

      protected final ConsumerMetadata metadata

    • subscriptions

      protected final SubscriptionState subscriptions

    • fetchConfig

      protected final FetchConfig fetchConfig

    • time

      protected final org.apache.kafka.common.utils.Time time

    • metricsManager

      protected final FetchMetricsManager metricsManager

    • fetchBuffer

      protected final FetchBuffer fetchBuffer

    • decompressionBufferSupplier

      protected final org.apache.kafka.common.utils.BufferSupplier decompressionBufferSupplier

    • nodesWithPendingFetchRequests

      protected final Set<Integer> nodesWithPendingFetchRequests

  • Constructor Details

  • Method Details

    • isUnavailable

      protected abstract boolean isUnavailable(org.apache.kafka.common.Node node)
      Check if the node is disconnected and unavailable for immediate reconnection (i.e. if it is in reconnect backoff window following the disconnect).
      Parameters:
      node - Node to check for availability
      See Also:
      • NetworkClientUtils.isUnavailable(KafkaClient, Node, Time)

    • maybeThrowAuthFailure

      protected abstract void maybeThrowAuthFailure(org.apache.kafka.common.Node node)
      Checks for an authentication error on a given node and throws the exception if it exists.
      Parameters:
      node - Node to check for a previous AuthenticationException; if found it is thrown
      See Also:
      • NetworkClientUtils.maybeThrowAuthFailure(KafkaClient, Node)

    • hasAvailableFetches

      public boolean hasAvailableFetches()
      Return whether we have any completed fetches that are fetchable. This method is thread-safe.
      Returns:
      true if there are completed fetches that can be returned, false otherwise

    • handleFetchSuccess

      protected void handleFetchSuccess(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, org.apache.kafka.clients.ClientResponse resp)
      Implements the core logic for a successful fetch response.
      Parameters:
      fetchTarget - Node from which the fetch data was requested
      data - FetchSessionHandler.FetchRequestData that represents the session data
      resp - ClientResponse from which the FetchResponse will be retrieved

    • handleFetchFailure

      protected void handleFetchFailure(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, Throwable t)
      Implements the core logic for a failed fetch response.
      Parameters:
      fetchTarget - Node from which the fetch data was requested
      data - FetchSessionHandler.FetchRequestData from request
      t - Throwable representing the error that resulted in the failure

    • handleCloseFetchSessionSuccess

      protected void handleCloseFetchSessionSuccess(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, org.apache.kafka.clients.ClientResponse ignored)

    • handleCloseFetchSessionFailure

      public void handleCloseFetchSessionFailure(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData data, Throwable t)

    • createFetchRequest

      protected org.apache.kafka.common.requests.FetchRequest.Builder createFetchRequest(org.apache.kafka.common.Node fetchTarget, org.apache.kafka.clients.FetchSessionHandler.FetchRequestData requestData)
      Creates a new fetch request in preparation for sending to the Kafka cluster.
      Parameters:
      fetchTarget - Node from which the fetch data will be requested
      requestData - FetchSessionHandler.FetchRequestData that represents the session data
      Returns:
      FetchRequest.Builder that can be submitted to the broker

    • prepareCloseFetchSessionRequests

      protected Map<org.apache.kafka.common.Node,org.apache.kafka.clients.FetchSessionHandler.FetchRequestData> prepareCloseFetchSessionRequests()

    • prepareFetchRequests

      protected Map<org.apache.kafka.common.Node,org.apache.kafka.clients.FetchSessionHandler.FetchRequestData> prepareFetchRequests()
      Create fetch requests for all nodes for which we have assigned partitions that have no existing requests in flight.

    • sessionHandler

      protected org.apache.kafka.clients.FetchSessionHandler sessionHandler(int node)

    • closeInternal

      protected void closeInternal(org.apache.kafka.common.utils.Timer timer)
      This method is called by close(Timer) which is guarded by the IdempotentCloser) such as to only be executed once the first time that any of the close() methods are called. Subclasses can override this method without the need for extra synchronization at the instance level.
      Parameters:
      timer - Timer to enforce time limit

    • close

      public void close(org.apache.kafka.common.utils.Timer timer)

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable