Package org.apache.kafka.clients.consumer.internals

Class ConsumerNetworkClient

java.lang.Object
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient
All Implemented Interfaces:
Closeable, AutoCloseable
public class ConsumerNetworkClient extends Object implements Closeable
Higher level consumer access to the network layer with basic support for request futures. This class is thread-safe, but provides no synchronization for response callbacks. This guarantees that no locks are held when they are invoked.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    ConsumerNetworkClient.PollCondition
    When invoking poll from a multi-threaded environment, it is possible that the condition that the caller is awaiting has already been satisfied prior to the invocation of poll.

  • Constructor Summary

    Constructors
    Constructor
    Description
    ConsumerNetworkClient(org.apache.kafka.common.utils.LogContext logContext, org.apache.kafka.clients.KafkaClient client, org.apache.kafka.clients.Metadata metadata, org.apache.kafka.common.utils.Time time, long retryBackoffMs, int requestTimeoutMs, int maxPollTimeoutMs)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    awaitMetadataUpdate(org.apache.kafka.common.utils.Timer timer)
    Block waiting on the metadata refresh with a timeout.
    boolean
    awaitPendingRequests(org.apache.kafka.common.Node node, org.apache.kafka.common.utils.Timer timer)
    Block until all pending requests from the given node have finished.
    void
    close()
     
    int
    defaultRequestTimeoutMs()
     
    void
    disableWakeups()
     
    void
    disconnectAsync(org.apache.kafka.common.Node node)
     
    boolean
    hasPendingRequests()
    Check whether there is pending request.
    boolean
    hasPendingRequests(org.apache.kafka.common.Node node)
    Check whether there is pending request to the given node.
    boolean
    hasReadyNodes(long now)
     
    boolean
    isUnavailable(org.apache.kafka.common.Node node)
    Check if the code is disconnected and unavailable for immediate reconnection (i.e.
    org.apache.kafka.common.Node
    leastLoadedNode()
     
    void
    maybeThrowAuthFailure(org.apache.kafka.common.Node node)
    Check for an authentication error on a given node and raise the exception if there is one.
    void
    maybeTriggerWakeup()
     
    int
    pendingRequestCount()
    Get the total count of pending requests from all nodes.
    int
    pendingRequestCount(org.apache.kafka.common.Node node)
    Get the count of pending requests to the given node.
    void
    poll(RequestFuture<?> future)
    Block indefinitely until the given request future has finished.
    boolean
    poll(RequestFuture<?> future, org.apache.kafka.common.utils.Timer timer)
    Block until the provided request future request has finished or the timeout has expired.
    boolean
    poll(RequestFuture<?> future, org.apache.kafka.common.utils.Timer timer, boolean disableWakeup)
    Block until the provided request future request has finished or the timeout has expired.
    void
    poll(org.apache.kafka.common.utils.Timer timer)
    Poll for any network IO.
    void
    poll(org.apache.kafka.common.utils.Timer timer, ConsumerNetworkClient.PollCondition pollCondition)
    Poll for any network IO.
    void
    poll(org.apache.kafka.common.utils.Timer timer, ConsumerNetworkClient.PollCondition pollCondition, boolean disableWakeup)
    Poll for any network IO.
    void
    pollNoWakeup()
    Poll for network IO and return immediately.
    RequestFuture<org.apache.kafka.clients.ClientResponse>
    send(org.apache.kafka.common.Node node, org.apache.kafka.common.requests.AbstractRequest.Builder<?> requestBuilder)
    Send a request with the default timeout.
    RequestFuture<org.apache.kafka.clients.ClientResponse>
    send(org.apache.kafka.common.Node node, org.apache.kafka.common.requests.AbstractRequest.Builder<?> requestBuilder, int requestTimeoutMs)
    Send a new request.
    void
    transmitSends()
    Poll for network IO in best-effort only trying to transmit the ready-to-send request Do not check any pending requests or metadata errors so that no exception should ever be thrown, also no wakeups be triggered and no interrupted exception either.
    void
    tryConnect(org.apache.kafka.common.Node node)
    Initiate a connection if currently possible.
    void
    wakeup()
    Wakeup an active poll.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • ConsumerNetworkClient

      public ConsumerNetworkClient(org.apache.kafka.common.utils.LogContext logContext, org.apache.kafka.clients.KafkaClient client, org.apache.kafka.clients.Metadata metadata, org.apache.kafka.common.utils.Time time, long retryBackoffMs, int requestTimeoutMs, int maxPollTimeoutMs)

  • Method Details

    • defaultRequestTimeoutMs

      public int defaultRequestTimeoutMs()

    • send

      public RequestFuture<org.apache.kafka.clients.ClientResponse> send(org.apache.kafka.common.Node node, org.apache.kafka.common.requests.AbstractRequest.Builder<?> requestBuilder)
      Send a request with the default timeout. See send(Node, AbstractRequest.Builder, int).

    • send

      public RequestFuture<org.apache.kafka.clients.ClientResponse> send(org.apache.kafka.common.Node node, org.apache.kafka.common.requests.AbstractRequest.Builder<?> requestBuilder, int requestTimeoutMs)
      Send a new request. Note that the request is not actually transmitted on the network until one of the poll(Timer) variants is invoked. At this point the request will either be transmitted successfully or will fail. Use the returned future to obtain the result of the send. Note that there is no need to check for disconnects explicitly on the ClientResponse object; instead, the future will be failed with a DisconnectException.
      Parameters:
      node - The destination of the request
      requestBuilder - A builder for the request payload
      requestTimeoutMs - Maximum time in milliseconds to await a response before disconnecting the socket and cancelling the request. The request may be cancelled sooner if the socket disconnects for any reason.
      Returns:
      A future which indicates the result of the send.

    • leastLoadedNode

      public org.apache.kafka.common.Node leastLoadedNode()

    • hasReadyNodes

      public boolean hasReadyNodes(long now)

    • awaitMetadataUpdate

      public boolean awaitMetadataUpdate(org.apache.kafka.common.utils.Timer timer)
      Block waiting on the metadata refresh with a timeout.
      Returns:
      true if update succeeded, false otherwise.

    • wakeup

      public void wakeup()
      Wakeup an active poll. This will cause the polling thread to throw an exception either on the current poll if one is active, or the next poll.

    • poll

      public void poll(RequestFuture<?> future)
      Block indefinitely until the given request future has finished.
      Parameters:
      future - The request future to await.
      Throws:
      org.apache.kafka.common.errors.WakeupException - if wakeup() is called from another thread
      org.apache.kafka.common.errors.InterruptException - if the calling thread is interrupted

    • poll

      public boolean poll(RequestFuture<?> future, org.apache.kafka.common.utils.Timer timer)
      Block until the provided request future request has finished or the timeout has expired.
      Parameters:
      future - The request future to wait for
      timer - Timer bounding how long this method can block
      Returns:
      true if the future is done, false otherwise
      Throws:
      org.apache.kafka.common.errors.WakeupException - if wakeup() is called from another thread
      org.apache.kafka.common.errors.InterruptException - if the calling thread is interrupted

    • poll

      public boolean poll(RequestFuture<?> future, org.apache.kafka.common.utils.Timer timer, boolean disableWakeup)
      Block until the provided request future request has finished or the timeout has expired.
      Parameters:
      future - The request future to wait for
      timer - Timer bounding how long this method can block
      disableWakeup - true if we should not check for wakeups, false otherwise
      Returns:
      true if the future is done, false otherwise
      Throws:
      org.apache.kafka.common.errors.WakeupException - if wakeup() is called from another thread and `disableWakeup` is false
      org.apache.kafka.common.errors.InterruptException - if the calling thread is interrupted

    • poll

      public void poll(org.apache.kafka.common.utils.Timer timer)
      Poll for any network IO.
      Parameters:
      timer - Timer bounding how long this method can block
      Throws:
      org.apache.kafka.common.errors.WakeupException - if wakeup() is called from another thread
      org.apache.kafka.common.errors.InterruptException - if the calling thread is interrupted

    • poll

      public void poll(org.apache.kafka.common.utils.Timer timer, ConsumerNetworkClient.PollCondition pollCondition)
      Poll for any network IO.
      Parameters:
      timer - Timer bounding how long this method can block
      pollCondition - Nullable blocking condition

    • poll

      public void poll(org.apache.kafka.common.utils.Timer timer, ConsumerNetworkClient.PollCondition pollCondition, boolean disableWakeup)
      Poll for any network IO.
      Parameters:
      timer - Timer bounding how long this method can block
      pollCondition - Nullable blocking condition
      disableWakeup - If TRUE disable triggering wake-ups

    • pollNoWakeup

      public void pollNoWakeup()
      Poll for network IO and return immediately. This will not trigger wakeups.

    • transmitSends

      public void transmitSends()
      Poll for network IO in best-effort only trying to transmit the ready-to-send request Do not check any pending requests or metadata errors so that no exception should ever be thrown, also no wakeups be triggered and no interrupted exception either.

    • awaitPendingRequests

      public boolean awaitPendingRequests(org.apache.kafka.common.Node node, org.apache.kafka.common.utils.Timer timer)
      Block until all pending requests from the given node have finished.
      Parameters:
      node - The node to await requests from
      timer - Timer bounding how long this method can block
      Returns:
      true If all requests finished, false if the timeout expired first

    • pendingRequestCount

      public int pendingRequestCount(org.apache.kafka.common.Node node)
      Get the count of pending requests to the given node. This includes both request that have been transmitted (i.e. in-flight requests) and those which are awaiting transmission.
      Parameters:
      node - The node in question
      Returns:
      The number of pending requests

    • hasPendingRequests

      public boolean hasPendingRequests(org.apache.kafka.common.Node node)
      Check whether there is pending request to the given node. This includes both request that have been transmitted (i.e. in-flight requests) and those which are awaiting transmission.
      Parameters:
      node - The node in question
      Returns:
      A boolean indicating whether there is pending request

    • pendingRequestCount

      public int pendingRequestCount()
      Get the total count of pending requests from all nodes. This includes both requests that have been transmitted (i.e. in-flight requests) and those which are awaiting transmission.
      Returns:
      The total count of pending requests

    • hasPendingRequests

      public boolean hasPendingRequests()
      Check whether there is pending request. This includes both requests that have been transmitted (i.e. in-flight requests) and those which are awaiting transmission.
      Returns:
      A boolean indicating whether there is pending request

    • disconnectAsync

      public void disconnectAsync(org.apache.kafka.common.Node node)

    • maybeTriggerWakeup

      public void maybeTriggerWakeup()

    • disableWakeups

      public void disableWakeups()

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • isUnavailable

      public boolean isUnavailable(org.apache.kafka.common.Node node)
      Check if the code is disconnected and unavailable for immediate reconnection (i.e. if it is in reconnect backoff window following the disconnect).

    • maybeThrowAuthFailure

      public void maybeThrowAuthFailure(org.apache.kafka.common.Node node)
      Check for an authentication error on a given node and raise the exception if there is one.

    • tryConnect

      public void tryConnect(org.apache.kafka.common.Node node)
      Initiate a connection if currently possible. This is only really useful for resetting the failed status of a socket. If there is an actual request to send, then send(Node, AbstractRequest.Builder) should be used.
      Parameters:
      node - The node to connect to