Package org.apache.kafka.clients.consumer.internals

Class AsyncKafkaConsumer<K,V>

java.lang.Object

org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer<K,V>

All Implemented Interfaces:

Closeable, AutoCloseable, Consumer<K,V>, ConsumerDelegate<K,V>

public class AsyncKafkaConsumer<K,V>
extends Object
implements ConsumerDelegate<K,V>

This Consumer implementation uses an event handler to process application events so that the network I/O can be processed in a dedicated network thread. Visit this document for implementation detail.

Note: this Consumer implementation is part of the revised consumer group protocol from KIP-848. This class should not be invoked directly; users should instead create a KafkaConsumer as before. This consumer implements the new consumer group protocol and is intended to be the default in coming releases.

Method Summary

Modifier and Type	Method	Description
void	assign(Collection<org.apache.kafka.common.TopicPartition> partitions)
Set<org.apache.kafka.common.TopicPartition>	assignment()
Map<org.apache.kafka.common.TopicPartition,Long>	beginningOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions)
Map<org.apache.kafka.common.TopicPartition,Long>	beginningOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions, Duration timeout)
String	clientId()
org.apache.kafka.common.Uuid	clientInstanceId(Duration timeout)	See KafkaConsumer.clientInstanceId(Duration)}
void	close()
void	close(Duration timeout)
void	commitAsync()	This method sends a commit event to the EventHandler and return.
void	commitAsync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets, OffsetCommitCallback callback)
void	commitAsync(OffsetCommitCallback callback)
void	commitSync()	Commit offsets returned on the last poll() for all the subscribed list of topics and partitions.
void	commitSync(Duration timeout)	This method sends a commit event to the EventHandler and waits for the event to finish.
void	commitSync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets)
void	commitSync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets, Duration timeout)
Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata>	committed(Set<org.apache.kafka.common.TopicPartition> partitions)
Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata>	committed(Set<org.apache.kafka.common.TopicPartition> partitions, Duration timeout)
OffsetAndMetadata	committed(org.apache.kafka.common.TopicPartition partition)	Deprecated.
OffsetAndMetadata	committed(org.apache.kafka.common.TopicPartition partition, Duration timeout)	Deprecated.
OptionalLong	currentLag(org.apache.kafka.common.TopicPartition topicPartition)
Map<org.apache.kafka.common.TopicPartition,Long>	endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions)
Map<org.apache.kafka.common.TopicPartition,Long>	endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions, Duration timeout)
void	enforceRebalance()
void	enforceRebalance(String reason)
ConsumerGroupMetadata	groupMetadata()
KafkaConsumerMetrics	kafkaConsumerMetrics()
Map<String,List<org.apache.kafka.common.PartitionInfo>>	listTopics()
Map<String,List<org.apache.kafka.common.PartitionInfo>>	listTopics(Duration timeout)
Map<org.apache.kafka.common.MetricName,? extends org.apache.kafka.common.Metric>	metrics()
org.apache.kafka.common.metrics.Metrics	metricsRegistry()
Map<org.apache.kafka.common.TopicPartition,OffsetAndTimestamp>	offsetsForTimes(Map<org.apache.kafka.common.TopicPartition,Long> timestampsToSearch)
Map<org.apache.kafka.common.TopicPartition,OffsetAndTimestamp>	offsetsForTimes(Map<org.apache.kafka.common.TopicPartition,Long> timestampsToSearch, Duration timeout)
List<org.apache.kafka.common.PartitionInfo>	partitionsFor(String topic)
List<org.apache.kafka.common.PartitionInfo>	partitionsFor(String topic, Duration timeout)
void	pause(Collection<org.apache.kafka.common.TopicPartition> partitions)
Set<org.apache.kafka.common.TopicPartition>	paused()
ConsumerRecords<K,V>	poll(long timeoutMs)	Deprecated.
ConsumerRecords<K,V>	poll(Duration timeout)	poll implementation using ApplicationEventHandler.
long	position(org.apache.kafka.common.TopicPartition partition)
long	position(org.apache.kafka.common.TopicPartition partition, Duration timeout)
void	resume(Collection<org.apache.kafka.common.TopicPartition> partitions)
void	seek(org.apache.kafka.common.TopicPartition partition, long offset)
void	seek(org.apache.kafka.common.TopicPartition partition, OffsetAndMetadata offsetAndMetadata)
void	seekToBeginning(Collection<org.apache.kafka.common.TopicPartition> partitions)
void	seekToEnd(Collection<org.apache.kafka.common.TopicPartition> partitions)
void	subscribe(Collection<String> topics)
void	subscribe(Collection<String> topics, ConsumerRebalanceListener listener)
void	subscribe(Pattern pattern)
void	subscribe(Pattern pattern, ConsumerRebalanceListener listener)
Set<String>	subscription()	Get the current subscription.
void	unsubscribe()
boolean	updateAssignmentMetadataIfNeeded(org.apache.kafka.common.utils.Timer timer)
void	wakeup()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

poll

public ConsumerRecords<K,V> poll(Duration timeout)

poll implementation using ApplicationEventHandler. 1. Poll for background events. If there's a fetch response event, process the record and return it. If it is another type of event, process it. 2. Send fetches if needed. If the timeout expires, return an empty ConsumerRecord.

Specified by:

poll in interface Consumer<K,V>

Parameters:

timeout - timeout of the poll loop

Returns:

ConsumerRecord. It can be empty if time timeout expires.

Throws:

org.apache.kafka.common.errors.WakeupException - if wakeup() is called before or while this function is called

org.apache.kafka.common.errors.InterruptException - if the calling thread is interrupted before or while this function is called

org.apache.kafka.common.errors.RecordTooLargeException - if the fetched record is larger than the maximum allowable size

org.apache.kafka.common.KafkaException - for any other unrecoverable errors

IllegalStateException - if the consumer is not subscribed to any topics or manually assigned any partitions to consume from or an unexpected error occurred

OffsetOutOfRangeException - if the fetch position of the consumer is out of range and no offset reset policy is configured.

org.apache.kafka.common.errors.TopicAuthorizationException - if the consumer is not authorized to read from a partition

org.apache.kafka.common.errors.SerializationException - if the fetched records cannot be deserialized

org.apache.kafka.common.errors.UnsupportedAssignorException - if the `group.remote.assignor` configuration is set to an assignor that is not available on the broker.

See Also:

• KafkaConsumer.poll(Duration)

commitSync

public void commitSync()

Commit offsets returned on the last poll() for all the subscribed list of topics and partitions.

Specified by:

commitSync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitSync()

commitAsync

public void commitAsync()

This method sends a commit event to the EventHandler and return.

Specified by:

commitAsync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitAsync()

commitAsync

public void commitAsync(OffsetCommitCallback callback)

Specified by:

commitAsync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitAsync(OffsetCommitCallback)

commitAsync

public void commitAsync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets,
 OffsetCommitCallback callback)

Specified by:

commitAsync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitAsync(Map, OffsetCommitCallback)

seek

public void seek(org.apache.kafka.common.TopicPartition partition,
 long offset)

Specified by:

seek in interface Consumer<K,V>

See Also:

• KafkaConsumer.seek(TopicPartition, long)

seek

public void seek(org.apache.kafka.common.TopicPartition partition,
 OffsetAndMetadata offsetAndMetadata)

Specified by:

seek in interface Consumer<K,V>

See Also:

• KafkaConsumer.seek(TopicPartition, OffsetAndMetadata)

seekToBeginning

public void seekToBeginning(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

seekToBeginning in interface Consumer<K,V>

See Also:

• KafkaConsumer.seekToBeginning(Collection)

seekToEnd

public void seekToEnd(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

seekToEnd in interface Consumer<K,V>

See Also:

• KafkaConsumer.seekToEnd(Collection)

position

public long position(org.apache.kafka.common.TopicPartition partition)

Specified by:

position in interface Consumer<K,V>

See Also:

• KafkaConsumer.position(TopicPartition)

position

public long position(org.apache.kafka.common.TopicPartition partition,
 Duration timeout)

Specified by:

position in interface Consumer<K,V>

See Also:

• KafkaConsumer.position(TopicPartition, Duration)

committed

@Deprecated
public OffsetAndMetadata committed(org.apache.kafka.common.TopicPartition partition)

Deprecated.

Specified by:

committed in interface Consumer<K,V>

See Also:

• KafkaConsumer.committed(TopicPartition)

committed

@Deprecated
public OffsetAndMetadata committed(org.apache.kafka.common.TopicPartition partition,
 Duration timeout)

Deprecated.

Specified by:

committed in interface Consumer<K,V>

See Also:

• KafkaConsumer.committed(TopicPartition, Duration)

committed

public Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> committed(Set<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

committed in interface Consumer<K,V>

See Also:

• KafkaConsumer.committed(Set)

committed

public Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> committed(Set<org.apache.kafka.common.TopicPartition> partitions,
 Duration timeout)

Specified by:

committed in interface Consumer<K,V>

See Also:

• KafkaConsumer.committed(Set, Duration)

metrics

public Map<org.apache.kafka.common.MetricName,? extends org.apache.kafka.common.Metric> metrics()

Specified by:

metrics in interface Consumer<K,V>

See Also:

• KafkaConsumer.metrics()

partitionsFor

public List<org.apache.kafka.common.PartitionInfo> partitionsFor(String topic)

Specified by:

partitionsFor in interface Consumer<K,V>

See Also:

• KafkaConsumer.partitionsFor(String)

partitionsFor

public List<org.apache.kafka.common.PartitionInfo> partitionsFor(String topic,
 Duration timeout)

Specified by:

partitionsFor in interface Consumer<K,V>

See Also:

• KafkaConsumer.partitionsFor(String, Duration)

listTopics

public Map<String,List<org.apache.kafka.common.PartitionInfo>> listTopics()

Specified by:

listTopics in interface Consumer<K,V>

See Also:

• KafkaConsumer.listTopics()

listTopics

public Map<String,List<org.apache.kafka.common.PartitionInfo>> listTopics(Duration timeout)

Specified by:

listTopics in interface Consumer<K,V>

See Also:

• KafkaConsumer.listTopics(Duration)

paused

public Set<org.apache.kafka.common.TopicPartition> paused()

Specified by:

paused in interface Consumer<K,V>

See Also:

• KafkaConsumer.paused()

pause

public void pause(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

pause in interface Consumer<K,V>

See Also:

• KafkaConsumer.pause(Collection)

resume

public void resume(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

resume in interface Consumer<K,V>

See Also:

• KafkaConsumer.resume(Collection)

offsetsForTimes

public Map<org.apache.kafka.common.TopicPartition,OffsetAndTimestamp> offsetsForTimes(Map<org.apache.kafka.common.TopicPartition,Long> timestampsToSearch)

Specified by:

offsetsForTimes in interface Consumer<K,V>

See Also:

• KafkaConsumer.offsetsForTimes(Map)

offsetsForTimes

public Map<org.apache.kafka.common.TopicPartition,OffsetAndTimestamp> offsetsForTimes(Map<org.apache.kafka.common.TopicPartition,Long> timestampsToSearch,
 Duration timeout)

Specified by:

offsetsForTimes in interface Consumer<K,V>

See Also:

• KafkaConsumer.offsetsForTimes(Map, Duration)

beginningOffsets

public Map<org.apache.kafka.common.TopicPartition,Long> beginningOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

beginningOffsets in interface Consumer<K,V>

See Also:

• KafkaConsumer.beginningOffsets(Collection)

beginningOffsets

public Map<org.apache.kafka.common.TopicPartition,Long> beginningOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions,
 Duration timeout)

Specified by:

beginningOffsets in interface Consumer<K,V>

See Also:

• KafkaConsumer.beginningOffsets(Collection, Duration)

endOffsets

public Map<org.apache.kafka.common.TopicPartition,Long> endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

endOffsets in interface Consumer<K,V>

See Also:

• KafkaConsumer.endOffsets(Collection)

endOffsets

public Map<org.apache.kafka.common.TopicPartition,Long> endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions,
 Duration timeout)

Specified by:

endOffsets in interface Consumer<K,V>

See Also:

• KafkaConsumer.endOffsets(Collection, Duration)

currentLag

public OptionalLong currentLag(org.apache.kafka.common.TopicPartition topicPartition)

Specified by:

currentLag in interface Consumer<K,V>

See Also:

• KafkaConsumer.currentLag(TopicPartition)

groupMetadata

public ConsumerGroupMetadata groupMetadata()

Specified by:

groupMetadata in interface Consumer<K,V>

See Also:

• KafkaConsumer.groupMetadata()

enforceRebalance

public void enforceRebalance()

Specified by:

enforceRebalance in interface Consumer<K,V>

See Also:

• KafkaConsumer.enforceRebalance()

enforceRebalance

public void enforceRebalance(String reason)

Specified by:

enforceRebalance in interface Consumer<K,V>

See Also:

• KafkaConsumer.enforceRebalance(String)

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface Consumer<K,V>

See Also:

• KafkaConsumer.close()

close

public void close(Duration timeout)

Specified by:

close in interface Consumer<K,V>

See Also:

• KafkaConsumer.close(Duration)

wakeup

public void wakeup()

Specified by:

wakeup in interface Consumer<K,V>

See Also:

• KafkaConsumer.wakeup()

commitSync

public void commitSync(Duration timeout)

This method sends a commit event to the EventHandler and waits for the event to finish.

Specified by:

commitSync in interface Consumer<K,V>

Parameters:

timeout - max wait time for the blocking operation.

See Also:

• KafkaConsumer.commitSync(Duration)

commitSync

public void commitSync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets)

Specified by:

commitSync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitSync(Map)

commitSync

public void commitSync(Map<org.apache.kafka.common.TopicPartition,OffsetAndMetadata> offsets,
 Duration timeout)

Specified by:

commitSync in interface Consumer<K,V>

See Also:

• KafkaConsumer.commitSync(Map, Duration)

clientInstanceId

public org.apache.kafka.common.Uuid clientInstanceId(Duration timeout)

Description copied from interface: Consumer

See KafkaConsumer.clientInstanceId(Duration)}

Specified by:

clientInstanceId in interface Consumer<K,V>

assignment

public Set<org.apache.kafka.common.TopicPartition> assignment()

Specified by:

assignment in interface Consumer<K,V>

See Also:

• KafkaConsumer.assignment()

subscription

public Set<String> subscription()

Get the current subscription. or an empty set if no such call has been made.

Specified by:

subscription in interface Consumer<K,V>

Returns:

The set of topics currently subscribed to

See Also:

• KafkaConsumer.subscription()

assign

public void assign(Collection<org.apache.kafka.common.TopicPartition> partitions)

Specified by:

assign in interface Consumer<K,V>

See Also:

• KafkaConsumer.assign(Collection)

unsubscribe

public void unsubscribe()

Specified by:

unsubscribe in interface Consumer<K,V>

See Also:

• KafkaConsumer.unsubscribe()

poll

@Deprecated
public ConsumerRecords<K,V> poll(long timeoutMs)

Deprecated.

Specified by:

poll in interface Consumer<K,V>

See Also:

• KafkaConsumer.poll(long)

updateAssignmentMetadataIfNeeded

public boolean updateAssignmentMetadataIfNeeded(org.apache.kafka.common.utils.Timer timer)

Specified by:

updateAssignmentMetadataIfNeeded in interface ConsumerDelegate<K,V>

subscribe

public void subscribe(Collection<String> topics)

Specified by:

subscribe in interface Consumer<K,V>

See Also:

• KafkaConsumer.subscribe(Collection)

subscribe

public void subscribe(Collection<String> topics,
 ConsumerRebalanceListener listener)

Specified by:

subscribe in interface Consumer<K,V>

See Also:

• KafkaConsumer.subscribe(Collection, ConsumerRebalanceListener)

subscribe

public void subscribe(Pattern pattern)

Specified by:

subscribe in interface Consumer<K,V>

See Also:

• KafkaConsumer.subscribe(Pattern)

subscribe

public void subscribe(Pattern pattern,
 ConsumerRebalanceListener listener)

Specified by:

subscribe in interface Consumer<K,V>

See Also:

• KafkaConsumer.subscribe(Pattern, ConsumerRebalanceListener)

clientId

public String clientId()

Specified by:

clientId in interface ConsumerDelegate<K,V>

metricsRegistry

public org.apache.kafka.common.metrics.Metrics metricsRegistry()

Specified by:

metricsRegistry in interface ConsumerDelegate<K,V>

kafkaConsumerMetrics

public KafkaConsumerMetrics kafkaConsumerMetrics()

Specified by:

kafkaConsumerMetrics in interface ConsumerDelegate<K,V>