trait KafkaConsume[F[_], K, V] extends AnyRef
- Alphabetic
- By Inheritance
- KafkaConsume
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def partitionedStream: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Stream
where the elements themselves areStream
s which continually request records for a single partition.Stream
where the elements themselves areStream
s which continually request records for a single partition. TheseStream
s will have to be processed in parallel, usingparJoin
orparJoinUnbounded
. Note that when usingparJoin(n)
andn
is smaller than the number of currently assigned partitions, then there will be assigned partitions which won't be processed. For that reason, preferparJoinUnbounded
and the actual limit will be the number of assigned partitions.If you do not want to process all partitions in parallel, then you can use records instead, where records for all partitions are in a single
Stream
.- Note
you have to first use
subscribe
orassign
the consumer before using thisStream
. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream
.
- abstract def partitionsMapStream: Stream[F, Map[TopicPartition, Stream[F, CommittableConsumerRecord[F, K, V]]]]
Stream
where each element contains aMap
with all newly assigned partitions.Stream
where each element contains aMap
with all newly assigned partitions. Keys of thisMap
areTopicPartition
s, and values are record streams for the particularTopicPartition
. These streams will be closed only when a partition is revoked.With the default assignor, all previous partitions are revoked at once, and a new set of partitions is assigned to a consumer on each rebalance. In this case, each returned
Map
contains the full partition assignment for the consumer. And all streams from the previous assignment are closed. It means, thatpartitionsMapStream
reflects the default assignment process in a streaming manner.This may not be the case when a custom assignor is configured in the consumer. When using the
CooperativeStickyAssignor
, for instance, partitions may be revoked individually. In this case, each element in the stream (eachMap
) will contain only streams for newly assigned partitions. Previously returned streams for partitions that are retained will remain active. Only streams for revoked partitions will be closed.This is the most generic
Stream
method. If you don't need such control, consider usingpartitionedStream
orstream
methods. They are both based on apartitionsMapStream
.- Note
you have to first use
subscribe
orassign
to subscribe the consumer before using thisStream
. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream
.- See also
- abstract def stopConsuming: F[Unit]
Stops consuming new messages from Kafka.
Stops consuming new messages from Kafka. This method could be used to implement a graceful shutdown.
This method has a few effects:
- After this call no more data will be fetched from Kafka through the
poll
method. - All currently running streams will continue to run until all in-flight messages will be
processed. It means that streams will be completed when all fetched messages will be
processed.
If some of the records methods will be called after stopConsuming call, these methods will return empty streams.
More than one call of stopConsuming will have no effect.
- After this call no more data will be fetched from Kafka through the
- abstract def stream: Stream[F, CommittableConsumerRecord[F, K, V]]
Alias for
partitionedStream.parJoinUnbounded
.Alias for
partitionedStream.parJoinUnbounded
.- Note
you have to first use
subscribe
orassign
the consumer before using thisStream
. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream
.- See also
partitionedRecords for more information
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def partitionedRecords: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Alias for partitionedStream
- final def records: Stream[F, CommittableConsumerRecord[F, K, V]]
Consume from all assigned partitions, producing a stream of CommittableConsumerRecords.
Consume from all assigned partitions, producing a stream of CommittableConsumerRecords. Alias for stream.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()