t

fs2.kafka.consumer

KafkaConsume

trait KafkaConsume[F[_], K, V] extends AnyRef

Source
KafkaConsume.scala
Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. KafkaConsume
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Abstract Value Members

  1. abstract def partitionedStream: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]

    Stream where the elements themselves are Streams which continually request records for a single partition.

    Stream where the elements themselves are Streams which continually request records for a single partition. These Streams will have to be processed in parallel, using parJoin or parJoinUnbounded. Note that when using parJoin(n) and n is smaller than the number of currently assigned partitions, then there will be assigned partitions which won't be processed. For that reason, prefer parJoinUnbounded and the actual limit will be the number of assigned partitions.

    If you do not want to process all partitions in parallel, then you can use records instead, where records for all partitions are in a single Stream.

    Note

    you have to first use subscribe or assign the consumer before using this Stream. If you forgot to subscribe, there will be a NotSubscribedException raised in the Stream.

  2. abstract def partitionsMapStream: Stream[F, Map[TopicPartition, Stream[F, CommittableConsumerRecord[F, K, V]]]]

    Stream where each element contains a Map with all newly assigned partitions.

    Stream where each element contains a Map with all newly assigned partitions. Keys of this Map are TopicPartitions, and values are record streams for the particular TopicPartition. These streams will be closed only when a partition is revoked.

    With the default assignor, all previous partitions are revoked at once, and a new set of partitions is assigned to a consumer on each rebalance. In this case, each returned Map contains the full partition assignment for the consumer. And all streams from the previous assignment are closed. It means, that partitionsMapStream reflects the default assignment process in a streaming manner.

    This may not be the case when a custom assignor is configured in the consumer. When using the CooperativeStickyAssignor, for instance, partitions may be revoked individually. In this case, each element in the stream (eachMap) will contain only streams for newly assigned partitions. Previously returned streams for partitions that are retained will remain active. Only streams for revoked partitions will be closed.

    This is the most generic Stream method. If you don't need such control, consider using partitionedStream or stream methods. They are both based on a partitionsMapStream.

    Note

    you have to first use subscribe or assign to subscribe the consumer before using this Stream. If you forgot to subscribe, there will be a NotSubscribedException raised in the Stream.

    See also

    records

    partitionedRecords

  3. abstract def stopConsuming: F[Unit]

    Stops consuming new messages from Kafka.

    Stops consuming new messages from Kafka. This method could be used to implement a graceful shutdown.

    This method has a few effects:

    • After this call no more data will be fetched from Kafka through the poll method.
    • All currently running streams will continue to run until all in-flight messages will be processed. It means that streams will be completed when all fetched messages will be processed.

    If some of the records methods will be called after stopConsuming call, these methods will return empty streams.

    More than one call of stopConsuming will have no effect.

  4. abstract def stream: Stream[F, CommittableConsumerRecord[F, K, V]]

    Alias for partitionedStream.parJoinUnbounded.

    Alias for partitionedStream.parJoinUnbounded.

    Note

    you have to first use subscribe or assign the consumer before using this Stream. If you forgot to subscribe, there will be a NotSubscribedException raised in the Stream.

    See also

    partitionedRecords for more information

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  9. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def partitionedRecords: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]

    Alias for partitionedStream

  16. final def records: Stream[F, CommittableConsumerRecord[F, K, V]]

    Consume from all assigned partitions, producing a stream of CommittableConsumerRecords.

    Consume from all assigned partitions, producing a stream of CommittableConsumerRecords. Alias for stream.

  17. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  20. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  21. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped