Experimental

org.apache.spark.rdd

Class RDDBarrier<T>

Object

org.apache.spark.rdd.RDDBarrier<T>

public class RDDBarrier<T>
extends Object

Wraps an RDD in a barrier stage, which forces Spark to launch tasks of this stage together. RDDBarrier instances are created by RDD.barrier().

Method Summary

Modifier and Type	Method and Description
<S> RDD<S>	ExperimentalmapPartitions(scala.Function1<scala.collection.Iterator<T>,scala.collection.Iterator<S>> f, boolean preservesPartitioning, scala.reflect.ClassTag<S> evidence$2) Returns a new RDD by applying a function to each partition of the wrapped RDD, where tasks are launched together in a barrier stage.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

Experimental

mapPartitions

public <S> RDD<S> mapPartitions(scala.Function1<scala.collection.Iterator<T>,scala.collection.Iterator<S>> f,
                                boolean preservesPartitioning,
                                scala.reflect.ClassTag<S> evidence$2)

Returns a new RDD by applying a function to each partition of the wrapped RDD, where tasks are launched together in a barrier stage. The interface is the same as RDD.mapPartitions(scala.Function1<scala.collection.Iterator<T>, scala.collection.Iterator<U>>, boolean, scala.reflect.ClassTag<U>). Please see the API doc there.

Parameters:

f - (undocumented)

preservesPartitioning - (undocumented)

evidence$2 - (undocumented)

Returns:

(undocumented)

See Also:

BarrierTaskContext