The Pregel API in GraphX

Now, let's formalize the programming interface for the Pregel operator. Here is its definition:

class GraphOps[VD, ED] {
  def pregel[A]
      (initialMsg: A,
       maxIter: Int = Int.MaxValue,
       activeDir: EdgeDirection = EdgeDirection.Out)
      (vprog: (VertexId, VD, A) => VD,
       sendMsg: EdgeTriplet[VD, ED] => Iterator[(VertexId, A)],
       mergeMsg: (A, A) => A)
    : Graph[VD, ED]
}

The pregel method is invoked on a property graph, and returns a new graph with the same type and structure. While the edges remain intact, the attributes of the vertices may change from one superset to the next one. Pregel takes the following two lists of arguments. The first list contains:

  • An initial message with a user-defined type A—this message is received by each vertex when the algorithm starts
  • A maximum number of iterations
  • The edge direction along which to send messages

Note

A Pregel algorithm terminates when either there are no more messages to be sent, or when a specified maximum number of iterations is reached. When implementing an algorithm, it is important to always limit the number of iterations, especially when the algorithm is not guaranteed to converge.

If no active edge direction is specified, Pregel assumes that messages are only sent for the outgoing edges of each vertex. Moreover, if a vertex did not receive a message in the previous superset, no message will be sent along its outgoing edge, at the end of the current superset.

In addition, the second list of arguments must include the three functions:

vprog: (VertexId, VD, A) => VD: this vertex program updates the attributes of all vertices who received messages from the previous iteration
mergeMsg: (A, A) => A): this function merges the messages to be received by each vertex.
sendMsg: EdgeTriplet[VD, ED] => Iterator[(VertexId, A)]: this function takes an edge triplet and creates the messages to be sent to the source node and/or destination node.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset