How to do it...

There are a few settings you can use to control speculation. The examples are provided only to show how to change values. Mostly, just turning on speculation is good enough:

  1. Setting spark.speculation (the default is false):
        $ spark-shell -conf spark.speculation=true
  1. Setting spark.speculation.interval (the default is 100 milliseconds) (denotes the rate at which Spark examines tasks to see whether speculation is needed): 
        $ spark-shell -conf spark.speculation.interval=200
  1. Setting spark.speculation.multiplier (the default is 1.5) (denotes how many times a task has to be slower than median to be a candidate for speculation):
        $ spark-shell -conf spark.speculation.multiplier=1.5
  1. Setting spark.speculation.quantile (the default is 0.75) (denotes the percentage of tasks to be completed before triggering speculation):
        $ spark-shell -conf spark.speculation.quantile=0.95
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset