org.apache.hadoop.mapred
Class ShuffleExceptionTracker

java.lang.Object
  extended by org.apache.hadoop.mapred.ShuffleExceptionTracker

public class ShuffleExceptionTracker
extends Object

This class is used to track shuffle exceptions. It contains routines to check an exception that occurred while fetching the Map output to see if it matches what was configured. It also provides functions to keep track of the number of exceptions that occurred and if a limit is set, it will abort the TT. The limit is a percent of exceptions of the last X number of requests.


Field Summary
static org.apache.commons.logging.Log LOG
           
 
Method Summary
 boolean checkException(IOException ie)
          Parse the exception to see if it matches the regular expression you configured.
protected  void doAbort()
           
 void exception()
          Mark the request as an exception occurred.
 int getNumRequests()
          Gets the number of requests we are tracking
 float getPercentExceptions()
          Gets the percent of the requests that had exceptions occur.
 void success()
          Mark the request as success.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Method Detail

getNumRequests

public int getNumRequests()
Gets the number of requests we are tracking

Returns:
number of requests

getPercentExceptions

public float getPercentExceptions()
Gets the percent of the requests that had exceptions occur.

Returns:
percent failures as float

success

public void success()
Mark the request as success.


exception

public void exception()
Mark the request as an exception occurred.


checkException

public boolean checkException(IOException ie)
Parse the exception to see if it matches the regular expression you configured. If both msgRegex and StackRegex are set then make sure both match, otherwise only one has to match. Abort if the limit is hit.

Parameters:
ie - - the shuffle exception that occurred
Returns:
true if the exception matches, false otherwise

doAbort

protected void doAbort()


Copyright © 2009 The Apache Software Foundation