F.DeFalco


I have a scenario in mind and was wondering if anyone had any suggestions on an approach.

Lets say I have a dataset where I have captured various attributes about blade servers in a rack. I have internal temperature, fan speed, disk reads, cpu temperature and failure events. Each of these are continuous variables except for the failure event, which has only two states, 'true' or 'false' which indicates whether a failure occurred at that point in time.

The table looks like this:

MachineID Timestamp Temperature FanSpeed Reads CPUTemp Failure

At the time of failure, only the MachineID, Timestamp and Failure values will be populated in the table. The failures at this time are not categorized or discriminated from each other in any way although they will be in the future. We're looking to use a mining algorithm to determine which variable is the best predictor of failure, or combination of variables. What do you think is the right approach How might this approach change once the failures do become categorized and differentiated

Thanks,

Frank




Re: Mining Approach?

Jamie MacLennan


This is an interesting problem and I would do some data preperation beforehand to solve this one.

Basically the problem comes to defining your "case". In this problem I would define a case as a "Machine/Time Period". The next problem is what do you want to predict For example, is predicting failure at the current timestamp useful Probably not. You probably want to predict failures ahead of time. This leads us to the problem of granularity. Is the timestamp too granular to be interesting Should you summarize the data to a higher level first

Let's say you want to predict if a failure will occur within the next five minutes based on minute-by-minute data for the last five minutes. In this situation you would need to aggregate your data at the minute level and create a "FailureInTheNextFiveMinutes" column. The previous five minutes would be represented as a nested table e.g.

Case Table

MachineID/Minute FailureInNextFiveMinutes

NestedTable

MachineID/Minute Minute Termperature FanSpeed Reads CPUTemp

In the nested table the first column is the reference to the case id and the "Minute" column is an integer 1-5 representing how long ago the row represents (e.g. 1 minute ago, 2 minutes ago). You would use the Minute column as the key.

Such a model would give you insights such as "If there are X number of reads 3 minutes ago and the temperature is above Y 1 minute ago you have a 60% chance of failure in the next five minutes".

Obviously you should tune this to fit your needs, but that's the general idea.

HTH

-Jamie







Re: Mining Approach?

F.DeFalco

Thanks for the reply.

Your suggestion confirms the direction we decided to take, in fact we probably simplfied it a little bit.

We're feeding a Neural Network with some 7 day aggregates (min,max,avg,std,var) on the different attributes we're capturing 14 days before failures. The hope is that we'll be able to predict issues 7 days before they actually occur, hence the historical snapshot.

I'm happy to report that the algorithm is performing fine, we're in the process of tweaking what we feed the algorithm to come up with the best prediction accuracy.

Frank






Re: Mining Approach?

ggciubuc

I have another point of view for this problem that suppose you can collect accurately training data sets.

Suppose you can get the following row periodically
MachineID Minute Termperature FanSpeed Reads CPUTemp CollectData Failure

where CollectData mean current data/time and Failure has the default value "No"

When the failure is done you have to have the last row before the crash so for that last row the value for Failure
column can be put "Yes". After you have an appropriate training data set (as a volume of data) you can build a data mining model with Failure as predictive column.

Gigi Ciubuc

www.sqlserver.ro