Miner Lambda

 

Miner Lambda Overview

The Miner, an important datamine child Lambda, manages the data mining operations for the datamine. The Miner Lambda operates on projects which are self-enclosed spreadsheets wherein theoretical concepts can be tested against the datamine. The Miner Lambda selects one of the available projects which is loaded into memory for editing, viewing, and testing. Each project is composed of a list of testing tables, a survival of the fittest Score Lambda, and a list of filter Lambdas.

Projects

Each datamine project provides a self-enclosed spreadsheet wherein the dataMineLib can learn to perform better on a set of testing tables. The definition of "perform better" is not fixed, but is entered into the project as a user defined Score Lambda. The Score Lambda is designed to provide a numeric value to each of the testing tables. The dataMineLib trains a set of Filter Lambdas to filter each of the testing tables so that the resulting filtered testing tables receive better scores from the Score Lambda. Once again, the set of filter Lambdas is not fixed. Filter Lambdas are entered into the project by the user (usually written in the javaScript language provided with the dataMineLib).

There is no restriction on the complexity of the filter Lambdas in a project. Obviously, filter Lambdas can be made to learn from experience; and, filter Lambdas can even create other filter Lambdas and automatically add them to the project (for example, where techniques such as Genetic Programming are used).

Each project provides a "survival of the fittest" spreadsheet where the best performing filter Lambdas can be sorted to the top (best performing being determined by the user entered Score Lambda) and the worst performing filter Lambdas can be eliminated. Within each project, as machine learning occurs, the top scoring filter Lambdas perform better. Once again, the learning strategies employed by the filter Lambdas are not fixed. Filter Lambdas are entered into the project by the user, and may employ the latest and most sophisticated learning algorithms available.

Blackboard

The Blackboard is the datamine logical database wherein the projects are stored. The blackboard being a logical database is not restricted to any one physical datamine extent.

Testing Tables

The testing tables provide the data upon which the filter Lambdas are trained. Each testing table is a standard table stored in the datamine.

Miner Score Lambda

The Score Lambda provides each table (in the list of testing tables) with a numeric score. As tables are filtered, their score may get better or worse. The purpose of the score Lambda is to guide the filter Lambdas (during machine learning).

Miner Filter Lambda

Each Filter Lambda filters each table (in the list of testing tables), attempting to get a higher numeric score. As tables are filtered, their score may get better or worse. As each filter Lambda learns, its score should improve. Filter Lambdas are the components which learn during the process of data mining.

Testing Runs

Each Testing Run loops until each filter Lambda has filtered every table (in the list of testing tables), receiving a numeric score. Each testing run fills in the project spreadsheet with the scores for each filter Lambda on each table. As each filter Lambda learns, its scores should improve. In addition, filter Lambdas can be compared with each other so that each project spreadsheet becomes a self-enclosed "survival of the fittest" environment wherein the best scoring filter Lambdas can be sorted to the top and the worst scoring filter Lambdas can be eliminated.

Wizards

The Wizard is a user defined Lambda which uses the dataMineLib to perform data mining experiments, and which provides filter Lambdas for data mining projects. Wizards may submit filter Lambdas, to a project, which refer back to the Wizard during machine learning. Wizards are often used as meta learning Lambdas.

Miner Lambda API

All table cursor objects support a simple Lambda Programmable Interface with functions for determining the size of tables, importing records to tables, exporting records from tables, writing records to tables, reading records from tables, etc. The functions available in the cursor API are listed in the following section:

abortProject

The abortProject function closes the current project without saving. All changes are lost.
Type: Function
Syntax: (dataMineLib.miner.abortProject)

Returns true

Note: This function assumes that a miner project has been opened for edit (see openProject).

addProjectFilter

The addProjectFilter function adds or updates a Filter Lambda to the current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.addProjectFilter filterName filterSource)

filterName The name of the Filter Lambda to be added to the current project.
filterSource The source (usually javaScript) for the Filter Lambda to be added to the current project.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

addProjectScore

The addProjectScore function adds or updates the Score Lambda for the current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.addProjectScore scoreName scoreSource)

scoreName The name of the Score Lambda for the current project.
scoreSource The source (usually javaScript) for the Score Lambda for the current project.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

addProjectTable

The addProjectTable function adds a testing table to the specified miner project. The new testing table is added to the end of the list of testing tables. The current project is saved and cleared (if any).
Type: Function
Syntax: (dataMineLib.miner.addProjectTable projectName tableName)

projectName The name of project to be updated.
tableName The name of the testing table to add to the project.
Returns true.

clearProjectScores

The clearProjectScores function clears the scores for the current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.clearProjectScores)

Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

currentProjectCursor

The currentProjectCursor property contains the cursor of the current project spreadsheet table. If there is no project selected, this property will be void.
Type: Property
Syntax: dataMineLib.miner.currentProjectCursor

currentProjectName

The currentProjectName property contains the name of the current project spreadsheet table. If there is no project selected, this property will be void.
Type: Property
Syntax: dataMineLib.miner. currentProjectName

currentTestingSW

The currentTestingSW property is true iff the current project spreadsheet table is in testing mode; otherwise, the value will be false. If there is no project selected, this property will be void.
Type: Property
Syntax: dataMineLib.miner. currentTestingSW

dropAutoFilters

The dropAutoFilters function deletes all automatically generated Filter Lambdas from the current project (Filter Lambdas whose names start with "_Auto"). The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.dropAutoFilters)

Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

dropProject

The dropProject function deletes the specified project from the data mine. The current project is saved and cleared (if any).
Type: Function
Syntax: (dataMineLib.miner.dropProject projectName)

projectName The project to be dropped from the datamine.
Returns true.

dropProjectFilter

The dropProjectFilter function deletes the specified Filter Lambda from the current project. The current project is not saved.
Type: Function
Syntax: (dataMineLib.miner.dropProjectFilter filterName)

filterName The Filter Lambda to be dropped from the current project.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

dropProjectTable

The dropProjectTable function drops a testing table from the specified miner project. The specified testing table is deleted from the list of the list of testing tables. The current project is saved and cleared (if any).
Type: Function
Syntax: (dataMineLib.miner.dropProjectTable projectName tableName)

projectName The name of project to be updated.
tableName The name of the testing table to be dropped from the project.
Returns true.

getFilterLambda

The getFilterLambda function returns the Filter Lambda, in the current project, for the specified filter Lambda name. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterLambda filterName)

filterName The name of the Filter Lambda whose Lambda is to be returned.
Returns The Filter Lambda corresponding to the specified filter name.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getFilterHistory

The getFilterHistory function returns the history structure for the specified Filter Lambda in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterHistory filterName)

filterName The name of the Filter Lambda whose History structure is to be returned.
Returns The History structure of the specified Filter Lambda.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getFilterNames

The getFilterNames function returns the sorted names of all Filter Lambdas in the current project. Both the sort order and the sort field are specified in the arguments. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterNames sortOrder sortField)

sortOrder The sort order: down or up.
sortField The sort field: Avg, Min, Max, Sum, Ssq, Std, Rar, Num, and Cnt.
Returns A vector of the sorted names of all Filter Lambdas in the current project.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getFilterScores

The getFilterScores function returns the scores structure for the specified Filter Lambda in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterScores filterName)

filterName The name of the Filter Lambda whose scores structure is to be returned.
Returns The scores structure of the specified Filter Lambda.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getFilterSource

The getFilterSource function returns the source string for the specified Filter Lambda in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterSource filterName)

filterName The name of the Filter Lambda whose source string is to be returned.
Returns The source string of the specified Filter Lambda.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getProjectNames

The getProjectNames function returns a vector of project names in the datamine. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getProjectNames)

Returns A vector of project names in the datamine.

getProjectTable

The getProjectTable function returns the name of the specified testing table in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getProjectTable tableIndex)

tableIndex The index of the testing table in the current project.
Returns The name of the specified testing table in the current project.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getScoreName

The getScoreName function returns the Score Lambda name for the current miner project. The current project is unaffected.
Type: Function
Syntax: (dataMineLib.miner.getScoreName)

Returns The Score Lambda name for the current miner project.

Note: This function assumes that a miner project has been opened for edit (see openProject).

getScoreSource

The getScoreSource function returns the source string for the Score Lambda in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getScoreSource)

Returns The source string of the project’s Score Lambda.

Note: This function assumes that a miner project has been opened for edit (see openProject).

newProject

The newProject function adds a new project to the datamine. The current project is saved (if any) and the current project is set to the newly created project. The project name, score Lambda name, score Lambda source, and the list of testing table are specified by the user.
Type: Function
Syntax: (dataMineLib.miner.newProject prjName tblNames scoreName scrSource filters)

prjName The name of the new project in the datamine (must not conflict with any existing table names).
tblTables The object vector of testing table names (must be an ObjVector).
scoreName The name of the new project’s Score Lambda.
scrSource The source code (usually written in javaScript) of the new project’s Score Lambda.
filters A structure containing the names and source code for the initial population of filter Lambdas for this project.
Returns true

openProject

The openProject function opens the specified project as the current project of the miner Lambda. The current project is saved (if any) and the specified project is set as the new miner project. The project may be opened for editing or for testing.
Type: Function
Syntax: (dataMineLib.miner.openProject projectName)
Syntax: (dataMineLib.miner.openProject projectName testingSW)

projectName The name of the new project in the datamine (must not conflict with any existing table names).
testingSW (Optional) Set to true iff the project is to be opened for testing, false iff the project is to be opened for editing.
Returns true

registerProjectsAsExtents

The registerProjectsAsExtents function registers all datamine Blackboard Project Tables to the browseLib. The various project tables can then be managed by the browseLib.
Type: Function
Syntax: (dataMineLib.miner.registerProjectsAsExtents

Returns true.

Note: This function only registers those project tables currently in the Blackboard.

renameProjectFilter

The renameProjectFilter function renames the specified Filter Lambda in the current project. The current project is not saved (see saveProject).

Note: This function assumes that a miner project has been opened for edit (see openProject).
Type: Function
Syntax: (dataMineLib.miner.renameProjectFilter oldName newName)

oldName The old name of the Filter Lambda in the current project.
newName The new name of the Filter Lambda in the current project.
Returns true.

renameProjectScore

The renameProjectScore function renames the Score Lambda for the current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.renameProjectScore scoreName)

scoreName The name of the Score Lambda for the current project.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

runProjectFilter

The runProjectFilter function runs the specified Filter Lambda, in the current project, against the specified table cursor. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.runProjectFilter filterName tableCursor)

filterName The name of the Filter Lambda in the current project.
tableCursor The table cursor to be filtered.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

saveFilterSource

The saveFilterSource function adds or updates a Filter Lambda to the current project (but only the source is changed). The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.saveFilterSource filterName filterSource)

filterName The name of the Filter Lambda to be added to the current project.
filterSource The source (usually javaScript) for the Filter Lambda to be added to the current project.
Returns true.

Note: This function assumes that a miner project has been opened for edit (see openProject).

saveProject

The saveProject function saves the current project. If the optional clear keyword is present, the project may be cleared after saving.
Type: Function
Syntax: (dataMineLib.miner.saveProject clear:)
Syntax: (dataMineLib.miner.saveProject)

clear: Optional keyword to request that the project be cleared as well as saved.
Returns true

Note: This function assumes that a miner project has been opened for edit (see openProject).

testRun

The testRun function tests every Filter Lambda, in the current Project, against every testing table. A score is recorded, in the current Project spreadsheet, for each Filter Lambda and each testing table. Prior to the testing run, the current Project is saved and closed.
Type: Function
Syntax: (dataMineLib.miner.testRun)

Returns true

Note: This function assumes that a miner project has been opened for edit (see openProject).