The Miner, an important datamine child Lambda, manages the data mining operations for the datamine. The Miner Lambda operates on projects which are self-enclosed spreadsheets wherein theoretical concepts can be tested against the datamine. The Miner Lambda selects one of the available projects which is loaded into memory for editing, viewing, and testing. Each project is composed of a list of testing tables, a survival of the fittest Score Lambda, and a list of filter Lambdas.
Each datamine project provides a self-enclosed spreadsheet wherein the dataMineLib can learn to perform better on a set of testing tables. The definition of "perform better" is not fixed, but is entered into the project as a user defined Score Lambda. The Score Lambda is designed to provide a numeric value to each of the testing tables. The dataMineLib trains a set of Filter Lambdas to filter each of the testing tables so that the resulting filtered testing tables receive better scores from the Score Lambda. Once again, the set of filter Lambdas is not fixed. Filter Lambdas are entered into the project by the user (usually written in the javaScript language provided with the dataMineLib).
There is no restriction on the complexity of the filter Lambdas in a project. Obviously, filter Lambdas can be made to learn from experience; and, filter Lambdas can even create other filter Lambdas and automatically add them to the project (for example, where techniques such as Genetic Programming are used).
Each project provides a "survival of the fittest" spreadsheet where the best performing filter Lambdas can be sorted to the top (best performing being determined by the user entered Score Lambda) and the worst performing filter Lambdas can be eliminated. Within each project, as machine learning occurs, the top scoring filter Lambdas perform better. Once again, the learning strategies employed by the filter Lambdas are not fixed. Filter Lambdas are entered into the project by the user, and may employ the latest and most sophisticated learning algorithms available.
The Blackboard is the datamine logical database wherein the projects are stored. The blackboard being a logical database is not restricted to any one physical datamine extent.
The testing tables provide the data upon which the filter Lambdas are trained. Each testing table is a standard table stored in the datamine.
The Score Lambda provides each table (in the list of testing tables) with a numeric score. As tables are filtered, their score may get better or worse. The purpose of the score Lambda is to guide the filter Lambdas (during machine learning).
Each Filter Lambda filters each table (in the list of testing tables), attempting to get a higher numeric score. As tables are filtered, their score may get better or worse. As each filter Lambda learns, its score should improve. Filter Lambdas are the components which learn during the process of data mining.
Each Testing Run loops until each filter Lambda has filtered every table (in the list of testing tables), receiving a numeric score. Each testing run fills in the project spreadsheet with the scores for each filter Lambda on each table. As each filter Lambda learns, its scores should improve. In addition, filter Lambdas can be compared with each other so that each project spreadsheet becomes a self-enclosed "survival of the fittest" environment wherein the best scoring filter Lambdas can be sorted to the top and the worst scoring filter Lambdas can be eliminated.
The Wizard is a user defined Lambda which uses the dataMineLib to perform data mining experiments, and which provides filter Lambdas for data mining projects. Wizards may submit filter Lambdas, to a project, which refer back to the Wizard during machine learning. Wizards are often used as meta learning Lambdas.
All table cursor objects support a simple Lambda Programmable Interface with functions for determining the size of tables, importing records to tables, exporting records from tables, writing records to tables, reading records from tables, etc. The functions available in the cursor API are listed in the following section:
The abortProject function closes the current project without saving.
All changes are lost.
Type: Function
Syntax: (dataMineLib.miner.abortProject)
| Returns | true |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The addProjectFilter function adds or updates a Filter Lambda to the
current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.addProjectFilter filterName filterSource)
| filterName | The name of the Filter Lambda to be added to the current project. |
| filterSource | The source (usually javaScript) for the Filter Lambda to be added to the current project. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The addProjectScore function adds or updates the Score Lambda for the
current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.addProjectScore scoreName scoreSource)
| scoreName | The name of the Score Lambda for the current project. |
| scoreSource | The source (usually javaScript) for the Score Lambda for the current project. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The addProjectTable function adds a testing table to the specified
miner project. The new testing table is added to the end of the list of testing
tables. The current project is saved and cleared (if any).
Type: Function
Syntax: (dataMineLib.miner.addProjectTable projectName tableName)
| projectName | The name of project to be updated. |
| tableName | The name of the testing table to add to the project. |
| Returns | true. |
The clearProjectScores function clears the scores for the current
project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.clearProjectScores)
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The currentProjectCursor property contains the cursor of the current
project spreadsheet table. If there is no project selected, this property will
be void.
Type: Property
Syntax: dataMineLib.miner.currentProjectCursor
The currentProjectName property contains the name of the current
project spreadsheet table. If there is no project selected, this property will
be void.
Type: Property
Syntax: dataMineLib.miner. currentProjectName
The currentTestingSW property is true iff the current project
spreadsheet table is in testing mode; otherwise, the value will be false. If
there is no project selected, this property will be void.
Type: Property
Syntax: dataMineLib.miner. currentTestingSW
The dropAutoFilters function deletes all automatically generated
Filter Lambdas from the current project (Filter Lambdas whose names start with
"_Auto"). The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.dropAutoFilters)
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The dropProject function deletes the specified project from the data
mine. The current project is saved and cleared (if any).
Type: Function
Syntax: (dataMineLib.miner.dropProject projectName)
| projectName | The project to be dropped from the datamine. |
| Returns | true. |
The dropProjectFilter function deletes the specified Filter Lambda
from the current project. The current project is not saved.
Type: Function
Syntax: (dataMineLib.miner.dropProjectFilter filterName)
| filterName | The Filter Lambda to be dropped from the current project. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The dropProjectTable function drops a testing table from the
specified miner project. The specified testing table is deleted from the list
of the list of testing tables. The current project is saved and cleared (if
any).
Type: Function
Syntax: (dataMineLib.miner.dropProjectTable projectName tableName)
| projectName | The name of project to be updated. |
| tableName | The name of the testing table to be dropped from the project. |
| Returns | true. |
The getFilterLambda function returns the Filter Lambda, in the current
project, for the specified filter Lambda name. The current project is unaffected
(if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterLambda filterName)
| filterName | The name of the Filter Lambda whose Lambda is to be returned. |
| Returns | The Filter Lambda corresponding to the specified filter name. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getFilterHistory function returns the history structure for the
specified Filter Lambda in the current project. The current project is
unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterHistory filterName)
| filterName | The name of the Filter Lambda whose History structure is to be returned. |
| Returns | The History structure of the specified Filter Lambda. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getFilterNames function returns the sorted names of all Filter
Lambdas in the current project. Both the sort order and the sort field are
specified in the arguments. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterNames sortOrder sortField)
| sortOrder | The sort order: down or up. |
| sortField | The sort field: Avg, Min, Max, Sum, Ssq, Std, Rar, Num, and Cnt. |
| Returns | A vector of the sorted names of all Filter Lambdas in the current project. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getFilterScores function returns the scores structure for the
specified Filter Lambda in the current project. The current project is
unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterScores filterName)
| filterName | The name of the Filter Lambda whose scores structure is to be returned. |
| Returns | The scores structure of the specified Filter Lambda. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getFilterSource function returns the source string for the
specified Filter Lambda in the current project. The current project is
unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getFilterSource filterName)
| filterName | The name of the Filter Lambda whose source string is to be returned. |
| Returns | The source string of the specified Filter Lambda. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getProjectNames function returns a vector of project names in the
datamine. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getProjectNames)
| Returns | A vector of project names in the datamine. |
The getProjectTable function returns the name of the specified
testing table in the current project. The current project is unaffected (if
any).
Type: Function
Syntax: (dataMineLib.miner.getProjectTable tableIndex)
| tableIndex | The index of the testing table in the current project. |
| Returns | The name of the specified testing table in the current project. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getScoreName function returns the Score Lambda name for the
current miner project. The current project is unaffected.
Type: Function
Syntax: (dataMineLib.miner.getScoreName)
| Returns | The Score Lambda name for the current miner project. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The getScoreSource function returns the source string for the Score
Lambda in the current project. The current project is unaffected (if any).
Type: Function
Syntax: (dataMineLib.miner.getScoreSource)
| Returns | The source string of the project’s Score Lambda. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The newProject function adds a new project to the datamine. The
current project is saved (if any) and the current project is set to the newly
created project. The project name, score Lambda name, score Lambda source, and
the list of testing table are specified by the user.
Type: Function
Syntax: (dataMineLib.miner.newProject prjName tblNames scoreName scrSource
filters)
| prjName | The name of the new project in the datamine (must not conflict with any existing table names). |
| tblTables | The object vector of testing table names (must be an ObjVector). |
| scoreName | The name of the new project’s Score Lambda. |
| scrSource | The source code (usually written in javaScript) of the new project’s Score Lambda. |
| filters | A structure containing the names and source code for the initial population of filter Lambdas for this project. |
| Returns | true |
The openProject function opens the specified project as the current
project of the miner Lambda. The current project is saved (if any) and the
specified project is set as the new miner project. The project may be opened
for editing or for testing.
Type: Function
Syntax: (dataMineLib.miner.openProject projectName)
Syntax: (dataMineLib.miner.openProject projectName testingSW)
| projectName | The name of the new project in the datamine (must not conflict with any existing table names). |
| testingSW | (Optional) Set to true iff the project is to be opened for testing, false iff the project is to be opened for editing. |
| Returns | true |
The registerProjectsAsExtents function registers all datamine
Blackboard Project Tables to the browseLib. The various project tables can
then be managed by the browseLib.
Type: Function
Syntax: (dataMineLib.miner.registerProjectsAsExtents
| Returns | true. |
Note: This function only registers those project tables currently in the Blackboard.
The renameProjectFilter function renames the specified Filter Lambda in the current project. The current project is not saved (see saveProject).
Note: This function assumes that a miner project has been opened for
edit (see openProject).
Type: Function
Syntax: (dataMineLib.miner.renameProjectFilter oldName newName)
| oldName | The old name of the Filter Lambda in the current project. |
| newName | The new name of the Filter Lambda in the current project. |
| Returns | true. |
The renameProjectScore function renames the Score Lambda for the
current project. The current project is not saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.renameProjectScore scoreName)
| scoreName | The name of the Score Lambda for the current project. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The runProjectFilter function runs the specified Filter Lambda, in the
current project, against the specified table cursor. The current project is not
saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.runProjectFilter filterName tableCursor)
| filterName | The name of the Filter Lambda in the current project. |
| tableCursor | The table cursor to be filtered. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The saveFilterSource function adds or updates a Filter Lambda to the
current project (but only the source is changed). The current project is not
saved (see saveProject).
Type: Function
Syntax: (dataMineLib.miner.saveFilterSource filterName filterSource)
| filterName | The name of the Filter Lambda to be added to the current project. |
| filterSource | The source (usually javaScript) for the Filter Lambda to be added to the current project. |
| Returns | true. |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The saveProject function saves the current project. If the optional
clear keyword is present, the project may be cleared after saving.
Type: Function
Syntax: (dataMineLib.miner.saveProject clear:)
Syntax: (dataMineLib.miner.saveProject)
| clear: | Optional keyword to request that the project be cleared as well as saved. |
| Returns | true |
Note: This function assumes that a miner project has been opened for edit (see openProject).
The testRun function tests every Filter Lambda, in the current
Project, against every testing table. A score is recorded, in the current
Project spreadsheet, for each Filter Lambda and each testing table. Prior to the
testing run, the current Project is saved and closed.
Type: Function
Syntax: (dataMineLib.miner.testRun)
| Returns | true |
Note: This function assumes that a miner project has been opened for edit (see openProject).