Skip to content

8b. Refine Ideation

In the previous tutorial, you ran an Ideation in automated mode, where the system independently generated a variety of features and models.

In this tutorial, you’ll learn how to refine that ideation by incorporating domain expertise or leveraging prior results to produce more meaningful and efficient outcomes.

You will learn how to:

  • Prune uninformative columns using prior feature selection results and enable filter detection during ideation configuration.
  • Clone or reset an existing ideation to edit column transformations.
  • Create a new feature selection and train new models.

Prune Uninformative Columns and Enable Filters

By default, Filter Detection is skipped because it can generate a large number of features. Before enabling it, first prune uninformative columns using the results from a prior feature selection to limit the number of generated features.

Once pruning is complete, you can run ideation up to the Filters stage to review and refine suggestions.

Steps

  1. Start a new ideation Go to Ideation and click New Ideation Button to begin a new process. Name

  2. Edit ideation details Update the ideation name and description by clicking Edit Button. Name

  3. Configure ideation setup Click Config Button to open the configuration panel.

    Under Semantic Detection Setup, select a prior feature selection to prune uninformative columns. Name

    In the Filter Setup section, uncheck “Skip Filters.” Name

  4. Run ideation up to the Filters stage Set the auto-run configuration to stop at the Filters stage, then click Start Button to start ideation. Name

  5. Review suggested filters Examine the filters generated at the Filters stage. Name

  6. Manage filters Click Manage Filters Button to view, cancel, or create filters as needed. Name

Once your review is complete, either proceed to the next refinement step by clicking Next Step Button or move to the Modeling phase by clicking Start Button.

Clone Ideation and Edit Column Transforms

Next, you’ll clone a prior ideation to modify column transformations.

  1. Clone Ideation Navigate back to Fully Automated Ideation and open its Transform step. Click Clone Button to clone the ideation at the Transform step. Name

  2. Review suggested transforms Choose a table of interest and examine the suggested transforms. Name

  3. Manage transforms Click Manage Transforms Button to view, cancel, or create new transforms. Name

After completing your review, either proceed to the next refinement step or move to the Modeling phase as before.

Run New Feature Selection and Model

You can run new Feature Selections in two ways:

  • Configure the Feature Selection Setup before starting a new ideation.
  • Run a new Feature Selection from a completed ideation.

In this section, you’ll create a new selection from an existing ideation.

  1. Navigate back to Fully Automated Ideation and open its Feature Ideation step. Name

  2. Click the Magic Wand Magic Wand to start a new Selection. Name

  3. Go to the Rule Based tab and configure the following:

    • Feature Selection Name = Top 1 per theme
    • No. of Top Features Overall = 500
    • No. of Top Features per Theme = 1
    • Selection Logic = AND

    Name

  4. Review selected features Once the selection completes, review the chosen features. Name

  5. Train a new model Go to the Modeling Training step and click New Model to train a model using the new selection. Name

  6. Review model performance Check the results in the Ideation Leaderboard. As expected, this simplified selection may perform poorly. Name

Add Features to the Feature Catalog

Features used in Ideation models are automatically added to the Catalog. You can also manually add additional features.

  1. Go to the EDA step and its Features tab. Click Filter Button and clear all filters and searches using the clear icons. Name

  2. Select the NEW Readiness filter (features not yet in the Catalog). Name

  3. Choose a few features (for example, two). Name

  4. Click Save Feature List to save them to the Feature Catalog.

  5. Confirm the save by clicking Save Features. Name